2022-08-01T22:46:27.879 INFO:root:teuthology version: 1.1.1.dev401+gdd7eb9d9 2022-08-01T22:46:27.881 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891 --verbose --owner scheduled_yuriw@teuthology --description fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --name yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi -- /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/orig.config.yaml 2022-08-01T22:46:27.940 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-08-01T22:46:28.093 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891 branch: wip-yuri6-testing-2022-08-01-0839-octopus description: fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: null first_in_suite: false job_id: '6954891' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri6-testing-2022-08-01-0839-octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2022-08-01-0839-octopus sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 owner: scheduled_yuriw@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 sleep_before_teardown: 0 suite: fs suite_branch: wip-yuri6-testing-2022-08-01-0839-octopus suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 targets: smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzz6chc/wq1TozbceqNOlZe9hGR4T5xcedJzyr+rA0Vg8YCeDHhjfBX9A90CediU+NH2sM8a7VqJGeZOTxfxRKByRExaPJcyc+rKZudCsNWSBp1UvtDOVxKgRZil4hT3/2BOlCwkwguwVAxaQibge6tYyHfDHEsE5mp/E4TTK2CcMxrToEcgJec775episHdFBB5YGHHRWXwQDT6Lcb7rfWY0V3/LjHBA++520J6D6QrHHlPXiTydNbweWoVIkRpxleVAwb6xcwotHc/xMHJMTG67NpWFLzQlySXMObHZB7Fa1f/h36nRinGY6Wvg+yjSi4PwYF6+CH9vvW3lP0nS6scAqgOFlXjIyjV4Q8ndQu2pn/udLfP2qwyylEL0mX+QwJTIWnNFFeVwi9wI7Ugh3w1Hgjt2D/ziWWT04mOuqqan4nnUfV5a7aBEYfbsXjngEUJoTPe+Sk79Hh3pWHKrSaJ75/KbHR5uqCZjW2iwFNbfRKDd6go8YBoxCX6wF/oc= smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA27u4nPPWAmT2f19t4k7S/N13/pEV4EXMw2HZ2iXmfqREBVYszdJuZiop4ncUpfEGHVwQTjPP0dRbmK06h6N1hToSYQCvCt1fm4rmH94FZO+gJp9Ay7pFTEnOJKf9DS7bqEbgAz9TIT8YEJPymj5ET6foP2Lf6CgwlN6ggbGQWxrVXuzqW5Prj9r5LnOZinKj7QEMvVmBOgx7Wn404GoYvZwBtq843kUKDnIVg44xp98UGD0ObPgvza7+wAIn16QxeZFOEP+2KI/F1RANOdHBPCr93UoS4l8cXIoNHHQTIRA56f9hFIv+bTH88oy2FNlr3HhdZ7m0UNGzgA5k2wDFwNbPaZOsh7vUyHUXEhEK6tQUbK8HP3+kIfY0xQkQJ1v40QuCJJrG0o7NRYdPPhVbdu+VEdkPbjt7R6qByHnspJ1Mz3DFHe7+J+x9m/8d3KL3VUA+4JLgqW7OpgJlqEBOYuDv23OTl/hN+JEjv6X1+7csBtzZsMyTAgBf0HMZNek= smithi130.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTxZNF1k1C8xA3EIsDv9994ft3Ax8y6u0M1raCXCYBKbRdpZmYGA5FY5i56RYxO2izD6K3pBgrjG03nw68aY0s6P6h4cuoANe+up34aNUvXKc+s66UWs4XgiAuIylcIY9GZyEDdLplTIXbgjJnzd/PTY9OTBCQffM4PMtfyMJv9Dk40WrCMo6XLrvYU+zlHEFA/YAgwnSIDTZ7+XFRa9nnD5386ryshXY0F3chXUTNLrqELizltzli9Ua9IT975muNDVmBFGUU29zTKKwkQ9oV1EjA1XQy90eK9DAKCWGrHpPaPwPszJ2+p2JC1Z7P/ah2ofeaaaAkWjHesl1pyRsmFaGpYgocuzBNebxP+xZ6AasfRqkm6RtI9zcCmA7+bSqrN3jU3p86ebzvnaSL8kO6D8/ESU4zYkjJQyQJrPd+PWWW6qvjmewSZHwjmIRm7Qi0u2VGhOLM2QY/+D2djcVER83KqNRwyY3s8Ld0ffHPUMyVbc/pjHVu+opbGmR3QV0= smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjmyxvBG42FX0ccPqpKTftkvOdu+icNorpE0+Gi1rzkncMT06zLDWiS8g5c/HSdLNDpvBmJ07NIAerPq0IuPnMrCblr+0coD0BV+dqQM47uPv2aRyfNaCZ/KPuhxcjeBKx2WfYGwXFgWYiLQPUoXWuqAWJXUkQM2zd/eexpyIx/z2SNlL1FSS6QdCaCZE/RbGyCU72nuTBxeRAoGOMUN7Wtw/N5q8bnotHuoA2pKfmnVijI5qQCC5xNkymHgLRCv5EV3pkijxqhgBBDrUUF1aASSKXzYoBHl7eq2jqCubQ+izhBxCStbF3LG6aLOa4GZ+IhzqLbXBkyRGAqV8lGfaRIvSwfNwrIRDFXNrH4aU5x6nYRWn19EYTJuANXZpmhcNp9rFAZ3Ly9Fsy4tHOkPP68jJCMaWRYqP9H3gUCs2nHiLFT/z/zqowfwH6YkAVq4pKYVYQKRNGmMeBVEOKf8dHuAKpBFTBE8MsleKBmJGjGHrgHCUaffwILdtbc3zSRnk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: dd7eb9d989259767783fb5ee9a761cd0732b84e6 timestamp: 2022-08-01_22:38:49 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1713 2022-08-01T22:46:28.094 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa; will attempt to use it 2022-08-01T22:46:28.095 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks 2022-08-01T22:46:28.097 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-08-01T22:46:28.102 INFO:teuthology.task.internal:Checking packages... 2022-08-01T22:46:28.144 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'fd96719b0e1200b56e7430604cfdd20ad281ca92' 2022-08-01T22:46:28.145 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-08-01T22:46:28.145 INFO:teuthology.packaging:ref: None 2022-08-01T22:46:28.146 INFO:teuthology.packaging:tag: None 2022-08-01T22:46:28.146 INFO:teuthology.packaging:branch: wip-yuri6-testing-2022-08-01-0839-octopus 2022-08-01T22:46:28.146 INFO:teuthology.packaging:sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:46:28.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri6-testing-2022-08-01-0839-octopus 2022-08-01T22:46:28.317 INFO:teuthology.task.internal:Found packages for ceph version 15.2.16-343-gfd96719b-1focal 2022-08-01T22:46:28.322 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-08-01T22:46:28.334 INFO:teuthology.task.internal:no buildpackages task found 2022-08-01T22:46:28.334 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-08-01T22:46:28.344 INFO:teuthology.task.internal:Saving configuration 2022-08-01T22:46:28.374 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-08-01T22:46:28.380 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-01T22:46:28.420 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 22:39:29.790818', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAJhc8GJ9PsU2IviYe/AVLUSB55cvz+lQ6zoj8omiL6Y'} 2022-08-01T22:46:28.450 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 22:39:29.793042', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKv0HyES+F/LS5a1ZdSmWjDv/KSwkYie2qIpRGPtrGj5'} 2022-08-01T22:46:28.487 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 22:39:29.791956', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH+VpXhY22uDejzKKOxHBqXHZkGHtM998MZOwj4cgeGU'} 2022-08-01T22:46:28.516 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 22:39:29.789292', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEd96nDV81LnTRCRGegC40geBnMp6IeXPZHwjVaH52gN'} 2022-08-01T22:46:28.517 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-08-01T22:46:28.528 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-08-01T22:46:28.528 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-08-01T22:46:28.529 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['client.0'] 2022-08-01T22:46:28.529 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['client.1'] 2022-08-01T22:46:28.529 INFO:teuthology.run_tasks:Running task console_log... 2022-08-01T22:46:28.791 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0918f46ae8>, signals=[15]) 2022-08-01T22:46:28.793 INFO:teuthology.run_tasks:Running task internal.connect... 2022-08-01T22:46:28.809 INFO:teuthology.task.internal:Opening connections... 2022-08-01T22:46:28.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T22:46:28.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:46:28.940 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T22:46:28.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:46:29.064 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2022-08-01T22:46:29.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:46:29.200 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2022-08-01T22:46:29.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:46:29.318 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-08-01T22:46:29.334 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2022-08-01T22:46:29.342 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2022-08-01T22:46:29.343 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2022-08-01T22:46:29.389 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2022-08-01T22:46:29.390 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T22:46:29.390 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2022-08-01T22:46:29.390 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2022-08-01T22:46:29.390 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T22:46:29.391 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="20.04" 2022-08-01T22:46:29.391 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T22:46:29.391 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T22:46:29.391 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T22:46:29.392 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T22:46:29.392 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=focal 2022-08-01T22:46:29.393 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=focal 2022-08-01T22:46:29.394 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2022-08-01T22:46:29.422 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2022-08-01T22:46:29.435 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2022-08-01T22:46:29.435 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2022-08-01T22:46:29.482 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Ubuntu" 2022-08-01T22:46:29.483 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T22:46:29.483 INFO:teuthology.orchestra.run.smithi055.stdout:ID=ubuntu 2022-08-01T22:46:29.484 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE=debian 2022-08-01T22:46:29.484 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T22:46:29.485 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="20.04" 2022-08-01T22:46:29.485 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T22:46:29.486 INFO:teuthology.orchestra.run.smithi055.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T22:46:29.486 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T22:46:29.487 INFO:teuthology.orchestra.run.smithi055.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T22:46:29.487 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_CODENAME=focal 2022-08-01T22:46:29.487 INFO:teuthology.orchestra.run.smithi055.stdout:UBUNTU_CODENAME=focal 2022-08-01T22:46:29.488 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2022-08-01T22:46:29.513 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2022-08-01T22:46:29.522 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2022-08-01T22:46:29.522 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2022-08-01T22:46:29.570 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="Ubuntu" 2022-08-01T22:46:29.571 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T22:46:29.571 INFO:teuthology.orchestra.run.smithi130.stdout:ID=ubuntu 2022-08-01T22:46:29.572 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE=debian 2022-08-01T22:46:29.572 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T22:46:29.574 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="20.04" 2022-08-01T22:46:29.574 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T22:46:29.575 INFO:teuthology.orchestra.run.smithi130.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T22:46:29.575 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T22:46:29.575 INFO:teuthology.orchestra.run.smithi130.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T22:46:29.576 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_CODENAME=focal 2022-08-01T22:46:29.576 INFO:teuthology.orchestra.run.smithi130.stdout:UBUNTU_CODENAME=focal 2022-08-01T22:46:29.576 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2022-08-01T22:46:29.597 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2022-08-01T22:46:29.606 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2022-08-01T22:46:29.606 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2022-08-01T22:46:29.655 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Ubuntu" 2022-08-01T22:46:29.655 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T22:46:29.656 INFO:teuthology.orchestra.run.smithi157.stdout:ID=ubuntu 2022-08-01T22:46:29.656 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE=debian 2022-08-01T22:46:29.656 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T22:46:29.657 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="20.04" 2022-08-01T22:46:29.657 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T22:46:29.657 INFO:teuthology.orchestra.run.smithi157.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T22:46:29.658 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T22:46:29.658 INFO:teuthology.orchestra.run.smithi157.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T22:46:29.659 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_CODENAME=focal 2022-08-01T22:46:29.659 INFO:teuthology.orchestra.run.smithi157.stdout:UBUNTU_CODENAME=focal 2022-08-01T22:46:29.660 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2022-08-01T22:46:29.689 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-08-01T22:46:29.701 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-08-01T22:46:29.709 INFO:teuthology.task.internal:Checking for old test directory... 2022-08-01T22:46:29.710 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2022-08-01T22:46:29.714 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2022-08-01T22:46:29.718 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2022-08-01T22:46:29.722 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2022-08-01T22:46:29.729 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-08-01T22:46:29.744 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-08-01T22:46:29.745 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2022-08-01T22:46:29.760 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2022-08-01T22:46:29.769 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2022-08-01T22:46:29.772 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2022-08-01T22:46:29.792 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-08-01T22:46:29.926 INFO:teuthology.run_tasks:Running task kernel... 2022-08-01T22:46:29.961 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-08-01T22:46:29.963 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-08-01T22:46:29.964 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T22:46:29.966 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T22:46:29.967 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T22:46:29.968 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T22:46:29.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:46:29.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:46:29.979 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2022-08-01T22:46:29.980 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2022-08-01T22:46:29.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:46:29.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:46:29.982 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2022-08-01T22:46:29.983 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2022-08-01T22:46:29.995 INFO:teuthology.orchestra.run.smithi157.stdout:5.4.0-122-generic 2022-08-01T22:46:29.996 INFO:teuthology.orchestra.run.smithi130.stdout:5.4.0-122-generic 2022-08-01T22:46:29.997 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.4.0-122-generic 2022-08-01T22:46:29.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get clean 2022-08-01T22:46:29.998 INFO:teuthology.task.kernel:Running kernel on smithi130: 5.4.0-122-generic 2022-08-01T22:46:29.998 DEBUG:teuthology.orchestra.run.smithi130:> sudo apt-get clean 2022-08-01T22:46:30.027 INFO:teuthology.orchestra.run.smithi055.stdout:5.4.0-122-generic 2022-08-01T22:46:30.028 INFO:teuthology.task.kernel:Running kernel on smithi055: 5.4.0-122-generic 2022-08-01T22:46:30.028 DEBUG:teuthology.orchestra.run.smithi055:> sudo apt-get clean 2022-08-01T22:46:30.029 INFO:teuthology.orchestra.run.smithi032.stdout:5.4.0-122-generic 2022-08-01T22:46:30.030 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.4.0-122-generic 2022-08-01T22:46:30.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2022-08-01T22:46:30.152 DEBUG:teuthology.orchestra.run.smithi055:> sudo apt-get update 2022-08-01T22:46:30.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2022-08-01T22:46:30.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2022-08-01T22:46:30.166 DEBUG:teuthology.orchestra.run.smithi130:> sudo apt-get update 2022-08-01T22:46:30.337 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T22:46:30.339 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:46:30.339 INFO:teuthology.orchestra.run.smithi130.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T22:46:30.341 INFO:teuthology.orchestra.run.smithi055.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T22:46:30.361 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:46:30.361 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T22:46:30.362 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T22:46:30.416 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T22:46:30.440 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:46:30.441 INFO:teuthology.orchestra.run.smithi055.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T22:46:30.450 INFO:teuthology.orchestra.run.smithi130.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:46:30.522 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T22:46:30.531 INFO:teuthology.orchestra.run.smithi130.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T22:46:30.652 INFO:teuthology.orchestra.run.smithi130.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,637 kB] 2022-08-01T22:46:30.666 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,637 kB] 2022-08-01T22:46:30.715 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,000 kB] 2022-08-01T22:46:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T22:46:30.747 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,637 kB] 2022-08-01T22:46:30.877 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T22:46:30.885 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T22:46:30.886 INFO:teuthology.orchestra.run.smithi130.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T22:46:31.059 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,637 kB] 2022-08-01T22:46:31.061 INFO:teuthology.orchestra.run.smithi130.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [473 kB] 2022-08-01T22:46:31.079 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [473 kB] 2022-08-01T22:46:31.085 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,000 kB] 2022-08-01T22:46:31.093 INFO:teuthology.orchestra.run.smithi130.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T22:46:31.151 INFO:teuthology.orchestra.run.smithi130.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [278 kB] 2022-08-01T22:46:31.160 INFO:teuthology.orchestra.run.smithi055.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T22:46:31.161 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [278 kB] 2022-08-01T22:46:31.200 INFO:teuthology.orchestra.run.smithi130.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T22:46:31.210 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T22:46:31.211 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,098 kB] 2022-08-01T22:46:31.217 INFO:teuthology.orchestra.run.smithi130.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,098 kB] 2022-08-01T22:46:31.255 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [473 kB] 2022-08-01T22:46:31.282 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,000 kB] 2022-08-01T22:46:31.286 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T22:46:31.357 INFO:teuthology.orchestra.run.smithi055.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T22:46:31.358 INFO:teuthology.orchestra.run.smithi055.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,187 kB] 2022-08-01T22:46:31.413 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [156 kB] 2022-08-01T22:46:31.422 INFO:teuthology.orchestra.run.smithi130.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [156 kB] 2022-08-01T22:46:31.439 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T22:46:31.461 INFO:teuthology.orchestra.run.smithi130.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T22:46:31.542 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [712 kB] 2022-08-01T22:46:31.578 INFO:teuthology.orchestra.run.smithi130.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [712 kB] 2022-08-01T22:46:31.590 INFO:teuthology.orchestra.run.smithi055.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [169 kB] 2022-08-01T22:46:31.630 INFO:teuthology.orchestra.run.smithi055.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T22:46:31.675 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [129 kB] 2022-08-01T22:46:31.695 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T22:46:31.782 INFO:teuthology.orchestra.run.smithi130.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [129 kB] 2022-08-01T22:46:31.797 INFO:teuthology.orchestra.run.smithi055.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T22:46:31.875 INFO:teuthology.orchestra.run.smithi130.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T22:46:31.905 INFO:teuthology.orchestra.run.smithi055.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T22:46:31.945 INFO:teuthology.orchestra.run.smithi055.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T22:46:31.946 INFO:teuthology.orchestra.run.smithi055.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-08-01T22:46:31.947 INFO:teuthology.orchestra.run.smithi055.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.4 kB] 2022-08-01T22:46:31.948 INFO:teuthology.orchestra.run.smithi055.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [16.0 kB] 2022-08-01T22:46:31.949 INFO:teuthology.orchestra.run.smithi055.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [860 B] 2022-08-01T22:46:32.015 INFO:teuthology.orchestra.run.smithi130.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,000 kB] 2022-08-01T22:46:32.038 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [278 kB] 2022-08-01T22:46:33.071 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T22:46:33.095 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,098 kB] 2022-08-01T22:46:33.343 INFO:teuthology.orchestra.run.smithi055.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [473 kB] 2022-08-01T22:46:33.780 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T22:46:33.951 INFO:teuthology.orchestra.run.smithi055.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [278 kB] 2022-08-01T22:46:34.148 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T22:46:34.340 INFO:teuthology.orchestra.run.smithi055.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T22:46:34.346 INFO:teuthology.orchestra.run.smithi055.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,098 kB] 2022-08-01T22:46:34.685 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T22:46:34.698 INFO:teuthology.orchestra.run.smithi130.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T22:46:35.182 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T22:46:35.183 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,187 kB] 2022-08-01T22:46:35.186 INFO:teuthology.orchestra.run.smithi130.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T22:46:35.194 INFO:teuthology.orchestra.run.smithi130.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,187 kB] 2022-08-01T22:46:35.367 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T22:46:35.428 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,187 kB] 2022-08-01T22:46:35.464 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [156 kB] 2022-08-01T22:46:35.547 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [712 kB] 2022-08-01T22:46:35.826 INFO:teuthology.orchestra.run.smithi055.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [156 kB] 2022-08-01T22:46:35.857 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T22:46:36.027 INFO:teuthology.orchestra.run.smithi055.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [712 kB] 2022-08-01T22:46:36.137 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [129 kB] 2022-08-01T22:46:36.178 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T22:46:36.777 INFO:teuthology.orchestra.run.smithi130.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [169 kB] 2022-08-01T22:46:36.827 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [169 kB] 2022-08-01T22:46:37.005 INFO:teuthology.orchestra.run.smithi055.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T22:46:37.036 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T22:46:37.050 INFO:teuthology.orchestra.run.smithi130.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T22:46:37.713 INFO:teuthology.orchestra.run.smithi055.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [129 kB] 2022-08-01T22:46:37.901 INFO:teuthology.orchestra.run.smithi055.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T22:46:37.937 INFO:teuthology.orchestra.run.smithi130.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T22:46:38.300 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T22:46:39.174 INFO:teuthology.orchestra.run.smithi130.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T22:46:39.196 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T22:46:39.458 INFO:teuthology.orchestra.run.smithi130.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T22:46:39.464 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [169 kB] 2022-08-01T22:46:39.466 INFO:teuthology.orchestra.run.smithi130.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-08-01T22:46:39.476 INFO:teuthology.orchestra.run.smithi130.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.4 kB] 2022-08-01T22:46:39.481 INFO:teuthology.orchestra.run.smithi130.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [16.0 kB] 2022-08-01T22:46:39.486 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T22:46:39.494 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-08-01T22:46:39.505 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.4 kB] 2022-08-01T22:46:39.511 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [16.0 kB] 2022-08-01T22:46:39.557 INFO:teuthology.orchestra.run.smithi130.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [860 B] 2022-08-01T22:46:39.584 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [860 B] 2022-08-01T22:46:39.791 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 11.7 MB in 8s (1,523 kB/s) 2022-08-01T22:46:40.013 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T22:46:41.165 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2022-08-01T22:46:41.190 DEBUG:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T22:46:41.266 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2022-08-01T22:46:41.450 INFO:teuthology.orchestra.run.smithi130.stdout:Fetched 11.7 MB in 9s (1,261 kB/s) 2022-08-01T22:46:41.451 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2022-08-01T22:46:41.451 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2022-08-01T22:46:41.458 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 11.7 MB in 9s (1,255 kB/s) 2022-08-01T22:46:41.677 INFO:teuthology.orchestra.run.smithi055.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T22:46:41.678 INFO:teuthology.orchestra.run.smithi055.stdout:linux-image-generic set to manually installed. 2022-08-01T22:46:41.679 INFO:teuthology.orchestra.run.smithi055.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:46:41.681 DEBUG:teuthology.orchestra.run.smithi055:> dpkg -s linux-image-generic 2022-08-01T22:46:41.703 INFO:teuthology.orchestra.run.smithi055.stdout:Package: linux-image-generic 2022-08-01T22:46:41.704 INFO:teuthology.orchestra.run.smithi055.stdout:Status: install ok installed 2022-08-01T22:46:41.704 INFO:teuthology.orchestra.run.smithi055.stdout:Priority: optional 2022-08-01T22:46:41.705 INFO:teuthology.orchestra.run.smithi055.stdout:Section: kernel 2022-08-01T22:46:41.705 INFO:teuthology.orchestra.run.smithi055.stdout:Installed-Size: 19 2022-08-01T22:46:41.705 INFO:teuthology.orchestra.run.smithi055.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T22:46:41.706 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: amd64 2022-08-01T22:46:41.706 INFO:teuthology.orchestra.run.smithi055.stdout:Source: linux-meta 2022-08-01T22:46:41.706 INFO:teuthology.orchestra.run.smithi055.stdout:Version: 5.4.0.122.123 2022-08-01T22:46:41.707 INFO:teuthology.orchestra.run.smithi055.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T22:46:41.707 INFO:teuthology.orchestra.run.smithi055.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T22:46:41.707 INFO:teuthology.orchestra.run.smithi055.stdout:Recommends: thermald 2022-08-01T22:46:41.708 INFO:teuthology.orchestra.run.smithi055.stdout:Description: Generic Linux kernel image 2022-08-01T22:46:41.708 INFO:teuthology.orchestra.run.smithi055.stdout: This package will always depend on the latest generic kernel image 2022-08-01T22:46:41.709 INFO:teuthology.orchestra.run.smithi055.stdout: available. 2022-08-01T22:46:41.710 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T22:46:41.710 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-08-01T22:46:41.711 DEBUG:teuthology.orchestra.run.smithi055:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T22:46:41.744 INFO:teuthology.orchestra.run.smithi055.stdout:ttyS1 2022-08-01T22:46:41.766 DEBUG:teuthology.parallel:result is None 2022-08-01T22:46:42.720 INFO:teuthology.orchestra.run.smithi130.stdout:Reading package lists... 2022-08-01T22:46:42.745 DEBUG:teuthology.orchestra.run.smithi130:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T22:46:42.776 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2022-08-01T22:46:42.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T22:46:42.814 INFO:teuthology.orchestra.run.smithi130.stdout:Reading package lists... 2022-08-01T22:46:42.873 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2022-08-01T22:46:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:Building dependency tree... 2022-08-01T22:46:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:Reading state information... 2022-08-01T22:46:43.050 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2022-08-01T22:46:43.051 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2022-08-01T22:46:43.177 INFO:teuthology.orchestra.run.smithi130.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T22:46:43.178 INFO:teuthology.orchestra.run.smithi130.stdout:linux-image-generic set to manually installed. 2022-08-01T22:46:43.178 INFO:teuthology.orchestra.run.smithi130.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:46:43.180 DEBUG:teuthology.orchestra.run.smithi130:> dpkg -s linux-image-generic 2022-08-01T22:46:43.203 INFO:teuthology.orchestra.run.smithi130.stdout:Package: linux-image-generic 2022-08-01T22:46:43.203 INFO:teuthology.orchestra.run.smithi130.stdout:Status: install ok installed 2022-08-01T22:46:43.204 INFO:teuthology.orchestra.run.smithi130.stdout:Priority: optional 2022-08-01T22:46:43.204 INFO:teuthology.orchestra.run.smithi130.stdout:Section: kernel 2022-08-01T22:46:43.204 INFO:teuthology.orchestra.run.smithi130.stdout:Installed-Size: 19 2022-08-01T22:46:43.204 INFO:teuthology.orchestra.run.smithi130.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T22:46:43.205 INFO:teuthology.orchestra.run.smithi130.stdout:Architecture: amd64 2022-08-01T22:46:43.205 INFO:teuthology.orchestra.run.smithi130.stdout:Source: linux-meta 2022-08-01T22:46:43.205 INFO:teuthology.orchestra.run.smithi130.stdout:Version: 5.4.0.122.123 2022-08-01T22:46:43.205 INFO:teuthology.orchestra.run.smithi130.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T22:46:43.205 INFO:teuthology.orchestra.run.smithi130.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T22:46:43.210 INFO:teuthology.orchestra.run.smithi130.stdout:Recommends: thermald 2022-08-01T22:46:43.211 INFO:teuthology.orchestra.run.smithi130.stdout:Description: Generic Linux kernel image 2022-08-01T22:46:43.211 INFO:teuthology.orchestra.run.smithi130.stdout: This package will always depend on the latest generic kernel image 2022-08-01T22:46:43.211 INFO:teuthology.orchestra.run.smithi130.stdout: available. 2022-08-01T22:46:43.212 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T22:46:43.213 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-08-01T22:46:43.213 DEBUG:teuthology.orchestra.run.smithi130:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T22:46:43.274 INFO:teuthology.orchestra.run.smithi130.stdout:ttyS1 2022-08-01T22:46:43.286 INFO:teuthology.orchestra.run.smithi157.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T22:46:43.287 INFO:teuthology.orchestra.run.smithi157.stdout:linux-image-generic set to manually installed. 2022-08-01T22:46:43.287 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:46:43.288 DEBUG:teuthology.orchestra.run.smithi157:> dpkg -s linux-image-generic 2022-08-01T22:46:43.301 DEBUG:teuthology.parallel:result is None 2022-08-01T22:46:43.310 INFO:teuthology.orchestra.run.smithi157.stdout:Package: linux-image-generic 2022-08-01T22:46:43.310 INFO:teuthology.orchestra.run.smithi157.stdout:Status: install ok installed 2022-08-01T22:46:43.311 INFO:teuthology.orchestra.run.smithi157.stdout:Priority: optional 2022-08-01T22:46:43.311 INFO:teuthology.orchestra.run.smithi157.stdout:Section: kernel 2022-08-01T22:46:43.311 INFO:teuthology.orchestra.run.smithi157.stdout:Installed-Size: 19 2022-08-01T22:46:43.311 INFO:teuthology.orchestra.run.smithi157.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T22:46:43.312 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: amd64 2022-08-01T22:46:43.312 INFO:teuthology.orchestra.run.smithi157.stdout:Source: linux-meta 2022-08-01T22:46:43.312 INFO:teuthology.orchestra.run.smithi157.stdout:Version: 5.4.0.122.123 2022-08-01T22:46:43.312 INFO:teuthology.orchestra.run.smithi157.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T22:46:43.313 INFO:teuthology.orchestra.run.smithi157.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T22:46:43.313 INFO:teuthology.orchestra.run.smithi157.stdout:Recommends: thermald 2022-08-01T22:46:43.313 INFO:teuthology.orchestra.run.smithi157.stdout:Description: Generic Linux kernel image 2022-08-01T22:46:43.313 INFO:teuthology.orchestra.run.smithi157.stdout: This package will always depend on the latest generic kernel image 2022-08-01T22:46:43.315 INFO:teuthology.orchestra.run.smithi157.stdout: available. 2022-08-01T22:46:43.315 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T22:46:43.316 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-08-01T22:46:43.316 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T22:46:43.347 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T22:46:43.376 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2022-08-01T22:46:43.403 DEBUG:teuthology.parallel:result is None 2022-08-01T22:46:45.828 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T22:46:46.536 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T22:46:46.591 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-08-01T22:46:46.655 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.4 kB] 2022-08-01T22:46:46.706 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [16.0 kB] 2022-08-01T22:46:46.763 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [860 B] 2022-08-01T22:46:48.646 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 11.7 MB in 17s (709 kB/s) 2022-08-01T22:46:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2022-08-01T22:46:49.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T22:46:50.036 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2022-08-01T22:46:50.217 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2022-08-01T22:46:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2022-08-01T22:46:50.449 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T22:46:50.449 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic set to manually installed. 2022-08-01T22:46:50.450 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:46:50.450 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2022-08-01T22:46:50.472 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2022-08-01T22:46:50.472 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2022-08-01T22:46:50.472 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2022-08-01T22:46:50.473 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2022-08-01T22:46:50.473 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 19 2022-08-01T22:46:50.473 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T22:46:50.473 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2022-08-01T22:46:50.474 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2022-08-01T22:46:50.474 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.4.0.122.123 2022-08-01T22:46:50.474 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T22:46:50.475 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T22:46:50.475 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2022-08-01T22:46:50.475 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2022-08-01T22:46:50.475 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2022-08-01T22:46:50.476 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2022-08-01T22:46:50.477 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T22:46:50.477 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-08-01T22:46:50.478 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T22:46:50.504 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2022-08-01T22:46:50.533 DEBUG:teuthology.parallel:result is None 2022-08-01T22:46:50.533 INFO:teuthology.run_tasks:Running task internal.base... 2022-08-01T22:46:50.541 INFO:teuthology.task.internal:Creating test directory... 2022-08-01T22:46:50.543 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T22:46:50.586 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T22:46:50.589 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T22:46:50.593 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T22:46:50.617 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-08-01T22:46:50.629 INFO:teuthology.run_tasks:Running task internal.archive... 2022-08-01T22:46:50.643 INFO:teuthology.task.internal:Creating archive directory... 2022-08-01T22:46:50.644 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T22:46:50.648 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T22:46:50.653 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T22:46:50.657 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T22:46:50.689 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-08-01T22:46:50.699 INFO:teuthology.task.internal:Enabling coredump saving... 2022-08-01T22:46:50.699 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T22:46:50.702 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T22:46:50.710 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T22:46:50.723 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T22:46:50.731 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.733 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.734 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.737 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.740 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.745 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.749 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.770 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T22:46:50.771 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-08-01T22:46:50.796 INFO:teuthology.task.internal:Configuring sudo... 2022-08-01T22:46:50.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T22:46:50.803 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T22:46:50.814 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T22:46:50.819 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T22:46:50.846 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-08-01T22:46:50.875 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-08-01T22:46:50.876 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T22:46:50.891 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T22:46:50.897 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T22:46:50.908 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T22:46:50.931 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T22:46:50.952 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T22:46:51.006 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:46:51.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T22:46:51.035 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T22:46:51.054 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T22:46:51.114 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:46:51.114 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T22:46:51.139 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T22:46:51.156 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T22:46:51.211 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:46:51.212 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T22:46:51.245 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T22:46:51.261 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T22:46:51.313 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:46:51.315 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T22:46:51.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2022-08-01T22:46:51.393 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2022-08-01T22:46:51.398 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2022-08-01T22:46:51.402 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2022-08-01T22:46:51.455 INFO:teuthology.run_tasks:Running task internal.timer... 2022-08-01T22:46:51.475 INFO:teuthology.task.internal:Starting timer... 2022-08-01T22:46:51.476 INFO:teuthology.run_tasks:Running task pcp... 2022-08-01T22:46:51.489 INFO:teuthology.run_tasks:Running task selinux... 2022-08-01T22:46:51.505 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2022-08-01T22:46:51.506 DEBUG:teuthology.task.selinux:Excluding smithi055: OS 'ubuntu' does not support SELinux 2022-08-01T22:46:51.507 DEBUG:teuthology.task.selinux:Excluding smithi130: OS 'ubuntu' does not support SELinux 2022-08-01T22:46:51.507 DEBUG:teuthology.task.selinux:Excluding smithi157: OS 'ubuntu' does not support SELinux 2022-08-01T22:46:51.507 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-08-01T22:46:51.507 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-08-01T22:46:51.508 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-08-01T22:46:51.508 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-08-01T22:46:51.530 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2022-08-01T22:46:51.550 INFO:teuthology.repo_utils:Fetching main from origin 2022-08-01T22:46:51.637 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-08-01T22:46:51.697 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-08-01T22:46:51.709 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi055.front.sepia.ceph.com,smithi130.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-08-01T22:52:24.996 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2022-08-01T22:52:25.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2022-08-01T22:52:25.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:52:25.105 DEBUG:teuthology.orchestra.run.smithi032:> true 2022-08-01T22:52:25.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2022-08-01T22:52:25.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:52:25.326 DEBUG:teuthology.orchestra.run.smithi055:> true 2022-08-01T22:52:25.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2022-08-01T22:52:25.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:52:25.555 DEBUG:teuthology.orchestra.run.smithi130:> true 2022-08-01T22:52:25.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2022-08-01T22:52:25.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T22:52:25.771 DEBUG:teuthology.orchestra.run.smithi157:> true 2022-08-01T22:52:25.893 INFO:teuthology.run_tasks:Running task clock... 2022-08-01T22:52:25.905 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-08-01T22:52:25.905 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T22:52:25.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T22:52:25.910 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T22:52:25.911 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T22:52:25.913 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T22:52:25.914 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T22:52:25.916 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T22:52:25.917 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T22:52:25.944 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T22:52:25.945 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Command line: ntpd -gq 2022-08-01T22:52:25.945 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: proto: precision = 0.172 usec (-22) 2022-08-01T22:52:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.947 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.947 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.948 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T22:52:25.949 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T22:52:25.949 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T22:52:25.949 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listen normally on 3 ens1f0 172.21.15.32:123 2022-08-01T22:52:25.949 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listen normally on 4 lo [::1]:123 2022-08-01T22:52:25.950 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%3]:123 2022-08-01T22:52:25.950 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:25 ntpd[17887]: Listening on routing socket on fd #22 for interface updates 2022-08-01T22:52:25.951 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T22:52:25.951 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Command line: ntpd -gq 2022-08-01T22:52:25.951 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: proto: precision = 0.080 usec (-23) 2022-08-01T22:52:25.952 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T22:52:25.952 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Command line: ntpd -gq 2022-08-01T22:52:25.952 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: proto: precision = 0.082 usec (-23) 2022-08-01T22:52:25.953 INFO:teuthology.orchestra.run.smithi130.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.953 INFO:teuthology.orchestra.run.smithi055.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.954 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.954 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.955 INFO:teuthology.orchestra.run.smithi130.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.955 INFO:teuthology.orchestra.run.smithi055.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.957 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.958 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T22:52:25.958 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T22:52:25.958 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T22:52:25.958 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listen normally on 3 enp3s0f1 172.21.15.130:123 2022-08-01T22:52:25.959 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listen normally on 4 lo [::1]:123 2022-08-01T22:52:25.959 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f37%5]:123 2022-08-01T22:52:25.959 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:25 ntpd[18142]: Listening on routing socket on fd #22 for interface updates 2022-08-01T22:52:25.959 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.959 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T22:52:25.960 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T22:52:25.960 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T22:52:25.960 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listen normally on 3 ens1f0 172.21.15.55:123 2022-08-01T22:52:25.960 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listen normally on 4 lo [::1]:123 2022-08-01T22:52:25.961 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2606%4]:123 2022-08-01T22:52:25.961 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:25 ntpd[17991]: Listening on routing socket on fd #22 for interface updates 2022-08-01T22:52:25.961 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T22:52:25.961 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Command line: ntpd -gq 2022-08-01T22:52:25.961 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: proto: precision = 0.095 usec (-23) 2022-08-01T22:52:25.962 INFO:teuthology.orchestra.run.smithi157.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.962 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T22:52:25.963 INFO:teuthology.orchestra.run.smithi157.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.964 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T22:52:25.964 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T22:52:25.964 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T22:52:25.965 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T22:52:25.965 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listen normally on 3 enp3s0f1 172.21.15.157:123 2022-08-01T22:52:25.965 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listen normally on 4 lo [::1]:123 2022-08-01T22:52:25.965 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ea1%5]:123 2022-08-01T22:52:25.965 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:25 ntpd[18226]: Listening on routing socket on fd #22 for interface updates 2022-08-01T22:52:26.949 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:26.949 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:26.949 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:26.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:26.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:26.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:26.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:26.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:26 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:28.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:28.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:28.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:28.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:28.954 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.954 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.954 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:28.954 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:28 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:28.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:28.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:28.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:28.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:28.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:28.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:28 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:29.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:29.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:29.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.950 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.950 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:29.950 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:29 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:29.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.954 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:29.954 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:29.954 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:29.954 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:29.954 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:29.955 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:29 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.945 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:30 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:30 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:30 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:30 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:30 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:30 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.949 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:30 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.954 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:30 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.955 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:30 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.955 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:30 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.955 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:30 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.955 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:30 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.956 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:30 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:30.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:30.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:30.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:30.962 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:30 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:31.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:31.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:31.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:31.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:31 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:31.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:31.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:31.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:31.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:31.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:31.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:31 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.945 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:32 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:32 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:32 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:32 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:32 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:32 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.949 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.950 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.951 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.952 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.953 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.954 INFO:teuthology.orchestra.run.smithi055.stdout: 1 Aug 22:52:32 ntpd[17991]: ntpd: time slew -0.005873 s 2022-08-01T22:52:32.954 INFO:teuthology.orchestra.run.smithi055.stdout:ntpd: time slew -0.005873s 2022-08-01T22:52:32.955 INFO:teuthology.orchestra.run.smithi055.stderr: 1 Aug 22:52:32 ntpd[17991]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T22:52:32.956 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:32 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.957 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:32 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.957 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:32 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.957 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:32 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.957 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:32 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.958 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:32 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.961 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.962 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.962 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.962 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.962 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:32.962 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:32.963 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:32.963 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:32 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:32.999 INFO:teuthology.orchestra.run.smithi055.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T22:52:32.999 INFO:teuthology.orchestra.run.smithi055.stdout:============================================================================== 2022-08-01T22:52:32.999 INFO:teuthology.orchestra.run.smithi055.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:32.999 INFO:teuthology.orchestra.run.smithi055.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:33.000 INFO:teuthology.orchestra.run.smithi055.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:33.000 INFO:teuthology.orchestra.run.smithi055.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:33.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:33.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:33.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:33.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:33 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:33.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:33.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:33.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:33.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:33.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:33.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:33 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:34.945 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:34 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:34 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:34 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:34 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:34 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:34.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:34 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:34.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:34 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:34 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:34 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:34 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:34 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:34.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:34 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:34.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.956 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.957 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:34.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:34.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:34.958 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:34.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:34.959 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:34.959 INFO:teuthology.orchestra.run.smithi157.stdout: 1 Aug 22:52:34 ntpd[18226]: ntpd: time slew -0.003347 s 2022-08-01T22:52:34.960 INFO:teuthology.orchestra.run.smithi157.stdout:ntpd: time slew -0.003347s 2022-08-01T22:52:34.960 INFO:teuthology.orchestra.run.smithi157.stderr: 1 Aug 22:52:34 ntpd[18226]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T22:52:35.008 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T22:52:35.009 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2022-08-01T22:52:35.009 INFO:teuthology.orchestra.run.smithi157.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:35.009 INFO:teuthology.orchestra.run.smithi157.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:35.009 INFO:teuthology.orchestra.run.smithi157.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:35.010 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:35.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:35.946 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:35.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:35.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:35.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:35.947 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:35.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:35.948 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:35.948 INFO:teuthology.orchestra.run.smithi032.stdout: 1 Aug 22:52:35 ntpd[17887]: ntpd: time slew -0.006943 s 2022-08-01T22:52:35.949 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew -0.006943s 2022-08-01T22:52:35.949 INFO:teuthology.orchestra.run.smithi032.stderr: 1 Aug 22:52:35 ntpd[17887]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T22:52:35.950 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:35.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:35.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:35.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:35.951 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T22:52:35.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T22:52:35.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T22:52:35.952 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T22:52:35.953 INFO:teuthology.orchestra.run.smithi130.stdout: 1 Aug 22:52:35 ntpd[18142]: ntpd: time slew -0.006697 s 2022-08-01T22:52:35.953 INFO:teuthology.orchestra.run.smithi130.stdout:ntpd: time slew -0.006697s 2022-08-01T22:52:35.953 INFO:teuthology.orchestra.run.smithi130.stderr: 1 Aug 22:52:35 ntpd[18142]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T22:52:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T22:52:36.006 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2022-08-01T22:52:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.010 INFO:teuthology.orchestra.run.smithi130.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T22:52:36.010 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================== 2022-08-01T22:52:36.010 INFO:teuthology.orchestra.run.smithi130.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.011 INFO:teuthology.orchestra.run.smithi130.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.011 INFO:teuthology.orchestra.run.smithi130.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.011 INFO:teuthology.orchestra.run.smithi130.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T22:52:36.012 INFO:teuthology.run_tasks:Running task install... 2022-08-01T22:52:36.020 DEBUG:teuthology.task.install:project ceph 2022-08-01T22:52:36.021 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}} 2022-08-01T22:52:36.022 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'} 2022-08-01T22:52:36.022 INFO:teuthology.task.install:Using flavor: default 2022-08-01T22:52:36.032 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-08-01T22:52:36.033 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2022-08-01T22:52:36.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo lsb_release -is 2022-08-01T22:52:36.084 INFO:teuthology.orchestra.run.smithi032.stdout:Ubuntu 2022-08-01T22:52:36.089 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.090 DEBUG:teuthology.orchestra.run.smithi055:> sudo lsb_release -is 2022-08-01T22:52:36.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2022-08-01T22:52:36.143 INFO:teuthology.orchestra.run.smithi055.stdout:Ubuntu 2022-08-01T22:52:36.149 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.149 DEBUG:teuthology.orchestra.run.smithi130:> sudo lsb_release -is 2022-08-01T22:52:36.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo apt-key list | grep Ceph 2022-08-01T22:52:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T22:52:36.202 INFO:teuthology.orchestra.run.smithi130.stdout:Ubuntu 2022-08-01T22:52:36.208 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.209 DEBUG:teuthology.orchestra.run.smithi157:> sudo lsb_release -is 2022-08-01T22:52:36.210 DEBUG:teuthology.orchestra.run.smithi130:> sudo apt-key list | grep Ceph 2022-08-01T22:52:36.248 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T22:52:36.249 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T22:52:36.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo lsb_release -is 2022-08-01T22:52:36.253 INFO:teuthology.orchestra.run.smithi055.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T22:52:36.262 INFO:teuthology.orchestra.run.smithi157.stdout:Ubuntu 2022-08-01T22:52:36.268 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-key list | grep Ceph 2022-08-01T22:52:36.303 INFO:teuthology.orchestra.run.smithi032.stdout:Ubuntu 2022-08-01T22:52:36.308 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.309 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T22:52:36.309 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T22:52:36.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:52:36.316 INFO:teuthology.orchestra.run.smithi130.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T22:52:36.317 INFO:teuthology.orchestra.run.smithi055.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T22:52:36.317 INFO:teuthology.orchestra.run.smithi055.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T22:52:36.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo lsb_release -is 2022-08-01T22:52:36.364 INFO:teuthology.orchestra.run.smithi055.stdout:Ubuntu 2022-08-01T22:52:36.366 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T22:52:36.369 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.370 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T22:52:36.370 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T22:52:36.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:52:36.375 INFO:teuthology.orchestra.run.smithi130.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T22:52:36.376 INFO:teuthology.orchestra.run.smithi130.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T22:52:36.376 DEBUG:teuthology.orchestra.run.smithi130:> sudo lsb_release -is 2022-08-01T22:52:36.426 INFO:teuthology.orchestra.run.smithi130.stdout:Ubuntu 2022-08-01T22:52:36.428 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T22:52:36.428 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T22:52:36.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo lsb_release -is 2022-08-01T22:52:36.432 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.433 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T22:52:36.433 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T22:52:36.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:52:36.482 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T22:52:36.483 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:36.488 INFO:teuthology.orchestra.run.smithi157.stdout:Ubuntu 2022-08-01T22:52:36.492 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:52:36.492 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T22:52:36.492 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T22:52:36.493 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:52:36.533 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T22:52:36.534 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:36.602 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:52:36.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T22:52:36.605 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T22:52:36.605 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:36.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2022-08-01T22:52:36.646 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:52:36.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T22:52:36.656 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T22:52:36.656 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:36.664 DEBUG:teuthology.orchestra.run.smithi055:> sudo apt-get update 2022-08-01T22:52:36.723 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:52:36.723 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T22:52:36.740 DEBUG:teuthology.orchestra.run.smithi130:> sudo apt-get update 2022-08-01T22:52:36.769 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:52:36.770 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T22:52:36.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2022-08-01T22:52:36.810 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:52:36.815 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T22:52:36.823 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:52:36.828 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T22:52:36.832 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T22:52:36.846 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T22:52:36.870 INFO:teuthology.orchestra.run.smithi055.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T22:52:36.879 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T22:52:36.882 INFO:teuthology.orchestra.run.smithi130.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:52:36.890 INFO:teuthology.orchestra.run.smithi130.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T22:52:36.894 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T22:52:36.902 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T22:52:36.906 INFO:teuthology.orchestra.run.smithi130.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T22:52:36.918 INFO:teuthology.orchestra.run.smithi055.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T22:52:36.923 INFO:teuthology.orchestra.run.smithi055.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T22:52:36.927 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T22:52:36.928 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T22:52:36.929 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T22:52:36.934 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T22:52:36.942 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T22:52:36.943 INFO:teuthology.orchestra.run.smithi130.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T22:52:36.952 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T22:52:36.953 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T22:52:36.967 INFO:teuthology.orchestra.run.smithi130.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T22:52:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:8 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T22:52:36.990 INFO:teuthology.orchestra.run.smithi055.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T22:52:36.991 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T22:52:36.993 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T22:52:36.994 INFO:teuthology.orchestra.run.smithi130.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T22:52:36.995 INFO:teuthology.orchestra.run.smithi130.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T22:52:37.015 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T22:52:37.018 INFO:teuthology.orchestra.run.smithi130.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T22:52:37.040 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T22:52:37.063 INFO:teuthology.orchestra.run.smithi130.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T22:52:37.064 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T22:52:37.098 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T22:52:37.476 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 26.6 kB in 1s (41.9 kB/s) 2022-08-01T22:52:37.486 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 26.6 kB in 1s (40.7 kB/s) 2022-08-01T22:52:37.537 INFO:teuthology.orchestra.run.smithi130.stdout:Fetched 26.6 kB in 1s (41.9 kB/s) 2022-08-01T22:52:37.578 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 26.6 kB in 1s (42.5 kB/s) 2022-08-01T22:52:38.393 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2022-08-01T22:52:38.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:38.435 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2022-08-01T22:52:38.436 INFO:teuthology.orchestra.run.smithi130.stdout:Reading package lists... 2022-08-01T22:52:38.459 DEBUG:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:38.460 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2022-08-01T22:52:38.461 DEBUG:teuthology.orchestra.run.smithi130:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:38.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T22:52:38.487 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2022-08-01T22:52:38.529 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2022-08-01T22:52:38.531 INFO:teuthology.orchestra.run.smithi130.stdout:Reading package lists... 2022-08-01T22:52:38.554 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2022-08-01T22:52:38.649 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2022-08-01T22:52:38.650 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2022-08-01T22:52:38.688 INFO:teuthology.orchestra.run.smithi130.stdout:Building dependency tree... 2022-08-01T22:52:38.689 INFO:teuthology.orchestra.run.smithi130.stdout:Reading state information... 2022-08-01T22:52:38.690 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2022-08-01T22:52:38.691 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2022-08-01T22:52:38.729 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2022-08-01T22:52:38.730 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2022-08-01T22:52:38.797 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:52:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:52:38.801 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:52:38.801 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2022-08-01T22:52:38.802 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.802 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.802 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T22:52:38.803 INFO:teuthology.orchestra.run.smithi032.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T22:52:38.803 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T22:52:38.803 INFO:teuthology.orchestra.run.smithi032.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T22:52:38.803 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T22:52:38.804 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T22:52:38.804 INFO:teuthology.orchestra.run.smithi032.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T22:52:38.804 INFO:teuthology.orchestra.run.smithi032.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T22:52:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T22:52:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T22:52:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T22:52:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T22:52:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T22:52:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T22:52:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T22:52:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T22:52:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T22:52:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet 2022-08-01T22:52:38.807 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2022-08-01T22:52:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T22:52:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T22:52:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T22:52:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T22:52:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T22:52:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T22:52:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T22:52:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T22:52:38.810 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2022-08-01T22:52:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2022-08-01T22:52:38.827 INFO:teuthology.orchestra.run.smithi130.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:52:38.831 INFO:teuthology.orchestra.run.smithi130.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:52:38.831 INFO:teuthology.orchestra.run.smithi130.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:52:38.831 INFO:teuthology.orchestra.run.smithi130.stdout:The following additional packages will be installed: 2022-08-01T22:52:38.831 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T22:52:38.832 INFO:teuthology.orchestra.run.smithi130.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T22:52:38.832 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T22:52:38.833 INFO:teuthology.orchestra.run.smithi130.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T22:52:38.833 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T22:52:38.834 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T22:52:38.834 INFO:teuthology.orchestra.run.smithi130.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T22:52:38.834 INFO:teuthology.orchestra.run.smithi130.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T22:52:38.834 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T22:52:38.835 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T22:52:38.835 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T22:52:38.835 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T22:52:38.836 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T22:52:38.836 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T22:52:38.836 INFO:teuthology.orchestra.run.smithi130.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T22:52:38.836 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T22:52:38.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T22:52:38.838 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet 2022-08-01T22:52:38.838 INFO:teuthology.orchestra.run.smithi130.stdout:Suggested packages: 2022-08-01T22:52:38.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T22:52:38.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T22:52:38.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T22:52:38.839 INFO:teuthology.orchestra.run.smithi130.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T22:52:38.840 INFO:teuthology.orchestra.run.smithi130.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T22:52:38.840 INFO:teuthology.orchestra.run.smithi130.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T22:52:38.840 INFO:teuthology.orchestra.run.smithi130.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T22:52:38.840 INFO:teuthology.orchestra.run.smithi130.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T22:52:38.841 INFO:teuthology.orchestra.run.smithi130.stdout:Recommended packages: 2022-08-01T22:52:38.841 INFO:teuthology.orchestra.run.smithi130.stdout: btrfs-tools 2022-08-01T22:52:38.843 INFO:teuthology.orchestra.run.smithi055.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:52:38.843 INFO:teuthology.orchestra.run.smithi055.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:52:38.843 INFO:teuthology.orchestra.run.smithi055.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:52:38.843 INFO:teuthology.orchestra.run.smithi055.stdout:The following additional packages will be installed: 2022-08-01T22:52:38.844 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.844 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.844 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T22:52:38.844 INFO:teuthology.orchestra.run.smithi055.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T22:52:38.844 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T22:52:38.845 INFO:teuthology.orchestra.run.smithi055.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T22:52:38.845 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T22:52:38.845 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T22:52:38.845 INFO:teuthology.orchestra.run.smithi055.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T22:52:38.846 INFO:teuthology.orchestra.run.smithi055.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T22:52:38.846 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T22:52:38.846 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T22:52:38.846 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T22:52:38.847 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T22:52:38.847 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T22:52:38.847 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T22:52:38.847 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T22:52:38.848 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T22:52:38.848 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T22:52:38.848 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet 2022-08-01T22:52:38.848 INFO:teuthology.orchestra.run.smithi055.stdout:Suggested packages: 2022-08-01T22:52:38.849 INFO:teuthology.orchestra.run.smithi055.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T22:52:38.849 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T22:52:38.849 INFO:teuthology.orchestra.run.smithi055.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T22:52:38.850 INFO:teuthology.orchestra.run.smithi055.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T22:52:38.850 INFO:teuthology.orchestra.run.smithi055.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T22:52:38.850 INFO:teuthology.orchestra.run.smithi055.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T22:52:38.851 INFO:teuthology.orchestra.run.smithi055.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T22:52:38.851 INFO:teuthology.orchestra.run.smithi055.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T22:52:38.851 INFO:teuthology.orchestra.run.smithi055.stdout:Recommended packages: 2022-08-01T22:52:38.851 INFO:teuthology.orchestra.run.smithi055.stdout: btrfs-tools 2022-08-01T22:52:38.875 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:52:38.877 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:52:38.877 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:52:38.878 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2022-08-01T22:52:38.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T22:52:38.879 INFO:teuthology.orchestra.run.smithi157.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T22:52:38.880 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T22:52:38.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T22:52:38.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T22:52:38.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T22:52:38.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T22:52:38.881 INFO:teuthology.orchestra.run.smithi157.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T22:52:38.881 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T22:52:38.881 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T22:52:38.881 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T22:52:38.881 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T22:52:38.882 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T22:52:38.882 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T22:52:38.882 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T22:52:38.882 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T22:52:38.883 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T22:52:38.883 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet 2022-08-01T22:52:38.883 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2022-08-01T22:52:38.884 INFO:teuthology.orchestra.run.smithi157.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T22:52:38.884 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T22:52:38.884 INFO:teuthology.orchestra.run.smithi157.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T22:52:38.884 INFO:teuthology.orchestra.run.smithi157.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T22:52:38.885 INFO:teuthology.orchestra.run.smithi157.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T22:52:38.885 INFO:teuthology.orchestra.run.smithi157.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T22:52:38.885 INFO:teuthology.orchestra.run.smithi157.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T22:52:38.885 INFO:teuthology.orchestra.run.smithi157.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T22:52:38.885 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2022-08-01T22:52:38.886 INFO:teuthology.orchestra.run.smithi157.stdout: btrfs-tools 2022-08-01T22:52:38.908 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2022-08-01T22:52:38.909 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T22:52:38.909 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T22:52:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T22:52:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T22:52:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T22:52:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T22:52:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T22:52:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T22:52:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T22:52:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T22:52:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T22:52:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T22:52:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T22:52:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T22:52:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T22:52:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T22:52:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T22:52:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T22:52:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T22:52:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T22:52:38.915 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2022-08-01T22:52:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 librbd1 2022-08-01T22:52:38.941 INFO:teuthology.orchestra.run.smithi130.stdout:The following NEW packages will be installed: 2022-08-01T22:52:38.941 INFO:teuthology.orchestra.run.smithi130.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T22:52:38.941 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.941 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.944 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T22:52:38.944 INFO:teuthology.orchestra.run.smithi130.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T22:52:38.944 INFO:teuthology.orchestra.run.smithi130.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T22:52:38.945 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T22:52:38.945 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T22:52:38.945 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T22:52:38.945 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T22:52:38.945 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T22:52:38.946 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T22:52:38.946 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T22:52:38.946 INFO:teuthology.orchestra.run.smithi130.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T22:52:38.946 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T22:52:38.947 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T22:52:38.947 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T22:52:38.947 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T22:52:38.947 INFO:teuthology.orchestra.run.smithi130.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T22:52:38.948 INFO:teuthology.orchestra.run.smithi130.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T22:52:38.948 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T22:52:38.948 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T22:52:38.948 INFO:teuthology.orchestra.run.smithi130.stdout:The following packages will be upgraded: 2022-08-01T22:52:38.948 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 librbd1 2022-08-01T22:52:38.949 INFO:teuthology.orchestra.run.smithi055.stdout:The following NEW packages will be installed: 2022-08-01T22:52:38.949 INFO:teuthology.orchestra.run.smithi055.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T22:52:38.950 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.950 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.951 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T22:52:38.952 INFO:teuthology.orchestra.run.smithi055.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T22:52:38.952 INFO:teuthology.orchestra.run.smithi055.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T22:52:38.952 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T22:52:38.952 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T22:52:38.953 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T22:52:38.953 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T22:52:38.953 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T22:52:38.954 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T22:52:38.954 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T22:52:38.955 INFO:teuthology.orchestra.run.smithi055.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T22:52:38.955 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T22:52:38.956 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T22:52:38.956 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T22:52:38.956 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T22:52:38.957 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T22:52:38.957 INFO:teuthology.orchestra.run.smithi055.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T22:52:38.958 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T22:52:38.958 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T22:52:38.959 INFO:teuthology.orchestra.run.smithi055.stdout:The following packages will be upgraded: 2022-08-01T22:52:38.960 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 librbd1 2022-08-01T22:52:38.975 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2022-08-01T22:52:38.975 INFO:teuthology.orchestra.run.smithi157.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T22:52:38.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T22:52:38.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T22:52:38.978 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T22:52:38.978 INFO:teuthology.orchestra.run.smithi157.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T22:52:38.978 INFO:teuthology.orchestra.run.smithi157.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T22:52:38.978 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T22:52:38.979 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T22:52:38.979 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T22:52:38.979 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T22:52:38.979 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T22:52:38.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T22:52:38.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T22:52:38.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T22:52:38.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T22:52:38.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T22:52:38.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T22:52:38.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T22:52:38.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T22:52:38.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T22:52:38.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T22:52:38.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T22:52:38.982 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2022-08-01T22:52:38.982 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 librbd1 2022-08-01T22:52:39.098 INFO:teuthology.orchestra.run.smithi032.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:52:39.098 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 153 MB of archives. 2022-08-01T22:52:39.099 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T22:52:39.099 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T22:52:39.121 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T22:52:39.130 INFO:teuthology.orchestra.run.smithi130.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:52:39.131 INFO:teuthology.orchestra.run.smithi130.stdout:Need to get 153 MB of archives. 2022-08-01T22:52:39.131 INFO:teuthology.orchestra.run.smithi130.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T22:52:39.131 INFO:teuthology.orchestra.run.smithi130.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T22:52:39.138 INFO:teuthology.orchestra.run.smithi055.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:52:39.138 INFO:teuthology.orchestra.run.smithi055.stdout:Need to get 153 MB of archives. 2022-08-01T22:52:39.139 INFO:teuthology.orchestra.run.smithi055.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T22:52:39.139 INFO:teuthology.orchestra.run.smithi055.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T22:52:39.160 INFO:teuthology.orchestra.run.smithi130.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T22:52:39.166 INFO:teuthology.orchestra.run.smithi055.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T22:52:39.166 INFO:teuthology.orchestra.run.smithi157.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:52:39.167 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 153 MB of archives. 2022-08-01T22:52:39.167 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T22:52:39.167 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T22:52:39.180 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T22:52:39.206 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T22:52:39.213 INFO:teuthology.orchestra.run.smithi130.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T22:52:39.217 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T22:52:39.220 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T22:52:39.249 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T22:52:39.250 INFO:teuthology.orchestra.run.smithi130.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T22:52:39.257 INFO:teuthology.orchestra.run.smithi055.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T22:52:39.283 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T22:52:39.350 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T22:52:39.384 INFO:teuthology.orchestra.run.smithi130.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T22:52:39.391 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T22:52:39.403 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T22:52:39.540 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T22:52:39.590 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T22:52:39.599 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T22:52:39.629 INFO:teuthology.orchestra.run.smithi130.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T22:52:39.637 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T22:52:39.638 INFO:teuthology.orchestra.run.smithi130.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T22:52:39.646 INFO:teuthology.orchestra.run.smithi055.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T22:52:39.656 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T22:52:39.664 INFO:teuthology.orchestra.run.smithi130.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T22:52:39.665 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T22:52:39.680 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T22:52:39.719 INFO:teuthology.orchestra.run.smithi130.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T22:52:39.728 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T22:52:39.746 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T22:52:39.752 INFO:teuthology.orchestra.run.smithi055.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T22:52:39.850 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T22:52:40.015 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T22:52:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T22:52:40.119 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T22:52:40.120 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T22:52:40.155 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T22:52:40.156 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T22:52:40.185 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T22:52:40.207 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T22:52:40.225 INFO:teuthology.orchestra.run.smithi130.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T22:52:40.231 INFO:teuthology.orchestra.run.smithi055.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T22:52:40.246 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T22:52:40.281 INFO:teuthology.orchestra.run.smithi130.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T22:52:40.342 INFO:teuthology.orchestra.run.smithi130.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T22:52:40.376 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T22:52:40.397 INFO:teuthology.orchestra.run.smithi130.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T22:52:40.398 INFO:teuthology.orchestra.run.smithi130.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T22:52:40.405 INFO:teuthology.orchestra.run.smithi130.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T22:52:40.407 INFO:teuthology.orchestra.run.smithi130.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T22:52:40.419 INFO:teuthology.orchestra.run.smithi130.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T22:52:40.428 INFO:teuthology.orchestra.run.smithi055.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T22:52:40.435 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T22:52:40.438 INFO:teuthology.orchestra.run.smithi130.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T22:52:40.459 INFO:teuthology.orchestra.run.smithi130.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T22:52:40.462 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T22:52:40.464 INFO:teuthology.orchestra.run.smithi055.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T22:52:40.513 INFO:teuthology.orchestra.run.smithi055.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T22:52:40.518 INFO:teuthology.orchestra.run.smithi055.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T22:52:40.519 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T22:52:40.529 INFO:teuthology.orchestra.run.smithi130.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T22:52:40.544 INFO:teuthology.orchestra.run.smithi130.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T22:52:40.545 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T22:52:40.548 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T22:52:40.553 INFO:teuthology.orchestra.run.smithi130.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T22:52:40.575 INFO:teuthology.orchestra.run.smithi055.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T22:52:40.577 INFO:teuthology.orchestra.run.smithi055.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T22:52:40.582 INFO:teuthology.orchestra.run.smithi130.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T22:52:40.583 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T22:52:40.585 INFO:teuthology.orchestra.run.smithi130.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T22:52:40.588 INFO:teuthology.orchestra.run.smithi130.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T22:52:40.590 INFO:teuthology.orchestra.run.smithi130.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T22:52:40.598 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T22:52:40.599 INFO:teuthology.orchestra.run.smithi130.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T22:52:40.601 INFO:teuthology.orchestra.run.smithi130.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T22:52:40.629 INFO:teuthology.orchestra.run.smithi055.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T22:52:40.630 INFO:teuthology.orchestra.run.smithi055.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T22:52:40.650 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T22:52:40.663 INFO:teuthology.orchestra.run.smithi130.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T22:52:40.672 INFO:teuthology.orchestra.run.smithi130.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T22:52:40.672 INFO:teuthology.orchestra.run.smithi130.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T22:52:40.688 INFO:teuthology.orchestra.run.smithi055.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T22:52:40.739 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T22:52:40.743 INFO:teuthology.orchestra.run.smithi130.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T22:52:40.749 INFO:teuthology.orchestra.run.smithi130.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T22:52:40.750 INFO:teuthology.orchestra.run.smithi130.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T22:52:40.750 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T22:52:40.760 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T22:52:40.760 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T22:52:40.766 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T22:52:40.767 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T22:52:40.768 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T22:52:40.784 INFO:teuthology.orchestra.run.smithi130.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T22:52:40.785 INFO:teuthology.orchestra.run.smithi130.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T22:52:40.786 INFO:teuthology.orchestra.run.smithi130.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T22:52:40.788 INFO:teuthology.orchestra.run.smithi130.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T22:52:40.801 INFO:teuthology.orchestra.run.smithi055.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T22:52:40.819 INFO:teuthology.orchestra.run.smithi055.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T22:52:40.823 INFO:teuthology.orchestra.run.smithi130.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T22:52:40.832 INFO:teuthology.orchestra.run.smithi055.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T22:52:40.836 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T22:52:40.837 INFO:teuthology.orchestra.run.smithi130.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T22:52:40.843 INFO:teuthology.orchestra.run.smithi130.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T22:52:40.844 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T22:52:40.847 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T22:52:40.868 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T22:52:40.874 INFO:teuthology.orchestra.run.smithi130.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T22:52:40.877 INFO:teuthology.orchestra.run.smithi130.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T22:52:40.903 INFO:teuthology.orchestra.run.smithi130.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T22:52:40.907 INFO:teuthology.orchestra.run.smithi130.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T22:52:40.910 INFO:teuthology.orchestra.run.smithi130.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T22:52:40.914 INFO:teuthology.orchestra.run.smithi130.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T22:52:40.914 INFO:teuthology.orchestra.run.smithi130.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T22:52:40.915 INFO:teuthology.orchestra.run.smithi130.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T22:52:40.933 INFO:teuthology.orchestra.run.smithi130.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T22:52:41.050 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T22:52:41.058 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T22:52:41.066 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T22:52:41.066 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T22:52:41.071 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T22:52:41.098 INFO:teuthology.orchestra.run.smithi055.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T22:52:41.106 INFO:teuthology.orchestra.run.smithi055.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T22:52:41.113 INFO:teuthology.orchestra.run.smithi055.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T22:52:41.119 INFO:teuthology.orchestra.run.smithi055.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T22:52:41.121 INFO:teuthology.orchestra.run.smithi055.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T22:52:41.137 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T22:52:41.146 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T22:52:41.152 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T22:52:41.163 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T22:52:41.168 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T22:52:41.170 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T22:52:41.192 INFO:teuthology.orchestra.run.smithi055.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T22:52:41.203 INFO:teuthology.orchestra.run.smithi055.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T22:52:41.210 INFO:teuthology.orchestra.run.smithi055.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T22:52:41.215 INFO:teuthology.orchestra.run.smithi055.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T22:52:41.218 INFO:teuthology.orchestra.run.smithi055.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T22:52:41.241 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T22:52:41.246 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T22:52:41.250 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T22:52:41.320 INFO:teuthology.orchestra.run.smithi055.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T22:52:41.339 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T22:52:41.350 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T22:52:41.352 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T22:52:41.399 INFO:teuthology.orchestra.run.smithi055.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T22:52:41.404 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T22:52:41.407 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T22:52:41.429 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T22:52:41.456 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T22:52:41.493 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T22:52:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T22:52:41.529 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T22:52:41.557 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T22:52:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T22:52:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T22:52:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T22:52:41.629 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T22:52:41.635 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T22:52:42.137 INFO:teuthology.orchestra.run.smithi130.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T22:52:42.197 INFO:teuthology.orchestra.run.smithi055.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T22:52:42.199 INFO:teuthology.orchestra.run.smithi055.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T22:52:42.221 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T22:52:42.223 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T22:52:42.224 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T22:52:42.224 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T22:52:42.236 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T22:52:42.246 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T22:52:42.277 INFO:teuthology.orchestra.run.smithi055.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T22:52:42.289 INFO:teuthology.orchestra.run.smithi055.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T22:52:42.300 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T22:52:42.311 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T22:52:42.333 INFO:teuthology.orchestra.run.smithi130.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T22:52:42.350 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T22:52:42.391 INFO:teuthology.orchestra.run.smithi055.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T22:52:42.393 INFO:teuthology.orchestra.run.smithi130.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T22:52:42.413 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T22:52:42.512 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T22:52:42.565 INFO:teuthology.orchestra.run.smithi130.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T22:52:42.566 INFO:teuthology.orchestra.run.smithi130.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T22:52:42.568 INFO:teuthology.orchestra.run.smithi130.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T22:52:42.569 INFO:teuthology.orchestra.run.smithi130.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T22:52:42.570 INFO:teuthology.orchestra.run.smithi130.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T22:52:42.571 INFO:teuthology.orchestra.run.smithi130.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T22:52:42.572 INFO:teuthology.orchestra.run.smithi130.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T22:52:42.574 INFO:teuthology.orchestra.run.smithi130.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T22:52:42.575 INFO:teuthology.orchestra.run.smithi130.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T22:52:42.621 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T22:52:42.624 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T22:52:42.646 INFO:teuthology.orchestra.run.smithi130.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T22:52:42.664 INFO:teuthology.orchestra.run.smithi055.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T22:52:42.665 INFO:teuthology.orchestra.run.smithi130.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T22:52:42.676 INFO:teuthology.orchestra.run.smithi130.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T22:52:42.679 INFO:teuthology.orchestra.run.smithi130.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T22:52:42.686 INFO:teuthology.orchestra.run.smithi130.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T22:52:42.692 INFO:teuthology.orchestra.run.smithi130.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T22:52:42.697 INFO:teuthology.orchestra.run.smithi130.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T22:52:42.705 INFO:teuthology.orchestra.run.smithi055.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T22:52:42.710 INFO:teuthology.orchestra.run.smithi055.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T22:52:42.732 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T22:52:42.737 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T22:52:42.752 INFO:teuthology.orchestra.run.smithi055.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T22:52:42.764 INFO:teuthology.orchestra.run.smithi130.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T22:52:42.765 INFO:teuthology.orchestra.run.smithi130.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T22:52:42.774 INFO:teuthology.orchestra.run.smithi130.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T22:52:42.791 INFO:teuthology.orchestra.run.smithi055.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T22:52:42.792 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T22:52:42.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T22:52:42.817 INFO:teuthology.orchestra.run.smithi055.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T22:52:42.819 INFO:teuthology.orchestra.run.smithi055.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T22:52:42.823 INFO:teuthology.orchestra.run.smithi055.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T22:52:42.835 INFO:teuthology.orchestra.run.smithi130.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T22:52:42.836 INFO:teuthology.orchestra.run.smithi130.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T22:52:42.837 INFO:teuthology.orchestra.run.smithi130.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T22:52:42.839 INFO:teuthology.orchestra.run.smithi130.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T22:52:42.905 INFO:teuthology.orchestra.run.smithi157.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T22:52:42.912 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T22:52:42.942 INFO:teuthology.orchestra.run.smithi130.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T22:52:42.943 INFO:teuthology.orchestra.run.smithi130.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T22:52:42.944 INFO:teuthology.orchestra.run.smithi130.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T22:52:42.945 INFO:teuthology.orchestra.run.smithi130.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T22:52:42.954 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T22:52:42.992 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T22:52:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T22:52:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T22:52:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T22:52:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T22:52:43.023 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T22:52:43.077 INFO:teuthology.orchestra.run.smithi157.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T22:52:43.123 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T22:52:43.128 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T22:52:43.129 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T22:52:43.179 INFO:teuthology.orchestra.run.smithi157.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T22:52:43.184 INFO:teuthology.orchestra.run.smithi157.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T22:52:43.197 INFO:teuthology.orchestra.run.smithi157.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T22:52:43.207 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T22:52:43.490 INFO:teuthology.orchestra.run.smithi130.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T22:52:43.686 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T22:52:44.034 INFO:teuthology.orchestra.run.smithi055.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T22:52:44.127 INFO:teuthology.orchestra.run.smithi130.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T22:52:44.129 INFO:teuthology.orchestra.run.smithi130.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T22:52:44.279 INFO:teuthology.orchestra.run.smithi055.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T22:52:44.313 INFO:teuthology.orchestra.run.smithi055.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T22:52:44.320 INFO:teuthology.orchestra.run.smithi130.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T22:52:44.475 INFO:teuthology.orchestra.run.smithi055.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T22:52:44.484 INFO:teuthology.orchestra.run.smithi055.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T22:52:44.485 INFO:teuthology.orchestra.run.smithi055.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T22:52:44.486 INFO:teuthology.orchestra.run.smithi055.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T22:52:44.487 INFO:teuthology.orchestra.run.smithi055.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T22:52:44.488 INFO:teuthology.orchestra.run.smithi055.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T22:52:44.489 INFO:teuthology.orchestra.run.smithi055.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T22:52:44.518 INFO:teuthology.orchestra.run.smithi055.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T22:52:44.520 INFO:teuthology.orchestra.run.smithi055.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T22:52:44.521 INFO:teuthology.orchestra.run.smithi055.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T22:52:44.565 INFO:teuthology.orchestra.run.smithi055.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T22:52:44.603 INFO:teuthology.orchestra.run.smithi055.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T22:52:44.607 INFO:teuthology.orchestra.run.smithi055.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T22:52:44.611 INFO:teuthology.orchestra.run.smithi055.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T22:52:44.613 INFO:teuthology.orchestra.run.smithi055.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T22:52:44.623 INFO:teuthology.orchestra.run.smithi055.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T22:52:44.628 INFO:teuthology.orchestra.run.smithi055.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T22:52:44.694 INFO:teuthology.orchestra.run.smithi055.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T22:52:44.695 INFO:teuthology.orchestra.run.smithi055.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T22:52:44.704 INFO:teuthology.orchestra.run.smithi055.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T22:52:44.763 INFO:teuthology.orchestra.run.smithi055.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T22:52:44.764 INFO:teuthology.orchestra.run.smithi055.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T22:52:44.764 INFO:teuthology.orchestra.run.smithi055.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T22:52:44.766 INFO:teuthology.orchestra.run.smithi055.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T22:52:44.869 INFO:teuthology.orchestra.run.smithi055.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T22:52:44.869 INFO:teuthology.orchestra.run.smithi055.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T22:52:44.871 INFO:teuthology.orchestra.run.smithi055.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T22:52:44.872 INFO:teuthology.orchestra.run.smithi055.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T22:52:44.933 INFO:teuthology.orchestra.run.smithi055.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T22:52:44.935 INFO:teuthology.orchestra.run.smithi055.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T22:52:44.936 INFO:teuthology.orchestra.run.smithi055.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T22:52:44.941 INFO:teuthology.orchestra.run.smithi055.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T22:52:45.064 INFO:teuthology.orchestra.run.smithi130.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T22:52:45.102 INFO:teuthology.orchestra.run.smithi157.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T22:52:45.279 INFO:teuthology.orchestra.run.smithi055.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T22:52:45.283 INFO:teuthology.orchestra.run.smithi055.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T22:52:45.515 INFO:teuthology.orchestra.run.smithi055.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T22:52:46.015 INFO:teuthology.orchestra.run.smithi157.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T22:52:46.025 INFO:teuthology.orchestra.run.smithi157.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T22:52:46.063 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T22:52:46.064 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T22:52:46.075 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T22:52:46.205 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T22:52:46.207 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T22:52:46.208 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T22:52:46.262 INFO:teuthology.orchestra.run.smithi157.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T22:52:46.448 INFO:teuthology.orchestra.run.smithi055.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T22:52:46.498 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T22:52:46.500 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T22:52:47.257 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T22:52:47.258 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T22:52:47.316 INFO:teuthology.orchestra.run.smithi157.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T22:52:48.293 INFO:teuthology.orchestra.run.smithi130.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T22:52:48.294 INFO:teuthology.orchestra.run.smithi130.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T22:52:48.336 INFO:teuthology.orchestra.run.smithi130.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T22:52:48.640 INFO:teuthology.orchestra.run.smithi130.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T22:52:48.661 INFO:teuthology.orchestra.run.smithi130.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T22:52:48.663 INFO:teuthology.orchestra.run.smithi130.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T22:52:49.130 INFO:teuthology.orchestra.run.smithi130.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T22:52:49.131 INFO:teuthology.orchestra.run.smithi130.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T22:52:50.307 INFO:teuthology.orchestra.run.smithi130.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T22:52:50.308 INFO:teuthology.orchestra.run.smithi130.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T22:52:50.772 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T22:52:50.773 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T22:52:50.774 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T22:52:50.835 INFO:teuthology.orchestra.run.smithi055.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T22:52:50.836 INFO:teuthology.orchestra.run.smithi055.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T22:52:50.847 INFO:teuthology.orchestra.run.smithi055.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T22:52:51.126 INFO:teuthology.orchestra.run.smithi157.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T22:52:51.127 INFO:teuthology.orchestra.run.smithi157.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T22:52:51.158 INFO:teuthology.orchestra.run.smithi157.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T22:52:51.195 INFO:teuthology.orchestra.run.smithi055.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T22:52:51.197 INFO:teuthology.orchestra.run.smithi055.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T22:52:51.219 INFO:teuthology.orchestra.run.smithi055.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T22:52:51.471 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T22:52:51.472 INFO:teuthology.orchestra.run.smithi157.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T22:52:51.474 INFO:teuthology.orchestra.run.smithi157.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T22:52:51.475 INFO:teuthology.orchestra.run.smithi157.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T22:52:51.664 INFO:teuthology.orchestra.run.smithi055.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T22:52:51.665 INFO:teuthology.orchestra.run.smithi055.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T22:52:51.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T22:52:51.798 INFO:teuthology.orchestra.run.smithi157.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T22:52:52.542 INFO:teuthology.orchestra.run.smithi055.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T22:52:52.543 INFO:teuthology.orchestra.run.smithi055.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T22:52:52.559 INFO:teuthology.orchestra.run.smithi157.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T22:52:52.578 INFO:teuthology.orchestra.run.smithi157.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T22:52:54.512 INFO:teuthology.orchestra.run.smithi130.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T22:52:54.512 INFO:teuthology.orchestra.run.smithi130.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T22:52:54.513 INFO:teuthology.orchestra.run.smithi130.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T22:52:55.262 INFO:teuthology.orchestra.run.smithi130.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T22:52:55.854 INFO:teuthology.orchestra.run.smithi130.stdout:Fetched 153 MB in 16s (9,402 kB/s) 2022-08-01T22:52:55.976 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T22:52:56.020 INFO:teuthology.orchestra.run.smithi130.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T22:52:56.023 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T22:52:56.069 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T22:52:56.144 INFO:teuthology.orchestra.run.smithi157.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T22:52:56.145 INFO:teuthology.orchestra.run.smithi157.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T22:52:56.145 INFO:teuthology.orchestra.run.smithi157.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T22:52:56.344 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T22:52:56.360 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T22:52:56.403 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:52:56.499 INFO:teuthology.orchestra.run.smithi055.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T22:52:56.500 INFO:teuthology.orchestra.run.smithi055.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T22:52:56.568 INFO:teuthology.orchestra.run.smithi157.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T22:52:56.572 INFO:teuthology.orchestra.run.smithi055.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T22:52:56.706 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T22:52:56.720 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T22:52:56.764 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:52:56.891 INFO:teuthology.orchestra.run.smithi055.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T22:52:57.142 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T22:52:57.154 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T22:52:57.234 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:52:57.526 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:52:57.548 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 153 MB in 18s (8,525 kB/s) 2022-08-01T22:52:57.629 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:52:57.700 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T22:52:57.741 INFO:teuthology.orchestra.run.smithi055.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T22:52:57.744 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T22:52:57.781 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T22:52:58.102 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T22:52:58.121 INFO:teuthology.orchestra.run.smithi157.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T22:52:58.166 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:52:58.197 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T22:52:58.206 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T22:52:58.234 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:52:58.301 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:52:58.529 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T22:52:58.548 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T22:52:58.588 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:52:58.905 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T22:52:58.919 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:52:58.924 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T22:52:58.939 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T22:52:58.954 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:52:59.043 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:52:59.257 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-rados. 2022-08-01T22:52:59.269 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:52:59.306 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:52:59.361 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:52:59.463 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:52:59.668 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T22:52:59.683 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:52:59.727 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:52:59.996 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:00.004 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T22:53:00.019 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:00.054 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:00.117 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:53:00.340 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T22:53:00.353 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:00.390 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:00.704 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T22:53:00.719 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:00.734 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T22:53:00.750 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T22:53:00.754 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:00.793 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T22:53:01.029 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T22:53:01.043 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:01.087 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:01.091 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-rados. 2022-08-01T22:53:01.104 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:01.141 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:01.197 INFO:teuthology.orchestra.run.smithi157.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T22:53:01.373 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T22:53:01.440 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T22:53:01.455 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T22:53:01.474 INFO:teuthology.orchestra.run.smithi157.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T22:53:01.490 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:01.504 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T22:53:01.520 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:01.568 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:01.587 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T22:53:01.809 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T22:53:01.824 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T22:53:01.838 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T22:53:01.852 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:01.868 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:53:01.879 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:02.034 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T22:53:02.050 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T22:53:02.052 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T22:53:02.053 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T22:53:02.054 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T22:53:02.056 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T22:53:02.057 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T22:53:02.061 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T22:53:02.062 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T22:53:02.114 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T22:53:02.144 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T22:53:02.147 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T22:53:02.151 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T22:53:02.153 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T22:53:02.163 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T22:53:02.171 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T22:53:02.212 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package librgw2. 2022-08-01T22:53:02.216 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T22:53:02.227 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:02.230 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T22:53:02.231 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:02.231 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T22:53:02.239 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T22:53:02.270 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:02.274 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:02.308 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T22:53:02.308 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T22:53:02.309 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T22:53:02.311 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T22:53:02.409 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T22:53:02.409 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T22:53:02.411 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T22:53:02.412 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T22:53:02.416 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T22:53:02.417 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T22:53:02.474 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T22:53:02.479 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T22:53:02.569 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T22:53:02.585 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T22:53:02.628 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T22:53:02.823 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T22:53:02.840 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:02.864 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:02.905 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T22:53:02.918 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:02.955 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:03.092 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 153 MB in 24s (6,508 kB/s) 2022-08-01T22:53:03.134 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T22:53:03.146 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T22:53:03.277 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:03.287 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T22:53:03.292 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T22:53:03.305 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T22:53:03.330 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T22:53:03.334 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T22:53:03.367 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:03.368 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T22:53:03.655 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T22:53:03.668 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T22:53:03.679 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package liboath0. 2022-08-01T22:53:03.691 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T22:53:03.728 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:03.730 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:53:03.870 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T22:53:03.884 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T22:53:03.953 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:53:03.973 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T22:53:03.988 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:04.018 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T22:53:04.023 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:04.028 INFO:teuthology.orchestra.run.smithi157.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T22:53:04.031 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T22:53:04.032 INFO:teuthology.orchestra.run.smithi157.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T22:53:04.074 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:53:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T22:53:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T22:53:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T22:53:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T22:53:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T22:53:04.282 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package librgw2. 2022-08-01T22:53:04.297 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:04.328 INFO:teuthology.orchestra.run.smithi157.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T22:53:04.336 INFO:teuthology.orchestra.run.smithi157.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T22:53:04.340 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:04.385 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T22:53:04.400 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T22:53:04.417 INFO:teuthology.orchestra.run.smithi157.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T22:53:04.476 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-common. 2022-08-01T22:53:04.492 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:04.494 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:53:04.536 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:04.725 INFO:teuthology.orchestra.run.smithi157.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T22:53:04.769 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:04.842 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T22:53:04.857 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:04.890 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:53:04.893 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:04.915 INFO:teuthology.orchestra.run.smithi157.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T22:53:05.097 INFO:teuthology.orchestra.run.smithi157.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T22:53:05.183 INFO:teuthology.orchestra.run.smithi157.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T22:53:05.237 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T22:53:05.251 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T22:53:05.295 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:05.450 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:05.578 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:53:05.580 INFO:teuthology.orchestra.run.smithi157.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T22:53:05.624 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package liboath0. 2022-08-01T22:53:05.640 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T22:53:05.674 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:05.775 INFO:teuthology.orchestra.run.smithi157.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T22:53:05.952 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T22:53:05.967 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:06.010 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:06.148 INFO:teuthology.orchestra.run.smithi157.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T22:53:06.158 INFO:teuthology.orchestra.run.smithi157.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T22:53:06.315 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T22:53:06.329 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:06.374 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:06.463 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-common. 2022-08-01T22:53:06.479 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:06.523 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:06.538 INFO:teuthology.orchestra.run.smithi157.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T22:53:06.541 INFO:teuthology.orchestra.run.smithi157.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T22:53:06.556 INFO:teuthology.orchestra.run.smithi157.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T22:53:06.560 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-base. 2022-08-01T22:53:06.561 INFO:teuthology.orchestra.run.smithi157.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T22:53:06.576 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:06.618 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:06.668 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2022-08-01T22:53:06.680 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:06.718 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:06.727 INFO:teuthology.orchestra.run.smithi157.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T22:53:07.079 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T22:53:07.093 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:07.138 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:07.374 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T22:53:07.386 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:07.415 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:07.505 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T22:53:07.517 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T22:53:07.555 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:53:07.762 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T22:53:07.774 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:07.817 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:07.942 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T22:53:07.958 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T22:53:08.000 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T22:53:08.079 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T22:53:08.092 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T22:53:08.121 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T22:53:08.252 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T22:53:08.265 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:08.294 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T22:53:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:08.483 INFO:teuthology.orchestra.run.smithi157.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T22:53:08.560 INFO:teuthology.orchestra.run.smithi157.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T22:53:08.561 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-base. 2022-08-01T22:53:08.574 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:08.583 INFO:teuthology.orchestra.run.smithi157.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T22:53:08.621 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:08.662 INFO:teuthology.orchestra.run.smithi157.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T22:53:08.826 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T22:53:08.836 INFO:teuthology.orchestra.run.smithi157.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T22:53:08.841 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T22:53:08.885 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:08.921 INFO:teuthology.orchestra.run.smithi157.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T22:53:08.996 INFO:teuthology.orchestra.run.smithi157.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T22:53:09.180 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T22:53:09.184 INFO:teuthology.orchestra.run.smithi157.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T22:53:09.192 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T22:53:09.221 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:53:09.324 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T22:53:09.337 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T22:53:09.374 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:53:09.566 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2022-08-01T22:53:09.581 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:09.615 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:09.702 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T22:53:09.716 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T22:53:09.760 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:53:09.855 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 153 MB in 30s (5,063 kB/s) 2022-08-01T22:53:09.996 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T22:53:10.036 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T22:53:10.039 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T22:53:10.041 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T22:53:10.057 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T22:53:10.064 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T22:53:10.076 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T22:53:10.077 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T22:53:10.100 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:53:10.113 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:10.210 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T22:53:10.226 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:10.268 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:10.413 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T22:53:10.425 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T22:53:10.426 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T22:53:10.438 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T22:53:10.463 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:53:10.475 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T22:53:10.512 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T22:53:10.525 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T22:53:10.562 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T22:53:10.579 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T22:53:10.595 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T22:53:10.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:10.765 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T22:53:10.780 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T22:53:10.803 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T22:53:10.816 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T22:53:10.826 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:53:10.850 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T22:53:10.853 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:53:10.862 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:10.906 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:10.949 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0. 2022-08-01T22:53:10.962 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T22:53:10.999 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:11.110 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T22:53:11.123 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-mako. 2022-08-01T22:53:11.124 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T22:53:11.135 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T22:53:11.164 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:53:11.194 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:53:11.219 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T22:53:11.236 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T22:53:11.277 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:53:11.310 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T22:53:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:11.368 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:11.459 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T22:53:11.472 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T22:53:11.500 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:53:11.555 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:11.606 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T22:53:11.619 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T22:53:11.656 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:53:11.681 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:53:11.795 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T22:53:11.808 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T22:53:11.813 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2022-08-01T22:53:11.829 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:11.845 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:53:11.863 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:11.993 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T22:53:12.005 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T22:53:12.076 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:12.173 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-webob. 2022-08-01T22:53:12.185 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T22:53:12.215 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:12.226 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T22:53:12.318 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T22:53:12.338 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T22:53:12.353 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T22:53:12.388 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T22:53:12.534 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T22:53:12.548 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T22:53:12.584 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:53:12.691 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T22:53:12.706 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T22:53:12.749 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:53:12.895 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T22:53:12.908 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T22:53:12.945 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T22:53:12.964 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T22:53:12.979 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:13.022 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:13.056 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-mako. 2022-08-01T22:53:13.069 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T22:53:13.111 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:53:13.301 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T22:53:13.314 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:13.349 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rados. 2022-08-01T22:53:13.373 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:13.393 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:13.408 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:13.456 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T22:53:13.478 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T22:53:13.514 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:53:13.718 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T22:53:13.719 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T22:53:13.732 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T22:53:13.733 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:13.759 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T22:53:13.773 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T22:53:13.776 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T22:53:13.794 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:13.811 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:53:14.055 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T22:53:14.067 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:14.087 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-paste. 2022-08-01T22:53:14.102 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T22:53:14.105 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:14.146 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-webob. 2022-08-01T22:53:14.146 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:53:14.162 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T22:53:14.203 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T22:53:14.465 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T22:53:14.478 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:14.486 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2022-08-01T22:53:14.501 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:14.515 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:14.531 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T22:53:14.541 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T22:53:14.546 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:14.547 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T22:53:14.555 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T22:53:14.589 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:53:14.599 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:53:14.854 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T22:53:14.865 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T22:53:14.868 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T22:53:14.886 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T22:53:14.903 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T22:53:14.909 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T22:53:14.935 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T22:53:14.948 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T22:53:14.985 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:53:15.197 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T22:53:15.211 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:15.230 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T22:53:15.243 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:15.246 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:15.282 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:15.296 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T22:53:15.312 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T22:53:15.347 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T22:53:15.457 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T22:53:15.470 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T22:53:15.507 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:53:15.575 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T22:53:15.587 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T22:53:15.590 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T22:53:15.606 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T22:53:15.625 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T22:53:15.640 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:15.649 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T22:53:15.662 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T22:53:15.691 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:53:15.902 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T22:53:15.915 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T22:53:15.952 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T22:53:15.961 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-paste. 2022-08-01T22:53:15.968 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T22:53:15.977 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T22:53:15.981 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T22:53:16.019 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:53:16.037 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T22:53:16.043 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:53:16.052 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T22:53:16.087 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:53:16.197 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T22:53:16.212 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:16.246 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:16.381 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T22:53:16.395 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T22:53:16.396 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librgw2. 2022-08-01T22:53:16.409 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:16.439 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:53:16.479 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:16.483 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T22:53:16.496 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:16.541 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:16.593 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T22:53:16.608 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T22:53:16.642 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:53:16.751 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T22:53:16.766 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T22:53:16.809 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:53:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T22:53:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T22:53:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:53:16.977 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T22:53:16.991 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:17.027 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:17.048 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T22:53:17.061 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:17.096 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T22:53:17.099 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:17.110 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T22:53:17.146 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T22:53:17.241 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T22:53:17.254 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T22:53:17.291 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:17.416 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T22:53:17.428 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T22:53:17.435 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T22:53:17.450 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T22:53:17.474 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:53:17.484 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:17.560 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T22:53:17.574 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T22:53:17.618 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T22:53:17.821 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T22:53:17.836 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T22:53:17.837 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liboath0. 2022-08-01T22:53:17.850 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T22:53:17.879 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:53:17.887 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T22:53:17.937 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T22:53:17.980 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:53:18.022 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T22:53:18.038 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:18.080 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:18.198 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T22:53:18.211 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:18.241 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T22:53:18.257 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:18.272 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:18.301 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:18.308 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2022-08-01T22:53:18.324 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T22:53:18.366 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:53:18.669 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T22:53:18.685 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T22:53:18.709 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-common. 2022-08-01T22:53:18.719 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:53:18.724 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:18.759 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:18.794 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T22:53:18.810 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:18.854 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:19.022 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T22:53:19.035 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T22:53:19.072 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:53:19.359 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2022-08-01T22:53:19.373 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T22:53:19.417 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T22:53:19.720 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T22:53:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T22:53:19.795 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:53:19.807 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T22:53:19.822 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:19.856 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:20.098 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T22:53:20.113 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T22:53:20.156 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T22:53:20.294 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph. 2022-08-01T22:53:20.307 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:20.344 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:20.470 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T22:53:20.483 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:20.521 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:20.697 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T22:53:20.709 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:20.747 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:20.855 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T22:53:20.867 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T22:53:20.873 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-base. 2022-08-01T22:53:20.888 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:20.905 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T22:53:20.941 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:21.092 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T22:53:21.109 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:21.141 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2022-08-01T22:53:21.156 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T22:53:21.162 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:21.199 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:53:21.620 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package cephadm. 2022-08-01T22:53:21.635 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:21.644 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T22:53:21.660 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T22:53:21.678 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:21.694 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:53:21.869 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T22:53:21.881 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T22:53:21.944 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:53:21.981 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T22:53:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T22:53:22.014 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T22:53:22.028 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T22:53:22.041 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:53:22.073 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T22:53:22.204 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph. 2022-08-01T22:53:22.220 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:22.263 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T22:53:22.366 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T22:53:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T22:53:22.380 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T22:53:22.392 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T22:53:22.407 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:22.417 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T22:53:22.422 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T22:53:22.450 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T22:53:22.605 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:22.649 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:22.667 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T22:53:22.681 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:22.716 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:22.731 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T22:53:22.743 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T22:53:22.786 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:53:22.787 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T22:53:22.803 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T22:53:22.846 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:23.019 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T22:53:23.034 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:23.077 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:23.079 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T22:53:23.095 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T22:53:23.103 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T22:53:23.113 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T22:53:23.137 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:53:23.156 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:53:23.157 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T22:53:23.171 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T22:53:23.207 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:53:23.460 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T22:53:23.465 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T22:53:23.472 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T22:53:23.478 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T22:53:23.510 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:53:23.515 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:53:23.553 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T22:53:23.567 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:23.598 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package cephadm. 2022-08-01T22:53:23.611 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:23.613 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:23.658 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:23.787 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-routes. 2022-08-01T22:53:23.799 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T22:53:23.828 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T22:53:23.860 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T22:53:23.876 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T22:53:23.918 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:23.986 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T22:53:24.000 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T22:53:24.043 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T22:53:24.081 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T22:53:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:24.139 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:24.165 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T22:53:24.180 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:24.215 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:24.246 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T22:53:24.263 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T22:53:24.305 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T22:53:24.387 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T22:53:24.400 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:24.437 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:24.666 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T22:53:24.681 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T22:53:24.725 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:53:24.733 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T22:53:24.746 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T22:53:24.801 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:24.994 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-mako. 2022-08-01T22:53:25.006 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T22:53:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:53:25.143 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T22:53:25.157 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:25.185 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T22:53:25.199 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T22:53:25.201 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:25.235 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:53:25.261 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T22:53:25.273 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:25.315 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:25.322 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T22:53:25.336 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T22:53:25.372 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:53:25.505 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T22:53:25.519 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T22:53:25.555 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:53:25.662 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T22:53:25.673 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T22:53:25.691 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T22:53:25.710 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:53:25.711 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T22:53:25.749 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:53:25.799 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-routes. 2022-08-01T22:53:25.815 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T22:53:25.866 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T22:53:25.963 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T22:53:25.976 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T22:53:26.013 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:53:26.019 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webob. 2022-08-01T22:53:26.036 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T22:53:26.068 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T22:53:26.212 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T22:53:26.224 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:26.261 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:26.349 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T22:53:26.364 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T22:53:26.374 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T22:53:26.389 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T22:53:26.407 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T22:53:26.440 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:53:26.743 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T22:53:26.760 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T22:53:26.801 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T22:53:27.163 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T22:53:27.178 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:27.218 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:27.298 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T22:53:27.312 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:27.356 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:27.474 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2022-08-01T22:53:27.488 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:27.532 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:27.541 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T22:53:27.556 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T22:53:27.599 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T22:53:27.658 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T22:53:27.672 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T22:53:27.675 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T22:53:27.691 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T22:53:27.716 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:53:27.733 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:27.835 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T22:53:27.851 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:27.868 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-paste. 2022-08-01T22:53:27.884 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T22:53:27.885 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:27.929 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:53:28.003 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T22:53:28.019 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T22:53:28.053 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:53:28.247 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T22:53:28.262 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T22:53:28.271 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T22:53:28.285 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:28.296 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:53:28.313 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T22:53:28.326 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T22:53:28.329 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:28.364 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T22:53:28.370 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T22:53:28.379 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T22:53:28.423 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T22:53:28.591 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T22:53:28.605 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T22:53:28.649 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:53:28.682 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T22:53:28.696 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T22:53:28.736 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:28.833 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2022-08-01T22:53:28.848 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:28.891 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:28.969 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T22:53:28.981 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T22:53:29.019 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T22:53:29.177 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T22:53:29.192 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T22:53:29.363 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T22:53:29.374 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T22:53:29.379 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T22:53:29.408 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T22:53:29.423 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:53:29.423 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:29.466 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:29.680 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T22:53:29.694 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:29.723 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T22:53:29.736 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T22:53:29.738 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:29.768 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T22:53:29.773 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:29.781 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T22:53:29.818 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:53:30.067 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T22:53:30.081 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T22:53:30.125 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:30.205 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T22:53:30.220 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:30.263 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:30.370 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T22:53:30.379 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T22:53:30.382 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T22:53:30.390 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T22:53:30.412 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:53:30.435 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T22:53:30.569 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T22:53:30.584 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T22:53:30.619 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:30.656 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T22:53:30.671 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T22:53:30.714 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:53:30.733 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T22:53:30.748 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:30.783 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:30.788 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T22:53:30.803 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T22:53:30.846 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:30.888 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T22:53:30.905 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T22:53:30.946 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:53:31.066 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2022-08-01T22:53:31.080 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T22:53:31.116 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T22:53:31.308 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T22:53:31.322 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T22:53:31.358 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:53:31.453 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T22:53:31.466 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:31.497 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T22:53:31.503 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:31.513 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:31.555 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:31.664 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T22:53:31.676 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T22:53:31.720 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T22:53:31.720 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T22:53:31.737 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:31.778 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:32.056 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T22:53:32.072 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T22:53:32.115 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T22:53:32.410 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T22:53:32.426 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T22:53:32.478 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T22:53:32.484 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:53:32.493 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:32.559 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:32.658 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T22:53:32.673 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T22:53:32.736 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:32.754 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T22:53:32.768 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T22:53:32.812 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:53:32.873 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T22:53:32.887 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T22:53:32.931 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:53:33.044 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T22:53:33.058 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T22:53:33.102 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:53:33.108 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T22:53:33.121 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T22:53:33.193 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:53:33.234 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T22:53:33.249 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T22:53:33.292 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:53:33.430 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T22:53:33.446 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T22:53:33.489 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:53:33.508 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T22:53:33.516 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T22:53:33.563 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:53:33.621 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T22:53:33.635 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T22:53:33.670 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T22:53:33.758 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T22:53:33.773 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T22:53:33.799 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T22:53:33.905 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T22:53:33.920 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T22:53:33.963 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:53:33.991 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph. 2022-08-01T22:53:34.006 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:34.041 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:34.111 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T22:53:34.127 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T22:53:34.178 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T22:53:34.361 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T22:53:34.377 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:34.419 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:34.480 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T22:53:34.494 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T22:53:34.496 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T22:53:34.507 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:34.543 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:34.562 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:53:34.829 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T22:53:34.831 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T22:53:34.844 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T22:53:34.845 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:34.850 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T22:53:34.863 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T22:53:34.879 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:53:34.888 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:34.899 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:53:35.232 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T22:53:35.246 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:35.247 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T22:53:35.262 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T22:53:35.262 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T22:53:35.276 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T22:53:35.306 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:35.307 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:35.323 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:53:35.334 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephadm. 2022-08-01T22:53:35.346 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:35.375 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:35.560 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package jq. 2022-08-01T22:53:35.575 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:35.618 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:35.656 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T22:53:35.670 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T22:53:35.703 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T22:53:35.714 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:53:35.723 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T22:53:35.764 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T22:53:35.817 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T22:53:35.832 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T22:53:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T22:53:35.876 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package socat. 2022-08-01T22:53:35.885 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T22:53:35.929 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T22:53:36.051 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T22:53:36.067 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T22:53:36.097 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T22:53:36.109 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:53:36.110 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:36.148 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:36.204 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T22:53:36.220 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T22:53:36.262 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:36.422 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T22:53:36.432 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T22:53:36.477 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:53:36.494 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T22:53:36.508 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T22:53:36.550 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:36.597 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T22:53:36.614 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:36.655 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:36.772 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T22:53:36.788 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T22:53:36.803 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T22:53:36.816 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T22:53:36.830 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:53:36.878 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:53:36.937 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T22:53:36.952 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:36.958 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T22:53:36.974 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T22:53:36.987 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:37.041 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:53:37.122 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T22:53:37.135 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T22:53:37.168 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package ceph-test. 2022-08-01T22:53:37.177 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:53:37.184 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:37.226 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:37.377 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T22:53:37.393 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:37.436 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:37.475 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-routes. 2022-08-01T22:53:37.489 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T22:53:37.534 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T22:53:37.705 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package jq. 2022-08-01T22:53:37.721 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:37.755 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:37.828 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T22:53:37.847 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:37.878 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:37.983 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package socat. 2022-08-01T22:53:37.998 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T22:53:38.041 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T22:53:38.193 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T22:53:38.206 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T22:53:38.242 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:38.530 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T22:53:38.545 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T22:53:38.546 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T22:53:38.559 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T22:53:38.588 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:53:38.595 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:53:38.853 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T22:53:38.857 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T22:53:38.869 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:38.874 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T22:53:38.903 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:38.908 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:53:38.932 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T22:53:38.946 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T22:53:39.007 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:53:39.189 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T22:53:39.203 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T22:53:39.246 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package ceph-test. 2022-08-01T22:53:39.250 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:53:39.263 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:39.294 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T22:53:39.300 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:39.309 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T22:53:39.343 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T22:53:39.583 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T22:53:39.596 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T22:53:39.613 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T22:53:39.626 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T22:53:39.633 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:53:39.663 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T22:53:39.937 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T22:53:39.950 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T22:53:39.966 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T22:53:39.980 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T22:53:40.008 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:53:40.019 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T22:53:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T22:53:40.317 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T22:53:40.361 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:53:40.647 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T22:53:40.665 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T22:53:40.705 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T22:53:40.725 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T22:53:40.740 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T22:53:40.786 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:53:40.949 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T22:53:40.966 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:40.999 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:41.101 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T22:53:41.115 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T22:53:41.159 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:53:41.202 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T22:53:41.218 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:41.260 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:41.295 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T22:53:41.309 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T22:53:41.353 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:41.471 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T22:53:41.488 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T22:53:41.529 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:53:41.686 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T22:53:41.693 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T22:53:41.748 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:41.924 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T22:53:41.940 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T22:53:41.982 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T22:53:42.035 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T22:53:42.050 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:42.096 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:42.111 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T22:53:42.126 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T22:53:42.169 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:53:42.344 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T22:53:42.358 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T22:53:42.402 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:42.429 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T22:53:42.448 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T22:53:42.487 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:53:42.497 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-attr. 2022-08-01T22:53:42.511 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T22:53:42.549 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T22:53:42.770 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T22:53:42.780 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T22:53:42.820 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T22:53:42.842 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T22:53:42.855 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T22:53:42.857 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:42.866 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T22:53:42.898 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:42.908 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:53:43.077 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T22:53:43.090 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:43.102 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T22:53:43.117 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:43.127 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:43.170 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:43.183 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2022-08-01T22:53:43.194 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:43.211 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T22:53:43.226 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T22:53:43.226 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:43.261 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T22:53:43.396 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T22:53:43.410 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:43.438 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:43.454 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2022-08-01T22:53:43.470 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T22:53:43.512 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T22:53:43.600 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T22:53:43.617 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T22:53:43.657 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:53:43.741 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T22:53:43.758 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T22:53:43.834 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:44.018 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T22:53:44.032 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T22:53:44.077 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:53:44.095 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T22:53:44.109 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T22:53:44.152 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T22:53:44.230 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T22:53:44.246 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T22:53:44.288 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:53:44.322 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T22:53:44.337 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T22:53:44.355 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T22:53:44.371 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T22:53:44.380 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:53:44.389 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T22:53:44.402 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T22:53:44.413 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:44.447 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T22:53:44.625 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-attr. 2022-08-01T22:53:44.642 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T22:53:44.674 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T22:53:44.750 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T22:53:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2022-08-01T22:53:44.764 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T22:53:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:44.775 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T22:53:44.792 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T22:53:44.801 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:44.810 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:53:44.835 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T22:53:45.019 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T22:53:45.032 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T22:53:45.077 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:53:45.179 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T22:53:45.197 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T22:53:45.229 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:53:45.356 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T22:53:45.370 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T22:53:45.406 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T22:53:45.445 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T22:53:45.462 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T22:53:45.503 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:53:45.581 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T22:53:45.596 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T22:53:45.631 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T22:53:45.707 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T22:53:45.721 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T22:53:45.757 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:53:45.806 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-py. 2022-08-01T22:53:45.827 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:45.864 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:53:45.926 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T22:53:45.939 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T22:53:45.975 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T22:53:46.226 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T22:53:46.236 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T22:53:46.240 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T22:53:46.253 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T22:53:46.254 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T22:53:46.268 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T22:53:46.284 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:53:46.294 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T22:53:46.328 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:53:46.590 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T22:53:46.605 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T22:53:46.615 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T22:53:46.620 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T22:53:46.631 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T22:53:46.636 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T22:53:46.651 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T22:53:46.673 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:53:46.678 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T22:53:46.958 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T22:53:46.973 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T22:53:47.012 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T22:53:47.024 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T22:53:47.031 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T22:53:47.037 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T22:53:47.041 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T22:53:47.081 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:53:47.085 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:53:47.295 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T22:53:47.310 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T22:53:47.352 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:53:47.421 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T22:53:47.438 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T22:53:47.479 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:53:47.528 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package radosgw. 2022-08-01T22:53:47.540 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:47.585 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:47.681 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-py. 2022-08-01T22:53:47.695 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:47.739 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:53:47.791 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T22:53:47.805 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T22:53:47.849 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:53:48.075 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T22:53:48.092 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T22:53:48.135 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:53:48.338 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T22:53:48.352 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T22:53:48.372 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T22:53:48.379 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T22:53:48.385 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:48.420 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:48.439 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T22:53:48.454 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T22:53:48.496 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T22:53:48.586 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T22:53:48.594 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:48.638 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:48.674 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T22:53:48.687 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:48.725 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:48.782 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T22:53:48.799 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T22:53:48.840 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:53:48.851 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T22:53:48.865 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T22:53:48.906 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:53:48.962 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T22:53:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T22:53:49.004 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:49.044 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:49.210 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T22:53:49.225 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:49.260 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:49.268 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:49.355 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package radosgw. 2022-08-01T22:53:49.368 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:49.369 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:49.372 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T22:53:49.387 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T22:53:49.412 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:49.465 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:49.495 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:49.537 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package jq. 2022-08-01T22:53:49.552 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T22:53:49.587 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:53:49.680 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:49.806 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package socat. 2022-08-01T22:53:49.824 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T22:53:49.862 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T22:53:49.864 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T22:53:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T22:53:49.919 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:53:50.249 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-attr. 2022-08-01T22:53:50.261 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T22:53:50.274 INFO:teuthology.orchestra.run.smithi130.stdout:Adding system user cephadm....done 2022-08-01T22:53:50.306 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T22:53:50.371 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T22:53:50.383 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T22:53:50.421 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:53:50.436 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:50.572 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T22:53:50.589 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:50.617 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T22:53:50.630 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:50.631 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T22:53:50.675 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T22:53:50.675 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:53:50.690 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T22:53:50.725 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:53:50.754 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:50.911 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T22:53:50.970 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T22:53:50.983 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T22:53:51.045 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T22:53:51.089 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:51.094 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-test. 2022-08-01T22:53:51.111 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:51.162 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:51.215 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:53:51.217 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:51.345 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:51.382 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T22:53:51.398 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T22:53:51.440 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:53:51.461 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:51.550 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T22:53:51.629 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:51.930 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T22:53:51.935 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:51.942 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T22:53:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T22:53:52.200 INFO:teuthology.orchestra.run.smithi055.stdout:Adding system user cephadm....done 2022-08-01T22:53:52.252 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:52.271 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T22:53:52.288 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T22:53:52.330 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T22:53:52.352 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:52.537 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T22:53:52.650 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T22:53:52.664 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T22:53:52.691 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T22:53:52.704 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:52.830 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:53:52.972 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:52.987 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T22:53:53.002 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T22:53:53.045 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:53:53.074 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:53:53.131 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:53:53.348 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2022-08-01T22:53:53.364 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T22:53:53.410 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T22:53:53.416 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:53.431 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:53:53.683 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:53.743 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T22:53:53.757 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T22:53:53.793 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T22:53:53.801 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:53:53.826 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:54.087 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T22:53:54.094 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:53:54.103 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T22:53:54.110 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:54.145 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T22:53:54.398 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:53:54.428 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T22:53:54.507 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T22:53:54.522 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T22:53:54.566 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:53:54.679 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:53:54.704 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:53:54.765 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T22:53:54.782 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T22:53:54.823 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T22:53:55.006 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:53:55.036 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2022-08-01T22:53:55.052 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:55.059 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T22:53:55.076 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T22:53:55.081 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:53:55.094 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:55.109 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:55.273 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:55.365 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:53:55.692 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:53:55.728 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T22:53:55.745 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:55.812 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:55.843 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:53:56.114 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T22:53:56.181 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T22:53:56.194 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:53:56.198 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T22:53:56.215 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T22:53:56.231 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:56.240 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T22:53:56.282 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:56.477 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T22:53:56.486 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:53:56.505 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T22:53:56.629 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T22:53:56.642 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T22:53:56.704 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:53:56.752 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:53:56.778 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T22:53:56.803 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:53:56.928 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:53:56.990 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-attr. 2022-08-01T22:53:57.005 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T22:53:57.049 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T22:53:57.054 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:53:57.055 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:53:57.080 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:53:57.244 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:53:57.248 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:57.393 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T22:53:57.407 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T22:53:57.422 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:53:57.451 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:53:57.533 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:53:57.698 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:53:57.754 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T22:53:57.770 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T22:53:57.803 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T22:53:57.830 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2022-08-01T22:53:57.925 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:53:57.971 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:53:58.049 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:53:58.157 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T22:53:58.172 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T22:53:58.201 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:53:58.215 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:53:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T22:53:58.325 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T22:53:58.611 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:53:58.640 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T22:53:58.676 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T22:53:58.710 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T22:53:58.717 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T22:53:58.760 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T22:53:58.766 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:53:58.977 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T22:53:58.980 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T22:53:59.046 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T22:53:59.062 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T22:53:59.096 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T22:53:59.106 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:53:59.112 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T22:53:59.278 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:53:59.417 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T22:53:59.421 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:53:59.450 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T22:53:59.465 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T22:53:59.496 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:53:59.508 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T22:53:59.714 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:53:59.745 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:53:59.771 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T22:53:59.842 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T22:53:59.844 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T22:53:59.858 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T22:53:59.902 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:54:00.102 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T22:54:00.106 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:54:00.135 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:54:00.230 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-py. 2022-08-01T22:54:00.245 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T22:54:00.261 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:54:00.297 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:54:00.391 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:54:00.407 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:54:00.596 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:54:00.609 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T22:54:00.624 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T22:54:00.659 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:54:00.683 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:54:00.784 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:54:00.914 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T22:54:00.928 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T22:54:00.942 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T22:54:00.978 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T22:54:00.985 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:01.153 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T22:54:01.215 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:54:01.261 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:54:01.262 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:01.333 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:54:01.361 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T22:54:01.373 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T22:54:01.386 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T22:54:01.426 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:54:01.539 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T22:54:01.609 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T22:54:01.663 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:54:01.819 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package radosgw. 2022-08-01T22:54:01.835 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:54:01.864 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:54:01.876 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:01.877 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:54:01.950 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:54:01.990 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T22:54:02.186 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:54:02.257 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:54:02.283 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:54:02.409 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:54:02.518 INFO:teuthology.orchestra.run.smithi130.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T22:54:02.633 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:54:02.761 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:54:02.788 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:54:02.897 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T22:54:02.912 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T22:54:02.915 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:54:02.919 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:54:02.988 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:03.045 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T22:54:03.233 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:54:03.245 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:54:03.297 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T22:54:03.341 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:54:03.481 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:54:03.504 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T22:54:03.576 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:54:03.618 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T22:54:03.677 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:54:03.727 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T22:54:03.756 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T22:54:03.803 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T22:54:03.846 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T22:54:03.870 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T22:54:04.058 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:54:04.072 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:04.137 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:54:04.140 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T22:54:04.367 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:54:04.433 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:54:04.455 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:04.559 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:54:04.581 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T22:54:04.617 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user cephadm....done 2022-08-01T22:54:04.662 INFO:teuthology.orchestra.run.smithi055.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T22:54:04.794 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T22:54:04.869 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:54:04.936 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:54:05.037 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:54:05.122 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T22:54:05.237 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:54:05.355 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:54:05.439 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T22:54:05.525 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:54:05.557 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T22:54:05.707 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:54:05.866 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:54:05.919 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T22:54:05.992 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T22:54:05.998 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:54:06.062 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:06.188 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:06.311 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:54:06.330 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T22:54:06.339 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T22:54:06.367 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T22:54:06.501 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:06.613 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T22:54:06.647 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:06.716 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T22:54:06.762 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T22:54:06.786 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T22:54:06.881 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T22:54:07.050 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:54:07.104 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:54:07.207 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T22:54:07.215 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T22:54:07.326 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T22:54:07.492 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T22:54:07.507 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:54:07.632 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:54:07.647 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:07.785 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:07.795 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:54:07.951 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:54:08.119 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T22:54:08.122 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:08.244 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T22:54:08.244 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T22:54:08.252 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:54:08.276 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:08.402 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:08.528 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T22:54:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:54:08.554 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T22:54:08.583 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:54:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:54:08.830 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T22:54:08.896 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T22:54:08.914 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T22:54:09.101 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T22:54:09.181 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:54:09.206 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:54:09.474 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T22:54:09.482 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T22:54:09.500 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:54:09.630 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:09.758 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:54:09.776 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T22:54:09.783 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T22:54:09.808 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:09.898 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:54:09.934 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:54:10.024 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:54:10.027 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:54:10.126 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T22:54:10.187 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:54:10.252 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:10.342 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:10.380 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:54:10.386 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T22:54:10.452 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T22:54:10.509 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:54:10.689 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:54:10.712 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T22:54:10.753 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T22:54:10.856 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:54:10.982 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:54:11.018 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T22:54:11.045 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:11.141 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T22:54:11.171 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:11.321 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T22:54:11.424 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:11.424 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:11.467 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:54:11.472 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T22:54:11.737 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:11.752 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:11.766 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T22:54:11.878 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T22:54:11.886 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:54:11.898 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:12.004 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:54:12.074 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:54:12.115 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:54:12.184 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T22:54:12.416 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:12.468 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T22:54:12.567 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T22:54:12.676 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T22:54:12.836 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T22:54:12.860 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T22:54:12.979 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:54:13.162 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.204 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T22:54:13.279 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.321 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.364 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.376 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.458 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.488 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:13.489 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:13.527 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:13.573 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.628 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T22:54:13.665 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T22:54:13.699 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T22:54:13.925 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:13.980 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T22:54:14.024 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:54:14.050 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T22:54:14.085 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T22:54:14.201 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T22:54:14.264 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:54:14.333 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T22:54:14.378 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:14.480 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T22:54:14.480 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:14.558 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T22:54:14.619 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T22:54:14.635 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:54:14.773 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:14.883 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:54:14.909 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:14.963 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T22:54:14.987 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:15.060 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:54:15.113 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:15.230 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:15.272 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T22:54:15.339 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T22:54:15.395 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:15.512 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:54:15.566 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T22:54:15.695 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T22:54:15.726 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T22:54:15.801 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:15.839 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:54:15.976 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T22:54:16.029 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:16.195 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:16.244 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T22:54:16.490 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:16.519 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T22:54:16.611 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:54:16.621 INFO:teuthology.orchestra.run.smithi130.stdout:Adding group ceph....done 2022-08-01T22:54:16.823 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T22:54:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:16.989 INFO:teuthology.orchestra.run.smithi130.stdout:Adding system user ceph....done 2022-08-01T22:54:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:54:17.114 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T22:54:17.130 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:54:17.173 INFO:teuthology.orchestra.run.smithi130.stdout:Setting system user ceph properties....done 2022-08-01T22:54:17.186 INFO:teuthology.orchestra.run.smithi130.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T22:54:17.231 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:54:17.240 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T22:54:17.329 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T22:54:17.546 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:17.553 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T22:54:17.642 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T22:54:17.676 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T22:54:17.939 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T22:54:17.980 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T22:54:18.046 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T22:54:18.095 INFO:teuthology.orchestra.run.smithi055.stdout:Adding group ceph....done 2022-08-01T22:54:18.096 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:18.144 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T22:54:18.212 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:18.278 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:18.412 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:18.438 INFO:teuthology.orchestra.run.smithi055.stdout:Adding system user ceph....done 2022-08-01T22:54:18.490 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:18.490 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:18.504 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T22:54:18.640 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:18.641 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:18.647 INFO:teuthology.orchestra.run.smithi055.stdout:Setting system user ceph properties....done 2022-08-01T22:54:18.666 INFO:teuthology.orchestra.run.smithi055.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T22:54:18.798 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:18.817 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T22:54:18.923 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T22:54:19.114 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:19.131 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:19.160 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T22:54:19.215 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:54:19.508 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T22:54:19.510 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T22:54:19.610 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:19.736 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:19.851 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:54:19.971 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.050 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:20.051 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:20.195 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:20.195 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:20.223 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.323 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.379 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.424 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.530 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T22:54:20.634 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.655 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T22:54:20.697 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:20.861 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:20.861 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:20.963 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T22:54:21.017 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T22:54:21.081 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T22:54:21.278 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:21.282 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T22:54:21.337 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:21.396 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:21.478 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:21.583 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:21.595 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T22:54:21.724 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:21.824 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:54:21.845 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:21.846 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:21.921 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.046 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T22:54:22.090 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:22.090 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:22.326 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.481 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T22:54:22.545 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:22.545 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:22.557 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.686 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.744 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.782 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T22:54:22.835 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:22.979 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:23.043 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:23.044 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:23.121 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:23.296 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2022-08-01T22:54:23.496 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:23.636 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:23.638 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:23.638 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:23.639 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2022-08-01T22:54:23.753 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:23.787 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:23.839 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2022-08-01T22:54:23.852 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T22:54:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T22:54:24.001 INFO:teuthology.orchestra.run.smithi130.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T22:54:24.010 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T22:54:24.055 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T22:54:24.105 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:24.159 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T22:54:24.231 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T22:54:24.374 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:24.380 INFO:teuthology.orchestra.run.smithi130.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:24.466 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T22:54:24.567 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T22:54:24.606 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:24.606 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:24.807 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:24.863 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T22:54:24.939 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:25.059 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:25.191 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:25.194 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:25.212 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:25.213 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:25.312 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:25.352 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:25.496 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T22:54:25.563 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:25.564 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T22:54:25.780 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:25.882 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:26.005 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:26.127 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T22:54:26.268 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T22:54:26.382 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T22:54:26.728 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:26.934 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T22:54:26.963 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:26.963 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:27.058 INFO:teuthology.orchestra.run.smithi130.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:27.277 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:27.453 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:27.487 INFO:teuthology.orchestra.run.smithi130.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:27.490 DEBUG:teuthology.orchestra.run.smithi130:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T22:54:27.529 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:27.566 INFO:teuthology.orchestra.run.smithi130.stdout:Reading package lists... 2022-08-01T22:54:27.680 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T22:54:27.688 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:27.689 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:27.744 INFO:teuthology.orchestra.run.smithi130.stdout:Building dependency tree... 2022-08-01T22:54:27.745 INFO:teuthology.orchestra.run.smithi130.stdout:Reading state information... 2022-08-01T22:54:27.902 INFO:teuthology.orchestra.run.smithi130.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T22:54:27.903 INFO:teuthology.orchestra.run.smithi130.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T22:54:27.903 INFO:teuthology.orchestra.run.smithi130.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T22:54:27.904 INFO:teuthology.orchestra.run.smithi130.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:54:27.904 INFO:teuthology.orchestra.run.smithi130.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:54:27.904 INFO:teuthology.orchestra.run.smithi130.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:54:27.905 INFO:teuthology.orchestra.run.smithi130.stdout:Suggested packages: 2022-08-01T22:54:27.905 INFO:teuthology.orchestra.run.smithi130.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T22:54:27.934 INFO:teuthology.orchestra.run.smithi130.stdout:The following NEW packages will be installed: 2022-08-01T22:54:27.934 INFO:teuthology.orchestra.run.smithi130.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T22:54:27.935 INFO:teuthology.orchestra.run.smithi130.stdout: libssl-dev 2022-08-01T22:54:27.990 INFO:teuthology.orchestra.run.smithi130.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:54:27.991 INFO:teuthology.orchestra.run.smithi130.stdout:Need to get 2,914 kB of archives. 2022-08-01T22:54:27.991 INFO:teuthology.orchestra.run.smithi130.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T22:54:27.991 INFO:teuthology.orchestra.run.smithi130.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T22:54:28.082 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T22:54:28.137 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:28.266 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:28.310 INFO:teuthology.orchestra.run.smithi130.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T22:54:28.367 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:28.526 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:28.694 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:28.747 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:28.748 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:28.791 INFO:teuthology.orchestra.run.smithi130.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T22:54:28.821 INFO:teuthology.orchestra.run.smithi130.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T22:54:28.830 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:28.882 INFO:teuthology.orchestra.run.smithi130.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T22:54:28.941 INFO:teuthology.orchestra.run.smithi130.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T22:54:28.944 INFO:teuthology.orchestra.run.smithi130.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T22:54:28.970 INFO:teuthology.orchestra.run.smithi130.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T22:54:28.972 INFO:teuthology.orchestra.run.smithi130.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T22:54:28.974 INFO:teuthology.orchestra.run.smithi130.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T22:54:28.978 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T22:54:29.148 INFO:teuthology.orchestra.run.smithi055.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:29.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T22:54:29.190 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:29.225 INFO:teuthology.orchestra.run.smithi055.stdout:Reading package lists... 2022-08-01T22:54:29.318 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:29.408 INFO:teuthology.orchestra.run.smithi055.stdout:Building dependency tree... 2022-08-01T22:54:29.418 INFO:teuthology.orchestra.run.smithi055.stdout:Reading state information... 2022-08-01T22:54:29.498 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:29.572 INFO:teuthology.orchestra.run.smithi055.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T22:54:29.572 INFO:teuthology.orchestra.run.smithi055.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T22:54:29.573 INFO:teuthology.orchestra.run.smithi055.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T22:54:29.573 INFO:teuthology.orchestra.run.smithi055.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:54:29.575 INFO:teuthology.orchestra.run.smithi055.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:54:29.576 INFO:teuthology.orchestra.run.smithi055.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:54:29.577 INFO:teuthology.orchestra.run.smithi055.stdout:Suggested packages: 2022-08-01T22:54:29.577 INFO:teuthology.orchestra.run.smithi055.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T22:54:29.608 INFO:teuthology.orchestra.run.smithi055.stdout:The following NEW packages will be installed: 2022-08-01T22:54:29.609 INFO:teuthology.orchestra.run.smithi055.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T22:54:29.610 INFO:teuthology.orchestra.run.smithi055.stdout: libssl-dev 2022-08-01T22:54:29.720 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:29.720 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:29.791 INFO:teuthology.orchestra.run.smithi055.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:54:29.792 INFO:teuthology.orchestra.run.smithi055.stdout:Need to get 2,914 kB of archives. 2022-08-01T22:54:29.792 INFO:teuthology.orchestra.run.smithi055.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T22:54:29.793 INFO:teuthology.orchestra.run.smithi055.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T22:54:29.923 INFO:teuthology.orchestra.run.smithi130.stdout:Fetched 2,914 kB in 2s (1,793 kB/s) 2022-08-01T22:54:29.924 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:30.064 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package flex. 2022-08-01T22:54:30.109 INFO:teuthology.orchestra.run.smithi130.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T22:54:30.112 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:30.153 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T22:54:30.160 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:30.311 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:30.419 INFO:teuthology.orchestra.run.smithi157.stdout:Adding group ceph....done 2022-08-01T22:54:30.437 INFO:teuthology.orchestra.run.smithi055.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T22:54:30.450 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:30.590 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T22:54:30.650 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package bison. 2022-08-01T22:54:30.673 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T22:54:30.712 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:30.811 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user ceph....done 2022-08-01T22:54:30.890 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:30.987 INFO:teuthology.orchestra.run.smithi157.stdout:Setting system user ceph properties....done 2022-08-01T22:54:31.002 INFO:teuthology.orchestra.run.smithi157.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T22:54:31.145 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T22:54:31.151 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T22:54:31.160 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T22:54:31.246 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:31.325 INFO:teuthology.orchestra.run.smithi055.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T22:54:31.346 INFO:teuthology.orchestra.run.smithi055.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T22:54:31.461 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T22:54:31.536 INFO:teuthology.orchestra.run.smithi055.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T22:54:31.544 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package dump. 2022-08-01T22:54:31.555 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T22:54:31.599 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T22:54:31.643 INFO:teuthology.orchestra.run.smithi055.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T22:54:31.647 INFO:teuthology.orchestra.run.smithi055.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T22:54:31.739 INFO:teuthology.orchestra.run.smithi055.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T22:54:31.743 INFO:teuthology.orchestra.run.smithi055.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T22:54:31.747 INFO:teuthology.orchestra.run.smithi055.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T22:54:31.924 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:31.986 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package indent. 2022-08-01T22:54:32.005 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T22:54:32.044 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T22:54:32.053 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:32.314 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T22:54:32.327 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T22:54:32.337 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:32.337 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T22:54:32.373 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:32.555 INFO:teuthology.orchestra.run.smithi055.stdout:Fetched 2,914 kB in 3s (1,133 kB/s) 2022-08-01T22:54:32.659 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T22:54:32.674 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T22:54:32.698 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package flex. 2022-08-01T22:54:32.708 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:32.746 INFO:teuthology.orchestra.run.smithi055.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T22:54:32.749 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:32.796 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T22:54:32.997 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:33.020 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T22:54:33.036 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:33.078 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:33.292 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package bison. 2022-08-01T22:54:33.306 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T22:54:33.355 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:33.372 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T22:54:33.375 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T22:54:33.389 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:33.430 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:33.585 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:33.675 INFO:teuthology.orchestra.run.smithi130.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T22:54:33.691 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T22:54:33.733 INFO:teuthology.orchestra.run.smithi130.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:33.754 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T22:54:33.770 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T22:54:33.830 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:33.871 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:34.018 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:34.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T22:54:34.086 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:34.086 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T22:54:34.100 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2022-08-01T22:54:34.182 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T22:54:34.216 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package dump. 2022-08-01T22:54:34.231 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T22:54:34.274 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T22:54:34.282 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:34.289 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2022-08-01T22:54:34.289 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2022-08-01T22:54:34.405 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:34.456 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T22:54:34.456 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T22:54:34.457 INFO:teuthology.orchestra.run.smithi032.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T22:54:34.457 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:54:34.457 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:54:34.458 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:54:34.458 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2022-08-01T22:54:34.459 INFO:teuthology.orchestra.run.smithi032.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T22:54:34.491 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2022-08-01T22:54:34.492 INFO:teuthology.orchestra.run.smithi032.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T22:54:34.492 INFO:teuthology.orchestra.run.smithi032.stdout: libssl-dev 2022-08-01T22:54:34.531 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:34.546 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:54:34.547 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 2,914 kB of archives. 2022-08-01T22:54:34.547 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T22:54:34.548 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T22:54:34.561 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:34.595 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package indent. 2022-08-01T22:54:34.610 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T22:54:34.655 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T22:54:34.691 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:34.724 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T22:54:34.760 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:34.760 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T22:54:34.817 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:34.881 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T22:54:34.891 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T22:54:34.896 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T22:54:34.897 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T22:54:34.930 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:34.941 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T22:54:34.942 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T22:54:34.965 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T22:54:34.966 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T22:54:35.010 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T22:54:35.015 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T22:54:35.018 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T22:54:35.038 INFO:teuthology.orchestra.run.smithi130.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T22:54:35.102 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:35.175 INFO:teuthology.orchestra.run.smithi130.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T22:54:35.207 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:35.226 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T22:54:35.240 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T22:54:35.274 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T22:54:35.283 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:35.375 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:35.380 INFO:teuthology.orchestra.run.smithi130.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:35.506 INFO:teuthology.orchestra.run.smithi130.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:35.650 INFO:teuthology.orchestra.run.smithi130.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:35.662 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T22:54:35.677 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:35.724 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:35.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:35.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T22:54:35.972 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T22:54:35.986 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:36.035 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:36.039 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 2,914 kB in 1s (2,478 kB/s) 2022-08-01T22:54:36.158 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package flex. 2022-08-01T22:54:36.202 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T22:54:36.206 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:36.239 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T22:54:36.267 INFO:teuthology.orchestra.run.smithi055.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T22:54:36.285 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T22:54:36.326 INFO:teuthology.orchestra.run.smithi055.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:36.335 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:36.445 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:36.604 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:36.784 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T22:54:36.832 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:36.833 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T22:54:36.946 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:37.069 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package bison. 2022-08-01T22:54:37.072 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:37.088 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T22:54:37.131 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:37.287 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:37.289 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:37.383 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:37.431 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:37.495 INFO:teuthology.orchestra.run.smithi130.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T22:54:37.509 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:37.565 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T22:54:37.582 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T22:54:37.583 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T22:54:37.635 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T22:54:37.665 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:37.727 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T22:54:37.730 INFO:teuthology.orchestra.run.smithi055.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T22:54:37.795 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:37.887 INFO:teuthology.orchestra.run.smithi055.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T22:54:37.947 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T22:54:38.003 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package dump. 2022-08-01T22:54:38.015 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T22:54:38.027 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:38.059 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T22:54:38.098 INFO:teuthology.orchestra.run.smithi055.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:38.202 INFO:teuthology.orchestra.run.smithi130.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:38.205 DEBUG:teuthology.parallel:result is None 2022-08-01T22:54:38.225 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:38.352 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package indent. 2022-08-01T22:54:38.368 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T22:54:38.369 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:38.406 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T22:54:38.640 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T22:54:38.647 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T22:54:38.682 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:38.919 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T22:54:38.933 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T22:54:38.977 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:39.347 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T22:54:39.361 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:39.405 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:39.640 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T22:54:39.658 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:39.711 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:39.918 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T22:54:39.933 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T22:54:39.989 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:40.255 INFO:teuthology.orchestra.run.smithi055.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T22:54:40.434 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T22:54:40.531 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:40.649 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:40.704 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:40.791 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:40.917 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:41.013 DEBUG:teuthology.parallel:result is None 2022-08-01T22:54:41.043 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:41.126 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:41.130 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T22:54:41.174 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T22:54:41.199 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2022-08-01T22:54:41.265 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T22:54:41.329 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:41.387 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2022-08-01T22:54:41.387 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2022-08-01T22:54:41.412 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T22:54:41.465 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T22:54:41.529 INFO:teuthology.orchestra.run.smithi157.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T22:54:41.530 INFO:teuthology.orchestra.run.smithi157.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T22:54:41.530 INFO:teuthology.orchestra.run.smithi157.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T22:54:41.531 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T22:54:41.531 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T22:54:41.532 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T22:54:41.532 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2022-08-01T22:54:41.532 INFO:teuthology.orchestra.run.smithi157.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T22:54:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2022-08-01T22:54:41.565 INFO:teuthology.orchestra.run.smithi157.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T22:54:41.565 INFO:teuthology.orchestra.run.smithi157.stdout: libssl-dev 2022-08-01T22:54:41.600 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:41.642 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T22:54:41.643 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 2,914 kB of archives. 2022-08-01T22:54:41.643 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T22:54:41.643 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T22:54:41.726 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:41.772 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T22:54:41.877 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:41.899 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T22:54:41.903 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T22:54:41.924 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T22:54:41.943 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T22:54:41.944 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T22:54:41.949 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T22:54:41.963 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T22:54:41.963 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T22:54:42.573 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 2,914 kB in 1s (4,428 kB/s) 2022-08-01T22:54:42.668 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package flex. 2022-08-01T22:54:42.713 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T22:54:42.716 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:42.779 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T22:54:43.262 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package bison. 2022-08-01T22:54:43.278 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T22:54:43.341 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:43.791 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T22:54:43.805 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T22:54:43.815 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T22:54:43.883 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:44.228 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package dump. 2022-08-01T22:54:44.245 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T22:54:44.286 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T22:54:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:44.549 DEBUG:teuthology.parallel:result is None 2022-08-01T22:54:44.573 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package indent. 2022-08-01T22:54:44.587 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T22:54:44.647 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T22:54:44.901 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T22:54:44.916 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T22:54:44.958 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:45.214 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T22:54:45.230 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T22:54:45.264 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:45.623 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T22:54:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:45.681 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:45.942 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T22:54:45.956 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T22:54:46.025 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:46.278 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T22:54:46.292 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T22:54:46.336 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:46.802 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T22:54:46.949 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T22:54:47.077 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T22:54:47.294 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T22:54:47.469 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T22:54:47.587 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:47.697 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T22:54:47.775 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T22:54:47.865 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T22:54:47.957 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T22:54:48.029 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T22:54:48.168 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T22:54:48.315 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T22:54:48.445 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T22:54:50.273 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T22:54:50.979 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T22:54:50.980 DEBUG:teuthology.parallel:result is None 2022-08-01T22:54:50.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo lsb_release -is 2022-08-01T22:54:51.035 INFO:teuthology.orchestra.run.smithi032.stdout:Ubuntu 2022-08-01T22:54:51.041 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:54:51.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:54:51.220 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2022-08-01T22:54:51.244 INFO:teuthology.orchestra.run.smithi032.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:51.245 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:51.246 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T22:54:51.247 DEBUG:teuthology.orchestra.run.smithi055:> sudo lsb_release -is 2022-08-01T22:54:51.301 INFO:teuthology.orchestra.run.smithi055.stdout:Ubuntu 2022-08-01T22:54:51.312 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:54:51.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:54:51.479 DEBUG:teuthology.orchestra.run.smithi055:> dpkg-query -W -f '${Version}' ceph 2022-08-01T22:54:51.502 INFO:teuthology.orchestra.run.smithi055.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:51.503 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:51.504 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T22:54:51.506 DEBUG:teuthology.orchestra.run.smithi130:> sudo lsb_release -is 2022-08-01T22:54:51.562 INFO:teuthology.orchestra.run.smithi130.stdout:Ubuntu 2022-08-01T22:54:51.568 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:54:51.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:54:51.733 DEBUG:teuthology.orchestra.run.smithi130:> dpkg-query -W -f '${Version}' ceph 2022-08-01T22:54:51.755 INFO:teuthology.orchestra.run.smithi130.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:51.755 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:51.756 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T22:54:51.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo lsb_release -is 2022-08-01T22:54:51.808 INFO:teuthology.orchestra.run.smithi157.stdout:Ubuntu 2022-08-01T22:54:51.817 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T22:54:51.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T22:54:51.976 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-query -W -f '${Version}' ceph 2022-08-01T22:54:52.003 INFO:teuthology.orchestra.run.smithi157.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:52.003 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T22:54:52.004 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T22:54:52.005 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-08-01T22:54:52.012 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:52.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T22:54:52.035 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:52.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T22:54:52.066 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:54:52.066 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T22:54:52.086 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:54:52.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T22:54:52.106 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-08-01T22:54:52.107 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:52.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T22:54:52.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T22:54:52.144 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:52.145 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T22:54:52.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T22:54:52.182 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:54:52.182 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T22:54:52.198 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T22:54:52.209 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:54:52.209 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T22:54:52.226 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T22:54:52.282 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-08-01T22:54:52.282 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:52.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T22:54:52.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T22:54:52.317 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:52.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T22:54:52.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T22:54:52.398 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:54:52.399 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T22:54:52.419 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T22:54:52.439 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:54:52.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T22:54:52.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T22:54:52.516 INFO:teuthology.run_tasks:Running task ceph... 2022-08-01T22:54:52.622 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-08-01T22:54:52.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2022-08-01T22:54:52.635 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /var/log/ceph 2022-08-01T22:54:52.638 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /var/log/ceph 2022-08-01T22:54:52.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2022-08-01T22:54:52.660 INFO:tasks.ceph:Disabling ceph logrotate... 2022-08-01T22:54:52.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T22:54:52.663 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T22:54:52.694 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T22:54:52.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T22:54:52.719 INFO:tasks.ceph:Creating extra log directories... 2022-08-01T22:54:52.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T22:54:52.729 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T22:54:52.735 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T22:54:52.763 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T22:54:52.790 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-08-01T22:54:52.791 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-08-01T22:54:52.791 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-08-01T22:54:52.792 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:52.793 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T22:54:52.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T22:54:52.889 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:52.889 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T22:54:52.898 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T22:54:52.936 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:54:52.937 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T22:54:52.952 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T22:54:52.997 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:54:52.998 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T22:54:53.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T22:54:53.064 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-08-01T22:54:53.070 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-08-01T22:54:53.071 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891', 'branch': 'wip-yuri6-testing-2022-08-01-0839-octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': None, 'first_in_suite': False, 'job_id': '6954891', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2022-08-01-0839-octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2022-08-01-0839-octopus', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 51, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1']], 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-yuri6-testing-2022-08-01-0839-octopus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92', 'targets': {'smithi032.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzz6chc/wq1TozbceqNOlZe9hGR4T5xcedJzyr+rA0Vg8YCeDHhjfBX9A90CediU+NH2sM8a7VqJGeZOTxfxRKByRExaPJcyc+rKZudCsNWSBp1UvtDOVxKgRZil4hT3/2BOlCwkwguwVAxaQibge6tYyHfDHEsE5mp/E4TTK2CcMxrToEcgJec775episHdFBB5YGHHRWXwQDT6Lcb7rfWY0V3/LjHBA++520J6D6QrHHlPXiTydNbweWoVIkRpxleVAwb6xcwotHc/xMHJMTG67NpWFLzQlySXMObHZB7Fa1f/h36nRinGY6Wvg+yjSi4PwYF6+CH9vvW3lP0nS6scAqgOFlXjIyjV4Q8ndQu2pn/udLfP2qwyylEL0mX+QwJTIWnNFFeVwi9wI7Ugh3w1Hgjt2D/ziWWT04mOuqqan4nnUfV5a7aBEYfbsXjngEUJoTPe+Sk79Hh3pWHKrSaJ75/KbHR5uqCZjW2iwFNbfRKDd6go8YBoxCX6wF/oc=', 'smithi055.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA27u4nPPWAmT2f19t4k7S/N13/pEV4EXMw2HZ2iXmfqREBVYszdJuZiop4ncUpfEGHVwQTjPP0dRbmK06h6N1hToSYQCvCt1fm4rmH94FZO+gJp9Ay7pFTEnOJKf9DS7bqEbgAz9TIT8YEJPymj5ET6foP2Lf6CgwlN6ggbGQWxrVXuzqW5Prj9r5LnOZinKj7QEMvVmBOgx7Wn404GoYvZwBtq843kUKDnIVg44xp98UGD0ObPgvza7+wAIn16QxeZFOEP+2KI/F1RANOdHBPCr93UoS4l8cXIoNHHQTIRA56f9hFIv+bTH88oy2FNlr3HhdZ7m0UNGzgA5k2wDFwNbPaZOsh7vUyHUXEhEK6tQUbK8HP3+kIfY0xQkQJ1v40QuCJJrG0o7NRYdPPhVbdu+VEdkPbjt7R6qByHnspJ1Mz3DFHe7+J+x9m/8d3KL3VUA+4JLgqW7OpgJlqEBOYuDv23OTl/hN+JEjv6X1+7csBtzZsMyTAgBf0HMZNek=', 'smithi130.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTxZNF1k1C8xA3EIsDv9994ft3Ax8y6u0M1raCXCYBKbRdpZmYGA5FY5i56RYxO2izD6K3pBgrjG03nw68aY0s6P6h4cuoANe+up34aNUvXKc+s66UWs4XgiAuIylcIY9GZyEDdLplTIXbgjJnzd/PTY9OTBCQffM4PMtfyMJv9Dk40WrCMo6XLrvYU+zlHEFA/YAgwnSIDTZ7+XFRa9nnD5386ryshXY0F3chXUTNLrqELizltzli9Ua9IT975muNDVmBFGUU29zTKKwkQ9oV1EjA1XQy90eK9DAKCWGrHpPaPwPszJ2+p2JC1Z7P/ah2ofeaaaAkWjHesl1pyRsmFaGpYgocuzBNebxP+xZ6AasfRqkm6RtI9zcCmA7+bSqrN3jU3p86ebzvnaSL8kO6D8/ESU4zYkjJQyQJrPd+PWWW6qvjmewSZHwjmIRm7Qi0u2VGhOLM2QY/+D2djcVER83KqNRwyY3s8Ld0ffHPUMyVbc/pjHVu+opbGmR3QV0=', 'smithi157.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjmyxvBG42FX0ccPqpKTftkvOdu+icNorpE0+Gi1rzkncMT06zLDWiS8g5c/HSdLNDpvBmJ07NIAerPq0IuPnMrCblr+0coD0BV+dqQM47uPv2aRyfNaCZ/KPuhxcjeBKx2WfYGwXFgWYiLQPUoXWuqAWJXUkQM2zd/eexpyIx/z2SNlL1FSS6QdCaCZE/RbGyCU72nuTBxeRAoGOMUN7Wtw/N5q8bnotHuoA2pKfmnVijI5qQCC5xNkymHgLRCv5EV3pkijxqhgBBDrUUF1aASSKXzYoBHl7eq2jqCubQ+izhBxCStbF3LG6aLOa4GZ+IhzqLbXBkyRGAqV8lGfaRIvSwfNwrIRDFXNrH4aU5x6nYRWn19EYTJuANXZpmhcNp9rFAZ3Ly9Fsy4tHOkPP68jJCMaWRYqP9H3gUCs2nHiLFT/z/zqowfwH6YkAVq4pKYVYQKRNGmMeBVEOKf8dHuAKpBFTBE8MsleKBmJGjGHrgHCUaffwILdtbc3zSRnk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'dd7eb9d989259767783fb5ee9a761cd0732b84e6', 'timestamp': '2022-08-01_22:38:49', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-08-01T22:54:53.073 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T22:54:53.084 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T22:54:53.093 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T22:54:53.098 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T22:54:53.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T22:54:53.142 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T22:54:53.146 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T22:54:53.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T22:54:53.169 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:53.170 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2022-08-01T22:54:53.182 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-08-01T22:54:53.183 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-08-01T22:54:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-08-01T22:54:53.237 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.237 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 360 Links: 1 2022-08-01T22:54:53.238 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.238 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2022-08-01 22:51:35.559950236 +0000 2022-08-01T22:54:53.238 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2022-08-01 22:51:35.199950240 +0000 2022-08-01T22:54:53.239 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2022-08-01 22:51:35.199950240 +0000 2022-08-01T22:54:53.239 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2022-08-01T22:54:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2022-08-01T22:54:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2022-08-01T22:54:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000307674 s, 1.7 MB/s 2022-08-01T22:54:53.253 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-08-01T22:54:53.262 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-08-01T22:54:53.263 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.263 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 367 Links: 1 2022-08-01T22:54:53.264 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.264 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2022-08-01 22:51:36.123950230 +0000 2022-08-01T22:54:53.265 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2022-08-01 22:51:35.791950233 +0000 2022-08-01T22:54:53.265 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2022-08-01 22:51:35.791950233 +0000 2022-08-01T22:54:53.266 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2022-08-01T22:54:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2022-08-01T22:54:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2022-08-01T22:54:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000293247 s, 1.7 MB/s 2022-08-01T22:54:53.275 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-08-01T22:54:53.330 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-08-01T22:54:53.331 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.331 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 371 Links: 1 2022-08-01T22:54:53.332 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.332 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2022-08-01 22:51:36.719950223 +0000 2022-08-01T22:54:53.333 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2022-08-01 22:51:36.363950227 +0000 2022-08-01T22:54:53.333 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2022-08-01 22:51:36.363950227 +0000 2022-08-01T22:54:53.334 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2022-08-01T22:54:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2022-08-01T22:54:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2022-08-01T22:54:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000289233 s, 1.8 MB/s 2022-08-01T22:54:53.341 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-08-01T22:54:53.350 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-08-01T22:54:53.351 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.351 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 378 Links: 1 2022-08-01T22:54:53.351 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.352 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2022-08-01 22:51:37.255950217 +0000 2022-08-01T22:54:53.352 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2022-08-01 22:51:36.923950221 +0000 2022-08-01T22:54:53.352 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2022-08-01 22:51:36.923950221 +0000 2022-08-01T22:54:53.352 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2022-08-01T22:54:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2022-08-01T22:54:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2022-08-01T22:54:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000295034 s, 1.7 MB/s 2022-08-01T22:54:53.362 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T22:54:53.363 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:53.363 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2022-08-01T22:54:53.376 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-08-01T22:54:53.377 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-08-01T22:54:53.389 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-08-01T22:54:53.390 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.390 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 358 Links: 1 2022-08-01T22:54:53.390 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.390 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2022-08-01 22:51:45.994435502 +0000 2022-08-01T22:54:53.390 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2022-08-01 22:51:45.990435502 +0000 2022-08-01T22:54:53.391 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2022-08-01 22:51:45.990435502 +0000 2022-08-01T22:54:53.391 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2022-08-01T22:54:53.396 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2022-08-01T22:54:53.396 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2022-08-01T22:54:53.397 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000476857 s, 1.1 MB/s 2022-08-01T22:54:53.402 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-08-01T22:54:53.451 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-08-01T22:54:53.452 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.452 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 364 Links: 1 2022-08-01T22:54:53.452 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.452 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2022-08-01 22:51:46.586435489 +0000 2022-08-01T22:54:53.452 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2022-08-01 22:51:46.582435489 +0000 2022-08-01T22:54:53.453 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2022-08-01 22:51:46.582435489 +0000 2022-08-01T22:54:53.453 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2022-08-01T22:54:53.462 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2022-08-01T22:54:53.463 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2022-08-01T22:54:53.463 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000577487 s, 887 kB/s 2022-08-01T22:54:53.469 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-08-01T22:54:53.478 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-08-01T22:54:53.479 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.479 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 374 Links: 1 2022-08-01T22:54:53.479 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.480 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2022-08-01 22:51:47.102435478 +0000 2022-08-01T22:54:53.480 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2022-08-01 22:51:47.098435478 +0000 2022-08-01T22:54:53.481 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2022-08-01 22:51:47.098435478 +0000 2022-08-01T22:54:53.481 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2022-08-01T22:54:53.489 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2022-08-01T22:54:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2022-08-01T22:54:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000468671 s, 1.1 MB/s 2022-08-01T22:54:53.493 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-08-01T22:54:53.545 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-08-01T22:54:53.546 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T22:54:53.546 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 377 Links: 1 2022-08-01T22:54:53.547 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T22:54:53.547 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2022-08-01 22:51:47.922435460 +0000 2022-08-01T22:54:53.548 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2022-08-01 22:51:47.618435466 +0000 2022-08-01T22:54:53.549 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2022-08-01 22:51:47.618435466 +0000 2022-08-01T22:54:53.549 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2022-08-01T22:54:53.556 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2022-08-01T22:54:53.556 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2022-08-01T22:54:53.556 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000481964 s, 1.1 MB/s 2022-08-01T22:54:53.560 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T22:54:53.561 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-08-01T22:54:53.561 INFO:tasks.ceph:Generating config... 2022-08-01T22:54:53.563 INFO:tasks.ceph:[client] client mount timeout = 600 2022-08-01T22:54:53.563 INFO:tasks.ceph:[client] debug client = 20 2022-08-01T22:54:53.564 INFO:tasks.ceph:[client] debug ms = 1 2022-08-01T22:54:53.564 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2022-08-01T22:54:53.564 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2022-08-01T22:54:53.565 INFO:tasks.ceph:[mds] debug mds = 20 2022-08-01T22:54:53.566 INFO:tasks.ceph:[mds] debug ms = 1 2022-08-01T22:54:53.566 INFO:tasks.ceph:[mds] mds bal frag = True 2022-08-01T22:54:53.566 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2022-08-01T22:54:53.567 INFO:tasks.ceph:[mds] mds bal merge size = 5 2022-08-01T22:54:53.567 INFO:tasks.ceph:[mds] mds bal split bits = 3 2022-08-01T22:54:53.567 INFO:tasks.ceph:[mds] mds bal split size = 100 2022-08-01T22:54:53.568 INFO:tasks.ceph:[mds] mds debug frag = True 2022-08-01T22:54:53.568 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-08-01T22:54:53.568 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-08-01T22:54:53.569 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-08-01T22:54:53.569 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-08-01T22:54:53.569 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2022-08-01T22:54:53.570 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2022-08-01T22:54:53.570 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-08-01T22:54:53.570 INFO:tasks.ceph:[mgr] debug ms = 1 2022-08-01T22:54:53.571 INFO:tasks.ceph:[mon] debug mon = 20 2022-08-01T22:54:53.571 INFO:tasks.ceph:[mon] debug ms = 1 2022-08-01T22:54:53.571 INFO:tasks.ceph:[mon] debug paxos = 20 2022-08-01T22:54:53.572 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-08-01T22:54:53.572 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-08-01T22:54:53.572 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2022-08-01T22:54:53.573 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-08-01T22:54:53.573 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-08-01T22:54:53.573 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-08-01T22:54:53.574 INFO:tasks.ceph:[osd] debug ms = 1 2022-08-01T22:54:53.574 INFO:tasks.ceph:[osd] debug osd = 20 2022-08-01T22:54:53.574 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-08-01T22:54:53.574 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-08-01T22:54:53.575 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-08-01T22:54:53.575 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-08-01T22:54:53.575 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-08-01T22:54:53.576 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-08-01T22:54:53.576 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-08-01T22:54:53.577 INFO:tasks.ceph:Setting up mon.a... 2022-08-01T22:54:53.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-08-01T22:54:53.616 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2022-08-01T22:54:53.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-08-01T22:54:53.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T22:54:53.710 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.55'), ('mon.c', '[v2:172.21.15.55:3301,v1:172.21.15.55:6790]')] 2022-08-01T22:54:53.711 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.55 --addv c '[v2:172.21.15.55:3301,v1:172.21.15.55:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:53.788 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:53.789 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T22:54:53.790 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2022-08-01T22:54:53.790 INFO:teuthology.orchestra.run.smithi032.stdout:fsid a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T22:54:53.791 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2022-08-01T22:54:53.791326+0000 2022-08-01T22:54:53.791 INFO:teuthology.orchestra.run.smithi032.stdout:created 2022-08-01T22:54:53.791326+0000 2022-08-01T22:54:53.791 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 0 (unknown) 2022-08-01T22:54:53.792 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2022-08-01T22:54:53.792 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.b 2022-08-01T22:54:53.792 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] mon.c 2022-08-01T22:54:53.793 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-08-01T22:54:53.795 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e... 2022-08-01T22:54:53.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T22:54:53.801 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T22:54:53.804 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T22:54:53.807 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T22:54:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2022-08-01T22:54:53.833 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2022-08-01T22:54:53.834 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T22:54:53.834 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2022-08-01T22:54:53.834 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.835 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2022-08-01T22:54:53.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.836 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2022-08-01T22:54:53.836 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.836 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2022-08-01T22:54:53.837 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2022-08-01T22:54:53.837 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.837 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2022-08-01T22:54:53.838 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2022-08-01T22:54:53.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.838 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T22:54:53.839 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2022-08-01T22:54:53.839 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.840 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T22:54:53.840 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T22:54:53.840 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.841 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2022-08-01T22:54:53.841 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.841 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2022-08-01T22:54:53.842 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2022-08-01T22:54:53.842 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2022-08-01T22:54:53.843 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2022-08-01T22:54:53.843 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2022-08-01T22:54:53.843 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2022-08-01T22:54:53.844 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2022-08-01T22:54:53.844 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T22:54:53.844 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T22:54:53.845 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.845 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T22:54:53.846 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.846 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2022-08-01T22:54:53.846 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.847 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2022-08-01T22:54:53.847 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.847 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2022-08-01T22:54:53.848 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2022-08-01T22:54:53.848 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2022-08-01T22:54:53.848 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T22:54:53.849 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T22:54:53.849 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.850 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2022-08-01T22:54:53.850 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2022-08-01T22:54:53.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.851 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2022-08-01T22:54:53.851 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2022-08-01T22:54:53.852 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.852 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2022-08-01T22:54:53.853 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2022-08-01T22:54:53.853 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.854 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2022-08-01T22:54:53.854 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2022-08-01T22:54:53.854 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2022-08-01T22:54:53.855 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.857 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2022-08-01T22:54:53.857 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2022-08-01T22:54:53.857 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.857 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2022-08-01T22:54:53.858 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.858 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2022-08-01T22:54:53.858 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.858 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2022-08-01T22:54:53.858 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.859 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2022-08-01T22:54:53.859 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2022-08-01T22:54:53.859 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.859 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2022-08-01T22:54:53.860 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2022-08-01T22:54:53.860 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2022-08-01T22:54:53.860 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression mode = aggressive 2022-08-01T22:54:53.860 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2022-08-01T22:54:53.860 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2022-08-01T22:54:53.861 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2022-08-01T22:54:53.861 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T22:54:53.861 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2022-08-01T22:54:53.861 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2022-08-01T22:54:53.862 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T22:54:53.862 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2022-08-01T22:54:53.862 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2022-08-01T22:54:53.862 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2022-08-01T22:54:53.862 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2022-08-01T22:54:53.863 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2022-08-01T22:54:53.863 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.863 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2022-08-01T22:54:53.863 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T22:54:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2022-08-01T22:54:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2022-08-01T22:54:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2022-08-01T22:54:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.864 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.865 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2022-08-01T22:54:53.865 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.865 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2022-08-01T22:54:53.865 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T22:54:53.865 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2022-08-01T22:54:53.866 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2022-08-01T22:54:53.866 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2022-08-01T22:54:53.866 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2022-08-01T22:54:53.866 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2022-08-01T22:54:53.867 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.867 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2022-08-01T22:54:53.867 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2022-08-01T22:54:53.867 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.867 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.868 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T22:54:53.868 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T22:54:53.868 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T22:54:53.868 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.869 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T22:54:53.869 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T22:54:53.869 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T22:54:53.869 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2022-08-01T22:54:53.869 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:54:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2022-08-01T22:54:53.870 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2022-08-01T22:54:53.870 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2022-08-01T22:54:53.870 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2022-08-01T22:54:53.871 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T22:54:53.871 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T22:54:53.871 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2022-08-01T22:54:53.871 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2022-08-01T22:54:53.871 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T22:54:53.872 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.872 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.872 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2022-08-01T22:54:53.872 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2022-08-01T22:54:53.873 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2022-08-01T22:54:53.873 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2022-08-01T22:54:53.873 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2022-08-01T22:54:53.873 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2022-08-01T22:54:53.873 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2022-08-01T22:54:53.874 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2022-08-01T22:54:53.874 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T22:54:53.874 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal frag = True 2022-08-01T22:54:53.874 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal fragment size max = 10000 2022-08-01T22:54:53.875 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal merge size = 5 2022-08-01T22:54:53.875 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split bits = 3 2022-08-01T22:54:53.875 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split size = 100 2022-08-01T22:54:53.875 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2022-08-01T22:54:53.875 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2022-08-01T22:54:53.876 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2022-08-01T22:54:53.876 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2022-08-01T22:54:53.876 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2022-08-01T22:54:53.876 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.876 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.880 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2022-08-01T22:54:53.880 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2022-08-01T22:54:53.880 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T22:54:53.880 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2022-08-01T22:54:53.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.881 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2022-08-01T22:54:53.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.881 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2022-08-01T22:54:53.882 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.882 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2022-08-01T22:54:53.882 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2022-08-01T22:54:53.882 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.882 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2022-08-01T22:54:53.883 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2022-08-01T22:54:53.883 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.883 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T22:54:53.883 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2022-08-01T22:54:53.884 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.884 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T22:54:53.884 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T22:54:53.884 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.885 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2022-08-01T22:54:53.885 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.885 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2022-08-01T22:54:53.885 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2022-08-01T22:54:53.885 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2022-08-01T22:54:53.886 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2022-08-01T22:54:53.886 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2022-08-01T22:54:53.887 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2022-08-01T22:54:53.887 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2022-08-01T22:54:53.888 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T22:54:53.888 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T22:54:53.888 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.889 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T22:54:53.889 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.889 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2022-08-01T22:54:53.889 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.890 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2022-08-01T22:54:53.890 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.890 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2022-08-01T22:54:53.890 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2022-08-01T22:54:53.890 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2022-08-01T22:54:53.891 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T22:54:53.891 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T22:54:53.891 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.891 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2022-08-01T22:54:53.891 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2022-08-01T22:54:53.892 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.892 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2022-08-01T22:54:53.892 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2022-08-01T22:54:53.892 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.893 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2022-08-01T22:54:53.893 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2022-08-01T22:54:53.893 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.893 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2022-08-01T22:54:53.893 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2022-08-01T22:54:53.894 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2022-08-01T22:54:53.894 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.894 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2022-08-01T22:54:53.894 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2022-08-01T22:54:53.895 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.895 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2022-08-01T22:54:53.895 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.895 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2022-08-01T22:54:53.895 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.896 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2022-08-01T22:54:53.896 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.896 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2022-08-01T22:54:53.896 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2022-08-01T22:54:53.897 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.897 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2022-08-01T22:54:53.897 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2022-08-01T22:54:53.897 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2022-08-01T22:54:53.897 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore compression mode = aggressive 2022-08-01T22:54:53.898 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2022-08-01T22:54:53.898 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2022-08-01T22:54:53.898 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2022-08-01T22:54:53.898 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T22:54:53.899 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2022-08-01T22:54:53.899 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2022-08-01T22:54:53.899 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T22:54:53.899 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2022-08-01T22:54:53.900 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2022-08-01T22:54:53.900 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2022-08-01T22:54:53.900 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2022-08-01T22:54:53.900 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2022-08-01T22:54:53.900 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.901 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2022-08-01T22:54:53.901 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T22:54:53.901 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2022-08-01T22:54:53.901 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2022-08-01T22:54:53.901 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2022-08-01T22:54:53.902 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.902 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.902 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2022-08-01T22:54:53.902 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.903 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2022-08-01T22:54:53.903 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T22:54:53.903 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2022-08-01T22:54:53.903 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2022-08-01T22:54:53.904 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2022-08-01T22:54:53.904 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2022-08-01T22:54:53.904 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2022-08-01T22:54:53.904 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.905 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2022-08-01T22:54:53.905 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2022-08-01T22:54:53.905 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.905 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.906 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T22:54:53.906 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T22:54:53.906 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T22:54:53.906 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.907 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T22:54:53.907 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T22:54:53.908 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T22:54:53.908 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2022-08-01T22:54:53.908 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T22:54:53.908 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2022-08-01T22:54:53.909 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2022-08-01T22:54:53.909 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2022-08-01T22:54:53.909 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2022-08-01T22:54:53.909 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T22:54:53.910 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T22:54:53.910 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2022-08-01T22:54:53.910 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2022-08-01T22:54:53.910 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T22:54:53.910 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.911 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.911 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2022-08-01T22:54:53.911 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2022-08-01T22:54:53.911 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2022-08-01T22:54:53.912 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2022-08-01T22:54:53.912 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2022-08-01T22:54:53.912 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2022-08-01T22:54:53.912 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2022-08-01T22:54:53.912 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2022-08-01T22:54:53.913 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T22:54:53.913 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal frag = True 2022-08-01T22:54:53.913 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal fragment size max = 10000 2022-08-01T22:54:53.913 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal merge size = 5 2022-08-01T22:54:53.913 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split bits = 3 2022-08-01T22:54:53.914 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split size = 100 2022-08-01T22:54:53.914 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2022-08-01T22:54:53.914 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2022-08-01T22:54:53.914 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2022-08-01T22:54:53.915 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2022-08-01T22:54:53.915 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2022-08-01T22:54:53.915 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.915 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.916 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2022-08-01T22:54:53.916 INFO:teuthology.orchestra.run.smithi130.stdout: chdir = "" 2022-08-01T22:54:53.916 INFO:teuthology.orchestra.run.smithi130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T22:54:53.916 INFO:teuthology.orchestra.run.smithi130.stdout: auth supported = cephx 2022-08-01T22:54:53.916 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.917 INFO:teuthology.orchestra.run.smithi130.stdout: filestore xattr use omap = true 2022-08-01T22:54:53.917 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.917 INFO:teuthology.orchestra.run.smithi130.stdout: mon clock drift allowed = 1.000 2022-08-01T22:54:53.917 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.918 INFO:teuthology.orchestra.run.smithi130.stdout: osd crush chooseleaf type = 0 2022-08-01T22:54:53.918 INFO:teuthology.orchestra.run.smithi130.stdout: auth debug = true 2022-08-01T22:54:53.918 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.918 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on old message = true 2022-08-01T22:54:53.918 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on bug = true 2022-08-01T22:54:53.919 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.919 INFO:teuthology.orchestra.run.smithi130.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T22:54:53.919 INFO:teuthology.orchestra.run.smithi130.stdout: mon pg warn max object skew = 0 2022-08-01T22:54:53.919 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.920 INFO:teuthology.orchestra.run.smithi130.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T22:54:53.920 INFO:teuthology.orchestra.run.smithi130.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T22:54:53.920 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.920 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default size = 2 2022-08-01T22:54:53.920 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.921 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow primary affinity = true 2022-08-01T22:54:53.921 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow pg remap = true 2022-08-01T22:54:53.921 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on legacy crush tunables = false 2022-08-01T22:54:53.921 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on crush straw calc version zero = false 2022-08-01T22:54:53.922 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on no sortbitwise = false 2022-08-01T22:54:53.922 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on osd down out interval zero = false 2022-08-01T22:54:53.922 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on too few osds = false 2022-08-01T22:54:53.922 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T22:54:53.922 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T22:54:53.923 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.923 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T22:54:53.923 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.923 INFO:teuthology.orchestra.run.smithi130.stdout: osd default data pool replay window = 5 2022-08-01T22:54:53.924 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.924 INFO:teuthology.orchestra.run.smithi130.stdout: mon allow pool delete = true 2022-08-01T22:54:53.924 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.924 INFO:teuthology.orchestra.run.smithi130.stdout: mon cluster log file level = debug 2022-08-01T22:54:53.924 INFO:teuthology.orchestra.run.smithi130.stdout: debug asserts on shutdown = true 2022-08-01T22:54:53.925 INFO:teuthology.orchestra.run.smithi130.stdout: mon health detail to clog = false 2022-08-01T22:54:53.925 INFO:teuthology.orchestra.run.smithi130.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T22:54:53.925 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T22:54:53.925 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.925 INFO:teuthology.orchestra.run.smithi130.stdout:[osd] 2022-08-01T22:54:53.926 INFO:teuthology.orchestra.run.smithi130.stdout: osd journal size = 100 2022-08-01T22:54:53.926 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.926 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub load threshold = 5.0 2022-08-01T22:54:53.926 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub max interval = 600 2022-08-01T22:54:53.927 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.927 INFO:teuthology.orchestra.run.smithi130.stdout: osd recover clone overlap = true 2022-08-01T22:54:53.927 INFO:teuthology.orchestra.run.smithi130.stdout: osd recovery max chunk = 1048576 2022-08-01T22:54:53.927 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.927 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug shutdown = true 2022-08-01T22:54:53.928 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug op order = true 2022-08-01T22:54:53.928 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug verify stray on activate = true 2022-08-01T22:54:53.928 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.928 INFO:teuthology.orchestra.run.smithi130.stdout: osd open classes on start = true 2022-08-01T22:54:53.929 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug pg log writeout = true 2022-08-01T22:54:53.929 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.929 INFO:teuthology.orchestra.run.smithi130.stdout: osd deep scrub update digest min age = 30 2022-08-01T22:54:53.930 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.930 INFO:teuthology.orchestra.run.smithi130.stdout: osd map max advance = 10 2022-08-01T22:54:53.930 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.930 INFO:teuthology.orchestra.run.smithi130.stdout: journal zero on create = true 2022-08-01T22:54:53.930 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.931 INFO:teuthology.orchestra.run.smithi130.stdout: filestore ondisk finisher threads = 3 2022-08-01T22:54:53.931 INFO:teuthology.orchestra.run.smithi130.stdout: filestore apply finisher threads = 3 2022-08-01T22:54:53.931 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.931 INFO:teuthology.orchestra.run.smithi130.stdout: bdev debug aio = true 2022-08-01T22:54:53.932 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug misdirected ops = true 2022-08-01T22:54:53.933 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore block size = 96636764160 2022-08-01T22:54:53.933 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore compression mode = aggressive 2022-08-01T22:54:53.934 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore fsck on mount = True 2022-08-01T22:54:53.934 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluefs = 20 2022-08-01T22:54:53.935 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluestore = 20 2022-08-01T22:54:53.935 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T22:54:53.936 INFO:teuthology.orchestra.run.smithi130.stdout: debug osd = 20 2022-08-01T22:54:53.936 INFO:teuthology.orchestra.run.smithi130.stdout: debug rocksdb = 10 2022-08-01T22:54:53.937 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T22:54:53.937 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd full ratio = 0.9 2022-08-01T22:54:53.937 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd nearfull ratio = 0.8 2022-08-01T22:54:53.939 INFO:teuthology.orchestra.run.smithi130.stdout: osd failsafe full ratio = 0.95 2022-08-01T22:54:53.940 INFO:teuthology.orchestra.run.smithi130.stdout: osd objectstore = bluestore 2022-08-01T22:54:53.940 INFO:teuthology.orchestra.run.smithi130.stdout: osd op complaint time = 180 2022-08-01T22:54:53.941 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.941 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr] 2022-08-01T22:54:53.941 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T22:54:53.942 INFO:teuthology.orchestra.run.smithi130.stdout: debug mgr = 20 2022-08-01T22:54:53.942 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2022-08-01T22:54:53.943 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2022-08-01T22:54:53.943 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.943 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.943 INFO:teuthology.orchestra.run.smithi130.stdout: mgr/telemetry/nag = false 2022-08-01T22:54:53.944 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.945 INFO:teuthology.orchestra.run.smithi130.stdout:[mon] 2022-08-01T22:54:53.945 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T22:54:53.946 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2022-08-01T22:54:53.946 INFO:teuthology.orchestra.run.smithi130.stdout: debug paxos = 20 2022-08-01T22:54:53.946 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2022-08-01T22:54:53.946 INFO:teuthology.orchestra.run.smithi130.stdout: mon data avail warn = 5 2022-08-01T22:54:53.947 INFO:teuthology.orchestra.run.smithi130.stdout: mon mgr mkfs grace = 240 2022-08-01T22:54:53.947 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.948 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd reporter subtree level = osd 2022-08-01T22:54:53.948 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd prime pg temp = true 2022-08-01T22:54:53.948 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.949 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.949 INFO:teuthology.orchestra.run.smithi130.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T22:54:53.949 INFO:teuthology.orchestra.run.smithi130.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T22:54:53.950 INFO:teuthology.orchestra.run.smithi130.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T22:54:53.951 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.951 INFO:teuthology.orchestra.run.smithi130.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T22:54:53.952 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T22:54:53.952 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T22:54:53.952 INFO:teuthology.orchestra.run.smithi130.stdout: mon op complaint time = 120 2022-08-01T22:54:53.952 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T22:54:53.953 INFO:teuthology.orchestra.run.smithi130.stdout:[client] 2022-08-01T22:54:53.953 INFO:teuthology.orchestra.run.smithi130.stdout: rgw cache enabled = true 2022-08-01T22:54:53.953 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable ops log = true 2022-08-01T22:54:53.953 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable usage log = true 2022-08-01T22:54:53.953 INFO:teuthology.orchestra.run.smithi130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T22:54:53.954 INFO:teuthology.orchestra.run.smithi130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T22:54:53.954 INFO:teuthology.orchestra.run.smithi130.stdout: client mount timeout = 600 2022-08-01T22:54:53.954 INFO:teuthology.orchestra.run.smithi130.stdout: debug client = 20 2022-08-01T22:54:53.954 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T22:54:53.955 INFO:teuthology.orchestra.run.smithi130.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.956 INFO:teuthology.orchestra.run.smithi130.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.956 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.a] 2022-08-01T22:54:53.956 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.b] 2022-08-01T22:54:53.956 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.c] 2022-08-01T22:54:53.956 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.a] 2022-08-01T22:54:53.957 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.c] 2022-08-01T22:54:53.957 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.b] 2022-08-01T22:54:53.957 INFO:teuthology.orchestra.run.smithi130.stdout:[mds] 2022-08-01T22:54:53.957 INFO:teuthology.orchestra.run.smithi130.stdout: debug mds = 20 2022-08-01T22:54:53.958 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T22:54:53.958 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal frag = True 2022-08-01T22:54:53.958 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal fragment size max = 10000 2022-08-01T22:54:53.958 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal merge size = 5 2022-08-01T22:54:53.959 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal split bits = 3 2022-08-01T22:54:53.959 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal split size = 100 2022-08-01T22:54:53.959 INFO:teuthology.orchestra.run.smithi130.stdout: mds debug frag = True 2022-08-01T22:54:53.959 INFO:teuthology.orchestra.run.smithi130.stdout: mds debug scatterstat = True 2022-08-01T22:54:53.959 INFO:teuthology.orchestra.run.smithi130.stdout: mds op complaint time = 180 2022-08-01T22:54:53.960 INFO:teuthology.orchestra.run.smithi130.stdout: mds verify scatter = True 2022-08-01T22:54:53.960 INFO:teuthology.orchestra.run.smithi130.stdout: osd op complaint time = 180 2022-08-01T22:54:53.960 INFO:teuthology.orchestra.run.smithi130.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.960 INFO:teuthology.orchestra.run.smithi130.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.961 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2022-08-01T22:54:53.961 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2022-08-01T22:54:53.961 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T22:54:53.961 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2022-08-01T22:54:53.962 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.962 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2022-08-01T22:54:53.962 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.962 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2022-08-01T22:54:53.962 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.963 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2022-08-01T22:54:53.963 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2022-08-01T22:54:53.963 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.963 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2022-08-01T22:54:53.964 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2022-08-01T22:54:53.964 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.964 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T22:54:53.964 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2022-08-01T22:54:53.965 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.965 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T22:54:53.965 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T22:54:53.965 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.966 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2022-08-01T22:54:53.966 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.967 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2022-08-01T22:54:53.967 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2022-08-01T22:54:53.967 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2022-08-01T22:54:53.967 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2022-08-01T22:54:53.968 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2022-08-01T22:54:53.968 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2022-08-01T22:54:53.968 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2022-08-01T22:54:53.968 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T22:54:53.968 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T22:54:53.969 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.969 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T22:54:53.969 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.969 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2022-08-01T22:54:53.970 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.970 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2022-08-01T22:54:53.970 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.971 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2022-08-01T22:54:53.971 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2022-08-01T22:54:53.971 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2022-08-01T22:54:53.971 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T22:54:53.971 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T22:54:53.972 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.972 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2022-08-01T22:54:53.972 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2022-08-01T22:54:53.972 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.973 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2022-08-01T22:54:53.973 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2022-08-01T22:54:53.973 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.973 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2022-08-01T22:54:53.973 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2022-08-01T22:54:53.974 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.974 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2022-08-01T22:54:53.974 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2022-08-01T22:54:53.974 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2022-08-01T22:54:53.974 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.975 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2022-08-01T22:54:53.975 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2022-08-01T22:54:53.975 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.975 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2022-08-01T22:54:53.975 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.976 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2022-08-01T22:54:53.976 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.976 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2022-08-01T22:54:53.976 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.976 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2022-08-01T22:54:53.977 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2022-08-01T22:54:53.977 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.977 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2022-08-01T22:54:53.977 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2022-08-01T22:54:53.978 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2022-08-01T22:54:53.978 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore compression mode = aggressive 2022-08-01T22:54:53.978 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2022-08-01T22:54:53.978 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2022-08-01T22:54:53.978 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2022-08-01T22:54:53.979 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T22:54:53.979 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2022-08-01T22:54:53.979 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2022-08-01T22:54:53.979 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T22:54:53.979 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2022-08-01T22:54:53.980 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2022-08-01T22:54:53.980 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2022-08-01T22:54:53.980 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2022-08-01T22:54:53.980 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2022-08-01T22:54:53.980 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.981 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2022-08-01T22:54:53.981 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T22:54:53.981 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2022-08-01T22:54:53.981 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2022-08-01T22:54:53.981 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2022-08-01T22:54:53.982 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.982 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.982 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2022-08-01T22:54:53.982 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.983 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2022-08-01T22:54:53.983 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T22:54:53.983 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2022-08-01T22:54:53.983 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2022-08-01T22:54:53.983 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2022-08-01T22:54:53.984 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2022-08-01T22:54:53.984 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2022-08-01T22:54:53.984 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2022-08-01T22:54:53.984 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2022-08-01T22:54:53.984 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2022-08-01T22:54:53.985 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2022-08-01T22:54:53.985 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.985 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T22:54:53.986 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T22:54:53.986 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T22:54:53.986 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.987 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T22:54:53.987 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T22:54:53.988 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T22:54:53.988 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2022-08-01T22:54:53.989 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T22:54:53.989 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2022-08-01T22:54:53.989 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2022-08-01T22:54:53.990 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2022-08-01T22:54:53.990 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2022-08-01T22:54:53.990 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T22:54:53.991 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T22:54:53.991 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2022-08-01T22:54:53.991 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2022-08-01T22:54:53.992 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T22:54:53.992 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 15m 2022-08-01T22:54:53.993 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 15m 2022-08-01T22:54:53.993 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2022-08-01T22:54:53.993 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2022-08-01T22:54:53.994 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2022-08-01T22:54:53.994 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2022-08-01T22:54:53.994 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2022-08-01T22:54:53.995 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2022-08-01T22:54:53.995 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2022-08-01T22:54:53.996 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2022-08-01T22:54:53.996 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T22:54:53.997 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal frag = True 2022-08-01T22:54:53.997 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal fragment size max = 10000 2022-08-01T22:54:53.997 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal merge size = 5 2022-08-01T22:54:53.998 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal split bits = 3 2022-08-01T22:54:53.998 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal split size = 100 2022-08-01T22:54:53.999 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2022-08-01T22:54:53.999 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2022-08-01T22:54:53.999 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2022-08-01T22:54:54.000 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2022-08-01T22:54:54.000 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2022-08-01T22:54:54.000 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 15m 2022-08-01T22:54:54.000 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 15m 2022-08-01T22:54:54.001 INFO:tasks.ceph:Creating admin key on mon.a... 2022-08-01T22:54:54.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-08-01T22:54:54.051 INFO:tasks.ceph:Copying monmap to all nodes... 2022-08-01T22:54:54.073 DEBUG:teuthology.orchestra.remote:smithi032:/etc/ceph/ceph.keyring is 206B 2022-08-01T22:54:54.100 DEBUG:teuthology.orchestra.remote:smithi032:/home/ubuntu/cephtest/ceph.monmap is 379B 2022-08-01T22:54:54.115 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T22:54:54.116 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:54.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T22:54:54.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T22:54:54.147 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2022-08-01T22:54:54.148 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:54.198 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T22:54:54.199 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:54.200 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T22:54:54.200 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T22:54:54.235 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2022-08-01T22:54:54.236 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:54.254 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi130.front.sepia.ceph.com 2022-08-01T22:54:54.255 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:54:54.256 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T22:54:54.257 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T22:54:54.287 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2022-08-01T22:54:54.288 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:54.298 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2022-08-01T22:54:54.298 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:54:54.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T22:54:54.300 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T22:54:54.360 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2022-08-01T22:54:54.360 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:54.367 INFO:tasks.ceph:Setting up mon nodes... 2022-08-01T22:54:54.368 INFO:tasks.ceph:Setting up mgr nodes... 2022-08-01T22:54:54.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-08-01T22:54:54.418 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-08-01T22:54:54.419 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-08-01T22:54:54.473 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-08-01T22:54:54.476 INFO:tasks.ceph:Setting up mds nodes... 2022-08-01T22:54:54.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-08-01T22:54:54.521 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-08-01T22:54:54.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-08-01T22:54:54.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-08-01T22:54:54.630 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-08-01T22:54:54.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-08-01T22:54:54.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-08-01T22:54:54.693 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-08-01T22:54:54.696 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-08-01T22:54:54.715 INFO:tasks.ceph_client:Setting up client nodes... 2022-08-01T22:54:54.716 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-08-01T22:54:54.763 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-08-01T22:54:54.779 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-08-01T22:54:54.827 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-08-01T22:54:54.841 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-08-01T22:54:54.841 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-08-01T22:54:54.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-08-01T22:54:54.859 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T22:54:54.860 INFO:tasks.ceph:role: osd.0 2022-08-01T22:54:54.860 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T22:54:54.861 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-08-01T22:54:54.910 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:54:54.910 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:54:54.910 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:54:54.911 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2022-08-01T22:54:54.911 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:54:54.911 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:54:54.912 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:54:54.912 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:54:54.912 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:54:54.913 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:54:54.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2022-08-01T22:54:54.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-08-01T22:54:54.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-08-01T22:54:54.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:54:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:54:54.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-08-01T22:54:54.999 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T22:54:55.000 INFO:tasks.ceph:role: osd.1 2022-08-01T22:54:55.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T22:54:55.002 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-08-01T22:54:55.087 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:54:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:54:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:54:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2022-08-01T22:54:55.088 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:54:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:54:55.089 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:54:55.089 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:54:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:54:55.089 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:54:55.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2022-08-01T22:54:55.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-08-01T22:54:55.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-08-01T22:54:55.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:54:55.132 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:54:55.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-08-01T22:54:55.193 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T22:54:55.193 INFO:tasks.ceph:role: osd.2 2022-08-01T22:54:55.195 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T22:54:55.196 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-08-01T22:54:55.240 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:54:55.240 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:54:55.240 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:54:55.241 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2022-08-01T22:54:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:54:55.241 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:54:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:54:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:54:55.242 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:54:55.243 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:54:55.244 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2022-08-01T22:54:55.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-08-01T22:54:55.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-08-01T22:54:55.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:54:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:54:55.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-08-01T22:54:55.343 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T22:54:55.343 INFO:tasks.ceph:role: osd.3 2022-08-01T22:54:55.344 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T22:54:55.344 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-08-01T22:54:55.433 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:54:55.433 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:54:55.434 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:54:55.434 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2022-08-01T22:54:55.434 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:54:55.435 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:54:55.435 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:54:55.435 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:54:55.435 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:54:55.436 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:54:55.437 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2022-08-01T22:54:55.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-08-01T22:54:55.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-08-01T22:54:55.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:54:55.519 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:54:55.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.581+0000 7ff329b6fd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-08-01T22:54:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.581+0000 7ff329b6fd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-08-01T22:54:55.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.581+0000 7ff329b6fd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-08-01T22:54:55.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.581+0000 7ff329b6fd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-08-01T22:54:55.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.581+0000 7ff329b6fd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-08-01T22:54:55.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.581+0000 7ff329b6fd80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-08-01T22:54:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:55.597+0000 7ff329b6fd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:54:58.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-08-01T22:54:58.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:54:58.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.125+0000 7f9287dced80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-08-01T22:54:58.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.125+0000 7f9287dced80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-08-01T22:54:58.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.125+0000 7f9287dced80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-08-01T22:54:58.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.125+0000 7f9287dced80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-08-01T22:54:58.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.125+0000 7f9287dced80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-08-01T22:54:58.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.125+0000 7f9287dced80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-08-01T22:54:58.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:54:58.141+0000 7f9287dced80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:00.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-08-01T22:55:00.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:00.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.685+0000 7fb7f9aadd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-08-01T22:55:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.685+0000 7fb7f9aadd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-08-01T22:55:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.685+0000 7fb7f9aadd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-08-01T22:55:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.685+0000 7fb7f9aadd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-08-01T22:55:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.685+0000 7fb7f9aadd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-08-01T22:55:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.685+0000 7fb7f9aadd80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-08-01T22:55:00.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:00.701+0000 7fb7f9aadd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:03.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-08-01T22:55:03.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.265+0000 7f79cc437d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-08-01T22:55:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.265+0000 7f79cc437d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-08-01T22:55:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.265+0000 7f79cc437d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-08-01T22:55:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.265+0000 7f79cc437d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-08-01T22:55:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.265+0000 7f79cc437d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-08-01T22:55:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.265+0000 7f79cc437d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-08-01T22:55:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:03.281+0000 7f79cc437d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:05.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-08-01T22:55:05.751 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-08-01T22:55:05.769 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T22:55:05.770 INFO:tasks.ceph:role: osd.4 2022-08-01T22:55:05.770 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T22:55:05.770 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-08-01T22:55:07.113 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:55:07.113 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:55:07.114 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:55:07.114 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2022-08-01T22:55:07.114 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:55:07.114 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:55:07.115 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:55:07.115 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:55:07.115 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:55:07.115 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:55:07.117 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2022-08-01T22:55:07.117 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-08-01T22:55:07.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-08-01T22:55:07.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:55:07.201 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:55:07.201 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-08-01T22:55:07.216 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T22:55:07.217 INFO:tasks.ceph:role: osd.5 2022-08-01T22:55:07.217 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T22:55:07.218 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-08-01T22:55:08.603 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:55:08.604 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:55:08.605 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:55:08.605 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2022-08-01T22:55:08.605 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:55:08.606 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:55:08.606 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:55:08.607 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:55:08.607 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:55:08.608 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:55:08.609 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2022-08-01T22:55:08.610 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-08-01T22:55:08.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-08-01T22:55:08.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:55:08.692 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:55:08.692 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-08-01T22:55:08.712 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T22:55:08.712 INFO:tasks.ceph:role: osd.6 2022-08-01T22:55:08.713 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T22:55:08.713 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-08-01T22:55:10.051 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:55:10.052 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:55:10.052 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:55:10.053 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2022-08-01T22:55:10.054 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:55:10.055 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:55:10.055 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:55:10.055 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:55:10.056 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:55:10.056 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:55:10.058 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2022-08-01T22:55:10.059 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-08-01T22:55:10.089 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-08-01T22:55:10.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:55:10.102 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:55:10.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-08-01T22:55:10.163 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T22:55:10.164 INFO:tasks.ceph:role: osd.7 2022-08-01T22:55:10.165 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T22:55:10.165 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-08-01T22:55:11.507 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T22:55:11.507 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T22:55:11.508 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T22:55:11.508 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2022-08-01T22:55:11.508 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T22:55:11.508 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2022-08-01T22:55:11.509 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T22:55:11.509 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T22:55:11.509 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T22:55:11.509 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T22:55:11.510 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2022-08-01T22:55:11.510 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-08-01T22:55:11.540 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-08-01T22:55:11.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:55:11.599 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T22:55:11.600 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:11.660 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.656+0000 7fa7470bcd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-08-01T22:55:11.661 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.660+0000 7fa7470bcd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-08-01T22:55:11.661 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.660+0000 7fa7470bcd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-08-01T22:55:11.662 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.660+0000 7fa7470bcd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-08-01T22:55:11.662 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.660+0000 7fa7470bcd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-08-01T22:55:11.662 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.660+0000 7fa7470bcd80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-08-01T22:55:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:11.684+0000 7fa7470bcd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:14.136 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-08-01T22:55:14.155 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.244+0000 7fcd6e05bd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-08-01T22:55:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.244+0000 7fcd6e05bd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-08-01T22:55:14.249 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.244+0000 7fcd6e05bd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-08-01T22:55:14.249 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.244+0000 7fcd6e05bd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-08-01T22:55:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.244+0000 7fcd6e05bd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-08-01T22:55:14.251 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.248+0000 7fcd6e05bd80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-08-01T22:55:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:14.288+0000 7fcd6e05bd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:16.731 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-08-01T22:55:16.748 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:16.788 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.788+0000 7fa8b3c29d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-08-01T22:55:16.790 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.788+0000 7fa8b3c29d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-08-01T22:55:16.791 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.788+0000 7fa8b3c29d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-08-01T22:55:16.792 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.788+0000 7fa8b3c29d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-08-01T22:55:16.792 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.788+0000 7fa8b3c29d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-08-01T22:55:16.793 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.788+0000 7fa8b3c29d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-08-01T22:55:16.832 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:16.828+0000 7fa8b3c29d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:19.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-08-01T22:55:19.329 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:19.367 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.364+0000 7efd879cfd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-08-01T22:55:19.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.364+0000 7efd879cfd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-08-01T22:55:19.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.364+0000 7efd879cfd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-08-01T22:55:19.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.364+0000 7efd879cfd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-08-01T22:55:19.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.364+0000 7efd879cfd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-08-01T22:55:19.369 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.364+0000 7efd879cfd80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-08-01T22:55:19.392 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:19.388+0000 7efd879cfd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T22:55:21.838 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-08-01T22:55:21.896 INFO:tasks.ceph:Reading keys from all nodes... 2022-08-01T22:55:21.896 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:21.905 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.xamO14M2uH 2022-08-01T22:55:21.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.xamO14M2uH 2022-08-01T22:55:21.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.xamO14M2uH 2022-08-01T22:55:21.998 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.xamO14M2uH is 56B 2022-08-01T22:55:22.065 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.xamO14M2uH 2022-08-01T22:55:22.079 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.GOZF8ineMR 2022-08-01T22:55:22.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.GOZF8ineMR 2022-08-01T22:55:22.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.GOZF8ineMR 2022-08-01T22:55:22.216 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.GOZF8ineMR is 56B 2022-08-01T22:55:22.231 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.GOZF8ineMR 2022-08-01T22:55:22.239 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:22.289 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.I206g68FB0 2022-08-01T22:55:22.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/mds/ceph-c/keyring /tmp/tmp.I206g68FB0 2022-08-01T22:55:22.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.I206g68FB0 2022-08-01T22:55:22.328 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.I206g68FB0 is 56B 2022-08-01T22:55:22.346 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.I206g68FB0 2022-08-01T22:55:22.352 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:22.402 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.uxzLBiRjd2 2022-08-01T22:55:22.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.uxzLBiRjd2 2022-08-01T22:55:22.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.uxzLBiRjd2 2022-08-01T22:55:22.487 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.uxzLBiRjd2 is 56B 2022-08-01T22:55:22.504 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.uxzLBiRjd2 2022-08-01T22:55:22.512 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:22.566 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.E5aSStSlcQ 2022-08-01T22:55:22.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.E5aSStSlcQ 2022-08-01T22:55:22.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.E5aSStSlcQ 2022-08-01T22:55:22.698 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.E5aSStSlcQ is 56B 2022-08-01T22:55:22.716 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.E5aSStSlcQ 2022-08-01T22:55:22.725 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:22.776 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.iqDLiCO9xo 2022-08-01T22:55:22.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.iqDLiCO9xo 2022-08-01T22:55:22.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.iqDLiCO9xo 2022-08-01T22:55:22.876 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.iqDLiCO9xo is 56B 2022-08-01T22:55:22.943 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.iqDLiCO9xo 2022-08-01T22:55:22.952 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:23.001 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.VJEwhigDPR 2022-08-01T22:55:23.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.VJEwhigDPR 2022-08-01T22:55:23.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.VJEwhigDPR 2022-08-01T22:55:23.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:23.092 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:23.097 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:23.100 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:23.148 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.VJEwhigDPR is 56B 2022-08-01T22:55:23.209 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.VJEwhigDPR 2022-08-01T22:55:23.218 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:23.227 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.dSXdndHh6f 2022-08-01T22:55:23.228 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.dSXdndHh6f 2022-08-01T22:55:23.244 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.dSXdndHh6f 2022-08-01T22:55:23.325 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.dSXdndHh6f is 56B 2022-08-01T22:55:23.395 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.dSXdndHh6f 2022-08-01T22:55:23.405 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:23.461 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.0Gof4afn4x 2022-08-01T22:55:23.461 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.0Gof4afn4x 2022-08-01T22:55:23.479 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.0Gof4afn4x 2022-08-01T22:55:23.560 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.0Gof4afn4x is 56B 2022-08-01T22:55:23.594 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.0Gof4afn4x 2022-08-01T22:55:23.608 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:23.658 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.eJ6BA9IhaX 2022-08-01T22:55:23.659 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.eJ6BA9IhaX 2022-08-01T22:55:23.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.eJ6BA9IhaX 2022-08-01T22:55:23.740 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.eJ6BA9IhaX is 56B 2022-08-01T22:55:23.760 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.eJ6BA9IhaX 2022-08-01T22:55:23.771 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:23.821 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.HXJeF2c34R 2022-08-01T22:55:23.821 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.HXJeF2c34R 2022-08-01T22:55:23.837 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.HXJeF2c34R 2022-08-01T22:55:23.967 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.HXJeF2c34R is 56B 2022-08-01T22:55:24.000 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.HXJeF2c34R 2022-08-01T22:55:24.018 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:24.072 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.uzog856g8T 2022-08-01T22:55:24.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.uzog856g8T 2022-08-01T22:55:24.098 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.uzog856g8T 2022-08-01T22:55:24.177 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.uzog856g8T is 56B 2022-08-01T22:55:24.233 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.uzog856g8T 2022-08-01T22:55:24.242 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:24.292 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.qSSszhXkVm 2022-08-01T22:55:24.293 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.qSSszhXkVm 2022-08-01T22:55:24.310 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.qSSszhXkVm 2022-08-01T22:55:24.383 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.qSSszhXkVm is 56B 2022-08-01T22:55:24.398 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.qSSszhXkVm 2022-08-01T22:55:24.420 DEBUG:teuthology.orchestra.remote:smithi130:/etc/ceph/ceph.client.0.keyring is 59B 2022-08-01T22:55:24.471 DEBUG:teuthology.orchestra.remote:smithi157:/etc/ceph/ceph.client.1.keyring is 59B 2022-08-01T22:55:24.487 INFO:tasks.ceph:Adding keys to all mons... 2022-08-01T22:55:24.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2022-08-01T22:55:24.491 DEBUG:teuthology.orchestra.run.smithi055:> sudo tee -a /etc/ceph/ceph.keyring 2022-08-01T22:55:24.512 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.y] 2022-08-01T22:55:24.513 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ== 2022-08-01T22:55:24.513 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2022-08-01T22:55:24.513 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw== 2022-08-01T22:55:24.514 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2022-08-01T22:55:24.514 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw== 2022-08-01T22:55:24.514 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.0] 2022-08-01T22:55:24.514 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw== 2022-08-01T22:55:24.515 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.1] 2022-08-01T22:55:24.515 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw== 2022-08-01T22:55:24.515 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.2] 2022-08-01T22:55:24.515 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A== 2022-08-01T22:55:24.515 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.3] 2022-08-01T22:55:24.516 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ== 2022-08-01T22:55:24.516 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.x] 2022-08-01T22:55:24.516 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ== 2022-08-01T22:55:24.517 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2022-08-01T22:55:24.517 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA== 2022-08-01T22:55:24.517 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.4] 2022-08-01T22:55:24.517 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA== 2022-08-01T22:55:24.518 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.5] 2022-08-01T22:55:24.518 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw== 2022-08-01T22:55:24.518 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.6] 2022-08-01T22:55:24.518 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q== 2022-08-01T22:55:24.519 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.7] 2022-08-01T22:55:24.519 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw== 2022-08-01T22:55:24.519 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2022-08-01T22:55:24.520 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q== 2022-08-01T22:55:24.520 INFO:teuthology.orchestra.run.smithi055.stdout:[client.1] 2022-08-01T22:55:24.520 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g== 2022-08-01T22:55:24.520 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2022-08-01T22:55:24.521 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ== 2022-08-01T22:55:24.521 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2022-08-01T22:55:24.521 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw== 2022-08-01T22:55:24.521 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2022-08-01T22:55:24.521 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw== 2022-08-01T22:55:24.522 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2022-08-01T22:55:24.522 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw== 2022-08-01T22:55:24.522 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2022-08-01T22:55:24.523 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw== 2022-08-01T22:55:24.523 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2022-08-01T22:55:24.523 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A== 2022-08-01T22:55:24.523 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2022-08-01T22:55:24.523 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ== 2022-08-01T22:55:24.524 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2022-08-01T22:55:24.524 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ== 2022-08-01T22:55:24.524 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2022-08-01T22:55:24.525 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA== 2022-08-01T22:55:24.525 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2022-08-01T22:55:24.525 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA== 2022-08-01T22:55:24.525 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2022-08-01T22:55:24.526 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw== 2022-08-01T22:55:24.526 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2022-08-01T22:55:24.526 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q== 2022-08-01T22:55:24.526 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2022-08-01T22:55:24.527 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw== 2022-08-01T22:55:24.527 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2022-08-01T22:55:24.528 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q== 2022-08-01T22:55:24.528 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2022-08-01T22:55:24.529 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g== 2022-08-01T22:55:24.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T22:55:24.534 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T22:55:24.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T22:55:24.632 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T22:55:24.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T22:55:24.687 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T22:55:24.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.777 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.826 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.912 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:24.962 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T22:55:25.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T22:55:25.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T22:55:25.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T22:55:25.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.193 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.246 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.334 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T22:55:25.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T22:55:25.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T22:55:25.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T22:55:25.535 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T22:55:25.584 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-08-01T22:55:25.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-08-01T22:55:25.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-08-01T22:55:25.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-08-01T22:55:25.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-08-01T22:55:25.746 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-08-01T22:55:25.831 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-08-01T22:55:25.845 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-08-01T22:55:25.858 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-08-01T22:55:25.933 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-08-01T22:55:25.951 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:25.955 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-08-01T22:55:25.962 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-08-01T22:55:25.963 INFO:tasks.ceph.mon.a:Restarting daemon 2022-08-01T22:55:25.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-08-01T22:55:25.967 INFO:tasks.ceph.mon.a:Started 2022-08-01T22:55:25.968 INFO:tasks.ceph.mon.b:Restarting daemon 2022-08-01T22:55:25.968 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-08-01T22:55:26.005 INFO:tasks.ceph.mon.b:Started 2022-08-01T22:55:26.006 INFO:tasks.ceph.mon.c:Restarting daemon 2022-08-01T22:55:26.006 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-08-01T22:55:26.009 INFO:tasks.ceph.mon.c:Started 2022-08-01T22:55:26.010 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-08-01T22:55:26.010 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-08-01T22:55:26.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-08-01T22:55:26.013 INFO:tasks.ceph.mgr.y:Started 2022-08-01T22:55:26.014 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-08-01T22:55:26.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-08-01T22:55:26.017 INFO:tasks.ceph.mgr.x:Started 2022-08-01T22:55:26.018 INFO:tasks.ceph:Setting crush tunables to default 2022-08-01T22:55:26.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2022-08-01T22:55:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 Processor -- start 2022-08-01T22:55:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 -- start start 2022-08-01T22:55:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c84061d90 con 0x7f5c840b5ed0 2022-08-01T22:55:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c84142f60 con 0x7f5c840b6ee0 2022-08-01T22:55:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c88909700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840ba5a0 con 0x7f5c840619b0 2022-08-01T22:55:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39860/0 (socket says 172.21.15.32:39860) 2022-08-01T22:55:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 learned_addr learned my addr 172.21.15.32:0/3305789608 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c82d9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c8359e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c840b5ed0 0x7f5c840b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55636/0 (socket says 172.21.15.32:55636) 2022-08-01T22:55:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T22:55:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.121+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.125+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-08-01T22:55:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.125+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.125+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.125+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.125+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-08-01T22:55:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:26.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:26.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:26.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-08-01T22:55:26.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T22:55:26.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:26.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:26.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.329+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-08-01T22:55:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:26.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:26.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-08-01T22:55:26.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T22:55:26.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:26.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:26.733+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-08-01T22:55:27.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:27.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:27.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T22:55:27.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c7bfff700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:27.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:27.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-08-01T22:55:27.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:27.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:27.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:27.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:27.537+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-08-01T22:55:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 msgr2=0x7f5c840b48a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-08-01T22:55:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c840619b0 0x7f5c840b48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 msgr2=0x7f5c84143890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-08-01T22:55:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.32:3300/0 conn(0x7f5c840b6ee0 0x7f5c84143890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 -- 172.21.15.32:0/3305789608 >> v1:172.21.15.32:6789/0 conn(0x7f5c840b5ed0 legacy=0x7f5c840b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 -- 172.21.15.32:0/3305789608 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c84143150 con 0x7f5c68005e30 2022-08-01T22:55:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.121+0000 7f5c8159a700 1 -- 172.21.15.32:0/3305789608 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c840ba5a0 con 0x7f5c680033b0 2022-08-01T22:55:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.125+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:29.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.125+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:29.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.125+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:29.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.125+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:29.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.125+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-08-01T22:55:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.329+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.329+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.329+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.329+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.329+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-08-01T22:55:29.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.733+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:29.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.733+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:29.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.733+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:29.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.733+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:29.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:29.733+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-08-01T22:55:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:30.537+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:30.537+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T22:55:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:30.537+0000 7f5c82d9d700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T22:55:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:30.537+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T22:55:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:30.537+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-08-01T22:55:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.373+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901868327 0 0) 0x7f5c84143150 con 0x7f5c68005e30 2022-08-01T22:55:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.373+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c64003260 con 0x7f5c68005e30 2022-08-01T22:55:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 <== mon.1 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 152031552 0 0) 0x7f5c64003260 con 0x7f5c68005e30 2022-08-01T22:55:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 msgr2=0x7f5c68005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-08-01T22:55:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 --2- 172.21.15.32:0/3305789608 >> v2:172.21.15.55:3300/0 conn(0x7f5c680033b0 0x7f5c68005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6800ac20 con 0x7f5c68005e30 2022-08-01T22:55:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 <== mon.1 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f5c6c00d3b0 con 0x7f5c68005e30 2022-08-01T22:55:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 >> v1:172.21.15.55:6789/0 conn(0x7f5c68005e30 legacy=0x7f5c680082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 -- 172.21.15.32:0/3305789608 reap_dead start 2022-08-01T22:55:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64003750 0x7f5c64003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64004500 0x7f5c64004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004ec0 0x7f5c6400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840ba5a0 con 0x7f5c64004ec0 2022-08-01T22:55:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6400be30 con 0x7f5c64003750 2022-08-01T22:55:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6400bff0 con 0x7f5c64004500 2022-08-01T22:55:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004ec0 0x7f5c6400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c82d9d700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64003750 0x7f5c64003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:31.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64004500 0x7f5c64004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:31.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64004500 msgr2=0x7f5c64004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:31.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64004500 0x7f5c64004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64003750 msgr2=0x7f5c64003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64003750 0x7f5c64003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 -- 172.21.15.32:0/3305789608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64003260 con 0x7f5c64004ec0 2022-08-01T22:55:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.377+0000 7f5c8359e700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004ec0 0x7f5c6400b750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5c840b45b0 tx=0x7f5c7c007bc0).ready entity=mon.0 client_cookie=13141128c8954613 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7c0048a0 con 0x7f5c64004ec0 2022-08-01T22:55:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c004a00 con 0x7f5c64004ec0 2022-08-01T22:55:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c8259c700 1 -- 172.21.15.32:0/3305789608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7c004d60 con 0x7f5c64004ec0 2022-08-01T22:55:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 -- 172.21.15.32:0/3305789608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004ec0 msgr2=0x7f5c6400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004ec0 0x7f5c6400b750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5c840b45b0 tx=0x7f5c7c007bc0).stop 2022-08-01T22:55:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 -- 172.21.15.32:0/3305789608 shutdown_connections 2022-08-01T22:55:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004ec0 0x7f5c6400b750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64004500 0x7f5c64004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 --2- 172.21.15.32:0/3305789608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64003750 0x7f5c64003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 -- 172.21.15.32:0/3305789608 >> 172.21.15.32:0/3305789608 conn(0x7f5c8406af80 msgr2=0x7f5c8405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 -- 172.21.15.32:0/3305789608 shutdown_connections 2022-08-01T22:55:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.381+0000 7f5c88909700 1 -- 172.21.15.32:0/3305789608 wait complete. 2022-08-01T22:55:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 Processor -- start 2022-08-01T22:55:31.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 -- start start 2022-08-01T22:55:31.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64003750 0x7f5c8413cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:31.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 0x7f5c8413d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:31.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64004ec0 0x7f5c840aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8413b3e0 con 0x7f5c64004500 2022-08-01T22:55:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c841559f0 con 0x7f5c64004ec0 2022-08-01T22:55:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c84155c90 con 0x7f5c64003750 2022-08-01T22:55:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 0x7f5c8413d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c82d9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64003750 0x7f5c8413cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 0x7f5c8413d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39932/0 (socket says 172.21.15.32:39932) 2022-08-01T22:55:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 -- 172.21.15.32:0/1658174004 learned_addr learned my addr 172.21.15.32:0/1658174004 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c8359e700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64004ec0 0x7f5c840aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 -- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64003750 msgr2=0x7f5c8413cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64003750 0x7f5c8413cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 -- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64004ec0 msgr2=0x7f5c840aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64004ec0 0x7f5c840aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 -- 172.21.15.32:0/1658174004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84155f30 con 0x7f5c64004500 2022-08-01T22:55:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 0x7f5c8413d650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f5c6c014000 tx=0x7f5c6c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c6c017910 con 0x7f5c64004500 2022-08-01T22:55:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c017a70 con 0x7f5c64004500 2022-08-01T22:55:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.385+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c841561d0 con 0x7f5c64004500 2022-08-01T22:55:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.389+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c6c017dd0 con 0x7f5c64004500 2022-08-01T22:55:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.389+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8413cc60 con 0x7f5c64004500 2022-08-01T22:55:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.389+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f5c6c026070 con 0x7f5c64004500 2022-08-01T22:55:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.389+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f5c6c000e40 con 0x7f5c64004500 2022-08-01T22:55:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.389+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c84156360 con 0x7f5c64004500 2022-08-01T22:55:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.393+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f5c6c024060 con 0x7f5c64004500 2022-08-01T22:55:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.449+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5c840574e0 con 0x7f5c64004500 2022-08-01T22:55:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c80d99700 1 -- 172.21.15.32:0/1658174004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5c6c02d020 con 0x7f5c64004500 2022-08-01T22:55:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 msgr2=0x7f5c8413d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 0x7f5c8413d650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f5c6c014000 tx=0x7f5c6c008040).stop 2022-08-01T22:55:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 shutdown_connections 2022-08-01T22:55:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c64004ec0 0x7f5c840aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64004500 0x7f5c8413d650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 --2- 172.21.15.32:0/1658174004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c64003750 0x7f5c8413cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 >> 172.21.15.32:0/1658174004 conn(0x7f5c8406af80 msgr2=0x7f5c840b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 shutdown_connections 2022-08-01T22:55:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:31.457+0000 7f5c88909700 1 -- 172.21.15.32:0/1658174004 wait complete. 2022-08-01T22:55:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2022-08-01T22:55:31.473 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-08-01T22:55:31.474 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:31.485 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.qFW74xQraA 2022-08-01T22:55:31.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.qFW74xQraA 2022-08-01T22:55:31.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.qFW74xQraA 2022-08-01T22:55:31.568 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.qFW74xQraA is 37B 2022-08-01T22:55:31.619 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.qFW74xQraA 2022-08-01T22:55:31.635 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:31.644 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.EeHrQ6m1FD 2022-08-01T22:55:31.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.EeHrQ6m1FD 2022-08-01T22:55:31.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.EeHrQ6m1FD 2022-08-01T22:55:31.766 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.EeHrQ6m1FD is 37B 2022-08-01T22:55:31.781 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.EeHrQ6m1FD 2022-08-01T22:55:31.788 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:31.837 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.YmwHuD3VqD 2022-08-01T22:55:31.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.YmwHuD3VqD 2022-08-01T22:55:31.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.YmwHuD3VqD 2022-08-01T22:55:31.883 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.YmwHuD3VqD is 37B 2022-08-01T22:55:31.939 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.YmwHuD3VqD 2022-08-01T22:55:31.952 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T22:55:32.005 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.eanwZlR1DY 2022-08-01T22:55:32.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.eanwZlR1DY 2022-08-01T22:55:32.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.eanwZlR1DY 2022-08-01T22:55:32.100 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.eanwZlR1DY is 37B 2022-08-01T22:55:32.159 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.eanwZlR1DY 2022-08-01T22:55:32.168 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:32.177 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.J0bK8cWYwY 2022-08-01T22:55:32.178 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.J0bK8cWYwY 2022-08-01T22:55:32.239 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.J0bK8cWYwY 2022-08-01T22:55:32.264 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.J0bK8cWYwY is 37B 2022-08-01T22:55:32.281 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.J0bK8cWYwY 2022-08-01T22:55:32.287 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:32.335 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.hvuzqSi80H 2022-08-01T22:55:32.335 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.hvuzqSi80H 2022-08-01T22:55:32.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.hvuzqSi80H 2022-08-01T22:55:32.414 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.hvuzqSi80H is 37B 2022-08-01T22:55:32.439 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.hvuzqSi80H 2022-08-01T22:55:32.447 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:32.496 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.XfM8uRhSWw 2022-08-01T22:55:32.496 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.XfM8uRhSWw 2022-08-01T22:55:32.512 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.XfM8uRhSWw 2022-08-01T22:55:32.590 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.XfM8uRhSWw is 37B 2022-08-01T22:55:32.611 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.XfM8uRhSWw 2022-08-01T22:55:32.620 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T22:55:32.666 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.l68KfOgXw3 2022-08-01T22:55:32.667 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.l68KfOgXw3 2022-08-01T22:55:32.682 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.l68KfOgXw3 2022-08-01T22:55:32.750 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.l68KfOgXw3 is 37B 2022-08-01T22:55:32.768 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.l68KfOgXw3 2022-08-01T22:55:32.774 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new 047a2077-8683-4670-a43a-a616da639621 0 2022-08-01T22:55:32.928 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.920+0000 7f421759e700 1 Processor -- start 2022-08-01T22:55:32.929 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.920+0000 7f421759e700 1 -- start start 2022-08-01T22:55:32.929 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 0x7f42181435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:32.929 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42180ad000 0x7f421813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:32.929 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4218143ab0 con 0x7f42180b08a0 2022-08-01T22:55:32.930 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42180b0c80 con 0x7f42180ad000 2022-08-01T22:55:32.930 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180ae810 con 0x7f42180b3260 2022-08-01T22:55:32.930 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 0x7f42181435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:32.931 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 0x7f42181435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57528/0 (socket says 172.21.15.55:57528) 2022-08-01T22:55:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 -- 172.21.15.55:0/1098396113 learned_addr learned my addr 172.21.15.55:0/1098396113 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4216d9d700 1 --2- 172.21.15.55:0/1098396113 >> v2:172.21.15.32:3300/0 conn(0x7f42180ad000 0x7f421813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 -- 172.21.15.55:0/1098396113 >> v2:172.21.15.32:3300/0 conn(0x7f42180ad000 msgr2=0x7f421813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 --2- 172.21.15.55:0/1098396113 >> v2:172.21.15.32:3300/0 conn(0x7f42180ad000 0x7f421813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 -- 172.21.15.55:0/1098396113 >> v1:172.21.15.32:6789/0 conn(0x7f42180b08a0 legacy=0x7f4218142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:32.954 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 -- 172.21.15.55:0/1098396113 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180aeab0 con 0x7f42180b3260 2022-08-01T22:55:32.955 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f4215d9b700 1 --2- 172.21.15.55:0/1098396113 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 0x7f42181435c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f420c002f70 tx=0x7f420c006eb0).ready entity=mon.2 client_cookie=ff6907d3136155c3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:32.955 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421559a700 1 -- 172.21.15.55:0/1098396113 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c00d570 con 0x7f42180b3260 2022-08-01T22:55:32.956 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421559a700 1 -- 172.21.15.55:0/1098396113 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c000e90 con 0x7f42180b3260 2022-08-01T22:55:32.956 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421559a700 1 -- 172.21.15.55:0/1098396113 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c00d900 con 0x7f42180b3260 2022-08-01T22:55:32.956 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- 172.21.15.55:0/1098396113 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 msgr2=0x7f42181435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:32.957 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 --2- 172.21.15.55:0/1098396113 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 0x7f42181435c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f420c002f70 tx=0x7f420c006eb0).stop 2022-08-01T22:55:32.962 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- 172.21.15.55:0/1098396113 shutdown_connections 2022-08-01T22:55:32.963 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 --2- 172.21.15.55:0/1098396113 >> v2:172.21.15.32:3300/0 conn(0x7f42180ad000 0x7f421813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:32.964 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 --2- 172.21.15.55:0/1098396113 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b3260 0x7f42181435c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:32.964 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- 172.21.15.55:0/1098396113 >> 172.21.15.55:0/1098396113 conn(0x7f42180612d0 msgr2=0x7f4218062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- 172.21.15.55:0/1098396113 shutdown_connections 2022-08-01T22:55:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.924+0000 7f421759e700 1 -- 172.21.15.55:0/1098396113 wait complete. 2022-08-01T22:55:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 Processor -- start 2022-08-01T22:55:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 -- start start 2022-08-01T22:55:32.977 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42180ad000 0x7f4218143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:32.977 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 0x7f42180a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:32.977 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b3260 0x7f42180a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:32.977 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180a10d0 con 0x7f42180b3260 2022-08-01T22:55:32.978 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218154580 con 0x7f42180ad000 2022-08-01T22:55:32.978 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4218154820 con 0x7f42180b08a0 2022-08-01T22:55:32.978 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f4215d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 0x7f42180a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:32.978 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f421659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42180ad000 0x7f4218143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f4215d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 0x7f42180a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57534/0 (socket says 172.21.15.55:57534) 2022-08-01T22:55:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f4215d9b700 1 -- 172.21.15.55:0/2850795696 learned_addr learned my addr 172.21.15.55:0/2850795696 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.928+0000 7f4216d9d700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b3260 0x7f42180a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f4215d9b700 1 -- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42180ad000 msgr2=0x7f4218143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f4215d9b700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42180ad000 0x7f4218143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:32.988 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f4215d9b700 1 -- 172.21.15.55:0/2850795696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b3260 msgr2=0x7f42180a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:32.988 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f4215d9b700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b3260 0x7f42180a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:32.988 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f4215d9b700 1 -- 172.21.15.55:0/2850795696 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c00b1e0 con 0x7f42180b08a0 2022-08-01T22:55:32.988 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f421659c700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42180ad000 0x7f4218143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:32.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f4215d9b700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 0x7f42180a3a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f420c002f70 tx=0x7f420c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:32.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c011760 con 0x7f42180b08a0 2022-08-01T22:55:32.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c0118c0 con 0x7f42180b08a0 2022-08-01T22:55:32.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c011c20 con 0x7f42180b08a0 2022-08-01T22:55:32.990 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421809fbe0 con 0x7f42180b08a0 2022-08-01T22:55:32.990 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42180a16e0 con 0x7f42180b08a0 2022-08-01T22:55:32.990 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f420c01a040 con 0x7f42180b08a0 2022-08-01T22:55:32.990 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f420c01ea00 con 0x7f42180b08a0 2022-08-01T22:55:32.991 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.932+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42180a2490 con 0x7f42180b08a0 2022-08-01T22:55:32.991 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:32.936+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f420c023070 con 0x7f42180b08a0 2022-08-01T22:55:33.006 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.004+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd new", "uuid": "047a2077-8683-4670-a43a-a616da639621", "id": 0} v 0) v1 -- 0x7f42180574e0 con 0x7f42180b08a0 2022-08-01T22:55:33.014 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.012+0000 7f42077fe700 1 -- 172.21.15.55:0/2850795696 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "047a2077-8683-4670-a43a-a616da639621", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f420c02f440 con 0x7f42180b08a0 2022-08-01T22:55:33.015 INFO:teuthology.orchestra.run.smithi055.stdout:0 2022-08-01T22:55:33.016 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.012+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 msgr2=0x7f42180a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.016 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.012+0000 7f421759e700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 0x7f42180a3a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f420c002f70 tx=0x7f420c015000).stop 2022-08-01T22:55:33.017 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.012+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 shutdown_connections 2022-08-01T22:55:33.017 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.012+0000 7f421759e700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b3260 0x7f42180a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.017 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.016+0000 7f421759e700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42180b08a0 0x7f42180a3a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.017 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.016+0000 7f421759e700 1 --2- 172.21.15.55:0/2850795696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42180ad000 0x7f4218143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.020 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.016+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 >> 172.21.15.55:0/2850795696 conn(0x7f42180612d0 msgr2=0x7f4218141300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.020 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.016+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 shutdown_connections 2022-08-01T22:55:33.021 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.016+0000 7f421759e700 1 -- 172.21.15.55:0/2850795696 wait complete. 2022-08-01T22:55:33.034 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new 200572dd-9760-43f7-9568-b2c7553721b3 1 2022-08-01T22:55:33.149 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 Processor -- start 2022-08-01T22:55:33.149 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 -- start start 2022-08-01T22:55:33.149 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04740af550 0x7f04740b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f04741445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04740aa630 con 0x7f0474144ad0 2022-08-01T22:55:33.152 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740b15e0 con 0x7f04740af550 2022-08-01T22:55:33.152 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0479367700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04740af930 con 0x7f0474060330 2022-08-01T22:55:33.153 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f04737fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f04741445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.153 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0478b66700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0474144ad0 0x7f04740a8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56020/0 (socket says 172.21.15.55:56020) 2022-08-01T22:55:33.154 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0478b66700 1 -- 172.21.15.55:0/1311226909 learned_addr learned my addr 172.21.15.55:0/1311226909 (peer_addr_for_me v1:172.21.15.55:0/0) 2022-08-01T22:55:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f04737fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f04741445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57538/0 (socket says 172.21.15.55:57538) 2022-08-01T22:55:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.132+0000 7f0473fff700 1 --2- 172.21.15.55:0/1311226909 >> v2:172.21.15.32:3300/0 conn(0x7f04740af550 0x7f04740b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0472ffd700 1 -- 172.21.15.55:0/1311226909 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726131709 0 0) 0x7f04740aa630 con 0x7f0474144ad0 2022-08-01T22:55:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0472ffd700 1 -- 172.21.15.55:0/1311226909 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0460003260 con 0x7f0474144ad0 2022-08-01T22:55:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 -- 172.21.15.55:0/1311226909 >> v2:172.21.15.32:3300/0 conn(0x7f04740af550 msgr2=0x7f04740b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 --2- 172.21.15.55:0/1311226909 >> v2:172.21.15.32:3300/0 conn(0x7f04740af550 0x7f04740b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 -- 172.21.15.55:0/1311226909 >> v1:172.21.15.55:6789/0 conn(0x7f0474144ad0 legacy=0x7f04740a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 -- 172.21.15.55:0/1311226909 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047405fd10 con 0x7f0474060330 2022-08-01T22:55:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0473fff700 1 --2- 172.21.15.55:0/1311226909 >> v2:172.21.15.32:3300/0 conn(0x7f04740af550 0x7f04740b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 --2- 172.21.15.55:0/1311226909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f04741445e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0468002f70 tx=0x7f0468006df0).ready entity=mon.2 client_cookie=6de5f36a70099542 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0472ffd700 1 -- 172.21.15.55:0/1311226909 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046800d570 con 0x7f0474060330 2022-08-01T22:55:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0472ffd700 1 -- 172.21.15.55:0/1311226909 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0468000e90 con 0x7f0474060330 2022-08-01T22:55:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0472ffd700 1 -- 172.21.15.55:0/1311226909 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046800d8c0 con 0x7f0474060330 2022-08-01T22:55:33.162 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- 172.21.15.55:0/1311226909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 msgr2=0x7f04741445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.162 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 --2- 172.21.15.55:0/1311226909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f04741445e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0468002f70 tx=0x7f0468006df0).stop 2022-08-01T22:55:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- 172.21.15.55:0/1311226909 shutdown_connections 2022-08-01T22:55:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 --2- 172.21.15.55:0/1311226909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f04741445e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 --2- 172.21.15.55:0/1311226909 >> v2:172.21.15.32:3300/0 conn(0x7f04740af550 0x7f04740b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.164 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- 172.21.15.55:0/1311226909 >> 172.21.15.55:0/1311226909 conn(0x7f04740612d0 msgr2=0x7f0474062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.164 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- 172.21.15.55:0/1311226909 shutdown_connections 2022-08-01T22:55:33.165 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- 172.21.15.55:0/1311226909 wait complete. 2022-08-01T22:55:33.166 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 Processor -- start 2022-08-01T22:55:33.166 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- start start 2022-08-01T22:55:33.167 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f0474143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.167 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04740af550 0x7f04741503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.168 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0474144ad0 0x7f04741508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.168 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04741534b0 con 0x7f0474144ad0 2022-08-01T22:55:33.169 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474064d20 con 0x7f04740af550 2022-08-01T22:55:33.169 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0479367700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0474064fc0 con 0x7f0474060330 2022-08-01T22:55:33.170 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0473fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f0474143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.170 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04740af550 0x7f04741503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.171 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0473fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f0474143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57544/0 (socket says 172.21.15.55:57544) 2022-08-01T22:55:33.171 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0473fff700 1 -- 172.21.15.55:0/1528283979 learned_addr learned my addr 172.21.15.55:0/1528283979 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.173 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f0478b66700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0474144ad0 0x7f04741508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.173 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 -- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 msgr2=0x7f0474143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.173 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.136+0000 7f04737fe700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f0474143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.175 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04737fe700 1 -- 172.21.15.55:0/1528283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0474144ad0 msgr2=0x7f04741508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.175 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04737fe700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0474144ad0 0x7f04741508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.176 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04737fe700 1 -- 172.21.15.55:0/1528283979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046800b180 con 0x7f04740af550 2022-08-01T22:55:33.176 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f0478b66700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0474144ad0 0x7f04741508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:33.177 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04737fe700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04740af550 0x7f04741503e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f0468002f70 tx=0x7f0468014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.177 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046800dea0 con 0x7f04740af550 2022-08-01T22:55:33.177 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0468013e90 con 0x7f04740af550 2022-08-01T22:55:33.178 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047405f840 con 0x7f04740af550 2022-08-01T22:55:33.178 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f046801e5f0 con 0x7f04740af550 2022-08-01T22:55:33.179 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047414db20 con 0x7f04740af550 2022-08-01T22:55:33.179 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f0468025030 con 0x7f04740af550 2022-08-01T22:55:33.179 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f0468028440 con 0x7f04740af550 2022-08-01T22:55:33.180 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.140+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047405fa70 con 0x7f04740af550 2022-08-01T22:55:33.180 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.144+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0468023070 con 0x7f04740af550 2022-08-01T22:55:33.214 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.212+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", "id": 1} v 0) v1 -- 0x7f04740574e0 con 0x7f04740af550 2022-08-01T22:55:33.223 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f04717fa700 1 -- 172.21.15.55:0/1528283979 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0468019030 con 0x7f04740af550 2022-08-01T22:55:33.224 INFO:teuthology.orchestra.run.smithi055.stdout:1 2022-08-01T22:55:33.225 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04740af550 msgr2=0x7f04741503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.226 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04740af550 0x7f04741503e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f0468002f70 tx=0x7f0468014000).stop 2022-08-01T22:55:33.226 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 shutdown_connections 2022-08-01T22:55:33.227 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0474144ad0 0x7f04741508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.228 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04740af550 0x7f04741503e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.228 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 --2- 172.21.15.55:0/1528283979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0474060330 0x7f0474143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.228 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 >> 172.21.15.55:0/1528283979 conn(0x7f04740612d0 msgr2=0x7f0474143cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.228 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.220+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 shutdown_connections 2022-08-01T22:55:33.228 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.224+0000 7f0479367700 1 -- 172.21.15.55:0/1528283979 wait complete. 2022-08-01T22:55:33.235 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new a8c47037-5440-477f-8684-a1f3f8fb26f4 2 2022-08-01T22:55:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 Processor -- start 2022-08-01T22:55:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- start start 2022-08-01T22:55:33.360 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 0x7fe670060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.361 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe670061190 0x7fe670066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.361 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700654c0 con 0x7fe6700b2730 2022-08-01T22:55:33.361 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe670062190 con 0x7fe670061190 2022-08-01T22:55:33.362 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe670061600 con 0x7fe6700b3190 2022-08-01T22:55:33.362 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 0x7fe670060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.363 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 0x7fe670060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57550/0 (socket says 172.21.15.55:57550) 2022-08-01T22:55:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 -- 172.21.15.55:0/3117468215 learned_addr learned my addr 172.21.15.55:0/3117468215 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.365 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675a25700 1 --2- 172.21.15.55:0/3117468215 >> v2:172.21.15.32:3300/0 conn(0x7fe670061190 0x7fe670066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.365 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 -- 172.21.15.55:0/3117468215 >> v2:172.21.15.32:3300/0 conn(0x7fe670061190 msgr2=0x7fe670066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:33.366 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 --2- 172.21.15.55:0/3117468215 >> v2:172.21.15.32:3300/0 conn(0x7fe670061190 0x7fe670066dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.366 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 -- 172.21.15.55:0/3117468215 >> v1:172.21.15.32:6789/0 conn(0x7fe6700b2730 legacy=0x7fe6700666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.366 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 -- 172.21.15.55:0/3117468215 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700b3570 con 0x7fe6700b3190 2022-08-01T22:55:33.367 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675a25700 1 --2- 172.21.15.55:0/3117468215 >> v2:172.21.15.32:3300/0 conn(0x7fe670061190 0x7fe670066dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:33.367 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe675224700 1 --2- 172.21.15.55:0/3117468215 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 0x7fe670060ca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe66c002f70 tx=0x7fe66c006eb0).ready entity=mon.2 client_cookie=1ed6ac8dc45aec55 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.367 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe65ffff700 1 -- 172.21.15.55:0/3117468215 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe66c00d570 con 0x7fe6700b3190 2022-08-01T22:55:33.367 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe65ffff700 1 -- 172.21.15.55:0/3117468215 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c000e90 con 0x7fe6700b3190 2022-08-01T22:55:33.367 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe65ffff700 1 -- 172.21.15.55:0/3117468215 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe66c00d900 con 0x7fe6700b3190 2022-08-01T22:55:33.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- 172.21.15.55:0/3117468215 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 msgr2=0x7fe670060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 --2- 172.21.15.55:0/3117468215 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 0x7fe670060ca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe66c002f70 tx=0x7fe66c006eb0).stop 2022-08-01T22:55:33.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- 172.21.15.55:0/3117468215 shutdown_connections 2022-08-01T22:55:33.368 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 --2- 172.21.15.55:0/3117468215 >> v2:172.21.15.32:3300/0 conn(0x7fe670061190 0x7fe670066dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.369 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 --2- 172.21.15.55:0/3117468215 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6700b3190 0x7fe670060ca0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.369 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- 172.21.15.55:0/3117468215 >> 172.21.15.55:0/3117468215 conn(0x7fe670063870 msgr2=0x7fe670063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.369 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- 172.21.15.55:0/3117468215 shutdown_connections 2022-08-01T22:55:33.369 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.340+0000 7fe676226700 1 -- 172.21.15.55:0/3117468215 wait complete. 2022-08-01T22:55:33.370 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 Processor -- start 2022-08-01T22:55:33.370 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- start start 2022-08-01T22:55:33.370 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe670061190 0x7fe6700addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.371 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6700b2730 0x7fe6700abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.371 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6700b3190 0x7fe6700b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.371 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700b0aa0 con 0x7fe6700b2730 2022-08-01T22:55:33.371 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700af590 con 0x7fe6700b3190 2022-08-01T22:55:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700af830 con 0x7fe670061190 2022-08-01T22:55:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe675224700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe670061190 0x7fe6700addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe675224700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe670061190 0x7fe6700addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57556/0 (socket says 172.21.15.55:57556) 2022-08-01T22:55:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe675224700 1 -- 172.21.15.55:0/965026766 learned_addr learned my addr 172.21.15.55:0/965026766 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.373 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe675a25700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6700b3190 0x7fe6700b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.373 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6700b2730 0x7fe6700abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.373 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 -- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe670061190 msgr2=0x7fe6700addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.373 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe670061190 0x7fe6700addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 -- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6700b3190 msgr2=0x7fe6700b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6700b3190 0x7fe6700b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 -- 172.21.15.55:0/965026766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668009470 con 0x7fe6700b2730 2022-08-01T22:55:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe674a23700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6700b2730 0x7fe6700abbe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe6700634e0 tx=0x7fe660007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe660012400 con 0x7fe6700b2730 2022-08-01T22:55:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660012560 con 0x7fe6700b2730 2022-08-01T22:55:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c00b1e0 con 0x7fe6700b2730 2022-08-01T22:55:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6600128c0 con 0x7fe6700b2730 2022-08-01T22:55:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe670155ad0 con 0x7fe6700b2730 2022-08-01T22:55:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe660019030 con 0x7fe6700b2730 2022-08-01T22:55:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fe660012bf0 con 0x7fe6700b2730 2022-08-01T22:55:33.377 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.344+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67005fbf0 con 0x7fe6700b2730 2022-08-01T22:55:33.377 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.352+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe660017070 con 0x7fe6700b2730 2022-08-01T22:55:33.424 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.420+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", "id": 2} v 0) v1 -- 0x7fe6700574e0 con 0x7fe6700b2730 2022-08-01T22:55:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe65e7fc700 1 -- 172.21.15.55:0/965026766 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe660025450 con 0x7fe6700b2730 2022-08-01T22:55:33.431 INFO:teuthology.orchestra.run.smithi055.stdout:2 2022-08-01T22:55:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6700b2730 msgr2=0x7fe6700abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6700b2730 0x7fe6700abbe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe6700634e0 tx=0x7fe660007b90).stop 2022-08-01T22:55:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 shutdown_connections 2022-08-01T22:55:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6700b3190 0x7fe6700b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6700b2730 0x7fe6700abbe0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 --2- 172.21.15.55:0/965026766 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe670061190 0x7fe6700addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 >> 172.21.15.55:0/965026766 conn(0x7fe670063870 msgr2=0x7fe6701490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 shutdown_connections 2022-08-01T22:55:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.428+0000 7fe676226700 1 -- 172.21.15.55:0/965026766 wait complete. 2022-08-01T22:55:33.445 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new 645afcc6-0658-4477-9722-c6e21159c5b3 3 2022-08-01T22:55:33.548 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 Processor -- start 2022-08-01T22:55:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 -- start start 2022-08-01T22:55:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f513c0b6420 0x7f513c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 0x7f513c061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0b60e0 con 0x7f513c061fe0 2022-08-01T22:55:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c0b2100 con 0x7f513c0b6420 2022-08-01T22:55:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f51417d2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b23a0 con 0x7f513c0b41f0 2022-08-01T22:55:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 0x7f513c061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 0x7f513c061af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57564/0 (socket says 172.21.15.55:57564) 2022-08-01T22:55:33.551 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f513b7fe700 1 -- 172.21.15.55:0/2672894558 learned_addr learned my addr 172.21.15.55:0/2672894558 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.551 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.544+0000 7f513bfff700 1 --2- 172.21.15.55:0/2672894558 >> v2:172.21.15.32:3300/0 conn(0x7f513c0b6420 0x7f513c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.551 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513b7fe700 1 -- 172.21.15.55:0/2672894558 >> v2:172.21.15.32:3300/0 conn(0x7f513c0b6420 msgr2=0x7f513c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:33.552 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513b7fe700 1 --2- 172.21.15.55:0/2672894558 >> v2:172.21.15.32:3300/0 conn(0x7f513c0b6420 0x7f513c061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.553 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513b7fe700 1 -- 172.21.15.55:0/2672894558 >> v1:172.21.15.32:6789/0 conn(0x7f513c061fe0 legacy=0x7f513c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.553 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513b7fe700 1 -- 172.21.15.55:0/2672894558 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0b2640 con 0x7f513c0b41f0 2022-08-01T22:55:33.553 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513b7fe700 1 --2- 172.21.15.55:0/2672894558 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 0x7f513c061af0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f5130002f70 tx=0x7f5130006eb0).ready entity=mon.2 client_cookie=b2ff7b49f9f34630 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.553 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513affd700 1 -- 172.21.15.55:0/2672894558 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f513000d5c0 con 0x7f513c0b41f0 2022-08-01T22:55:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513affd700 1 -- 172.21.15.55:0/2672894558 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130000e90 con 0x7f513c0b41f0 2022-08-01T22:55:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f513affd700 1 -- 172.21.15.55:0/2672894558 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f513000d950 con 0x7f513c0b41f0 2022-08-01T22:55:33.559 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 -- 172.21.15.55:0/2672894558 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 msgr2=0x7f513c061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.560 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 --2- 172.21.15.55:0/2672894558 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 0x7f513c061af0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f5130002f70 tx=0x7f5130006eb0).stop 2022-08-01T22:55:33.560 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 -- 172.21.15.55:0/2672894558 shutdown_connections 2022-08-01T22:55:33.560 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 --2- 172.21.15.55:0/2672894558 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c0b41f0 0x7f513c061af0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.560 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 --2- 172.21.15.55:0/2672894558 >> v2:172.21.15.32:3300/0 conn(0x7f513c0b6420 0x7f513c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.561 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 -- 172.21.15.55:0/2672894558 >> 172.21.15.55:0/2672894558 conn(0x7f513c05c620 msgr2=0x7f513c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.561 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 -- 172.21.15.55:0/2672894558 shutdown_connections 2022-08-01T22:55:33.561 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 -- 172.21.15.55:0/2672894558 wait complete. 2022-08-01T22:55:33.561 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 Processor -- start 2022-08-01T22:55:33.562 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.548+0000 7f51417d2700 1 -- start start 2022-08-01T22:55:33.562 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c061fe0 0x7f513c0b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.562 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f513c0b41f0 0x7f513c12aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.563 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513c0b6420 0x7f513c12b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.563 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0af7c0 con 0x7f513c0b41f0 2022-08-01T22:55:33.564 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b0b50 con 0x7f513c0b6420 2022-08-01T22:55:33.564 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b0170 con 0x7f513c061fe0 2022-08-01T22:55:33.564 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f513bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c061fe0 0x7f513c0b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.564 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f513bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c061fe0 0x7f513c0b8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57568/0 (socket says 172.21.15.55:57568) 2022-08-01T22:55:33.565 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513c0b6420 0x7f513c12b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.565 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f513bfff700 1 -- 172.21.15.55:0/862652076 learned_addr learned my addr 172.21.15.55:0/862652076 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.565 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f513b7fe700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f513c0b41f0 0x7f513c12aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.565 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 -- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c061fe0 msgr2=0x7f513c0b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.566 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c061fe0 0x7f513c0b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.566 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 -- 172.21.15.55:0/862652076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f513c0b41f0 msgr2=0x7f513c12aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.566 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f513c0b41f0 0x7f513c12aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.567 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 -- 172.21.15.55:0/862652076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513000b1e0 con 0x7f513c0b6420 2022-08-01T22:55:33.567 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f5140fd1700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513c0b6420 0x7f513c12b3c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f513c06ac70 tx=0x7f5134007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.567 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f513400f470 con 0x7f513c0b6420 2022-08-01T22:55:33.567 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513400f5d0 con 0x7f513c0b6420 2022-08-01T22:55:33.568 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c0b0410 con 0x7f513c0b6420 2022-08-01T22:55:33.568 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f513400f930 con 0x7f513c0b6420 2022-08-01T22:55:33.568 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c12b8b0 con 0x7f513c0b6420 2022-08-01T22:55:33.568 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f5134010070 con 0x7f513c0b6420 2022-08-01T22:55:33.569 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f513400fc60 con 0x7f513c0b6420 2022-08-01T22:55:33.569 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.552+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c13d260 con 0x7f513c0b6420 2022-08-01T22:55:33.569 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.556+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f5134017070 con 0x7f513c0b6420 2022-08-01T22:55:33.626 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.624+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", "id": 3} v 0) v1 -- 0x7f513c0b28e0 con 0x7f513c0b6420 2022-08-01T22:55:33.635 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.632+0000 7f51397fa700 1 -- 172.21.15.55:0/862652076 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f513c0b60e0 con 0x7f513c0b6420 2022-08-01T22:55:33.636 INFO:teuthology.orchestra.run.smithi055.stdout:3 2022-08-01T22:55:33.637 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513c0b6420 msgr2=0x7f513c12b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.637 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513c0b6420 0x7f513c12b3c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f513c06ac70 tx=0x7f5134007eb0).stop 2022-08-01T22:55:33.638 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 shutdown_connections 2022-08-01T22:55:33.638 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513c0b6420 0x7f513c12b3c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.638 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f513c0b41f0 0x7f513c12aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.638 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 --2- 172.21.15.55:0/862652076 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f513c061fe0 0x7f513c0b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 >> 172.21.15.55:0/862652076 conn(0x7f513c05c620 msgr2=0x7f513c0b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 shutdown_connections 2022-08-01T22:55:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.636+0000 7f51417d2700 1 -- 172.21.15.55:0/862652076 wait complete. 2022-08-01T22:55:33.649 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new 8bb02127-2e21-4432-b767-9a7559abfe67 4 2022-08-01T22:55:33.765 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.748+0000 7ff8225ab700 1 Processor -- start 2022-08-01T22:55:33.766 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.748+0000 7ff8225ab700 1 -- start start 2022-08-01T22:55:33.767 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff81c0b31d0 0x7ff81c066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.767 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 0x7ff81c140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.767 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c067030 con 0x7ff81c0a9da0 2022-08-01T22:55:33.768 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c064a70 con 0x7ff81c0b31d0 2022-08-01T22:55:33.768 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c061f60 con 0x7ff81c1402d0 2022-08-01T22:55:33.769 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff821daa700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 0x7ff81c140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.769 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff821daa700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 0x7ff81c140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57578/0 (socket says 172.21.15.55:57578) 2022-08-01T22:55:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff821daa700 1 -- 172.21.15.55:0/4282183942 learned_addr learned my addr 172.21.15.55:0/4282183942 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff820da8700 1 --1- 172.21.15.55:0/4282183942 >> v1:172.21.15.55:6789/0 conn(0x7ff81c0a9da0 0x7ff81c060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56052/0 (socket says 172.21.15.55:56052) 2022-08-01T22:55:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8215a9700 1 --2- 172.21.15.55:0/4282183942 >> v2:172.21.15.55:3300/0 conn(0x7ff81c0b31d0 0x7ff81c066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8215a9700 1 -- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 msgr2=0x7ff81c140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8215a9700 1 --2- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 0x7ff81c140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8215a9700 1 -- 172.21.15.55:0/4282183942 >> v1:172.21.15.55:6789/0 conn(0x7ff81c0a9da0 legacy=0x7ff81c060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8215a9700 1 -- 172.21.15.55:0/4282183942 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0b35b0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff821daa700 1 --2- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 0x7ff81c140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8215a9700 1 --2- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c066020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7ff818002f70 tx=0x7ff818006df0).ready entity=mon.1 client_cookie=12f2fe79527610b6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff80bfff700 1 -- 172.21.15.55:0/4282183942 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff81800d570 con 0x7ff81c0b31d0 2022-08-01T22:55:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff80bfff700 1 -- 172.21.15.55:0/4282183942 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818000e90 con 0x7ff81c0b31d0 2022-08-01T22:55:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff80bfff700 1 -- 172.21.15.55:0/4282183942 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff81800d8c0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 msgr2=0x7ff81c066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 --2- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c066020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7ff818002f70 tx=0x7ff818006df0).stop 2022-08-01T22:55:33.782 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- 172.21.15.55:0/4282183942 shutdown_connections 2022-08-01T22:55:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 --2- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c1402d0 0x7ff81c140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 --2- 172.21.15.55:0/4282183942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c066020 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.784 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- 172.21.15.55:0/4282183942 >> 172.21.15.55:0/4282183942 conn(0x7ff81c063870 msgr2=0x7ff81c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.785 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- 172.21.15.55:0/4282183942 shutdown_connections 2022-08-01T22:55:33.785 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.752+0000 7ff8225ab700 1 -- 172.21.15.55:0/4282183942 wait complete. 2022-08-01T22:55:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 Processor -- start 2022-08-01T22:55:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 -- start start 2022-08-01T22:55:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c0a9da0 0x7ff81c1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.788 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.788 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff81c1402d0 0x7ff81c13a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.789 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0a91e0 con 0x7ff81c1402d0 2022-08-01T22:55:33.789 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c138ac0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.790 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c13e190 con 0x7ff81c0a9da0 2022-08-01T22:55:33.790 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44986/0 (socket says 172.21.15.55:44986) 2022-08-01T22:55:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 -- 172.21.15.55:0/2370010892 learned_addr learned my addr 172.21.15.55:0/2370010892 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.792 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8215a9700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c0a9da0 0x7ff81c1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.792 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff821daa700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff81c1402d0 0x7ff81c13a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 -- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c0a9da0 msgr2=0x7ff81c1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c0a9da0 0x7ff81c1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 -- 172.21.15.55:0/2370010892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff81c1402d0 msgr2=0x7ff81c13a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff81c1402d0 0x7ff81c13a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 -- 172.21.15.55:0/2370010892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009530 con 0x7ff81c0b31d0 2022-08-01T22:55:33.796 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff821daa700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff81c1402d0 0x7ff81c13a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:55:33.796 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff820da8700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c145f40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff81c0634e0 tx=0x7ff80c007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff80c0060d0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c006230 con 0x7ff81c0b31d0 2022-08-01T22:55:33.798 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff80c0046f0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.799 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81800b180 con 0x7ff81c0b31d0 2022-08-01T22:55:33.800 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c137640 con 0x7ff81c0b31d0 2022-08-01T22:55:33.800 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7ff80c01a070 con 0x7ff81c0b31d0 2022-08-01T22:55:33.801 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.756+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7ff80c0063a0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.801 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.760+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c13a850 con 0x7ff81c0b31d0 2022-08-01T22:55:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.760+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7ff80c018060 con 0x7ff81c0b31d0 2022-08-01T22:55:33.841 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.840+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", "id": 4} v 0) v1 -- 0x7ff81c13efa0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.852 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.848+0000 7ff80a7fc700 1 -- 172.21.15.55:0/2370010892 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff80c0220b0 con 0x7ff81c0b31d0 2022-08-01T22:55:33.853 INFO:teuthology.orchestra.run.smithi055.stdout:4 2022-08-01T22:55:33.859 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 msgr2=0x7ff81c145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.859 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c145f40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff81c0634e0 tx=0x7ff80c007c20).stop 2022-08-01T22:55:33.860 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 shutdown_connections 2022-08-01T22:55:33.860 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff81c1402d0 0x7ff81c13a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.860 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff81c0b31d0 0x7ff81c145f40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.860 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 --2- 172.21.15.55:0/2370010892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff81c0a9da0 0x7ff81c1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.861 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 >> 172.21.15.55:0/2370010892 conn(0x7ff81c063870 msgr2=0x7ff81c0b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.861 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 shutdown_connections 2022-08-01T22:55:33.861 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.852+0000 7ff8225ab700 1 -- 172.21.15.55:0/2370010892 wait complete. 2022-08-01T22:55:33.867 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new d961d2aa-c6df-4770-90e1-0d2a43f360ad 5 2022-08-01T22:55:33.963 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.956+0000 7f44649c6700 1 Processor -- start 2022-08-01T22:55:33.964 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.956+0000 7f44649c6700 1 -- start start 2022-08-01T22:55:33.964 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 0x7f44600b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.964 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44600ad340 con 0x7f44600b57b0 2022-08-01T22:55:33.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44600b5be0 con 0x7f4460080c30 2022-08-01T22:55:33.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44600ae750 con 0x7f44600b0740 2022-08-01T22:55:33.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 0x7f44600b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4460080c30 0x7f44600af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56064/0 (socket says 172.21.15.55:56064) 2022-08-01T22:55:33.965 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 0x7f44600b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57590/0 (socket says 172.21.15.55:57590) 2022-08-01T22:55:33.966 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445f7fe700 1 -- 172.21.15.55:0/1190259038 learned_addr learned my addr 172.21.15.55:0/1190259038 (peer_addr_for_me v1:172.21.15.55:0/0) 2022-08-01T22:55:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445effd700 1 -- 172.21.15.55:0/1190259038 >> v1:172.21.15.55:6789/0 conn(0x7f4460080c30 legacy=0x7f44600af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445effd700 1 -- 172.21.15.55:0/1190259038 >> v1:172.21.15.32:6789/0 conn(0x7f44600b57b0 legacy=0x7f446015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445effd700 1 -- 172.21.15.55:0/1190259038 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44600ae9f0 con 0x7f44600b0740 2022-08-01T22:55:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445effd700 1 --2- 172.21.15.55:0/1190259038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 0x7f44600b52c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4454002f70 tx=0x7f4454006e80).ready entity=mon.2 client_cookie=a66ca7c2b543bd9f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445e7fc700 1 -- 172.21.15.55:0/1190259038 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445400f730 con 0x7f44600b0740 2022-08-01T22:55:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445e7fc700 1 -- 172.21.15.55:0/1190259038 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4454000e90 con 0x7f44600b0740 2022-08-01T22:55:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f445e7fc700 1 -- 172.21.15.55:0/1190259038 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445400fa80 con 0x7f44600b0740 2022-08-01T22:55:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- 172.21.15.55:0/1190259038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 msgr2=0x7f44600b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.973 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 --2- 172.21.15.55:0/1190259038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 0x7f44600b52c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4454002f70 tx=0x7f4454006e80).stop 2022-08-01T22:55:33.973 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- 172.21.15.55:0/1190259038 shutdown_connections 2022-08-01T22:55:33.973 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 --2- 172.21.15.55:0/1190259038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44600b0740 0x7f44600b52c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.973 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- 172.21.15.55:0/1190259038 >> 172.21.15.55:0/1190259038 conn(0x7f446005f840 msgr2=0x7f446005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- 172.21.15.55:0/1190259038 shutdown_connections 2022-08-01T22:55:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.960+0000 7f44649c6700 1 -- 172.21.15.55:0/1190259038 wait complete. 2022-08-01T22:55:33.982 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.964+0000 7f44649c6700 1 Processor -- start 2022-08-01T22:55:33.983 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.964+0000 7f44649c6700 1 -- start start 2022-08-01T22:55:33.983 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 0x7f446014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.983 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44600b0740 0x7f44600aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.984 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44600b57b0 0x7f44600aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:33.985 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446015fae0 con 0x7f44600b0740 2022-08-01T22:55:33.985 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44600aacc0 con 0x7f44600b57b0 2022-08-01T22:55:33.986 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4460151440 con 0x7f4460080c30 2022-08-01T22:55:33.986 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 0x7f446014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.987 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44600b57b0 0x7f44600aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.987 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44600b0740 0x7f44600aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:33.988 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 0x7f446014eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57592/0 (socket says 172.21.15.55:57592) 2022-08-01T22:55:33.988 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44600b57b0 0x7f44600aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45000/0 (socket says 172.21.15.55:45000) 2022-08-01T22:55:33.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44600b0740 0x7f44600aa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.55:41636/0 (socket says 172.21.15.55:41636) 2022-08-01T22:55:33.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 -- 172.21.15.55:0/1367355091 learned_addr learned my addr 172.21.15.55:0/1367355091 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:33.989 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 -- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44600b57b0 msgr2=0x7f44600aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.990 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44600b57b0 0x7f44600aa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.990 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 -- 172.21.15.55:0/1367355091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44600b0740 msgr2=0x7f44600aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:33.991 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44600b0740 0x7f44600aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:33.991 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 -- 172.21.15.55:0/1367355091 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445400c310 con 0x7f4460080c30 2022-08-01T22:55:33.991 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445f7fe700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 0x7f446014eb90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f446006ae20 tx=0x7f4450007c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:33.992 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44500070d0 con 0x7f4460080c30 2022-08-01T22:55:33.992 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446015fcd0 con 0x7f4460080c30 2022-08-01T22:55:33.993 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450007230 con 0x7f4460080c30 2022-08-01T22:55:33.994 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4450005610 con 0x7f4460080c30 2022-08-01T22:55:33.995 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446015ff30 con 0x7f4460080c30 2022-08-01T22:55:33.996 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f4450010070 con 0x7f4460080c30 2022-08-01T22:55:33.996 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f4450005d50 con 0x7f4460080c30 2022-08-01T22:55:33.997 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.968+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44600ac330 con 0x7f4460080c30 2022-08-01T22:55:33.997 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:33.972+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4450018020 con 0x7f4460080c30 2022-08-01T22:55:34.038 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.036+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", "id": 5} v 0) v1 -- 0x7f4460151580 con 0x7f4460080c30 2022-08-01T22:55:34.054 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f445cff9700 1 -- 172.21.15.55:0/1367355091 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4450013450 con 0x7f4460080c30 2022-08-01T22:55:34.055 INFO:teuthology.orchestra.run.smithi055.stdout:5 2022-08-01T22:55:34.058 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 msgr2=0x7f446014eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.063 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 0x7f446014eb90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f446006ae20 tx=0x7f4450007c40).stop 2022-08-01T22:55:34.063 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 shutdown_connections 2022-08-01T22:55:34.064 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44600b57b0 0x7f44600aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.064 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44600b0740 0x7f44600aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.065 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 --2- 172.21.15.55:0/1367355091 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4460080c30 0x7f446014eb90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.065 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 >> 172.21.15.55:0/1367355091 conn(0x7f446005f840 msgr2=0x7f446005c690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:34.066 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 shutdown_connections 2022-08-01T22:55:34.066 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.052+0000 7f44649c6700 1 -- 172.21.15.55:0/1367355091 wait complete. 2022-08-01T22:55:34.069 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new 162a1876-d2c7-4886-9434-b4114d80e92b 6 2022-08-01T22:55:34.175 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 Processor -- start 2022-08-01T22:55:34.176 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 -- start start 2022-08-01T22:55:34.177 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9320 0x7f758c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.177 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f758c0b0560 0x7f758c140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.177 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c0b12d0 con 0x7f758c0b0d10 2022-08-01T22:55:34.178 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0a9700 con 0x7f758c0b0560 2022-08-01T22:55:34.179 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759333e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b1500 con 0x7f758c0a9320 2022-08-01T22:55:34.180 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759233c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9320 0x7f758c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f758c0b0560 0x7f758c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759233c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9320 0x7f758c1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57598/0 (socket says 172.21.15.55:57598) 2022-08-01T22:55:34.182 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f758c0b0560 0x7f758c140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45004/0 (socket says 172.21.15.55:45004) 2022-08-01T22:55:34.182 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759233c700 1 -- 172.21.15.55:0/137636958 learned_addr learned my addr 172.21.15.55:0/137636958 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:34.183 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7591b3b700 1 --1- 172.21.15.55:0/137636958 >> v1:172.21.15.55:6789/0 conn(0x7f758c0b0d10 0x7f758c146bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56080/0 (socket says 172.21.15.55:56080) 2022-08-01T22:55:34.194 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 -- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9320 msgr2=0x7f758c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 --2- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9320 0x7f758c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 -- 172.21.15.55:0/137636958 >> v1:172.21.15.55:6789/0 conn(0x7f758c0b0d10 legacy=0x7f758c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 -- 172.21.15.55:0/137636958 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0b17a0 con 0x7f758c0b0560 2022-08-01T22:55:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f7592b3d700 1 --2- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0b0560 0x7f758c140350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f75840032f0 tx=0x7f75840091d0).ready entity=mon.1 client_cookie=58b7f028451186cf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759133a700 1 -- 172.21.15.55:0/137636958 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f758400b6c0 con 0x7f758c0b0560 2022-08-01T22:55:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759133a700 1 -- 172.21.15.55:0/137636958 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400b820 con 0x7f758c0b0560 2022-08-01T22:55:34.198 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.172+0000 7f759133a700 1 -- 172.21.15.55:0/137636958 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f758400b990 con 0x7f758c0b0560 2022-08-01T22:55:34.198 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0b0560 msgr2=0x7f758c140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.199 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 --2- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0b0560 0x7f758c140350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f75840032f0 tx=0x7f75840091d0).stop 2022-08-01T22:55:34.199 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- 172.21.15.55:0/137636958 shutdown_connections 2022-08-01T22:55:34.200 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 --2- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0b0560 0x7f758c140350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.200 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 --2- 172.21.15.55:0/137636958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9320 0x7f758c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- 172.21.15.55:0/137636958 >> 172.21.15.55:0/137636958 conn(0x7f758c0612d0 msgr2=0x7f758c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- 172.21.15.55:0/137636958 shutdown_connections 2022-08-01T22:55:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- 172.21.15.55:0/137636958 wait complete. 2022-08-01T22:55:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 Processor -- start 2022-08-01T22:55:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- start start 2022-08-01T22:55:34.204 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 0x7f758c0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0b0560 0x7f758c0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 0x7f758c0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.206 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0a9e20 con 0x7f758c0b0560 2022-08-01T22:55:34.206 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0ab8f0 con 0x7f758c0a9320 2022-08-01T22:55:34.207 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759333e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0b4eb0 con 0x7f758c0b0d10 2022-08-01T22:55:34.207 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759233c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 0x7f758c0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.208 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f7592b3d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 0x7f758c0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.208 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f7592b3d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 0x7f758c0ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57608/0 (socket says 172.21.15.55:57608) 2022-08-01T22:55:34.209 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f759233c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 0x7f758c0b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45008/0 (socket says 172.21.15.55:45008) 2022-08-01T22:55:34.209 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.176+0000 7f7592b3d700 1 -- 172.21.15.55:0/2566428418 learned_addr learned my addr 172.21.15.55:0/2566428418 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:34.210 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7591b3b700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0b0560 0x7f758c0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.210 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7592b3d700 1 -- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 msgr2=0x7f758c0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.210 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7592b3d700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 0x7f758c0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.211 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7592b3d700 1 -- 172.21.15.55:0/2566428418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0b0560 msgr2=0x7f758c0abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.211 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7592b3d700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0b0560 0x7f758c0abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.212 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7592b3d700 1 -- 172.21.15.55:0/2566428418 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009470 con 0x7f758c0b0d10 2022-08-01T22:55:34.212 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f759233c700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 0x7f758c0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:34.214 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7591b3b700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0b0560 0x7f758c0abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:34.214 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f7592b3d700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 0x7f758c0ac290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f75840032f0 tx=0x7f7584016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:34.214 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7584021440 con 0x7f758c0b0d10 2022-08-01T22:55:34.214 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840215a0 con 0x7f758c0b0d10 2022-08-01T22:55:34.215 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0b5150 con 0x7f758c0b0d10 2022-08-01T22:55:34.215 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7584021900 con 0x7f758c0b0d10 2022-08-01T22:55:34.215 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c0b37c0 con 0x7f758c0b0d10 2022-08-01T22:55:34.215 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f758400f070 con 0x7f758c0b0d10 2022-08-01T22:55:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f7584020a00 con 0x7f758c0b0d10 2022-08-01T22:55:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.180+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c0b52e0 con 0x7f758c0b0d10 2022-08-01T22:55:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.184+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f7584015020 con 0x7f758c0b0d10 2022-08-01T22:55:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.212+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f7584020bc0 con 0x7f758c0b0d10 2022-08-01T22:55:34.255 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.252+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd new", "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", "id": 6} v 0) v1 -- 0x7f758c0574e0 con 0x7f758c0b0d10 2022-08-01T22:55:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.280+0000 7f757b7fe700 1 -- 172.21.15.55:0/2566428418 <== mon.2 v2:172.21.15.55:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f758401d880 con 0x7f758c0b0d10 2022-08-01T22:55:34.286 INFO:teuthology.orchestra.run.smithi055.stdout:6 2022-08-01T22:55:34.287 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 msgr2=0x7f758c0ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.288 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 0x7f758c0ac290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f75840032f0 tx=0x7f7584016000).stop 2022-08-01T22:55:34.288 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 shutdown_connections 2022-08-01T22:55:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b0d10 0x7f758c0ac290 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0b0560 0x7f758c0abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 --2- 172.21.15.55:0/2566428418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9320 0x7f758c0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.291 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 >> 172.21.15.55:0/2566428418 conn(0x7f758c0612d0 msgr2=0x7f758c0af690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:34.291 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 shutdown_connections 2022-08-01T22:55:34.301 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.284+0000 7f759333e700 1 -- 172.21.15.55:0/2566428418 wait complete. 2022-08-01T22:55:34.303 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd new d1c950ea-35ee-4ac6-8e31-e832328f349f 7 2022-08-01T22:55:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 Processor -- start 2022-08-01T22:55:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 -- start start 2022-08-01T22:55:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7aa40a9320 0x7f7aa41466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7aa40b0d10 0x7f7aa4146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa40b12d0 con 0x7f7aa40b0560 2022-08-01T22:55:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa40a9700 con 0x7f7aa40a9320 2022-08-01T22:55:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa8e0b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa40b1500 con 0x7f7aa40b0d10 2022-08-01T22:55:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7a9bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7aa40b0d10 0x7f7aa4146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa3fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7aa40b0560 0x7f7aa4140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56122/0 (socket says 172.21.15.55:56122) 2022-08-01T22:55:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa3fff700 1 -- 172.21.15.55:0/319143216 learned_addr learned my addr 172.21.15.55:0/319143216 (peer_addr_for_me v1:172.21.15.55:0/0) 2022-08-01T22:55:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.400+0000 7f7aa37fe700 1 --2- 172.21.15.55:0/319143216 >> v2:172.21.15.32:3300/0 conn(0x7f7aa40a9320 0x7f7aa41466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7a9bfff700 1 -- 172.21.15.55:0/319143216 >> v2:172.21.15.32:3300/0 conn(0x7f7aa40a9320 msgr2=0x7f7aa41466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7a9bfff700 1 --2- 172.21.15.55:0/319143216 >> v2:172.21.15.32:3300/0 conn(0x7f7aa40a9320 0x7f7aa41466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7a9bfff700 1 -- 172.21.15.55:0/319143216 >> v1:172.21.15.55:6789/0 conn(0x7f7aa40b0560 legacy=0x7f7aa4140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/319143216 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663680542 0 0) 0x7f7aa40b12d0 con 0x7f7aa40b0560 2022-08-01T22:55:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7a9bfff700 1 -- 172.21.15.55:0/319143216 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa40b17a0 con 0x7f7aa40b0d10 2022-08-01T22:55:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7a9bfff700 1 --2- 172.21.15.55:0/319143216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0d10 0x7f7aa4146bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7a8c002f70 tx=0x7f7a8c006df0).ready entity=mon.1 client_cookie=3423df646dafb955 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/319143216 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8c00d570 con 0x7f7aa40b0d10 2022-08-01T22:55:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/319143216 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c000e90 con 0x7f7aa40b0d10 2022-08-01T22:55:34.409 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/319143216 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8c00d8c0 con 0x7f7aa40b0d10 2022-08-01T22:55:34.412 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/319143216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0d10 msgr2=0x7f7aa4146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.412 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/319143216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0d10 0x7f7aa4146bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7a8c002f70 tx=0x7f7a8c006df0).stop 2022-08-01T22:55:34.413 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/319143216 shutdown_connections 2022-08-01T22:55:34.413 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/319143216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0d10 0x7f7aa4146bd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.413 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/319143216 >> v2:172.21.15.32:3300/0 conn(0x7f7aa40a9320 0x7f7aa41466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.413 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/319143216 >> 172.21.15.55:0/319143216 conn(0x7f7aa40612d0 msgr2=0x7f7aa4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:34.414 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/319143216 shutdown_connections 2022-08-01T22:55:34.414 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/319143216 wait complete. 2022-08-01T22:55:34.414 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 Processor -- start 2022-08-01T22:55:34.414 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- start start 2022-08-01T22:55:34.415 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7aa40a9320 0x7f7aa4145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.415 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 0x7f7aa40b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.415 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa40b0d10 0x7f7aa40b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:34.415 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4152840 con 0x7f7aa40b0d10 2022-08-01T22:55:34.415 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4152ae0 con 0x7f7aa40b0560 2022-08-01T22:55:34.416 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.404+0000 7f7aa8e0b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4152d80 con 0x7f7aa40a9320 2022-08-01T22:55:34.416 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7aa40a9320 0x7f7aa4145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.416 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 0x7f7aa40b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.416 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7aa40a9320 0x7f7aa4145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.55:57646/0 (socket says 172.21.15.55:57646) 2022-08-01T22:55:34.417 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa37fe700 1 -- 172.21.15.55:0/3713364084 learned_addr learned my addr 172.21.15.55:0/3713364084 (peer_addr_for_me v2:172.21.15.55:0/0) 2022-08-01T22:55:34.417 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 0x7f7aa40b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45052/0 (socket says 172.21.15.55:45052) 2022-08-01T22:55:34.417 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa40b0d10 0x7f7aa40b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:34.417 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 -- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7aa40a9320 msgr2=0x7f7aa4145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.417 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7aa40a9320 0x7f7aa4145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.418 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 -- 172.21.15.55:0/3713364084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa40b0d10 msgr2=0x7f7aa40b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.418 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa40b0d10 0x7f7aa40b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.418 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 -- 172.21.15.55:0/3713364084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c00b180 con 0x7f7aa40b0560 2022-08-01T22:55:34.419 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7a9bfff700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 0x7f7aa40b37c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7a8c002f70 tx=0x7f7a8c014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:34.419 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8c013da0 con 0x7f7aa40b0560 2022-08-01T22:55:34.419 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c01e400 con 0x7f7aa40b0560 2022-08-01T22:55:34.419 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa4153020 con 0x7f7aa40b0560 2022-08-01T22:55:34.420 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8c01e760 con 0x7f7aa40b0560 2022-08-01T22:55:34.420 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa4152450 con 0x7f7aa40b0560 2022-08-01T22:55:34.420 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f7a8c025030 con 0x7f7aa40b0560 2022-08-01T22:55:34.420 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa40b1980 con 0x7f7aa40b0560 2022-08-01T22:55:34.420 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.408+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1935+0+0 (secure 0 0 0) 0x7f7a8c03b140 con 0x7f7aa40b0560 2022-08-01T22:55:34.421 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.412+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f7a8c01c770 con 0x7f7aa40b0560 2022-08-01T22:55:34.486 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.480+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", "id": 7} v 0) v1 -- 0x7f7aa41531b0 con 0x7f7aa40b0560 2022-08-01T22:55:34.499 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.492+0000 7f7aa17fa700 1 -- 172.21.15.55:0/3713364084 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7a8c023020 con 0x7f7aa40b0560 2022-08-01T22:55:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:7 2022-08-01T22:55:34.503 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 msgr2=0x7f7aa40b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:34.503 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 0x7f7aa40b37c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7a8c002f70 tx=0x7f7a8c014000).stop 2022-08-01T22:55:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 shutdown_connections 2022-08-01T22:55:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa40b0d10 0x7f7aa40b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b0560 0x7f7aa40b37c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 --2- 172.21.15.55:0/3713364084 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7aa40a9320 0x7f7aa4145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 >> 172.21.15.55:0/3713364084 conn(0x7f7aa40612d0 msgr2=0x7f7aa40b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:34.506 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 shutdown_connections 2022-08-01T22:55:34.506 INFO:teuthology.orchestra.run.smithi055.stderr:2022-08-01T22:55:34.496+0000 7f7aa8e0b700 1 -- 172.21.15.55:0/3713364084 wait complete. 2022-08-01T22:55:34.525 INFO:tasks.ceph.osd.0:Restarting daemon 2022-08-01T22:55:34.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-08-01T22:55:34.545 INFO:tasks.ceph.osd.0:Started 2022-08-01T22:55:34.545 INFO:tasks.ceph.osd.1:Restarting daemon 2022-08-01T22:55:34.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-08-01T22:55:34.551 INFO:tasks.ceph.osd.1:Started 2022-08-01T22:55:34.552 INFO:tasks.ceph.osd.2:Restarting daemon 2022-08-01T22:55:34.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-08-01T22:55:34.557 INFO:tasks.ceph.osd.2:Started 2022-08-01T22:55:34.557 INFO:tasks.ceph.osd.3:Restarting daemon 2022-08-01T22:55:34.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-08-01T22:55:34.564 INFO:tasks.ceph.osd.3:Started 2022-08-01T22:55:34.564 INFO:tasks.ceph.osd.4:Restarting daemon 2022-08-01T22:55:34.565 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-08-01T22:55:34.568 INFO:tasks.ceph.osd.4:Started 2022-08-01T22:55:34.569 INFO:tasks.ceph.osd.5:Restarting daemon 2022-08-01T22:55:34.569 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-08-01T22:55:34.572 INFO:tasks.ceph.osd.5:Started 2022-08-01T22:55:34.573 INFO:tasks.ceph.osd.6:Restarting daemon 2022-08-01T22:55:34.573 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-08-01T22:55:34.577 INFO:tasks.ceph.osd.6:Started 2022-08-01T22:55:34.578 INFO:tasks.ceph.osd.7:Restarting daemon 2022-08-01T22:55:34.588 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-08-01T22:55:34.599 INFO:tasks.ceph.osd.7:Started 2022-08-01T22:55:34.600 INFO:tasks.ceph:Waiting for OSDs to come up 2022-08-01T22:55:34.905 INFO:tasks.ceph.osd.1.smithi032.stderr:2022-08-01T22:55:34.905+0000 7f36c2f08d80 -1 Falling back to public interface 2022-08-01T22:55:34.958 INFO:tasks.ceph.osd.5.smithi055.stderr:2022-08-01T22:55:34.956+0000 7f464f637d80 -1 Falling back to public interface 2022-08-01T22:55:35.158 INFO:tasks.ceph.osd.2.smithi032.stderr:2022-08-01T22:55:35.157+0000 7ff302f0ad80 -1 Falling back to public interface 2022-08-01T22:55:35.190 INFO:tasks.ceph.osd.6.smithi055.stderr:2022-08-01T22:55:35.188+0000 7fd8f7d07d80 -1 Falling back to public interface 2022-08-01T22:55:35.197 INFO:tasks.ceph.osd.3.smithi032.stderr:2022-08-01T22:55:35.197+0000 7f1a4eb9fd80 -1 Falling back to public interface 2022-08-01T22:55:35.206 INFO:tasks.ceph.osd.0.smithi032.stderr:2022-08-01T22:55:35.205+0000 7fa2297fad80 -1 Falling back to public interface 2022-08-01T22:55:35.227 INFO:tasks.ceph.osd.4.smithi055.stderr:2022-08-01T22:55:35.224+0000 7efc18236d80 -1 Falling back to public interface 2022-08-01T22:55:35.229 INFO:tasks.ceph.osd.7.smithi055.stderr:2022-08-01T22:55:35.224+0000 7f20ae9a0d80 -1 Falling back to public interface 2022-08-01T22:55:35.408 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:35.526 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.525+0000 7fdb0a5b2700 1 Processor -- start 2022-08-01T22:55:35.526 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.525+0000 7fdb0a5b2700 1 -- start start 2022-08-01T22:55:35.543 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb040b07f0 0x7fdb0405f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:35.543 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b1520 0x7fdb040af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:35.544 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb040b64a0 con 0x7fdb040b60c0 2022-08-01T22:55:35.544 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb040b0c90 con 0x7fdb040b07f0 2022-08-01T22:55:35.544 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb040b0f30 con 0x7fdb040b1520 2022-08-01T22:55:35.544 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb040b07f0 0x7fdb0405f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:35.545 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb08b4f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b1520 0x7fdb040af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:35.545 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb040b07f0 0x7fdb0405f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39966/0 (socket says 172.21.15.32:39966) 2022-08-01T22:55:35.545 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 -- 172.21.15.32:0/2359298522 learned_addr learned my addr 172.21.15.32:0/2359298522 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:35.545 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb02ffd700 1 -- 172.21.15.32:0/2359298522 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69035629 0 0) 0x7fdb040b64a0 con 0x7fdb040b60c0 2022-08-01T22:55:35.546 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 -- 172.21.15.32:0/2359298522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b1520 msgr2=0x7fdb040af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:35.546 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 --2- 172.21.15.32:0/2359298522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b1520 0x7fdb040af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.546 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 -- 172.21.15.32:0/2359298522 >> v1:172.21.15.55:6789/0 conn(0x7fdb040b60c0 legacy=0x7fdb04148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:35.546 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 -- 172.21.15.32:0/2359298522 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb04147910 con 0x7fdb040b07f0 2022-08-01T22:55:35.547 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb08b4f700 1 --2- 172.21.15.32:0/2359298522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b1520 0x7fdb040af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:35.547 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb037fe700 1 --2- 172.21.15.32:0/2359298522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b07f0 0x7fdb0405f780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fdaf8002f70 tx=0x7fdaf8006eb0).ready entity=mon.0 client_cookie=593afaf2797803b9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:35.547 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb02ffd700 1 -- 172.21.15.32:0/2359298522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf800d570 con 0x7fdb040b07f0 2022-08-01T22:55:35.547 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb02ffd700 1 -- 172.21.15.32:0/2359298522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8000e90 con 0x7fdb040b07f0 2022-08-01T22:55:35.548 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb02ffd700 1 -- 172.21.15.32:0/2359298522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf800d8c0 con 0x7fdb040b07f0 2022-08-01T22:55:35.548 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/2359298522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b07f0 msgr2=0x7fdb0405f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:35.548 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/2359298522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b07f0 0x7fdb0405f780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fdaf8002f70 tx=0x7fdaf8006eb0).stop 2022-08-01T22:55:35.548 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/2359298522 shutdown_connections 2022-08-01T22:55:35.548 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/2359298522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b07f0 0x7fdb0405f780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.549 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/2359298522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b1520 0x7fdb040af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.549 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.529+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/2359298522 >> 172.21.15.32:0/2359298522 conn(0x7fdb04063250 msgr2=0x7fdb04063660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:35.549 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/2359298522 shutdown_connections 2022-08-01T22:55:35.549 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/2359298522 wait complete. 2022-08-01T22:55:35.552 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 Processor -- start 2022-08-01T22:55:35.553 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- start start 2022-08-01T22:55:35.553 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb040b07f0 0x7fdb04157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:35.554 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 0x7fdb04064d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:35.554 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b60c0 0x7fdb040674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:35.554 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04064900 con 0x7fdb040b1520 2022-08-01T22:55:35.555 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04064b30 con 0x7fdb040b07f0 2022-08-01T22:55:35.555 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04063f10 con 0x7fdb040b60c0 2022-08-01T22:55:35.556 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 0x7fdb04064d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:35.556 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 0x7fdb04064d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39970/0 (socket says 172.21.15.32:39970) 2022-08-01T22:55:35.556 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 -- 172.21.15.32:0/1295490394 learned_addr learned my addr 172.21.15.32:0/1295490394 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:35.557 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb08b4f700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b60c0 0x7fdb040674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:35.557 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb03fff700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb040b07f0 0x7fdb04157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:35.558 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 -- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b60c0 msgr2=0x7fdb040674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:35.558 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b60c0 0x7fdb040674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.559 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 -- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb040b07f0 msgr2=0x7fdb04157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:35.559 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb040b07f0 0x7fdb04157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.560 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 -- 172.21.15.32:0/1295490394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009470 con 0x7fdb040b1520 2022-08-01T22:55:35.560 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb037fe700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 0x7fdb04064d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fdaf8002f70 tx=0x7fdaf8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:35.561 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf8011990 con 0x7fdb040b1520 2022-08-01T22:55:35.561 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8011af0 con 0x7fdb040b1520 2022-08-01T22:55:35.561 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf800b1e0 con 0x7fdb040b1520 2022-08-01T22:55:35.562 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf8011e50 con 0x7fdb040b1520 2022-08-01T22:55:35.562 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.533+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb04065bf0 con 0x7fdb040b1520 2022-08-01T22:55:35.563 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.537+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7fdaf8025030 con 0x7fdb040b1520 2022-08-01T22:55:35.563 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.537+0000 7fdb017fa700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdae40538b0 0x7fdae4055d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:35.563 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.537+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7fdaf806bec0 con 0x7fdb040b1520 2022-08-01T22:55:35.565 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.537+0000 7fdb03fff700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdae40538b0 0x7fdae4055d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:35.565 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.537+0000 7fdb03fff700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdae40538b0 0x7fdae4055d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdaf400b000 tx=0x7fdaf4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:35.565 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.545+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb040ad1a0 con 0x7fdb040b1520 2022-08-01T22:55:35.567 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.549+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdaf8023020 con 0x7fdb040b1520 2022-08-01T22:55:35.714 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.713+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb04148760 con 0x7fdb040b1520 2022-08-01T22:55:35.715 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.713+0000 7fdb017fa700 1 -- 172.21.15.32:0/1295490394 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8275 (secure 0 0 0) 0x7fdaf8047890 con 0x7fdb040b1520 2022-08-01T22:55:35.715 INFO:teuthology.misc.health.smithi032.stdout: 2022-08-01T22:55:35.716 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:34.488346+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:35.717 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdae40538b0 msgr2=0x7fdae4055d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:35.717 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdae40538b0 0x7fdae4055d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdaf400b000 tx=0x7fdaf4009040).stop 2022-08-01T22:55:35.717 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 msgr2=0x7fdb04064d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:35.719 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 0x7fdb04064d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fdaf8002f70 tx=0x7fdaf8015000).stop 2022-08-01T22:55:35.719 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 shutdown_connections 2022-08-01T22:55:35.720 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdae40538b0 0x7fdae4055d70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.720 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb040b60c0 0x7fdb040674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.720 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb040b1520 0x7fdb04064d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.720 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 --2- 172.21.15.32:0/1295490394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb040b07f0 0x7fdb04157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:35.721 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 >> 172.21.15.32:0/1295490394 conn(0x7fdb04063250 msgr2=0x7fdb041462e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:35.722 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 shutdown_connections 2022-08-01T22:55:35.722 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:35.717+0000 7fdb0a5b2700 1 -- 172.21.15.32:0/1295490394 wait complete. 2022-08-01T22:55:35.726 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-08-01T22:55:36.115 INFO:tasks.ceph.osd.1.smithi032.stderr:2022-08-01T22:55:36.113+0000 7f36c2f08d80 -1 osd.1 0 log_to_monitors {default=true} 2022-08-01T22:55:36.147 INFO:tasks.ceph.osd.5.smithi055.stderr:2022-08-01T22:55:36.144+0000 7f464f637d80 -1 osd.5 0 log_to_monitors {default=true} 2022-08-01T22:55:36.322 INFO:tasks.ceph.osd.2.smithi032.stderr:2022-08-01T22:55:36.321+0000 7ff302f0ad80 -1 osd.2 0 log_to_monitors {default=true} 2022-08-01T22:55:36.349 INFO:tasks.ceph.osd.6.smithi055.stderr:2022-08-01T22:55:36.348+0000 7fd8f7d07d80 -1 osd.6 0 log_to_monitors {default=true} 2022-08-01T22:55:36.400 INFO:tasks.ceph.osd.4.smithi055.stderr:2022-08-01T22:55:36.396+0000 7efc18236d80 -1 osd.4 0 log_to_monitors {default=true} 2022-08-01T22:55:36.403 INFO:tasks.ceph.osd.3.smithi032.stderr:2022-08-01T22:55:36.401+0000 7f1a4eb9fd80 -1 osd.3 0 log_to_monitors {default=true} 2022-08-01T22:55:36.424 INFO:tasks.ceph.osd.0.smithi032.stderr:2022-08-01T22:55:36.425+0000 7fa2297fad80 -1 osd.0 0 log_to_monitors {default=true} 2022-08-01T22:55:36.427 INFO:tasks.ceph.osd.7.smithi055.stderr:2022-08-01T22:55:36.420+0000 7f20ae9a0d80 -1 osd.7 0 log_to_monitors {default=true} 2022-08-01T22:55:38.080 INFO:tasks.ceph.osd.1.smithi032.stderr:2022-08-01T22:55:38.077+0000 7f36b4fb2700 -1 osd.1 0 waiting for initial osdmap 2022-08-01T22:55:38.081 INFO:tasks.ceph.osd.2.smithi032.stderr:2022-08-01T22:55:38.077+0000 7ff2f4fb4700 -1 osd.2 0 waiting for initial osdmap 2022-08-01T22:55:38.085 INFO:tasks.ceph.osd.7.smithi055.stderr:2022-08-01T22:55:38.084+0000 7f20a0a4a700 -1 osd.7 0 waiting for initial osdmap 2022-08-01T22:55:38.086 INFO:tasks.ceph.osd.3.smithi032.stderr:2022-08-01T22:55:38.085+0000 7f1a40c49700 -1 osd.3 0 waiting for initial osdmap 2022-08-01T22:55:38.087 INFO:tasks.ceph.osd.1.smithi032.stderr:2022-08-01T22:55:38.089+0000 7f36bc42c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.088 INFO:tasks.ceph.osd.0.smithi032.stderr:2022-08-01T22:55:38.089+0000 7fa21b8a4700 -1 osd.0 0 waiting for initial osdmap 2022-08-01T22:55:38.089 INFO:tasks.ceph.osd.2.smithi032.stderr:2022-08-01T22:55:38.089+0000 7ff2fc42e700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.090 INFO:tasks.ceph.osd.4.smithi055.stderr:2022-08-01T22:55:38.084+0000 7efc0a2e0700 -1 osd.4 0 waiting for initial osdmap 2022-08-01T22:55:38.091 INFO:tasks.ceph.osd.6.smithi055.stderr:2022-08-01T22:55:38.088+0000 7fd8e9db1700 -1 osd.6 0 waiting for initial osdmap 2022-08-01T22:55:38.092 INFO:tasks.ceph.osd.5.smithi055.stderr:2022-08-01T22:55:38.088+0000 7f46416e1700 -1 osd.5 0 waiting for initial osdmap 2022-08-01T22:55:38.093 INFO:tasks.ceph.osd.3.smithi032.stderr:2022-08-01T22:55:38.093+0000 7f1a480c3700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.094 INFO:tasks.ceph.osd.0.smithi032.stderr:2022-08-01T22:55:38.093+0000 7fa222d1e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.104 INFO:tasks.ceph.osd.7.smithi055.stderr:2022-08-01T22:55:38.100+0000 7f20a7ec4700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.105 INFO:tasks.ceph.osd.4.smithi055.stderr:2022-08-01T22:55:38.104+0000 7efc1175a700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.106 INFO:tasks.ceph.osd.6.smithi055.stderr:2022-08-01T22:55:38.104+0000 7fd8f122b700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:38.107 INFO:tasks.ceph.osd.5.smithi055.stderr:2022-08-01T22:55:38.104+0000 7f4648b5b700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T22:55:42.531 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:42.634 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 Processor -- start 2022-08-01T22:55:42.634 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 -- start start 2022-08-01T22:55:42.635 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f340c05fac0 0x7f340c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.635 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c060bd0 0x7f340c05c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.635 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0b0c20 con 0x7f340c0603a0 2022-08-01T22:55:42.635 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c0b08a0 con 0x7f340c05fac0 2022-08-01T22:55:42.636 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0aa560 con 0x7f340c060bd0 2022-08-01T22:55:42.636 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f340c05fac0 0x7f340c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.636 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f340c05fac0 0x7f340c0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40188/0 (socket says 172.21.15.32:40188) 2022-08-01T22:55:42.636 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 -- 172.21.15.32:0/899327638 learned_addr learned my addr 172.21.15.32:0/899327638 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:42.637 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340b7fe700 1 --2- 172.21.15.32:0/899327638 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c060bd0 0x7f340c05c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.637 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 -- 172.21.15.32:0/899327638 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c060bd0 msgr2=0x7f340c05c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:42.637 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 --2- 172.21.15.32:0/899327638 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c060bd0 0x7f340c05c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.639 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 -- 172.21.15.32:0/899327638 >> v1:172.21.15.55:6789/0 conn(0x7f340c0603a0 legacy=0x7f340c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.640 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 -- 172.21.15.32:0/899327638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0a9c70 con 0x7f340c05fac0 2022-08-01T22:55:42.640 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f340affd700 1 --2- 172.21.15.32:0/899327638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c05fac0 0x7f340c0b1700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f340c0b1c10 tx=0x7f33f4006e50).ready entity=mon.0 client_cookie=dfefddbf79d28e0c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:42.640 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f3409ffb700 1 -- 172.21.15.32:0/899327638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33f400d5c0 con 0x7f340c05fac0 2022-08-01T22:55:42.640 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f3409ffb700 1 -- 172.21.15.32:0/899327638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4000e90 con 0x7f340c05fac0 2022-08-01T22:55:42.641 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.633+0000 7f3409ffb700 1 -- 172.21.15.32:0/899327638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33f400d910 con 0x7f340c05fac0 2022-08-01T22:55:42.646 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 -- 172.21.15.32:0/899327638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c05fac0 msgr2=0x7f340c0b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.646 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 --2- 172.21.15.32:0/899327638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c05fac0 0x7f340c0b1700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f340c0b1c10 tx=0x7f33f4006e50).stop 2022-08-01T22:55:42.646 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 -- 172.21.15.32:0/899327638 shutdown_connections 2022-08-01T22:55:42.646 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 --2- 172.21.15.32:0/899327638 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c060bd0 0x7f340c05c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.647 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 --2- 172.21.15.32:0/899327638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c05fac0 0x7f340c0b1700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.647 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 -- 172.21.15.32:0/899327638 >> 172.21.15.32:0/899327638 conn(0x7f340c061670 msgr2=0x7f340c062ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:42.647 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 -- 172.21.15.32:0/899327638 shutdown_connections 2022-08-01T22:55:42.647 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 -- 172.21.15.32:0/899327638 wait complete. 2022-08-01T22:55:42.648 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 Processor -- start 2022-08-01T22:55:42.648 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.637+0000 7f340bfff700 1 -- start start 2022-08-01T22:55:42.649 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f340c05fac0 0x7f340c13e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.649 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c0603a0 0x7f340c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.649 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 0x7f340c150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.649 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c140330 con 0x7f340c060bd0 2022-08-01T22:55:42.650 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c150fb0 con 0x7f340c05fac0 2022-08-01T22:55:42.650 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c1548b0 con 0x7f340c0603a0 2022-08-01T22:55:42.650 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 0x7f340c150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.650 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 0x7f340c150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40198/0 (socket says 172.21.15.32:40198) 2022-08-01T22:55:42.650 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 -- 172.21.15.32:0/4014592498 learned_addr learned my addr 172.21.15.32:0/4014592498 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:42.651 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340affd700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f340c05fac0 0x7f340c13e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.651 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340a7fc700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c0603a0 0x7f340c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.651 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 -- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c0603a0 msgr2=0x7f340c1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.660 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c0603a0 0x7f340c1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.661 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 -- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f340c05fac0 msgr2=0x7f340c13e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.661 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f340c05fac0 0x7f340c13e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.661 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 -- 172.21.15.32:0/4014592498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f400b1e0 con 0x7f340c060bd0 2022-08-01T22:55:42.661 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340b7fe700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 0x7f340c150ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f33fc00a000 tx=0x7f33fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:42.662 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33fc012610 con 0x7f340c060bd0 2022-08-01T22:55:42.662 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc012770 con 0x7f340c060bd0 2022-08-01T22:55:42.662 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c140560 con 0x7f340c060bd0 2022-08-01T22:55:42.663 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33fc012ad0 con 0x7f340c060bd0 2022-08-01T22:55:42.663 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c13e9a0 con 0x7f340c060bd0 2022-08-01T22:55:42.663 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c0a9f10 con 0x7f340c060bd0 2022-08-01T22:55:42.663 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.641+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33fc01b410 con 0x7f340c060bd0 2022-08-01T22:55:42.664 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.645+0000 7f3403fff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33f0053b00 0x7f33f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.664 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.645+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f33fc06ba80 con 0x7f340c060bd0 2022-08-01T22:55:42.664 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.645+0000 7f340affd700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33f0053b00 0x7f33f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.664 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.645+0000 7f340affd700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33f0053b00 0x7f33f0055fc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f33f401c000 tx=0x7f33f4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:42.674 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.649+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33fc042cc0 con 0x7f340c060bd0 2022-08-01T22:55:42.781 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.777+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f340c0574e0 con 0x7f340c060bd0 2022-08-01T22:55:42.782 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f3403fff700 1 -- 172.21.15.32:0/4014592498 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11805 (secure 0 0 0) 0x7f33fc0462e0 con 0x7f340c060bd0 2022-08-01T22:55:42.783 INFO:teuthology.misc.health.smithi032.stdout: 2022-08-01T22:55:42.783 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":15,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:41.104139+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:42.786 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33f0053b00 msgr2=0x7f33f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.787 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33f0053b00 0x7f33f0055fc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f33f401c000 tx=0x7f33f4015000).stop 2022-08-01T22:55:42.787 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 msgr2=0x7f340c150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.788 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 0x7f340c150ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f33fc00a000 tx=0x7f33fc008040).stop 2022-08-01T22:55:42.788 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 shutdown_connections 2022-08-01T22:55:42.790 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33f0053b00 0x7f33f0055fc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.791 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f340c060bd0 0x7f340c150ac0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.791 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f340c0603a0 0x7f340c1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.792 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 --2- 172.21.15.32:0/4014592498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f340c05fac0 0x7f340c13e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.792 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 >> 172.21.15.32:0/4014592498 conn(0x7f340c061670 msgr2=0x7f340c13cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:42.792 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 shutdown_connections 2022-08-01T22:55:42.793 INFO:teuthology.misc.health.smithi032.stderr:2022-08-01T22:55:42.781+0000 7f340bfff700 1 -- 172.21.15.32:0/4014592498 wait complete. 2022-08-01T22:55:42.806 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-08-01T22:55:42.807 INFO:tasks.ceph:Creating RBD pool 2022-08-01T22:55:42.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool create rbd 8 2022-08-01T22:55:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70e4ce1700 1 Processor -- start 2022-08-01T22:55:42.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70e4ce1700 1 -- start start 2022-08-01T22:55:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70e4ce1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70e00a9da0 0x7f70e00b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70e4ce1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e0143ab0 con 0x7f70e00b0740 2022-08-01T22:55:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70e4ce1700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e0143d50 con 0x7f70e00b38a0 2022-08-01T22:55:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70e4ce1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e0143ff0 con 0x7f70e00a9da0 2022-08-01T22:55:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70de59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f70e00b0740 0x7f70e013d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55976/0 (socket says 172.21.15.32:55976) 2022-08-01T22:55:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70de59c700 1 -- 172.21.15.32:0/2364767783 learned_addr learned my addr 172.21.15.32:0/2364767783 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70ded9d700 1 --2- 172.21.15.32:0/2364767783 >> v2:172.21.15.32:3300/0 conn(0x7f70e00a9da0 0x7f70e00b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70dd59a700 1 -- 172.21.15.32:0/2364767783 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196614848 0 0) 0x7f70e0143ab0 con 0x7f70e00b0740 2022-08-01T22:55:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70dd59a700 1 -- 172.21.15.32:0/2364767783 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c4003260 con 0x7f70e00b0740 2022-08-01T22:55:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70ded9d700 1 -- 172.21.15.32:0/2364767783 >> v1:172.21.15.55:6789/0 conn(0x7f70e00b38a0 legacy=0x7f70e00b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70ded9d700 1 -- 172.21.15.32:0/2364767783 >> v1:172.21.15.32:6789/0 conn(0x7f70e00b0740 legacy=0x7f70e013d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70ded9d700 1 -- 172.21.15.32:0/2364767783 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0144290 con 0x7f70e00a9da0 2022-08-01T22:55:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70ded9d700 1 --2- 172.21.15.32:0/2364767783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00a9da0 0x7f70e00b0b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f70d80032f0 tx=0x7f70d80091d0).ready entity=mon.0 client_cookie=3e4b293277cf834b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70dd59a700 1 -- 172.21.15.32:0/2364767783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d800b6c0 con 0x7f70e00a9da0 2022-08-01T22:55:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70dd59a700 1 -- 172.21.15.32:0/2364767783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d800b820 con 0x7f70e00a9da0 2022-08-01T22:55:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.917+0000 7f70dd59a700 1 -- 172.21.15.32:0/2364767783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d800b990 con 0x7f70e00a9da0 2022-08-01T22:55:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- 172.21.15.32:0/2364767783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00a9da0 msgr2=0x7f70e00b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/2364767783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00a9da0 0x7f70e00b0b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f70d80032f0 tx=0x7f70d80091d0).stop 2022-08-01T22:55:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- 172.21.15.32:0/2364767783 shutdown_connections 2022-08-01T22:55:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/2364767783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00a9da0 0x7f70e00b0b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- 172.21.15.32:0/2364767783 >> 172.21.15.32:0/2364767783 conn(0x7f70e0060950 msgr2=0x7f70e0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- 172.21.15.32:0/2364767783 shutdown_connections 2022-08-01T22:55:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- 172.21.15.32:0/2364767783 wait complete. 2022-08-01T22:55:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 Processor -- start 2022-08-01T22:55:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- start start 2022-08-01T22:55:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70e00a9da0 0x7f70e0063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 0x7f70e00ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70e00b38a0 0x7f70e014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00aca00 con 0x7f70e00b0740 2022-08-01T22:55:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00ad190 con 0x7f70e00b38a0 2022-08-01T22:55:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70e4ce1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00ad430 con 0x7f70e00a9da0 2022-08-01T22:55:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70ddd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 0x7f70e00ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70ddd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 0x7f70e00ad8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40210/0 (socket says 172.21.15.32:40210) 2022-08-01T22:55:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70ddd9b700 1 -- 172.21.15.32:0/121370385 learned_addr learned my addr 172.21.15.32:0/121370385 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70de59c700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70e00a9da0 0x7f70e0063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.921+0000 7f70ded9d700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70e00b38a0 0x7f70e014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70ddd9b700 1 -- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70e00a9da0 msgr2=0x7f70e0063da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70ddd9b700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70e00a9da0 0x7f70e0063da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70ddd9b700 1 -- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70e00b38a0 msgr2=0x7f70e014e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70ddd9b700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70e00b38a0 0x7f70e014e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70ddd9b700 1 -- 172.21.15.32:0/121370385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009470 con 0x7f70e00b0740 2022-08-01T22:55:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70ddd9b700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 0x7f70e00ad8d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f70e006ac20 tx=0x7f70d4007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d4007e40 con 0x7f70e00b0740 2022-08-01T22:55:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4007030 con 0x7f70e00b0740 2022-08-01T22:55:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e0065e70 con 0x7f70e00b0740 2022-08-01T22:55:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d4007390 con 0x7f70e00b0740 2022-08-01T22:55:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e014e9b0 con 0x7f70e00b0740 2022-08-01T22:55:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00ad5f0 con 0x7f70e00b0740 2022-08-01T22:55:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f70d4010030 con 0x7f70e00b0740 2022-08-01T22:55:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70cf7fe700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f70bc053a70 0x7f70bc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f70d406b480 con 0x7f70e00b0740 2022-08-01T22:55:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.925+0000 7f70de59c700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f70bc053a70 0x7f70bc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.929+0000 7f70de59c700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f70bc053a70 0x7f70bc055f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f70d000d000 tx=0x7f70d0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:42.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:42.937+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70d4042830 con 0x7f70e00b0740 2022-08-01T22:55:43.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.061+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f70e00acbd0 con 0x7f70e00b0740 2022-08-01T22:55:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.173+0000 7f70cf7fe700 1 -- 172.21.15.32:0/121370385 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f70d4045e50 con 0x7f70e00b0740 2022-08-01T22:55:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f70bc053a70 msgr2=0x7f70bc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f70bc053a70 0x7f70bc055f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f70d000d000 tx=0x7f70d0009040).stop 2022-08-01T22:55:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 msgr2=0x7f70e00ad8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 0x7f70e00ad8d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f70e006ac20 tx=0x7f70d4007bd0).stop 2022-08-01T22:55:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 shutdown_connections 2022-08-01T22:55:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f70bc053a70 0x7f70bc055f30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70e00b38a0 0x7f70e014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70e00b0740 0x7f70e00ad8d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 --2- 172.21.15.32:0/121370385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70e00a9da0 0x7f70e0063da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 >> 172.21.15.32:0/121370385 conn(0x7f70e0060950 msgr2=0x7f70e013ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 shutdown_connections 2022-08-01T22:55:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.181+0000 7f70e4ce1700 1 -- 172.21.15.32:0/121370385 wait complete. 2022-08-01T22:55:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'rbd' created 2022-08-01T22:55:43.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-08-01T22:55:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5429f8c700 1 Processor -- start 2022-08-01T22:55:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5429f8c700 1 -- start start 2022-08-01T22:55:43.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5429f8c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54240b29e0 0x7f54240b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:43.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5429f8c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240b2dc0 con 0x7f542413c870 2022-08-01T22:55:43.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5429f8c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240b17f0 con 0x7f54240ad0b0 2022-08-01T22:55:43.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5429f8c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542413d3c0 con 0x7f54240b29e0 2022-08-01T22:55:43.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5428f8a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54240b29e0 0x7f54240b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:43.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5428f8a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54240b29e0 0x7f54240b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40216/0 (socket says 172.21.15.32:40216) 2022-08-01T22:55:43.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5428f8a700 1 -- 172.21.15.32:0/1723252504 learned_addr learned my addr 172.21.15.32:0/1723252504 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5423fff700 1 --1- 172.21.15.32:0/1723252504 >> v1:172.21.15.32:6789/0 conn(0x7f542413c870 0x7f542413ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55992/0 (socket says 172.21.15.32:55992) 2022-08-01T22:55:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5428f8a700 1 -- 172.21.15.32:0/1723252504 >> v1:172.21.15.55:6789/0 conn(0x7f54240ad0b0 legacy=0x7f54240ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5428f8a700 1 -- 172.21.15.32:0/1723252504 >> v1:172.21.15.32:6789/0 conn(0x7f542413c870 legacy=0x7f542413ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.297+0000 7f5428f8a700 1 -- 172.21.15.32:0/1723252504 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240b1a90 con 0x7f54240b29e0 2022-08-01T22:55:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5428f8a700 1 --2- 172.21.15.32:0/1723252504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54240b29e0 0x7f54240b0c30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f5418002f70 tx=0x7f5418006eb0).ready entity=mon.0 client_cookie=de03f103b17c2b61 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:43.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f54237fe700 1 -- 172.21.15.32:0/1723252504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541800d660 con 0x7f54240b29e0 2022-08-01T22:55:43.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f54237fe700 1 -- 172.21.15.32:0/1723252504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418000e90 con 0x7f54240b29e0 2022-08-01T22:55:43.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f54237fe700 1 -- 172.21.15.32:0/1723252504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541800d9f0 con 0x7f54240b29e0 2022-08-01T22:55:43.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 -- 172.21.15.32:0/1723252504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54240b29e0 msgr2=0x7f54240b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 --2- 172.21.15.32:0/1723252504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54240b29e0 0x7f54240b0c30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f5418002f70 tx=0x7f5418006eb0).stop 2022-08-01T22:55:43.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 -- 172.21.15.32:0/1723252504 shutdown_connections 2022-08-01T22:55:43.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 --2- 172.21.15.32:0/1723252504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54240b29e0 0x7f54240b0c30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 -- 172.21.15.32:0/1723252504 >> 172.21.15.32:0/1723252504 conn(0x7f54240613c0 msgr2=0x7f5424062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:43.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 -- 172.21.15.32:0/1723252504 shutdown_connections 2022-08-01T22:55:43.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 -- 172.21.15.32:0/1723252504 wait complete. 2022-08-01T22:55:43.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 Processor -- start 2022-08-01T22:55:43.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.301+0000 7f5429f8c700 1 -- start start 2022-08-01T22:55:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54240ad0b0 0x7f542413ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54240b29e0 0x7f5424141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 0x7f542414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542414f490 con 0x7f542413c870 2022-08-01T22:55:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424143930 con 0x7f54240ad0b0 2022-08-01T22:55:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5424143380 con 0x7f54240b29e0 2022-08-01T22:55:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 0x7f542414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:43.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 0x7f542414f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40226/0 (socket says 172.21.15.32:40226) 2022-08-01T22:55:43.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 -- 172.21.15.32:0/4288754933 learned_addr learned my addr 172.21.15.32:0/4288754933 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:43.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5423fff700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54240b29e0 0x7f5424141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5428f8a700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54240ad0b0 0x7f542413ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 -- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54240b29e0 msgr2=0x7f5424141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54240b29e0 0x7f5424141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 -- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54240ad0b0 msgr2=0x7f542413ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54240ad0b0 0x7f542413ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 -- 172.21.15.32:0/4288754933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541800b1e0 con 0x7f542413c870 2022-08-01T22:55:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f542978b700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 0x7f542414f880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5424063770 tx=0x7f541c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c00f470 con 0x7f542413c870 2022-08-01T22:55:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00f5d0 con 0x7f542413c870 2022-08-01T22:55:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5424153570 con 0x7f542413c870 2022-08-01T22:55:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c00f930 con 0x7f542413c870 2022-08-01T22:55:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.305+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54241421e0 con 0x7f542413c870 2022-08-01T22:55:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.309+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5424141910 con 0x7f542413c870 2022-08-01T22:55:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.309+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f541c019030 con 0x7f542413c870 2022-08-01T22:55:43.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.309+0000 7f5421ffb700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5404053b00 0x7f5404055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:43.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.309+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f541c06b270 con 0x7f542413c870 2022-08-01T22:55:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.309+0000 7f5428f8a700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5404053b00 0x7f5404055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.309+0000 7f5428f8a700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5404053b00 0x7f5404055fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f541801c000 tx=0x7f5418015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.317+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f541c042400 con 0x7f542413c870 2022-08-01T22:55:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:43.445+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f54240574e0 con 0x7f542413c870 2022-08-01T22:55:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.177+0000 7f5421ffb700 1 -- 172.21.15.32:0/4288754933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f541c045a20 con 0x7f542413c870 2022-08-01T22:55:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5404053b00 msgr2=0x7f5404055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5404053b00 0x7f5404055fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f541801c000 tx=0x7f5418015000).stop 2022-08-01T22:55:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 msgr2=0x7f542414f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 0x7f542414f880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5424063770 tx=0x7f541c007f40).stop 2022-08-01T22:55:44.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 shutdown_connections 2022-08-01T22:55:44.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5404053b00 0x7f5404055fc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f542413c870 0x7f542414f880 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54240b29e0 0x7f5424141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 --2- 172.21.15.32:0/4288754933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54240ad0b0 0x7f542413ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 >> 172.21.15.32:0/4288754933 conn(0x7f54240613c0 msgr2=0x7f54240ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 shutdown_connections 2022-08-01T22:55:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.185+0000 7f5429f8c700 1 -- 172.21.15.32:0/4288754933 wait complete. 2022-08-01T22:55:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:enabled application 'rbd' on pool 'rbd' 2022-08-01T22:55:44.199 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-08-01T22:55:44.199 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T22:55:44.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T22:55:44.202 INFO:tasks.ceph.mds.a:Started 2022-08-01T22:55:44.203 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T22:55:44.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T22:55:44.206 INFO:tasks.ceph.mds.c:Started 2022-08-01T22:55:44.207 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T22:55:44.207 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T22:55:44.210 INFO:tasks.ceph.mds.b:Started 2022-08-01T22:55:44.211 INFO:tasks.ceph:Setting up CephFS filesystem... 2022-08-01T22:55:44.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:44.268 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T22:55:44.270 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T22:55:44.286 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T22:55:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 Processor -- start 2022-08-01T22:55:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 -- start start 2022-08-01T22:55:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdc480ad4b0 0x7fdc480af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc480a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc480aff30 con 0x7fdc480af450 2022-08-01T22:55:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4805c7a0 con 0x7fdc480ad4b0 2022-08-01T22:55:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4ddd1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48140990 con 0x7fdc480ad9e0 2022-08-01T22:55:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc480a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdc480ad4b0 0x7fdc480af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdc480ad4b0 0x7fdc480af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:50050/0 (socket says 172.21.15.32:50050) 2022-08-01T22:55:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc480a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52004/0 (socket says 172.21.15.32:52004) 2022-08-01T22:55:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4cdcf700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdc480af450 0x7fdc481432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:48710/0 (socket says 172.21.15.32:48710) 2022-08-01T22:55:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3ffff700 1 -- 172.21.15.32:0/1508840460 learned_addr learned my addr 172.21.15.32:0/1508840460 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3f7fe700 1 -- 172.21.15.32:0/1508840460 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412383924 0 0) 0x7fdc480aff30 con 0x7fdc480af450 2022-08-01T22:55:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3f7fe700 1 -- 172.21.15.32:0/1508840460 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc34003260 con 0x7fdc480af450 2022-08-01T22:55:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 -- 172.21.15.32:0/1508840460 >> v2:172.21.15.55:3300/0 conn(0x7fdc480ad4b0 msgr2=0x7fdc480af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 --2- 172.21.15.32:0/1508840460 >> v2:172.21.15.55:3300/0 conn(0x7fdc480ad4b0 0x7fdc480af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 -- 172.21.15.32:0/1508840460 >> v1:172.21.15.55:6789/0 conn(0x7fdc480af450 legacy=0x7fdc481432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 -- 172.21.15.32:0/1508840460 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc481439f0 con 0x7fdc480ad9e0 2022-08-01T22:55:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc4d5d0700 1 --2- 172.21.15.32:0/1508840460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc480a8e40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fdc480ad900 tx=0x7fdc400091d0).ready entity=mon.2 client_cookie=972def8b34de0771 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3f7fe700 1 -- 172.21.15.32:0/1508840460 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc4000b600 con 0x7fdc480ad9e0 2022-08-01T22:55:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.313+0000 7fdc3f7fe700 1 -- 172.21.15.32:0/1508840460 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4000b760 con 0x7fdc480ad9e0 2022-08-01T22:55:44.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc3f7fe700 1 -- 172.21.15.32:0/1508840460 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc4000b8d0 con 0x7fdc480ad9e0 2022-08-01T22:55:44.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/1508840460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 msgr2=0x7fdc480a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/1508840460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc480a8e40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fdc480ad900 tx=0x7fdc400091d0).stop 2022-08-01T22:55:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/1508840460 shutdown_connections 2022-08-01T22:55:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/1508840460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc480a8e40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/1508840460 >> v2:172.21.15.55:3300/0 conn(0x7fdc480ad4b0 0x7fdc480af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/1508840460 >> 172.21.15.32:0/1508840460 conn(0x7fdc480612d0 msgr2=0x7fdc48062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:44.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/1508840460 shutdown_connections 2022-08-01T22:55:44.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/1508840460 wait complete. 2022-08-01T22:55:44.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 Processor -- start 2022-08-01T22:55:44.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- start start 2022-08-01T22:55:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 0x7fdc4813f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc48142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc480af450 0x7fdc48142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48141a30 con 0x7fdc480ad4b0 2022-08-01T22:55:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480abcb0 con 0x7fdc480af450 2022-08-01T22:55:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4ddd1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48141790 con 0x7fdc480ad9e0 2022-08-01T22:55:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 0x7fdc4813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 0x7fdc4813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40268/0 (socket says 172.21.15.32:40268) 2022-08-01T22:55:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 -- 172.21.15.32:0/2975489385 learned_addr learned my addr 172.21.15.32:0/2975489385 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4d5d0700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc480af450 0x7fdc48142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc3ffff700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc48142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 -- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 msgr2=0x7fdc48142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc48142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 -- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc480af450 msgr2=0x7fdc48142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc480af450 0x7fdc48142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 -- 172.21.15.32:0/2975489385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc40009470 con 0x7fdc480ad4b0 2022-08-01T22:55:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.317+0000 7fdc4cdcf700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 0x7fdc4813f740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdc44001110 tx=0x7fdc44007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc44007e10 con 0x7fdc480ad4b0 2022-08-01T22:55:44.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc440072e0 con 0x7fdc480ad4b0 2022-08-01T22:55:44.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480b63f0 con 0x7fdc480ad4b0 2022-08-01T22:55:44.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc44005740 con 0x7fdc480ad4b0 2022-08-01T22:55:44.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4813f010 con 0x7fdc480ad4b0 2022-08-01T22:55:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdc4401a060 con 0x7fdc480ad4b0 2022-08-01T22:55:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3dffb700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdc28053aa0 0x7fdc28055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdc4406b3a0 con 0x7fdc480ad4b0 2022-08-01T22:55:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc480abe20 con 0x7fdc480ad4b0 2022-08-01T22:55:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3ffff700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdc28053aa0 0x7fdc28055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.321+0000 7fdc3ffff700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdc28053aa0 0x7fdc28055f60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fdc38008000 tx=0x7fdc38008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.329+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc44018020 con 0x7fdc480ad4b0 2022-08-01T22:55:44.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.457+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc480574e0 con 0x7fdc480ad4b0 2022-08-01T22:55:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc3dffb700 1 -- 172.21.15.32:0/2975489385 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fdc44018020 con 0x7fdc480ad4b0 2022-08-01T22:55:44.459 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:44.460 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:44.174058+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdc28053aa0 msgr2=0x7fdc28055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdc28053aa0 0x7fdc28055f60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fdc38008000 tx=0x7fdc38008040).stop 2022-08-01T22:55:44.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 msgr2=0x7fdc4813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 0x7fdc4813f740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fdc44001110 tx=0x7fdc44007ad0).stop 2022-08-01T22:55:44.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 shutdown_connections 2022-08-01T22:55:44.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdc28053aa0 0x7fdc28055f60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc480af450 0x7fdc48142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdc480ad9e0 0x7fdc48142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 --2- 172.21.15.32:0/2975489385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc480ad4b0 0x7fdc4813f740 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 >> 172.21.15.32:0/2975489385 conn(0x7fdc480612d0 msgr2=0x7fdc48146660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 shutdown_connections 2022-08-01T22:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.461+0000 7fdc4ddd1700 1 -- 172.21.15.32:0/2975489385 wait complete. 2022-08-01T22:55:44.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T22:55:44.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:55:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.605+0000 7f646d12b700 1 Processor -- start 2022-08-01T22:55:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.605+0000 7f646d12b700 1 -- start start 2022-08-01T22:55:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6468144ad0 0x7f64680abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f646813e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6468144eb0 con 0x7f64680aa050 2022-08-01T22:55:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64680ac390 con 0x7f6468144ad0 2022-08-01T22:55:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6468145110 con 0x7f64680b1110 2022-08-01T22:55:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64680aa050 0x7f64680b0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56054/0 (socket says 172.21.15.32:56054) 2022-08-01T22:55:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646659c700 1 -- 172.21.15.32:0/3544120556 learned_addr learned my addr 172.21.15.32:0/3544120556 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f6466d9d700 1 --2- 172.21.15.32:0/3544120556 >> v2:172.21.15.55:3300/0 conn(0x7f6468144ad0 0x7f64680abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646759e700 1 --2- 172.21.15.32:0/3544120556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f646813e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f6465d9b700 1 -- 172.21.15.32:0/3544120556 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199484529 0 0) 0x7f6468144eb0 con 0x7f64680aa050 2022-08-01T22:55:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f6465d9b700 1 -- 172.21.15.32:0/3544120556 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6448003260 con 0x7f64680aa050 2022-08-01T22:55:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646759e700 1 -- 172.21.15.32:0/3544120556 >> v2:172.21.15.55:3300/0 conn(0x7f6468144ad0 msgr2=0x7f64680abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646759e700 1 --2- 172.21.15.32:0/3544120556 >> v2:172.21.15.55:3300/0 conn(0x7f6468144ad0 0x7f64680abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646759e700 1 -- 172.21.15.32:0/3544120556 >> v1:172.21.15.32:6789/0 conn(0x7f64680aa050 legacy=0x7f64680b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646759e700 1 -- 172.21.15.32:0/3544120556 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64681453b0 con 0x7f64680b1110 2022-08-01T22:55:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646759e700 1 --2- 172.21.15.32:0/3544120556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f646813e2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f6460002f70 tx=0x7f6460006df0).ready entity=mon.2 client_cookie=f21bbd9a2e0ea8b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f6465d9b700 1 -- 172.21.15.32:0/3544120556 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646000d4a0 con 0x7f64680b1110 2022-08-01T22:55:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f6465d9b700 1 -- 172.21.15.32:0/3544120556 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6460000e90 con 0x7f64680b1110 2022-08-01T22:55:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f6465d9b700 1 -- 172.21.15.32:0/3544120556 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646000d7f0 con 0x7f64680b1110 2022-08-01T22:55:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- 172.21.15.32:0/3544120556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 msgr2=0x7f646813e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 --2- 172.21.15.32:0/3544120556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f646813e2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f6460002f70 tx=0x7f6460006df0).stop 2022-08-01T22:55:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- 172.21.15.32:0/3544120556 shutdown_connections 2022-08-01T22:55:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 --2- 172.21.15.32:0/3544120556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f646813e2b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 --2- 172.21.15.32:0/3544120556 >> v2:172.21.15.55:3300/0 conn(0x7f6468144ad0 0x7f64680abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- 172.21.15.32:0/3544120556 >> 172.21.15.32:0/3544120556 conn(0x7f64680b2b00 msgr2=0x7f64680b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- 172.21.15.32:0/3544120556 shutdown_connections 2022-08-01T22:55:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.609+0000 7f646d12b700 1 -- 172.21.15.32:0/3544120556 wait complete. 2022-08-01T22:55:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 Processor -- start 2022-08-01T22:55:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 -- start start 2022-08-01T22:55:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 0x7f6468064940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f6468064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6468144ad0 0x7f6468150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468065320 con 0x7f64680aa050 2022-08-01T22:55:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468141300 con 0x7f6468144ad0 2022-08-01T22:55:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646d12b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646813f500 con 0x7f64680b1110 2022-08-01T22:55:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 0x7f6468064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 0x7f6468064940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40282/0 (socket says 172.21.15.32:40282) 2022-08-01T22:55:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 -- 172.21.15.32:0/2845808935 learned_addr learned my addr 172.21.15.32:0/2845808935 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646659c700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f6468064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646759e700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6468144ad0 0x7f6468150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 -- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 msgr2=0x7f6468064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f6468064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 -- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6468144ad0 msgr2=0x7f6468150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6468144ad0 0x7f6468150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 -- 172.21.15.32:0/2845808935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646000b180 con 0x7f64680aa050 2022-08-01T22:55:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f646659c700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f6468064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f6466d9d700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 0x7f6468064940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f645800c000 tx=0x7f64580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.613+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6458012600 con 0x7f64680aa050 2022-08-01T22:55:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6458012760 con 0x7f64680aa050 2022-08-01T22:55:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646813fa60 con 0x7f64680aa050 2022-08-01T22:55:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6458012ac0 con 0x7f64680aa050 2022-08-01T22:55:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646813eea0 con 0x7f64680aa050 2022-08-01T22:55:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646813fbf0 con 0x7f64680aa050 2022-08-01T22:55:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6458022070 con 0x7f64680aa050 2022-08-01T22:55:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f645ffff700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f644c053b00 0x7f644c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f645806c110 con 0x7f64680aa050 2022-08-01T22:55:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.617+0000 7f646659c700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f644c053b00 0x7f644c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.621+0000 7f646659c700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f644c053b00 0x7f644c055fc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f645000d000 tx=0x7f6450009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.625+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f64580432c0 con 0x7f64680aa050 2022-08-01T22:55:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.745+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6468145650 con 0x7f64680aa050 2022-08-01T22:55:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.745+0000 7f645ffff700 1 -- 172.21.15.32:0/2845808935 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f64580468e0 con 0x7f64680aa050 2022-08-01T22:55:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:55:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f644c053b00 msgr2=0x7f644c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f644c053b00 0x7f644c055fc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f645000d000 tx=0x7f6450009040).stop 2022-08-01T22:55:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 msgr2=0x7f6468064940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 0x7f6468064940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f645800c000 tx=0x7f64580091d0).stop 2022-08-01T22:55:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 shutdown_connections 2022-08-01T22:55:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f644c053b00 0x7f644c055fc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6468144ad0 0x7f6468150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64680b1110 0x7f6468064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 --2- 172.21.15.32:0/2845808935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64680aa050 0x7f6468064940 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 >> 172.21.15.32:0/2845808935 conn(0x7f64680b2b00 msgr2=0x7f64680b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 shutdown_connections 2022-08-01T22:55:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.749+0000 7f646d12b700 1 -- 172.21.15.32:0/2845808935 wait complete. 2022-08-01T22:55:44.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:55:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5283fff700 1 Processor -- start 2022-08-01T22:55:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5283fff700 1 -- start start 2022-08-01T22:55:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5283fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f52840b01a0 0x7f52840a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5283fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52840a7250 con 0x7f52840b2710 2022-08-01T22:55:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5283fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52840a7480 con 0x7f52840afd60 2022-08-01T22:55:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5283fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52840a7720 con 0x7f52840b01a0 2022-08-01T22:55:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5282ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52840b2710 0x7f528414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56066/0 (socket says 172.21.15.32:56066) 2022-08-01T22:55:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5282ffd700 1 -- 172.21.15.32:0/2991292141 learned_addr learned my addr 172.21.15.32:0/2991292141 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f52827fc700 1 --2- 172.21.15.32:0/2991292141 >> v2:172.21.15.55:3300/0 conn(0x7f52840b01a0 0x7f52840a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5281ffb700 1 -- 172.21.15.32:0/2991292141 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108514062 0 0) 0x7f52840a7250 con 0x7f52840b2710 2022-08-01T22:55:44.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5281ffb700 1 -- 172.21.15.32:0/2991292141 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5268003260 con 0x7f52840b2710 2022-08-01T22:55:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f52827fc700 1 -- 172.21.15.32:0/2991292141 >> v1:172.21.15.55:6789/0 conn(0x7f52840afd60 legacy=0x7f5284151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f52827fc700 1 -- 172.21.15.32:0/2991292141 >> v1:172.21.15.32:6789/0 conn(0x7f52840b2710 legacy=0x7f528414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f52827fc700 1 -- 172.21.15.32:0/2991292141 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52840a79c0 con 0x7f52840b01a0 2022-08-01T22:55:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f52827fc700 1 --2- 172.21.15.32:0/2991292141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840b01a0 0x7f52840a8540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f52740032f0 tx=0x7f52740091d0).ready entity=mon.1 client_cookie=ea9b0cdd286a91aa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5281ffb700 1 -- 172.21.15.32:0/2991292141 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f527400b6c0 con 0x7f52840b01a0 2022-08-01T22:55:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5281ffb700 1 -- 172.21.15.32:0/2991292141 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527400b820 con 0x7f52840b01a0 2022-08-01T22:55:44.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.881+0000 7f5281ffb700 1 -- 172.21.15.32:0/2991292141 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f527400b990 con 0x7f52840b01a0 2022-08-01T22:55:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- 172.21.15.32:0/2991292141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840b01a0 msgr2=0x7f52840a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 --2- 172.21.15.32:0/2991292141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840b01a0 0x7f52840a8540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f52740032f0 tx=0x7f52740091d0).stop 2022-08-01T22:55:44.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- 172.21.15.32:0/2991292141 shutdown_connections 2022-08-01T22:55:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 --2- 172.21.15.32:0/2991292141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840b01a0 0x7f52840a8540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- 172.21.15.32:0/2991292141 >> 172.21.15.32:0/2991292141 conn(0x7f5284060310 msgr2=0x7f52840b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:44.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- 172.21.15.32:0/2991292141 shutdown_connections 2022-08-01T22:55:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- 172.21.15.32:0/2991292141 wait complete. 2022-08-01T22:55:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 Processor -- start 2022-08-01T22:55:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- start start 2022-08-01T22:55:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840afd60 0x7f52840aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52840b01a0 0x7f52840b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 0x7f52840b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52840b5270 con 0x7f52840b2710 2022-08-01T22:55:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52840b3f90 con 0x7f52840afd60 2022-08-01T22:55:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5283fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52840b2f60 con 0x7f52840b01a0 2022-08-01T22:55:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 0x7f52840b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 0x7f52840b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40300/0 (socket says 172.21.15.32:40300) 2022-08-01T22:55:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 -- 172.21.15.32:0/1531069504 learned_addr learned my addr 172.21.15.32:0/1531069504 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52827fc700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52840b01a0 0x7f52840b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f5282ffd700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840afd60 0x7f52840aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 -- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52840b01a0 msgr2=0x7f52840b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52840b01a0 0x7f52840b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 -- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840afd60 msgr2=0x7f52840aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:44.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.885+0000 7f52837fe700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840afd60 0x7f52840aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f52837fe700 1 -- 172.21.15.32:0/1531069504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274009470 con 0x7f52840b2710 2022-08-01T22:55:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f52837fe700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 0x7f52840b4d80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5284061c10 tx=0x7f527c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f527c012400 con 0x7f52840b2710 2022-08-01T22:55:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c012560 con 0x7f52840b2710 2022-08-01T22:55:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52840b3200 con 0x7f52840b2710 2022-08-01T22:55:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f527c0128c0 con 0x7f52840b2710 2022-08-01T22:55:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52840644e0 con 0x7f52840b2710 2022-08-01T22:55:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52840574e0 con 0x7f52840b2710 2022-08-01T22:55:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f527c019030 con 0x7f52840b2710 2022-08-01T22:55:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5273fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5260053a10 0x7f5260055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f527c06aec0 con 0x7f52840b2710 2022-08-01T22:55:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.889+0000 7f5282ffd700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5260053a10 0x7f5260055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.893+0000 7f5282ffd700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5260053a10 0x7f5260055ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f527800b000 tx=0x7f5278009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:44.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:44.897+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f527c042110 con 0x7f52840b2710 2022-08-01T22:55:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5284069a70 con 0x7f52840b2710 2022-08-01T22:55:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5273fff700 1 -- 172.21.15.32:0/1531069504 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f527c045730 con 0x7f52840b2710 2022-08-01T22:55:45.021 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:55:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5260053a10 msgr2=0x7f5260055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5260053a10 0x7f5260055ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f527800b000 tx=0x7f5278009040).stop 2022-08-01T22:55:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 msgr2=0x7f52840b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 0x7f52840b4d80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5284061c10 tx=0x7f527c007b90).stop 2022-08-01T22:55:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 shutdown_connections 2022-08-01T22:55:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5260053a10 0x7f5260055ed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52840b2710 0x7f52840b4d80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52840b01a0 0x7f52840b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 --2- 172.21.15.32:0/1531069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52840afd60 0x7f52840aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 >> 172.21.15.32:0/1531069504 conn(0x7f5284060310 msgr2=0x7f528414a060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 shutdown_connections 2022-08-01T22:55:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.021+0000 7f5283fff700 1 -- 172.21.15.32:0/1531069504 wait complete. 2022-08-01T22:55:45.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T22:55:45.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 Processor -- start 2022-08-01T22:55:45.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- start start 2022-08-01T22:55:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30701415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30700b3360 con 0x7f3070141ab0 2022-08-01T22:55:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30700b7d80 con 0x7f30700b79a0 2022-08-01T22:55:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3070067750 con 0x7f30700b1300 2022-08-01T22:55:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3070141ab0 0x7f307014bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56082/0 (socket says 172.21.15.32:56082) 2022-08-01T22:55:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306f7fe700 1 -- 172.21.15.32:0/2993646400 learned_addr learned my addr 172.21.15.32:0/2993646400 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306e7fc700 1 --2- 172.21.15.32:0/2993646400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30701415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323600183 0 0) 0x7f30700b3360 con 0x7f3070141ab0 2022-08-01T22:55:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3054003260 con 0x7f3070141ab0 2022-08-01T22:55:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027321065 0 0) 0x7f30700b7d80 con 0x7f30700b79a0 2022-08-01T22:55:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30700b3360 con 0x7f30700b79a0 2022-08-01T22:55:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306e7fc700 1 -- 172.21.15.32:0/2993646400 >> v1:172.21.15.55:6789/0 conn(0x7f30700b79a0 legacy=0x7f3070144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306e7fc700 1 -- 172.21.15.32:0/2993646400 >> v1:172.21.15.32:6789/0 conn(0x7f3070141ab0 legacy=0x7f307014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306e7fc700 1 -- 172.21.15.32:0/2993646400 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700ab000 con 0x7f30700b1300 2022-08-01T22:55:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306e7fc700 1 --2- 172.21.15.32:0/2993646400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30701415c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3060002f70 tx=0x7f3060006df0).ready entity=mon.2 client_cookie=60a5ccd047bd4f81 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306000d4f0 con 0x7f30700b1300 2022-08-01T22:55:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060000e90 con 0x7f30700b1300 2022-08-01T22:55:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f306dffb700 1 -- 172.21.15.32:0/2993646400 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306000d840 con 0x7f30700b1300 2022-08-01T22:55:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- 172.21.15.32:0/2993646400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 msgr2=0x7f30701415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 --2- 172.21.15.32:0/2993646400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30701415c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3060002f70 tx=0x7f3060006df0).stop 2022-08-01T22:55:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- 172.21.15.32:0/2993646400 shutdown_connections 2022-08-01T22:55:45.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 --2- 172.21.15.32:0/2993646400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30701415c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- 172.21.15.32:0/2993646400 >> 172.21.15.32:0/2993646400 conn(0x7f307005f840 msgr2=0x7f307005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- 172.21.15.32:0/2993646400 shutdown_connections 2022-08-01T22:55:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.137+0000 7f30753b9700 1 -- 172.21.15.32:0/2993646400 wait complete. 2022-08-01T22:55:45.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 Processor -- start 2022-08-01T22:55:45.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 -- start start 2022-08-01T22:55:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30700ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30700b79a0 0x7f307005bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3070141ab0 0x7f3070150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700b2620 con 0x7f3070141ab0 2022-08-01T22:55:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700af060 con 0x7f30700b79a0 2022-08-01T22:55:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f30753b9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30700adb50 con 0x7f30700b1300 2022-08-01T22:55:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3070141ab0 0x7f3070150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f306e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30700b79a0 0x7f307005bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f306effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30700ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f306effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30700ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52048/0 (socket says 172.21.15.32:52048) 2022-08-01T22:55:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.141+0000 7f306effd700 1 -- 172.21.15.32:0/3313003461 learned_addr learned my addr 172.21.15.32:0/3313003461 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306effd700 1 -- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30700b79a0 msgr2=0x7f307005bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306effd700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30700b79a0 0x7f307005bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306effd700 1 -- 172.21.15.32:0/3313003461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3070141ab0 msgr2=0x7f3070150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306effd700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3070141ab0 0x7f3070150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306effd700 1 -- 172.21.15.32:0/3313003461 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306000b180 con 0x7f30700b1300 2022-08-01T22:55:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306f7fe700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3070141ab0 0x7f3070150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f306effd700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30700ad500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3064001110 tx=0x7f3064007a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3064012400 con 0x7f30700b1300 2022-08-01T22:55:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064012560 con 0x7f30700b1300 2022-08-01T22:55:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30700addf0 con 0x7f30700b1300 2022-08-01T22:55:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30640128c0 con 0x7f30700b1300 2022-08-01T22:55:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307005c4c0 con 0x7f30700b1300 2022-08-01T22:55:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30700ae530 con 0x7f30700b1300 2022-08-01T22:55:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.145+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3064019030 con 0x7f30700b1300 2022-08-01T22:55:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.149+0000 7f305ffff700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f304c053a70 0x7f304c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.149+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f306406af40 con 0x7f30700b1300 2022-08-01T22:55:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.149+0000 7f306e7fc700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f304c053a70 0x7f304c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.149+0000 7f306e7fc700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f304c053a70 0x7f304c055f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3060018000 tx=0x7f3060014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.157+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30700ae530 con 0x7f30700b1300 2022-08-01T22:55:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f30700ae530 con 0x7f30700b1300 2022-08-01T22:55:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f305ffff700 1 -- 172.21.15.32:0/3313003461 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f30700ae530 con 0x7f30700b1300 2022-08-01T22:55:45.289 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:45.290 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T22:55:45.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:55:45.291 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T22:55:45.291 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T22:55:45.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:55:45.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:55:45.292 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T22:55:45.292 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T22:55:45.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:55:45.293 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T22:55:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f304c053a70 msgr2=0x7f304c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f304c053a70 0x7f304c055f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3060018000 tx=0x7f3060014000).stop 2022-08-01T22:55:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 msgr2=0x7f30700ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30700ad500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3064001110 tx=0x7f3064007a90).stop 2022-08-01T22:55:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 shutdown_connections 2022-08-01T22:55:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f304c053a70 0x7f304c055f30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3070141ab0 0x7f3070150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30700b79a0 0x7f307005bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 --2- 172.21.15.32:0/3313003461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f30700b1300 0x7f30700ad500 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.289+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 >> 172.21.15.32:0/3313003461 conn(0x7f307005f840 msgr2=0x7f30700b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.293+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 shutdown_connections 2022-08-01T22:55:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.293+0000 7f30753b9700 1 -- 172.21.15.32:0/3313003461 wait complete. 2022-08-01T22:55:45.303 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T22:55:45.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T22:55:45.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 Processor -- start 2022-08-01T22:55:45.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 -- start start 2022-08-01T22:55:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed540b0740 0x7fed540b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540a9da0 0x7fed540b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed54143ab0 con 0x7fed540b38a0 2022-08-01T22:55:45.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed54143d50 con 0x7fed540b0740 2022-08-01T22:55:45.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed5ac3d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed54143ff0 con 0x7fed540a9da0 2022-08-01T22:55:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed540b0740 0x7fed540b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed540b0740 0x7fed540b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40320/0 (socket says 172.21.15.32:40320) 2022-08-01T22:55:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 -- 172.21.15.32:0/3365129280 learned_addr learned my addr 172.21.15.32:0/3365129280 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed53fff700 1 --1- 172.21.15.32:0/3365129280 >> v1:172.21.15.32:6789/0 conn(0x7fed540b38a0 0x7fed5413d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56096/0 (socket says 172.21.15.32:56096) 2022-08-01T22:55:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed591da700 1 --2- 172.21.15.32:0/3365129280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540a9da0 0x7fed540b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 -- 172.21.15.32:0/3365129280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540a9da0 msgr2=0x7fed540b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 --2- 172.21.15.32:0/3365129280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540a9da0 0x7fed540b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 -- 172.21.15.32:0/3365129280 >> v1:172.21.15.32:6789/0 conn(0x7fed540b38a0 legacy=0x7fed5413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 -- 172.21.15.32:0/3365129280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54144290 con 0x7fed540b0740 2022-08-01T22:55:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed589d9700 1 --2- 172.21.15.32:0/3365129280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b0740 0x7fed540b0b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fed44002f70 tx=0x7fed44006f40).ready entity=mon.0 client_cookie=2a17d447dc84e34a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed537fe700 1 -- 172.21.15.32:0/3365129280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed4400dc70 con 0x7fed540b0740 2022-08-01T22:55:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed537fe700 1 -- 172.21.15.32:0/3365129280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44000e90 con 0x7fed540b0740 2022-08-01T22:55:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.409+0000 7fed537fe700 1 -- 172.21.15.32:0/3365129280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed4400be90 con 0x7fed540b0740 2022-08-01T22:55:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 -- 172.21.15.32:0/3365129280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b0740 msgr2=0x7fed540b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/3365129280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b0740 0x7fed540b0b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fed44002f70 tx=0x7fed44006f40).stop 2022-08-01T22:55:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 -- 172.21.15.32:0/3365129280 shutdown_connections 2022-08-01T22:55:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/3365129280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540a9da0 0x7fed540b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/3365129280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b0740 0x7fed540b0b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 -- 172.21.15.32:0/3365129280 >> 172.21.15.32:0/3365129280 conn(0x7fed54060950 msgr2=0x7fed54060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 -- 172.21.15.32:0/3365129280 shutdown_connections 2022-08-01T22:55:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 -- 172.21.15.32:0/3365129280 wait complete. 2022-08-01T22:55:45.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 Processor -- start 2022-08-01T22:55:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.413+0000 7fed5ac3d700 1 -- start start 2022-08-01T22:55:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed540a9da0 0x7fed540ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540b0740 0x7fed540ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 0x7fed540ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5413ee80 con 0x7fed540b38a0 2022-08-01T22:55:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed54156c80 con 0x7fed540a9da0 2022-08-01T22:55:45.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5413f070 con 0x7fed540b0740 2022-08-01T22:55:45.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 0x7fed540ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 0x7fed540ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40330/0 (socket says 172.21.15.32:40330) 2022-08-01T22:55:45.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 -- 172.21.15.32:0/667934882 learned_addr learned my addr 172.21.15.32:0/667934882 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:45.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed53fff700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540b0740 0x7fed540ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed589d9700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed540a9da0 0x7fed540ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 -- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540b0740 msgr2=0x7fed540ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540b0740 0x7fed540ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 -- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed540a9da0 msgr2=0x7fed540ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:45.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed540a9da0 0x7fed540ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:45.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 -- 172.21.15.32:0/667934882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4400b460 con 0x7fed540b38a0 2022-08-01T22:55:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed591da700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 0x7fed540ae630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fed4c00a000 tx=0x7fed4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed4c013620 con 0x7fed540b38a0 2022-08-01T22:55:45.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c013780 con 0x7fed540b38a0 2022-08-01T22:55:45.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5413f2a0 con 0x7fed540b38a0 2022-08-01T22:55:45.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed4c013ae0 con 0x7fed540b38a0 2022-08-01T22:55:45.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5413b5a0 con 0x7fed540b38a0 2022-08-01T22:55:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5413ddf0 con 0x7fed540b38a0 2022-08-01T22:55:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed4c013c40 con 0x7fed540b38a0 2022-08-01T22:55:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.417+0000 7fed51ffb700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fed34053a70 0x7fed34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.421+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fed4c06bb80 con 0x7fed540b38a0 2022-08-01T22:55:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.421+0000 7fed589d9700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fed34053a70 0x7fed34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.421+0000 7fed589d9700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fed34053a70 0x7fed34055f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fed44018000 tx=0x7fed540b0660).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:45.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.425+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed4c042c60 con 0x7fed540b38a0 2022-08-01T22:55:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:45.549+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fed54155f40 con 0x7fed540b38a0 2022-08-01T22:55:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.301+0000 7fed51ffb700 1 -- 172.21.15.32:0/667934882 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7fed4c06be60 con 0x7fed540b38a0 2022-08-01T22:55:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fed34053a70 msgr2=0x7fed34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fed34053a70 0x7fed34055f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fed44018000 tx=0x7fed540b0660).stop 2022-08-01T22:55:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 msgr2=0x7fed540ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 0x7fed540ae630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fed4c00a000 tx=0x7fed4c008040).stop 2022-08-01T22:55:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 shutdown_connections 2022-08-01T22:55:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fed34053a70 0x7fed34055f30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed540b38a0 0x7fed540ae630 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fed540b0740 0x7fed540ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 --2- 172.21.15.32:0/667934882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed540a9da0 0x7fed540ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 >> 172.21.15.32:0/667934882 conn(0x7fed54060950 msgr2=0x7fed541448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 shutdown_connections 2022-08-01T22:55:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.305+0000 7fed5ac3d700 1 -- 172.21.15.32:0/667934882 wait complete. 2022-08-01T22:55:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T22:55:46.322 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2022-08-01T22:55:46.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2022-08-01T22:55:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 Processor -- start 2022-08-01T22:55:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- start start 2022-08-01T22:55:46.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa3c40b6710 0x7fa3c40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3c40b6af0 0x7fa3c412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c4144240 con 0x7fa3c40b4550 2022-08-01T22:55:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c4143d00 con 0x7fa3c40b6af0 2022-08-01T22:55:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c4143fa0 con 0x7fa3c40b6710 2022-08-01T22:55:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3c40b6af0 0x7fa3c412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3c40b6af0 0x7fa3c412bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40338/0 (socket says 172.21.15.32:40338) 2022-08-01T22:55:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 -- 172.21.15.32:0/1855078494 learned_addr learned my addr 172.21.15.32:0/1855078494 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c2ffd700 1 --2- 172.21.15.32:0/1855078494 >> v2:172.21.15.55:3300/0 conn(0x7fa3c40b6710 0x7fa3c40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 -- 172.21.15.32:0/1855078494 >> v2:172.21.15.55:3300/0 conn(0x7fa3c40b6710 msgr2=0x7fa3c40b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 --2- 172.21.15.32:0/1855078494 >> v2:172.21.15.55:3300/0 conn(0x7fa3c40b6710 0x7fa3c40b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 -- 172.21.15.32:0/1855078494 >> v1:172.21.15.55:6789/0 conn(0x7fa3c40b4550 legacy=0x7fa3c4061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 -- 172.21.15.32:0/1855078494 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4061780 con 0x7fa3c40b6af0 2022-08-01T22:55:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c2ffd700 1 --2- 172.21.15.32:0/1855078494 >> v2:172.21.15.55:3300/0 conn(0x7fa3c40b6710 0x7fa3c40b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:55:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c37fe700 1 --2- 172.21.15.32:0/1855078494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b6af0 0x7fa3c412bb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa3c41432b0 tx=0x7fa3bc009290).ready entity=mon.0 client_cookie=73b3502eb3def983 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c1ffb700 1 -- 172.21.15.32:0/1855078494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3bc00b820 con 0x7fa3c40b6af0 2022-08-01T22:55:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c1ffb700 1 -- 172.21.15.32:0/1855078494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc00b980 con 0x7fa3c40b6af0 2022-08-01T22:55:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c1ffb700 1 -- 172.21.15.32:0/1855078494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3bc00bb30 con 0x7fa3c40b6af0 2022-08-01T22:55:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1855078494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b6af0 msgr2=0x7fa3c412bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1855078494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b6af0 0x7fa3c412bb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa3c41432b0 tx=0x7fa3bc009290).stop 2022-08-01T22:55:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1855078494 shutdown_connections 2022-08-01T22:55:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1855078494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b6af0 0x7fa3c412bb90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1855078494 >> v2:172.21.15.55:3300/0 conn(0x7fa3c40b6710 0x7fa3c40b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1855078494 >> 172.21.15.32:0/1855078494 conn(0x7fa3c405c620 msgr2=0x7fa3c405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1855078494 shutdown_connections 2022-08-01T22:55:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.437+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1855078494 wait complete. 2022-08-01T22:55:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 Processor -- start 2022-08-01T22:55:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 -- start start 2022-08-01T22:55:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 0x7fa3c412af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3c40b6710 0x7fa3c4144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3c40b6af0 0x7fa3c4156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4144b70 con 0x7fa3c40b4550 2022-08-01T22:55:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c41269b0 con 0x7fa3c40b6710 2022-08-01T22:55:46.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4126c50 con 0x7fa3c40b6af0 2022-08-01T22:55:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 0x7fa3c412af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 0x7fa3c412af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40340/0 (socket says 172.21.15.32:40340) 2022-08-01T22:55:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 -- 172.21.15.32:0/1240112762 learned_addr learned my addr 172.21.15.32:0/1240112762 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c37fe700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3c40b6af0 0x7fa3c4156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c27fc700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3c40b6710 0x7fa3c4144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 -- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3c40b6af0 msgr2=0x7fa3c4156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3c40b6af0 0x7fa3c4156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 -- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3c40b6710 msgr2=0x7fa3c4144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3c40b6710 0x7fa3c4144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 -- 172.21.15.32:0/1240112762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc009530 con 0x7fa3c40b4550 2022-08-01T22:55:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c2ffd700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 0x7fa3c412af10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa3b4007000 tx=0x7fa3b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3b4010580 con 0x7fa3c40b4550 2022-08-01T22:55:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b40106e0 con 0x7fa3c40b4550 2022-08-01T22:55:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c4155c50 con 0x7fa3c40b4550 2022-08-01T22:55:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3b4010a40 con 0x7fa3c40b4550 2022-08-01T22:55:46.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.441+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c4156880 con 0x7fa3c40b4550 2022-08-01T22:55:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.445+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c41560f0 con 0x7fa3c40b4550 2022-08-01T22:55:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.445+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa3b4010ba0 con 0x7fa3c40b4550 2022-08-01T22:55:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.445+0000 7fa3bbfff700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3a8053b30 0x7fa3a8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:46.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.445+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa3b406bd10 con 0x7fa3c40b4550 2022-08-01T22:55:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.445+0000 7fa3c27fc700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3a8053b30 0x7fa3a8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.445+0000 7fa3c27fc700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3a8053b30 0x7fa3a8055ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa3ac00d000 tx=0x7fa3ac007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.449+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3c41560f0 con 0x7fa3c40b4550 2022-08-01T22:55:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:46.585+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa3c41560f0 con 0x7fa3c40b4550 2022-08-01T22:55:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.305+0000 7fa3bbfff700 1 -- 172.21.15.32:0/1240112762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 154+0+0 (secure 0 0 0) 0x7fa3c41560f0 con 0x7fa3c40b4550 2022-08-01T22:55:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3a8053b30 msgr2=0x7fa3a8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3a8053b30 0x7fa3a8055ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa3ac00d000 tx=0x7fa3ac007b70).stop 2022-08-01T22:55:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 msgr2=0x7fa3c412af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 0x7fa3c412af10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa3b4007000 tx=0x7fa3b4008040).stop 2022-08-01T22:55:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 shutdown_connections 2022-08-01T22:55:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3a8053b30 0x7fa3a8055ff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3c40b6af0 0x7fa3c4156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3c40b6710 0x7fa3c4144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 --2- 172.21.15.32:0/1240112762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c40b4550 0x7fa3c412af10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 >> 172.21.15.32:0/1240112762 conn(0x7fa3c405c620 msgr2=0x7fa3c41432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 shutdown_connections 2022-08-01T22:55:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.313+0000 7fa3c98c6700 1 -- 172.21.15.32:0/1240112762 wait complete. 2022-08-01T22:55:47.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 erasure cephfs_data 2022-08-01T22:55:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 Processor -- start 2022-08-01T22:55:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 -- start start 2022-08-01T22:55:47.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f097805fb80 0x7f097813ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:47.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f0978140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09780afa50 con 0x7f09780ae3c0 2022-08-01T22:55:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09780b03c0 con 0x7f097805fb80 2022-08-01T22:55:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097f2b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097805ff60 con 0x7f09780b2050 2022-08-01T22:55:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097eab6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09780ae3c0 0x7f09781434e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56126/0 (socket says 172.21.15.32:56126) 2022-08-01T22:55:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097eab6700 1 -- 172.21.15.32:0/3510607999 learned_addr learned my addr 172.21.15.32:0/3510607999 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097dab4700 1 --2- 172.21.15.32:0/3510607999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f0978140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097e2b5700 1 --2- 172.21.15.32:0/3510607999 >> v2:172.21.15.55:3300/0 conn(0x7f097805fb80 0x7f097813ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097d2b3700 1 -- 172.21.15.32:0/3510607999 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599747248 0 0) 0x7f09780afa50 con 0x7f09780ae3c0 2022-08-01T22:55:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.453+0000 7f097d2b3700 1 -- 172.21.15.32:0/3510607999 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0960003260 con 0x7f09780ae3c0 2022-08-01T22:55:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097dab4700 1 -- 172.21.15.32:0/3510607999 >> v2:172.21.15.55:3300/0 conn(0x7f097805fb80 msgr2=0x7f097813ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097dab4700 1 --2- 172.21.15.32:0/3510607999 >> v2:172.21.15.55:3300/0 conn(0x7f097805fb80 0x7f097813ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097dab4700 1 -- 172.21.15.32:0/3510607999 >> v1:172.21.15.32:6789/0 conn(0x7f09780ae3c0 legacy=0x7f09781434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097dab4700 1 -- 172.21.15.32:0/3510607999 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09780ad1c0 con 0x7f09780b2050 2022-08-01T22:55:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097dab4700 1 --2- 172.21.15.32:0/3510607999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f0978140480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0968002f70 tx=0x7f0968006df0).ready entity=mon.2 client_cookie=c0861f7348e04636 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:47.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097d2b3700 1 -- 172.21.15.32:0/3510607999 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0968014070 con 0x7f09780b2050 2022-08-01T22:55:47.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097d2b3700 1 -- 172.21.15.32:0/3510607999 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968000e90 con 0x7f09780b2050 2022-08-01T22:55:47.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097d2b3700 1 -- 172.21.15.32:0/3510607999 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096800fa90 con 0x7f09780b2050 2022-08-01T22:55:47.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- 172.21.15.32:0/3510607999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 msgr2=0x7f0978140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:47.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 --2- 172.21.15.32:0/3510607999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f0978140480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0968002f70 tx=0x7f0968006df0).stop 2022-08-01T22:55:47.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- 172.21.15.32:0/3510607999 shutdown_connections 2022-08-01T22:55:47.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 --2- 172.21.15.32:0/3510607999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f0978140480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 --2- 172.21.15.32:0/3510607999 >> v2:172.21.15.55:3300/0 conn(0x7f097805fb80 0x7f097813ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- 172.21.15.32:0/3510607999 >> 172.21.15.32:0/3510607999 conn(0x7f09780613c0 msgr2=0x7f0978062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- 172.21.15.32:0/3510607999 shutdown_connections 2022-08-01T22:55:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- 172.21.15.32:0/3510607999 wait complete. 2022-08-01T22:55:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 Processor -- start 2022-08-01T22:55:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- start start 2022-08-01T22:55:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 0x7f0978143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09780ae3c0 0x7f0978067240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f097814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0978064540 con 0x7f097805fb80 2022-08-01T22:55:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0978065870 con 0x7f09780ae3c0 2022-08-01T22:55:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097f2b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09780642a0 con 0x7f09780b2050 2022-08-01T22:55:47.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097e2b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 0x7f0978143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097e2b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 0x7f0978143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40354/0 (socket says 172.21.15.32:40354) 2022-08-01T22:55:47.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097e2b5700 1 -- 172.21.15.32:0/300676290 learned_addr learned my addr 172.21.15.32:0/300676290 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.457+0000 7f097eab6700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f097814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:47.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097dab4700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09780ae3c0 0x7f0978067240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097e2b5700 1 -- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 msgr2=0x7f097814e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097e2b5700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f097814e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097e2b5700 1 -- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09780ae3c0 msgr2=0x7f0978067240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097e2b5700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09780ae3c0 0x7f0978067240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097e2b5700 1 -- 172.21.15.32:0/300676290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096800b180 con 0x7f097805fb80 2022-08-01T22:55:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097dab4700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09780ae3c0 0x7f0978067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097e2b5700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 0x7f0978143c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f097400c000 tx=0x7f0974006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0974010780 con 0x7f097805fb80 2022-08-01T22:55:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740108e0 con 0x7f097805fb80 2022-08-01T22:55:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0978065b10 con 0x7f097805fb80 2022-08-01T22:55:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0974010c40 con 0x7f097805fb80 2022-08-01T22:55:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0978152780 con 0x7f097805fb80 2022-08-01T22:55:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f097401c460 con 0x7f097805fb80 2022-08-01T22:55:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0978152520 con 0x7f097805fb80 2022-08-01T22:55:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f09677fe700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0958053980 0x7f0958055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5863+0+0 (secure 0 0 0) 0x7f097406c1e0 con 0x7f097805fb80 2022-08-01T22:55:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.461+0000 7f097dab4700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0958053980 0x7f0958055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.465+0000 7f097dab4700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0958053980 0x7f0958055e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f096801b000 tx=0x7f0968016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.469+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0974043170 con 0x7f097805fb80 2022-08-01T22:55:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:47.597+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7f09780afc80 con 0x7f097805fb80 2022-08-01T22:55:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.337+0000 7f09677fe700 1 -- 172.21.15.32:0/300676290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f0974046790 con 0x7f097805fb80 2022-08-01T22:55:49.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0958053980 msgr2=0x7f0958055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:49.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0958053980 0x7f0958055e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f096801b000 tx=0x7f0968016000).stop 2022-08-01T22:55:49.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 msgr2=0x7f0978143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:49.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 0x7f0978143c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f097400c000 tx=0x7f0974006df0).stop 2022-08-01T22:55:49.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 shutdown_connections 2022-08-01T22:55:49.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0958053980 0x7f0958055e40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09780b2050 0x7f097814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09780ae3c0 0x7f0978067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 --2- 172.21.15.32:0/300676290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f097805fb80 0x7f0978143c90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 >> 172.21.15.32:0/300676290 conn(0x7f09780613c0 msgr2=0x7f097813ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:49.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 shutdown_connections 2022-08-01T22:55:49.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.341+0000 7f097f2b7700 1 -- 172.21.15.32:0/300676290 wait complete. 2022-08-01T22:55:49.348 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T22:55:49.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true 2022-08-01T22:55:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.469+0000 7f590e66c700 1 Processor -- start 2022-08-01T22:55:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.469+0000 7f590e66c700 1 -- start start 2022-08-01T22:55:49.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590e66c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59080af550 0x7f59080b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590e66c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5908060330 0x7f59081445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590e66c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080aa630 con 0x7f5908144ad0 2022-08-01T22:55:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590e66c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080b15e0 con 0x7f5908060330 2022-08-01T22:55:49.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590e66c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080af930 con 0x7f59080af550 2022-08-01T22:55:49.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5908060330 0x7f59081445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:49.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5908060330 0x7f59081445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40364/0 (socket says 172.21.15.32:40364) 2022-08-01T22:55:49.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 -- 172.21.15.32:0/1529786086 learned_addr learned my addr 172.21.15.32:0/1529786086 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590d66a700 1 --2- 172.21.15.32:0/1529786086 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59080af550 0x7f59080b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 -- 172.21.15.32:0/1529786086 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59080af550 msgr2=0x7f59080b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 --2- 172.21.15.32:0/1529786086 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59080af550 0x7f59080b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 -- 172.21.15.32:0/1529786086 >> v1:172.21.15.55:6789/0 conn(0x7f5908144ad0 legacy=0x7f59080a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 -- 172.21.15.32:0/1529786086 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590805fd10 con 0x7f5908060330 2022-08-01T22:55:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f590ce69700 1 --2- 172.21.15.32:0/1529786086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5908060330 0x7f59081445e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f58f8002f70 tx=0x7f58f8006eb0).ready entity=mon.0 client_cookie=dbe45415670d92f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f58f7fff700 1 -- 172.21.15.32:0/1529786086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f800d570 con 0x7f5908060330 2022-08-01T22:55:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f58f7fff700 1 -- 172.21.15.32:0/1529786086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8000e90 con 0x7f5908060330 2022-08-01T22:55:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.473+0000 7f58f7fff700 1 -- 172.21.15.32:0/1529786086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f800d900 con 0x7f5908060330 2022-08-01T22:55:49.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 -- 172.21.15.32:0/1529786086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5908060330 msgr2=0x7f59081445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:49.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 --2- 172.21.15.32:0/1529786086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5908060330 0x7f59081445e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f58f8002f70 tx=0x7f58f8006eb0).stop 2022-08-01T22:55:49.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 -- 172.21.15.32:0/1529786086 shutdown_connections 2022-08-01T22:55:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 --2- 172.21.15.32:0/1529786086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5908060330 0x7f59081445e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 --2- 172.21.15.32:0/1529786086 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59080af550 0x7f59080b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 -- 172.21.15.32:0/1529786086 >> 172.21.15.32:0/1529786086 conn(0x7f59080612d0 msgr2=0x7f5908062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 -- 172.21.15.32:0/1529786086 shutdown_connections 2022-08-01T22:55:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 -- 172.21.15.32:0/1529786086 wait complete. 2022-08-01T22:55:49.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 Processor -- start 2022-08-01T22:55:49.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.477+0000 7f590e66c700 1 -- start start 2022-08-01T22:55:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5908060330 0x7f59081418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:49.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 0x7f59080ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5908144ad0 0x7f59081436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:49.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908140dd0 con 0x7f59080af550 2022-08-01T22:55:49.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59081404e0 con 0x7f5908060330 2022-08-01T22:55:49.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080afb80 con 0x7f5908144ad0 2022-08-01T22:55:49.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 0x7f59080ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:49.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 0x7f59080ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40370/0 (socket says 172.21.15.32:40370) 2022-08-01T22:55:49.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 -- 172.21.15.32:0/4033589674 learned_addr learned my addr 172.21.15.32:0/4033589674 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590d66a700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5908060330 0x7f59081418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590de6b700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5908144ad0 0x7f59081436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:49.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 -- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5908144ad0 msgr2=0x7f59081436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5908144ad0 0x7f59081436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 -- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5908060330 msgr2=0x7f59081418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5908060330 0x7f59081418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 -- 172.21.15.32:0/4033589674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f800b1e0 con 0x7f59080af550 2022-08-01T22:55:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590ce69700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 0x7f59080ae9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5908141840 tx=0x7f58f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:49.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f80117e0 con 0x7f59080af550 2022-08-01T22:55:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590805f840 con 0x7f59080af550 2022-08-01T22:55:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8011940 con 0x7f59080af550 2022-08-01T22:55:49.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f8011ca0 con 0x7f59080af550 2022-08-01T22:55:49.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5908063be0 con 0x7f59080af550 2022-08-01T22:55:49.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.481+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5908063da0 con 0x7f59080af550 2022-08-01T22:55:49.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.485+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58f8025030 con 0x7f59080af550 2022-08-01T22:55:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.485+0000 7f58f67fc700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58e8053aa0 0x7f58e8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.485+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f58f806d1b0 con 0x7f59080af550 2022-08-01T22:55:49.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.485+0000 7f590d66a700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58e8053aa0 0x7f58e8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:49.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.485+0000 7f590d66a700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58e8053aa0 0x7f58e8055f60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f590400b000 tx=0x7f5904006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:49.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.489+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58f8043f40 con 0x7f59080af550 2022-08-01T22:55:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:49.613+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f590805fa70 con 0x7f59080af550 2022-08-01T22:55:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.349+0000 7f58f67fc700 1 -- 172.21.15.32:0/4033589674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 166+0+0 (secure 0 0 0) 0x7f58f8047560 con 0x7f59080af550 2022-08-01T22:55:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58e8053aa0 msgr2=0x7f58e8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58e8053aa0 0x7f58e8055f60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f590400b000 tx=0x7f5904006df0).stop 2022-08-01T22:55:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 msgr2=0x7f59080ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 0x7f59080ae9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5908141840 tx=0x7f58f8015000).stop 2022-08-01T22:55:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 shutdown_connections 2022-08-01T22:55:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58e8053aa0 0x7f58e8055f60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5908144ad0 0x7f59081436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59080af550 0x7f59080ae9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 --2- 172.21.15.32:0/4033589674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5908060330 0x7f59081418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 >> 172.21.15.32:0/4033589674 conn(0x7f59080612d0 msgr2=0x7f59080b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 shutdown_connections 2022-08-01T22:55:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.353+0000 7f590e66c700 1 -- 172.21.15.32:0/4033589674 wait complete. 2022-08-01T22:55:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 4 allow_ec_overwrites to true 2022-08-01T22:55:50.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T22:55:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 Processor -- start 2022-08-01T22:55:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 -- start start 2022-08-01T22:55:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80b3630 0x7f18c8143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18c80aa0d0 0x7f18c80af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f18c80af6f0 0x7f18c805f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c81443f0 con 0x7f18c80aa0d0 2022-08-01T22:55:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c80b3a10 con 0x7f18c80af6f0 2022-08-01T22:55:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18ceaad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80afb80 con 0x7f18c80b3630 2022-08-01T22:55:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18c7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18c80aa0d0 0x7f18c80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18c7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18c80aa0d0 0x7f18c80af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40378/0 (socket says 172.21.15.32:40378) 2022-08-01T22:55:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.489+0000 7f18c7fff700 1 -- 172.21.15.32:0/689983697 learned_addr learned my addr 172.21.15.32:0/689983697 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18cc849700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80b3630 0x7f18c8143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18cd04a700 1 --2- 172.21.15.32:0/689983697 >> v2:172.21.15.55:3300/0 conn(0x7f18c80af6f0 0x7f18c805f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c7fff700 1 -- 172.21.15.32:0/689983697 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80b3630 msgr2=0x7f18c8143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c7fff700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80b3630 0x7f18c8143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c7fff700 1 -- 172.21.15.32:0/689983697 >> v2:172.21.15.55:3300/0 conn(0x7f18c80af6f0 msgr2=0x7f18c805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c7fff700 1 --2- 172.21.15.32:0/689983697 >> v2:172.21.15.55:3300/0 conn(0x7f18c80af6f0 0x7f18c805f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c7fff700 1 -- 172.21.15.32:0/689983697 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8146b40 con 0x7f18c80aa0d0 2022-08-01T22:55:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18cc849700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80b3630 0x7f18c8143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18cd04a700 1 --2- 172.21.15.32:0/689983697 >> v2:172.21.15.55:3300/0 conn(0x7f18c80af6f0 0x7f18c805f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c7fff700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c80af200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f18b0002f70 tx=0x7f18b0006eb0).ready entity=mon.0 client_cookie=cddad4abfc18e15d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c77fe700 1 -- 172.21.15.32:0/689983697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b000d570 con 0x7f18c80aa0d0 2022-08-01T22:55:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c77fe700 1 -- 172.21.15.32:0/689983697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b0000e90 con 0x7f18c80aa0d0 2022-08-01T22:55:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18c77fe700 1 -- 172.21.15.32:0/689983697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b000d900 con 0x7f18c80aa0d0 2022-08-01T22:55:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 -- 172.21.15.32:0/689983697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 msgr2=0x7f18c80af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c80af200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f18b0002f70 tx=0x7f18b0006eb0).stop 2022-08-01T22:55:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 -- 172.21.15.32:0/689983697 shutdown_connections 2022-08-01T22:55:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c80af200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 --2- 172.21.15.32:0/689983697 >> v2:172.21.15.55:3300/0 conn(0x7f18c80af6f0 0x7f18c805f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 --2- 172.21.15.32:0/689983697 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80b3630 0x7f18c8143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 -- 172.21.15.32:0/689983697 >> 172.21.15.32:0/689983697 conn(0x7f18c8060950 msgr2=0x7f18c8060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 -- 172.21.15.32:0/689983697 shutdown_connections 2022-08-01T22:55:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 -- 172.21.15.32:0/689983697 wait complete. 2022-08-01T22:55:50.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 Processor -- start 2022-08-01T22:55:50.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.493+0000 7f18ceaad700 1 -- start start 2022-08-01T22:55:50.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c813f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80af6f0 0x7f18c80b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80b3630 0x7f18c813bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8152640 con 0x7f18c80aa0d0 2022-08-01T22:55:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80b5d90 con 0x7f18c80b3630 2022-08-01T22:55:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80b4560 con 0x7f18c80af6f0 2022-08-01T22:55:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c813f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c813f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40382/0 (socket says 172.21.15.32:40382) 2022-08-01T22:55:50.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 -- 172.21.15.32:0/1987281074 learned_addr learned my addr 172.21.15.32:0/1987281074 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:50.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18c7fff700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80af6f0 0x7f18c80b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cd04a700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80b3630 0x7f18c813bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 -- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80af6f0 msgr2=0x7f18c80b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80af6f0 0x7f18c80b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 -- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80b3630 msgr2=0x7f18c813bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80b3630 0x7f18c813bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 -- 172.21.15.32:0/1987281074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b000b1e0 con 0x7f18c80aa0d0 2022-08-01T22:55:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18cc849700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c813f380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f18b8006940 tx=0x7f18b8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b8011780 con 0x7f18c80aa0d0 2022-08-01T22:55:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b80118e0 con 0x7f18c80aa0d0 2022-08-01T22:55:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c80b4800 con 0x7f18c80aa0d0 2022-08-01T22:55:50.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b8011c40 con 0x7f18c80aa0d0 2022-08-01T22:55:50.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.497+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c81424c0 con 0x7f18c80aa0d0 2022-08-01T22:55:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.501+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f18b8012420 con 0x7f18c80aa0d0 2022-08-01T22:55:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.501+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c80b4ca0 con 0x7f18c80aa0d0 2022-08-01T22:55:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.501+0000 7f18c5ffb700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f18ac053aa0 0x7f18ac055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.501+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f18b806bd90 con 0x7f18c80aa0d0 2022-08-01T22:55:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.501+0000 7f18c7fff700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f18ac053aa0 0x7f18ac055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.501+0000 7f18c7fff700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f18ac053aa0 0x7f18ac055f60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f18b001c000 tx=0x7f18b0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.509+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18b8042b60 con 0x7f18c80aa0d0 2022-08-01T22:55:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.677+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f18c8146de0 con 0x7f18c80aa0d0 2022-08-01T22:55:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.685+0000 7f18c5ffb700 1 -- 172.21.15.32:0/1987281074 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7f18b8046180 con 0x7f18c80aa0d0 2022-08-01T22:55:50.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f18ac053aa0 msgr2=0x7f18ac055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f18ac053aa0 0x7f18ac055f60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f18b001c000 tx=0x7f18b0015000).stop 2022-08-01T22:55:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 msgr2=0x7f18c813f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c813f380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f18b8006940 tx=0x7f18b8008040).stop 2022-08-01T22:55:50.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 shutdown_connections 2022-08-01T22:55:50.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f18ac053aa0 0x7f18ac055f60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80b3630 0x7f18c813bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f18c80af6f0 0x7f18c80b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 --2- 172.21.15.32:0/1987281074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c80aa0d0 0x7f18c813f380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.689+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 >> 172.21.15.32:0/1987281074 conn(0x7f18c8060950 msgr2=0x7f18c813d220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:50.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.693+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 shutdown_connections 2022-08-01T22:55:50.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.693+0000 7f18ceaad700 1 -- 172.21.15.32:0/1987281074 wait complete. 2022-08-01T22:55:50.697 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 3 and data pool 4 2022-08-01T22:55:50.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 Processor -- start 2022-08-01T22:55:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 -- start start 2022-08-01T22:55:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f797c0b6420 0x7f797c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b41f0 0x7f797c061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0b60e0 con 0x7f797c061fe0 2022-08-01T22:55:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c0b2100 con 0x7f797c0b6420 2022-08-01T22:55:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f798090c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0b23a0 con 0x7f797c0b41f0 2022-08-01T22:55:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f797c061fe0 0x7f797c1465c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56178/0 (socket says 172.21.15.32:56178) 2022-08-01T22:55:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797bfff700 1 -- 172.21.15.32:0/1173301594 learned_addr learned my addr 172.21.15.32:0/1173301594 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:50.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797b7fe700 1 --2- 172.21.15.32:0/1173301594 >> v2:172.21.15.55:3300/0 conn(0x7f797c0b6420 0x7f797c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797affd700 1 --2- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b41f0 0x7f797c061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797a7fc700 1 -- 172.21.15.32:0/1173301594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321974009 0 0) 0x7f7974001490 con 0x7f797c061fe0 2022-08-01T22:55:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797a7fc700 1 -- 172.21.15.32:0/1173301594 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f795c003260 con 0x7f797c061fe0 2022-08-01T22:55:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797a7fc700 1 -- 172.21.15.32:0/1173301594 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3990586401 0 0) 0x7f795c003260 con 0x7f797c061fe0 2022-08-01T22:55:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797b7fe700 1 -- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b41f0 msgr2=0x7f797c061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797b7fe700 1 --2- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b41f0 0x7f797c061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797b7fe700 1 -- 172.21.15.32:0/1173301594 >> v1:172.21.15.32:6789/0 conn(0x7f797c061fe0 legacy=0x7f797c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.805+0000 7f797b7fe700 1 -- 172.21.15.32:0/1173301594 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0b2640 con 0x7f797c0b6420 2022-08-01T22:55:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f797b7fe700 1 --2- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b6420 0x7f797c061600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7974001d10 tx=0x7f796c006df0).ready entity=mon.1 client_cookie=bca07127a04afc3d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f797a7fc700 1 -- 172.21.15.32:0/1173301594 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7974001b70 con 0x7f797c0b6420 2022-08-01T22:55:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f797a7fc700 1 -- 172.21.15.32:0/1173301594 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c000e90 con 0x7f797c0b6420 2022-08-01T22:55:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f797a7fc700 1 -- 172.21.15.32:0/1173301594 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796c012460 con 0x7f797c0b6420 2022-08-01T22:55:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 -- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b6420 msgr2=0x7f797c061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 --2- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b6420 0x7f797c061600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7974001d10 tx=0x7f796c006df0).stop 2022-08-01T22:55:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 -- 172.21.15.32:0/1173301594 shutdown_connections 2022-08-01T22:55:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 --2- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b41f0 0x7f797c061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 --2- 172.21.15.32:0/1173301594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b6420 0x7f797c061600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 -- 172.21.15.32:0/1173301594 >> 172.21.15.32:0/1173301594 conn(0x7f797c05c620 msgr2=0x7f797c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 -- 172.21.15.32:0/1173301594 shutdown_connections 2022-08-01T22:55:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.809+0000 7f798090c700 1 -- 172.21.15.32:0/1173301594 wait complete. 2022-08-01T22:55:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 Processor -- start 2022-08-01T22:55:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 -- start start 2022-08-01T22:55:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 0x7f797c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b41f0 0x7f797c144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b6420 0x7f797c12ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0afc00 con 0x7f797c061fe0 2022-08-01T22:55:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c153810 con 0x7f797c0b41f0 2022-08-01T22:55:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c141010 con 0x7f797c0b6420 2022-08-01T22:55:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 0x7f797c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 0x7f797c0b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40406/0 (socket says 172.21.15.32:40406) 2022-08-01T22:55:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 -- 172.21.15.32:0/2551832180 learned_addr learned my addr 172.21.15.32:0/2551832180 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797bfff700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b6420 0x7f797c12ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797affd700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b41f0 0x7f797c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 -- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b6420 msgr2=0x7f797c12ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b6420 0x7f797c12ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 -- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b41f0 msgr2=0x7f797c144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b41f0 0x7f797c144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 -- 172.21.15.32:0/2551832180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c00b180 con 0x7f797c061fe0 2022-08-01T22:55:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797affd700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b41f0 0x7f797c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797bfff700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b6420 0x7f797c12ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f797b7fe700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 0x7f797c0b04c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7974001d10 tx=0x7f796c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796c016030 con 0x7f797c061fe0 2022-08-01T22:55:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c013e40 con 0x7f797c061fe0 2022-08-01T22:55:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c0af560 con 0x7f797c061fe0 2022-08-01T22:55:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796c00de60 con 0x7f797c061fe0 2022-08-01T22:55:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.813+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c12b200 con 0x7f797c061fe0 2022-08-01T22:55:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.817+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c1539d0 con 0x7f797c061fe0 2022-08-01T22:55:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.817+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f796c024020 con 0x7f797c061fe0 2022-08-01T22:55:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.817+0000 7f7978ff9700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7960053a10 0x7f7960055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.817+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f796c017070 con 0x7f797c061fe0 2022-08-01T22:55:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.817+0000 7f797affd700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7960053a10 0x7f7960055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.817+0000 7f797affd700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7960053a10 0x7f7960055ed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f796400b000 tx=0x7f7964008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.825+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f796c043e80 con 0x7f797c061fe0 2022-08-01T22:55:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.949+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f797c0b28e0 con 0x7f797c061fe0 2022-08-01T22:55:50.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f7978ff9700 1 -- 172.21.15.32:0/2551832180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f796c0474a0 con 0x7f797c061fe0 2022-08-01T22:55:50.954 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:50.954 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7960053a10 msgr2=0x7f7960055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7960053a10 0x7f7960055ed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f796400b000 tx=0x7f7964008040).stop 2022-08-01T22:55:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 msgr2=0x7f797c0b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 0x7f797c0b04c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7974001d10 tx=0x7f796c014000).stop 2022-08-01T22:55:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 shutdown_connections 2022-08-01T22:55:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7960053a10 0x7f7960055ed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f797c0b6420 0x7f797c12ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f797c0b41f0 0x7f797c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 --2- 172.21.15.32:0/2551832180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f797c061fe0 0x7f797c0b04c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 >> 172.21.15.32:0/2551832180 conn(0x7f797c05c620 msgr2=0x7f797c0b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 shutdown_connections 2022-08-01T22:55:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:50.953+0000 7f798090c700 1 -- 172.21.15.32:0/2551832180 wait complete. 2022-08-01T22:55:50.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:51.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 Processor -- start 2022-08-01T22:55:51.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- start start 2022-08-01T22:55:51.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f099c0b6710 0x7f099c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c144240 con 0x7f099c0b4550 2022-08-01T22:55:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c143d00 con 0x7f099c0b6710 2022-08-01T22:55:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c143fa0 con 0x7f099c0b6af0 2022-08-01T22:55:51.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1151700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f099c0b6710 0x7f099c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a0950700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f099c0b4550 0x7f099c061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:48862/0 (socket says 172.21.15.32:48862) 2022-08-01T22:55:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a0950700 1 -- 172.21.15.32:0/4179800087 learned_addr learned my addr 172.21.15.32:0/4179800087 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1952700 1 --2- 172.21.15.32:0/4179800087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1151700 1 -- 172.21.15.32:0/4179800087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 msgr2=0x7f099c12bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:51.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1151700 1 --2- 172.21.15.32:0/4179800087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c12bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1151700 1 -- 172.21.15.32:0/4179800087 >> v1:172.21.15.55:6789/0 conn(0x7f099c0b4550 legacy=0x7f099c061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1151700 1 -- 172.21.15.32:0/4179800087 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c061780 con 0x7f099c0b6710 2022-08-01T22:55:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a1151700 1 --2- 172.21.15.32:0/4179800087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c0b5d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f099c1432b0 tx=0x7f098c006df0).ready entity=mon.0 client_cookie=29e1d60f2bd23b98 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f0993fff700 1 -- 172.21.15.32:0/4179800087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f098c00d570 con 0x7f099c0b6710 2022-08-01T22:55:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f0993fff700 1 -- 172.21.15.32:0/4179800087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c000e90 con 0x7f099c0b6710 2022-08-01T22:55:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f0993fff700 1 -- 172.21.15.32:0/4179800087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f098c00d8c0 con 0x7f099c0b6710 2022-08-01T22:55:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- 172.21.15.32:0/4179800087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 msgr2=0x7f099c0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 --2- 172.21.15.32:0/4179800087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c0b5d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f099c1432b0 tx=0x7f098c006df0).stop 2022-08-01T22:55:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- 172.21.15.32:0/4179800087 shutdown_connections 2022-08-01T22:55:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 --2- 172.21.15.32:0/4179800087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c12bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 --2- 172.21.15.32:0/4179800087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c0b5d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- 172.21.15.32:0/4179800087 >> 172.21.15.32:0/4179800087 conn(0x7f099c05c620 msgr2=0x7f099c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- 172.21.15.32:0/4179800087 shutdown_connections 2022-08-01T22:55:51.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.069+0000 7f09a33b5700 1 -- 172.21.15.32:0/4179800087 wait complete. 2022-08-01T22:55:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 Processor -- start 2022-08-01T22:55:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 -- start start 2022-08-01T22:55:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f099c0b4550 0x7f099c129c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c12bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c12a490 con 0x7f099c0b6710 2022-08-01T22:55:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c152f30 con 0x7f099c0b4550 2022-08-01T22:55:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1531d0 con 0x7f099c0b6af0 2022-08-01T22:55:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c12bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c12bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40422/0 (socket says 172.21.15.32:40422) 2022-08-01T22:55:51.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 -- 172.21.15.32:0/3624593461 learned_addr learned my addr 172.21.15.32:0/3624593461 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a1151700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f099c0b4550 0x7f099c129c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a1952700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 -- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 msgr2=0x7f099c146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 -- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f099c0b4550 msgr2=0x7f099c129c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f099c0b4550 0x7f099c129c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 -- 172.21.15.32:0/3624593461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c00b180 con 0x7f099c0b6710 2022-08-01T22:55:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a0950700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c12bcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f099c060a40 tx=0x7f0998007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:51.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0998001d70 con 0x7f099c0b6710 2022-08-01T22:55:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0998007110 con 0x7f099c0b6710 2022-08-01T22:55:51.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c12b5b0 con 0x7f099c0b6710 2022-08-01T22:55:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.073+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0998007470 con 0x7f099c0b6710 2022-08-01T22:55:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c12c1e0 con 0x7f099c0b6710 2022-08-01T22:55:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f099801b070 con 0x7f099c0b6710 2022-08-01T22:55:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c152ce0 con 0x7f099c0b6710 2022-08-01T22:55:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09927fc700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0984053b00 0x7f0984055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f099806b2c0 con 0x7f099c0b6710 2022-08-01T22:55:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09a1151700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0984053b00 0x7f0984055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.077+0000 7f09a1151700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0984053b00 0x7f0984055fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f098c01a000 tx=0x7f098c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.085+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0998042080 con 0x7f099c0b6710 2022-08-01T22:55:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.209+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f099c061a20 con 0x7f099c0b6710 2022-08-01T22:55:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09927fc700 1 -- 172.21.15.32:0/3624593461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f09980456a0 con 0x7f099c0b6710 2022-08-01T22:55:51.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:51.213 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0984053b00 msgr2=0x7f0984055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0984053b00 0x7f0984055fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f098c01a000 tx=0x7f098c014000).stop 2022-08-01T22:55:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 msgr2=0x7f099c12bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c12bcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f099c060a40 tx=0x7f0998007c30).stop 2022-08-01T22:55:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 shutdown_connections 2022-08-01T22:55:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0984053b00 0x7f0984055fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f099c0b6af0 0x7f099c146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b6710 0x7f099c12bcf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 --2- 172.21.15.32:0/3624593461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f099c0b4550 0x7f099c129c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.213+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 >> 172.21.15.32:0/3624593461 conn(0x7f099c05c620 msgr2=0x7f099c141120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.217+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 shutdown_connections 2022-08-01T22:55:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.217+0000 7f09a33b5700 1 -- 172.21.15.32:0/3624593461 wait complete. 2022-08-01T22:55:51.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T22:55:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 Processor -- start 2022-08-01T22:55:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 -- start start 2022-08-01T22:55:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 0x7f97b0064ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97b00b3a60 0x7f97b0065390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b0060670 con 0x7f97b01445d0 2022-08-01T22:55:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b0065880 con 0x7f97b00b3a60 2022-08-01T22:55:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97b49da700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0065e10 con 0x7f97b0060290 2022-08-01T22:55:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97aed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97b00b3a60 0x7f97b0065390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97ae59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 0x7f97b0064ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97ae59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 0x7f97b0064ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52166/0 (socket says 172.21.15.32:52166) 2022-08-01T22:55:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97ae59c700 1 -- 172.21.15.32:0/473430093 learned_addr learned my addr 172.21.15.32:0/473430093 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97add9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f97b01445d0 0x7f97b0144b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:48876/0 (socket says 172.21.15.32:48876) 2022-08-01T22:55:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.341+0000 7f97aed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97b00b3a60 0x7f97b0065390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:50218/0 (socket says 172.21.15.32:50218) 2022-08-01T22:55:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ae59c700 1 -- 172.21.15.32:0/473430093 >> v2:172.21.15.55:3300/0 conn(0x7f97b00b3a60 msgr2=0x7f97b0065390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ae59c700 1 --2- 172.21.15.32:0/473430093 >> v2:172.21.15.55:3300/0 conn(0x7f97b00b3a60 0x7f97b0065390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ae59c700 1 -- 172.21.15.32:0/473430093 >> v1:172.21.15.55:6789/0 conn(0x7f97b01445d0 legacy=0x7f97b0144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ae59c700 1 -- 172.21.15.32:0/473430093 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00b2540 con 0x7f97b0060290 2022-08-01T22:55:51.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ae59c700 1 --2- 172.21.15.32:0/473430093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 0x7f97b0064ea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f97a00032f0 tx=0x7f97a0009290).ready entity=mon.2 client_cookie=24fb30a13a7f1ac5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ad59a700 1 -- 172.21.15.32:0/473430093 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a000b8a0 con 0x7f97b0060290 2022-08-01T22:55:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ad59a700 1 -- 172.21.15.32:0/473430093 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000ba00 con 0x7f97b0060290 2022-08-01T22:55:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97ad59a700 1 -- 172.21.15.32:0/473430093 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a000bb70 con 0x7f97b0060290 2022-08-01T22:55:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 -- 172.21.15.32:0/473430093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 msgr2=0x7f97b0064ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 --2- 172.21.15.32:0/473430093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 0x7f97b0064ea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f97a00032f0 tx=0x7f97a0009290).stop 2022-08-01T22:55:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 -- 172.21.15.32:0/473430093 shutdown_connections 2022-08-01T22:55:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 --2- 172.21.15.32:0/473430093 >> v2:172.21.15.55:3300/0 conn(0x7f97b00b3a60 0x7f97b0065390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 --2- 172.21.15.32:0/473430093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b0060290 0x7f97b0064ea0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 -- 172.21.15.32:0/473430093 >> 172.21.15.32:0/473430093 conn(0x7f97b005d0f0 msgr2=0x7f97b005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 -- 172.21.15.32:0/473430093 shutdown_connections 2022-08-01T22:55:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 -- 172.21.15.32:0/473430093 wait complete. 2022-08-01T22:55:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 Processor -- start 2022-08-01T22:55:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.345+0000 7f97b49da700 1 -- start start 2022-08-01T22:55:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97b49da700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97b0060290 0x7f97b0143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97b49da700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b00b3a60 0x7f97b0150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97b49da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 0x7f97b0150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97b49da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b013dd10 con 0x7f97b00ae8b0 2022-08-01T22:55:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97b49da700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0154790 con 0x7f97b0060290 2022-08-01T22:55:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97b49da700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00af060 con 0x7f97b00b3a60 2022-08-01T22:55:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 0x7f97b0150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 0x7f97b0150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40438/0 (socket says 172.21.15.32:40438) 2022-08-01T22:55:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 -- 172.21.15.32:0/3058999443 learned_addr learned my addr 172.21.15.32:0/3058999443 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97add9b700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b00b3a60 0x7f97b0150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97ae59c700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97b0060290 0x7f97b0143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 -- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b00b3a60 msgr2=0x7f97b0150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b00b3a60 0x7f97b0150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 -- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97b0060290 msgr2=0x7f97b0143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97b0060290 0x7f97b0143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 -- 172.21.15.32:0/3058999443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009530 con 0x7f97b00ae8b0 2022-08-01T22:55:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97add9b700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b00b3a60 0x7f97b0150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.349+0000 7f97aed9d700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 0x7f97b0150b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f97a800c000 tx=0x7f97a8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a8010780 con 0x7f97b00ae8b0 2022-08-01T22:55:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b00af300 con 0x7f97b00ae8b0 2022-08-01T22:55:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b013e780 con 0x7f97b00ae8b0 2022-08-01T22:55:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80108e0 con 0x7f97b00ae8b0 2022-08-01T22:55:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a8010c40 con 0x7f97b00ae8b0 2022-08-01T22:55:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9788003410 con 0x7f97b00ae8b0 2022-08-01T22:55:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.353+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f97a801c460 con 0x7f97b00ae8b0 2022-08-01T22:55:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.357+0000 7f979f7fe700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:51.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.357+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f97a806c5b0 con 0x7f97b00ae8b0 2022-08-01T22:55:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.357+0000 7f97ae59c700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.357+0000 7f97ae59c700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f97a0019000 tx=0x7f97a0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.361+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f97a8043180 con 0x7f97b00ae8b0 2022-08-01T22:55:51.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:51.525+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9788003c40 con 0x7f97b00ae8b0 2022-08-01T22:55:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.361+0000 7f979f7fe700 1 -- 172.21.15.32:0/3058999443 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f97a80467a0 con 0x7f97b00ae8b0 2022-08-01T22:55:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 msgr2=0x7f978c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f97a0019000 tx=0x7f97a0016000).stop 2022-08-01T22:55:52.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 msgr2=0x7f97b0150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 0x7f97b0150b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f97a800c000 tx=0x7f97a8006df0).stop 2022-08-01T22:55:52.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 shutdown_connections 2022-08-01T22:55:52.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97b00ae8b0 0x7f97b0150b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97b00b3a60 0x7f97b0150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 --2- 172.21.15.32:0/3058999443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97b0060290 0x7f97b0143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 >> 172.21.15.32:0/3058999443 conn(0x7f97b005d0f0 msgr2=0x7f97b0064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:52.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 shutdown_connections 2022-08-01T22:55:52.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.365+0000 7f97b49da700 1 -- 172.21.15.32:0/3058999443 wait complete. 2022-08-01T22:55:52.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:55:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 Processor -- start 2022-08-01T22:55:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 -- start start 2022-08-01T22:55:52.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd4640ad750 0x7fd464061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4641405d0 0x7fd464140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4640aee80 con 0x7fd4640614b0 2022-08-01T22:55:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4640b70b0 con 0x7fd4640ad750 2022-08-01T22:55:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.489+0000 7fd469a07700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640af0b0 con 0x7fd4641405d0 2022-08-01T22:55:52.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd468a05700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4640614b0 0x7fd4640b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56216/0 (socket says 172.21.15.32:56216) 2022-08-01T22:55:52.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd468a05700 1 -- 172.21.15.32:0/2879983117 learned_addr learned my addr 172.21.15.32:0/2879983117 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:52.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd463fff700 1 --2- 172.21.15.32:0/2879983117 >> v2:172.21.15.55:3300/0 conn(0x7fd4640ad750 0x7fd464061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469206700 1 --2- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4641405d0 0x7fd464140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd4637fe700 1 -- 172.21.15.32:0/2879983117 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64631599 0 0) 0x7fd4640aee80 con 0x7fd4640614b0 2022-08-01T22:55:52.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd4637fe700 1 -- 172.21.15.32:0/2879983117 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd448003260 con 0x7fd4640614b0 2022-08-01T22:55:52.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd463fff700 1 -- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4641405d0 msgr2=0x7fd464140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:52.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd463fff700 1 --2- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4641405d0 0x7fd464140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd463fff700 1 -- 172.21.15.32:0/2879983117 >> v1:172.21.15.32:6789/0 conn(0x7fd4640614b0 legacy=0x7fd4640b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd463fff700 1 -- 172.21.15.32:0/2879983117 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4640af350 con 0x7fd4640ad750 2022-08-01T22:55:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd463fff700 1 --2- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640ad750 0x7fd464061890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd450002f70 tx=0x7fd450006df0).ready entity=mon.1 client_cookie=3ecdd87c7d490d35 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd4637fe700 1 -- 172.21.15.32:0/2879983117 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd45000d4a0 con 0x7fd4640ad750 2022-08-01T22:55:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd4637fe700 1 -- 172.21.15.32:0/2879983117 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd450000e90 con 0x7fd4640ad750 2022-08-01T22:55:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd4637fe700 1 -- 172.21.15.32:0/2879983117 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd45000d7f0 con 0x7fd4640ad750 2022-08-01T22:55:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 -- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640ad750 msgr2=0x7fd464061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 --2- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640ad750 0x7fd464061890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd450002f70 tx=0x7fd450006df0).stop 2022-08-01T22:55:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 -- 172.21.15.32:0/2879983117 shutdown_connections 2022-08-01T22:55:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 --2- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4641405d0 0x7fd464140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 --2- 172.21.15.32:0/2879983117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640ad750 0x7fd464061890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 -- 172.21.15.32:0/2879983117 >> 172.21.15.32:0/2879983117 conn(0x7fd464066f70 msgr2=0x7fd464067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 -- 172.21.15.32:0/2879983117 shutdown_connections 2022-08-01T22:55:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 -- 172.21.15.32:0/2879983117 wait complete. 2022-08-01T22:55:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 Processor -- start 2022-08-01T22:55:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.493+0000 7fd469a07700 1 -- start start 2022-08-01T22:55:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640614b0 0x7fd464141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4640ad750 0x7fd464152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 0x7fd464145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640aa1a0 con 0x7fd4641405d0 2022-08-01T22:55:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640aa3d0 con 0x7fd4640614b0 2022-08-01T22:55:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640aa670 con 0x7fd4640ad750 2022-08-01T22:55:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 0x7fd464145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd463fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4640ad750 0x7fd464152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 0x7fd464145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40452/0 (socket says 172.21.15.32:40452) 2022-08-01T22:55:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 -- 172.21.15.32:0/1557747127 learned_addr learned my addr 172.21.15.32:0/1557747127 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd468a05700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640614b0 0x7fd464141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 -- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4640ad750 msgr2=0x7fd464152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4640ad750 0x7fd464152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 -- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640614b0 msgr2=0x7fd464141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640614b0 0x7fd464141970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 -- 172.21.15.32:0/1557747127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45000b180 con 0x7fd4641405d0 2022-08-01T22:55:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469206700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 0x7fd464145340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fd45c006e00 tx=0x7fd45c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd45c012600 con 0x7fd4641405d0 2022-08-01T22:55:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c012760 con 0x7fd4641405d0 2022-08-01T22:55:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd464152890 con 0x7fd4641405d0 2022-08-01T22:55:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd45c012ac0 con 0x7fd4641405d0 2022-08-01T22:55:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd464141e60 con 0x7fd4641405d0 2022-08-01T22:55:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.497+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd464152a20 con 0x7fd4641405d0 2022-08-01T22:55:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.501+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd45c022070 con 0x7fd4641405d0 2022-08-01T22:55:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.501+0000 7fd461ffb700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd444053a70 0x7fd444055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.501+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd45c06c6f0 con 0x7fd4641405d0 2022-08-01T22:55:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.501+0000 7fd468a05700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd444053a70 0x7fd444055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.501+0000 7fd468a05700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd444053a70 0x7fd444055f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd45800d000 tx=0x7fd458009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.505+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd45c043410 con 0x7fd4641405d0 2022-08-01T22:55:52.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.669+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4640af5f0 con 0x7fd4641405d0 2022-08-01T22:55:52.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.669+0000 7fd461ffb700 1 -- 172.21.15.32:0/1557747127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2486 (secure 0 0 0) 0x7fd45c016020 con 0x7fd4641405d0 2022-08-01T22:55:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:52.671 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3750794992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3750794992},{"type":"v1","addr":"172.21.15.32:6835","nonce":3750794992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3123630923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3123630923},{"type":"v1","addr":"172.21.15.55:6835","nonce":3123630923}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:55:50.680148+0000","modified":"2022-08-01T22:55:52.352879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/2632088620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2632088620},{"type":"v1","addr":"172.21.15.32:6833","nonce":2632088620}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T22:55:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd444053a70 msgr2=0x7fd444055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd444053a70 0x7fd444055f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd45800d000 tx=0x7fd458009040).stop 2022-08-01T22:55:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 msgr2=0x7fd464145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 0x7fd464145340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fd45c006e00 tx=0x7fd45c0091d0).stop 2022-08-01T22:55:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 shutdown_connections 2022-08-01T22:55:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd444053a70 0x7fd444055f30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4641405d0 0x7fd464145340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4640ad750 0x7fd464152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 --2- 172.21.15.32:0/1557747127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4640614b0 0x7fd464141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 >> 172.21.15.32:0/1557747127 conn(0x7fd464066f70 msgr2=0x7fd464142500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 shutdown_connections 2022-08-01T22:55:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.673+0000 7fd469a07700 1 -- 172.21.15.32:0/1557747127 wait complete. 2022-08-01T22:55:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2022-08-01T22:55:52.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 Processor -- start 2022-08-01T22:55:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- start start 2022-08-01T22:55:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54780b6a90 0x7f54780acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54780aa050 0x7f54780aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54780b6e70 con 0x7f5478060aa0 2022-08-01T22:55:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547805cd30 con 0x7f54780aa050 2022-08-01T22:55:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5478061370 con 0x7f54780b6a90 2022-08-01T22:55:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f54777fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5478060aa0 0x7f547805c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56232/0 (socket says 172.21.15.32:56232) 2022-08-01T22:55:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547cc0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54780aa050 0x7f54780aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f54777fe700 1 -- 172.21.15.32:0/3795645853 learned_addr learned my addr 172.21.15.32:0/3795645853 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5477fff700 1 --2- 172.21.15.32:0/3795645853 >> v2:172.21.15.55:3300/0 conn(0x7f54780b6a90 0x7f54780acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5476ffd700 1 -- 172.21.15.32:0/3795645853 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298089711 0 0) 0x7f54780b6e70 con 0x7f5478060aa0 2022-08-01T22:55:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5476ffd700 1 -- 172.21.15.32:0/3795645853 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5464003260 con 0x7f5478060aa0 2022-08-01T22:55:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547cc0d700 1 -- 172.21.15.32:0/3795645853 >> v2:172.21.15.55:3300/0 conn(0x7f54780b6a90 msgr2=0x7f54780acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547cc0d700 1 --2- 172.21.15.32:0/3795645853 >> v2:172.21.15.55:3300/0 conn(0x7f54780b6a90 0x7f54780acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547cc0d700 1 -- 172.21.15.32:0/3795645853 >> v1:172.21.15.32:6789/0 conn(0x7f5478060aa0 legacy=0x7f547805c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547cc0d700 1 -- 172.21.15.32:0/3795645853 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478061610 con 0x7f54780aa050 2022-08-01T22:55:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5477fff700 1 --2- 172.21.15.32:0/3795645853 >> v2:172.21.15.55:3300/0 conn(0x7f54780b6a90 0x7f54780acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547cc0d700 1 --2- 172.21.15.32:0/3795645853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780aa050 0x7f54780aa4c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f54700032f0 tx=0x7f54700091d0).ready entity=mon.0 client_cookie=45e8af2fc6e94089 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5476ffd700 1 -- 172.21.15.32:0/3795645853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547000b600 con 0x7f54780aa050 2022-08-01T22:55:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5476ffd700 1 -- 172.21.15.32:0/3795645853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547000b760 con 0x7f54780aa050 2022-08-01T22:55:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f5476ffd700 1 -- 172.21.15.32:0/3795645853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547000b8d0 con 0x7f54780aa050 2022-08-01T22:55:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- 172.21.15.32:0/3795645853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780aa050 msgr2=0x7f54780aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 --2- 172.21.15.32:0/3795645853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780aa050 0x7f54780aa4c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f54700032f0 tx=0x7f54700091d0).stop 2022-08-01T22:55:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- 172.21.15.32:0/3795645853 shutdown_connections 2022-08-01T22:55:52.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 --2- 172.21.15.32:0/3795645853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780aa050 0x7f54780aa4c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 --2- 172.21.15.32:0/3795645853 >> v2:172.21.15.55:3300/0 conn(0x7f54780b6a90 0x7f54780acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- 172.21.15.32:0/3795645853 >> 172.21.15.32:0/3795645853 conn(0x7f54780b3610 msgr2=0x7f54780b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:52.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- 172.21.15.32:0/3795645853 shutdown_connections 2022-08-01T22:55:52.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.789+0000 7f547e670700 1 -- 172.21.15.32:0/3795645853 wait complete. 2022-08-01T22:55:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 Processor -- start 2022-08-01T22:55:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 -- start start 2022-08-01T22:55:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5478060aa0 0x7f54780b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54780aa050 0x7f547809ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 0x7f547809ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54780afda0 con 0x7f54780b6a90 2022-08-01T22:55:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54780ae600 con 0x7f54780aa050 2022-08-01T22:55:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54780aebd0 con 0x7f5478060aa0 2022-08-01T22:55:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 0x7f547809ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 0x7f547809ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40466/0 (socket says 172.21.15.32:40466) 2022-08-01T22:55:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 -- 172.21.15.32:0/4086230824 learned_addr learned my addr 172.21.15.32:0/4086230824 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f54777fe700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54780aa050 0x7f547809ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f5477fff700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5478060aa0 0x7f54780b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 -- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5478060aa0 msgr2=0x7f54780b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5478060aa0 0x7f54780b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 -- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54780aa050 msgr2=0x7f547809ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54780aa050 0x7f547809ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 -- 172.21.15.32:0/4086230824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546800b180 con 0x7f54780b6a90 2022-08-01T22:55:52.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547cc0d700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 0x7f547809ef30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f54700032f0 tx=0x7f5470016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:52.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5470011c10 con 0x7f54780b6a90 2022-08-01T22:55:52.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470011d70 con 0x7f54780b6a90 2022-08-01T22:55:52.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5470009470 con 0x7f54780b6a90 2022-08-01T22:55:52.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547001f5e0 con 0x7f54780b6a90 2022-08-01T22:55:52.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.793+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54780ae330 con 0x7f54780b6a90 2022-08-01T22:55:52.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.797+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54780aed90 con 0x7f54780b6a90 2022-08-01T22:55:52.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.797+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5470026030 con 0x7f54780b6a90 2022-08-01T22:55:52.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.797+0000 7f54757fa700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54580539e0 0x7f5458055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:52.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.797+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f547006cce0 con 0x7f54780b6a90 2022-08-01T22:55:52.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.797+0000 7f5477fff700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54580539e0 0x7f5458055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:52.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.797+0000 7f5477fff700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54580539e0 0x7f5458055ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f546800b000 tx=0x7f5468008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:52.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.805+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54780aed90 con 0x7f54780b6a90 2022-08-01T22:55:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.929+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54780aed90 con 0x7f54780b6a90 2022-08-01T22:55:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.929+0000 7f54757fa700 1 -- 172.21.15.32:0/4086230824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f54780aed90 con 0x7f54780b6a90 2022-08-01T22:55:52.932 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:52.933 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54580539e0 msgr2=0x7f5458055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54580539e0 0x7f5458055ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f546800b000 tx=0x7f5468008040).stop 2022-08-01T22:55:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 msgr2=0x7f547809ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 0x7f547809ef30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f54700032f0 tx=0x7f5470016000).stop 2022-08-01T22:55:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 shutdown_connections 2022-08-01T22:55:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54580539e0 0x7f5458055ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54780b6a90 0x7f547809ef30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54780aa050 0x7f547809ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 --2- 172.21.15.32:0/4086230824 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5478060aa0 0x7f54780b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 >> 172.21.15.32:0/4086230824 conn(0x7f54780b3610 msgr2=0x7f547813bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:52.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 shutdown_connections 2022-08-01T22:55:52.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:52.933+0000 7f547e670700 1 -- 172.21.15.32:0/4086230824 wait complete. 2022-08-01T22:55:52.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:52.947 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-08-01T22:55:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 Processor -- start 2022-08-01T22:55:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 -- start start 2022-08-01T22:55:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb97c0b41f0 0x7fb97c061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0b60e0 con 0x7fb97c061fe0 2022-08-01T22:55:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c0b2100 con 0x7fb97c0b41f0 2022-08-01T22:55:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0b23a0 con 0x7fb97c0b6420 2022-08-01T22:55:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb97c0b41f0 0x7fb97c061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97affd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb97c0b41f0 0x7fb97c061af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40474/0 (socket says 172.21.15.32:40474) 2022-08-01T22:55:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 -- 172.21.15.32:0/4218473140 learned_addr learned my addr 172.21.15.32:0/4218473140 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 -- 172.21.15.32:0/4218473140 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 msgr2=0x7fb97c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 --2- 172.21.15.32:0/4218473140 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 -- 172.21.15.32:0/4218473140 >> v1:172.21.15.55:6789/0 conn(0x7fb97c061fe0 legacy=0x7fb97c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 -- 172.21.15.32:0/4218473140 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0b2640 con 0x7fb97c0b41f0 2022-08-01T22:55:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.057+0000 7fb97a7fc700 1 --2- 172.21.15.32:0/4218473140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c0b41f0 0x7fb97c061af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fb970002f70 tx=0x7fb970006eb0).ready entity=mon.0 client_cookie=f9bea4cd3f749e37 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb979ffb700 1 -- 172.21.15.32:0/4218473140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97000d570 con 0x7fb97c0b41f0 2022-08-01T22:55:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb979ffb700 1 -- 172.21.15.32:0/4218473140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb970000e90 con 0x7fb97c0b41f0 2022-08-01T22:55:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb979ffb700 1 -- 172.21.15.32:0/4218473140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97000d900 con 0x7fb97c0b41f0 2022-08-01T22:55:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 -- 172.21.15.32:0/4218473140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c0b41f0 msgr2=0x7fb97c061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 --2- 172.21.15.32:0/4218473140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c0b41f0 0x7fb97c061af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fb970002f70 tx=0x7fb970006eb0).stop 2022-08-01T22:55:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 -- 172.21.15.32:0/4218473140 shutdown_connections 2022-08-01T22:55:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 --2- 172.21.15.32:0/4218473140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c0b41f0 0x7fb97c061af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 --2- 172.21.15.32:0/4218473140 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 -- 172.21.15.32:0/4218473140 >> 172.21.15.32:0/4218473140 conn(0x7fb97c05c620 msgr2=0x7fb97c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 -- 172.21.15.32:0/4218473140 shutdown_connections 2022-08-01T22:55:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 -- 172.21.15.32:0/4218473140 wait complete. 2022-08-01T22:55:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 Processor -- start 2022-08-01T22:55:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.061+0000 7fb97bfff700 1 -- start start 2022-08-01T22:55:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 0x7fb97c0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97c0b41f0 0x7fb97c0ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c143770 con 0x7fb97c061fe0 2022-08-01T22:55:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c145fb0 con 0x7fb97c0b41f0 2022-08-01T22:55:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 0x7fb97c0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1434d0 con 0x7fb97c0b6420 2022-08-01T22:55:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 0x7fb97c0b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40476/0 (socket says 172.21.15.32:40476) 2022-08-01T22:55:53.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 -- 172.21.15.32:0/114954040 learned_addr learned my addr 172.21.15.32:0/114954040 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97a7fc700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97c0b41f0 0x7fb97c0ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97b7fe700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 -- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 msgr2=0x7fb97c155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 -- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97c0b41f0 msgr2=0x7fb97c0ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97c0b41f0 0x7fb97c0ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 -- 172.21.15.32:0/114954040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97000b1e0 con 0x7fb97c061fe0 2022-08-01T22:55:53.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97affd700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 0x7fb97c0b0350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fb97400c000 tx=0x7fb9740091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97400c850 con 0x7fb97c061fe0 2022-08-01T22:55:53.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97400c9b0 con 0x7fb97c061fe0 2022-08-01T22:55:53.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97400cd10 con 0x7fb97c061fe0 2022-08-01T22:55:53.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c146210 con 0x7fb97c061fe0 2022-08-01T22:55:53.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c143150 con 0x7fb97c061fe0 2022-08-01T22:55:53.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c1463a0 con 0x7fb97c061fe0 2022-08-01T22:55:53.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.065+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb974022070 con 0x7fb97c061fe0 2022-08-01T22:55:53.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.069+0000 7fb96bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb958053a70 0x7fb958055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.069+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb97406c7d0 con 0x7fb97c061fe0 2022-08-01T22:55:53.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.069+0000 7fb97a7fc700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb958053a70 0x7fb958055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.069+0000 7fb97a7fc700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb958053a70 0x7fb958055f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb97001a000 tx=0x7fb970015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.073+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb974043510 con 0x7fb97c061fe0 2022-08-01T22:55:53.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:53.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:53.173 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:53.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:55:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.201+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb97c0b28e0 con 0x7fb97c061fe0 2022-08-01T22:55:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.201+0000 7fb96bfff700 1 -- 172.21.15.32:0/114954040 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fb974046b30 con 0x7fb97c061fe0 2022-08-01T22:55:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:53.207 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:53.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb958053a70 msgr2=0x7fb958055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb958053a70 0x7fb958055f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb97001a000 tx=0x7fb970015000).stop 2022-08-01T22:55:53.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 msgr2=0x7fb97c0b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 0x7fb97c0b0350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fb97400c000 tx=0x7fb9740091d0).stop 2022-08-01T22:55:53.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 shutdown_connections 2022-08-01T22:55:53.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb958053a70 0x7fb958055f30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb97c0b6420 0x7fb97c155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97c0b41f0 0x7fb97c0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 --2- 172.21.15.32:0/114954040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c061fe0 0x7fb97c0b0350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 >> 172.21.15.32:0/114954040 conn(0x7fb97c05c620 msgr2=0x7fb97c0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 shutdown_connections 2022-08-01T22:55:53.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.205+0000 7fb97bfff700 1 -- 172.21.15.32:0/114954040 wait complete. 2022-08-01T22:55:53.216 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:55:53.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:55:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.373+0000 7f7d1fa0d700 1 Processor -- start 2022-08-01T22:55:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.373+0000 7f7d1fa0d700 1 -- start start 2022-08-01T22:55:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1fa0d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 0x7f7d180abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1fa0d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d18144eb0 con 0x7f7d180b1110 2022-08-01T22:55:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1fa0d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d18145110 con 0x7f7d180aa050 2022-08-01T22:55:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1fa0d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180ac390 con 0x7f7d18144ad0 2022-08-01T22:55:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1dfaa700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d180b1110 0x7f7d1813e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56260/0 (socket says 172.21.15.32:56260) 2022-08-01T22:55:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1dfaa700 1 -- 172.21.15.32:0/3197756579 learned_addr learned my addr 172.21.15.32:0/3197756579 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1d7a9700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 0x7f7d180abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705766769 0 0) 0x7f7d18144eb0 con 0x7f7d180b1110 2022-08-01T22:55:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cfc003260 con 0x7f7d180b1110 2022-08-01T22:55:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111478721 0 0) 0x7f7d18145110 con 0x7f7d180aa050 2022-08-01T22:55:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d18144eb0 con 0x7f7d180aa050 2022-08-01T22:55:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888004317 0 0) 0x7f7cfc003260 con 0x7f7d180b1110 2022-08-01T22:55:53.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 msgr2=0x7f7d180abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 0x7f7d180abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 >> v1:172.21.15.55:6789/0 conn(0x7f7d180aa050 legacy=0x7f7d180b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d181453b0 con 0x7f7d180b1110 2022-08-01T22:55:53.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f7d10001680 con 0x7f7d180b1110 2022-08-01T22:55:53.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 >> v1:172.21.15.32:6789/0 conn(0x7f7d180b1110 legacy=0x7f7d1813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 0x7f7cfc003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc003e70 0x7f7cfc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0049f0 0x7f7cfc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180ac390 con 0x7f7cfc003e70 2022-08-01T22:55:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc00b830 con 0x7f7cfc0049f0 2022-08-01T22:55:53.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc00b9f0 con 0x7f7d18144ad0 2022-08-01T22:55:53.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc003e70 0x7f7cfc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1d7a9700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 0x7f7cfc003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1dfaa700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0049f0 0x7f7cfc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 -- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 msgr2=0x7f7cfc003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18144ad0 0x7f7cfc003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 -- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0049f0 msgr2=0x7f7cfc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0049f0 0x7f7cfc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 -- 172.21.15.32:0/3197756579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc003260 con 0x7f7cfc003e70 2022-08-01T22:55:53.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1dfaa700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0049f0 0x7f7cfc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:53.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d1cfa8700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc003e70 0x7f7cfc004500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7d08001110 tx=0x7f7d08007a00).ready entity=mon.0 client_cookie=2e8a4abc38388161 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d08007d10 con 0x7f7cfc003e70 2022-08-01T22:55:53.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08007030 con 0x7f7cfc003e70 2022-08-01T22:55:53.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.377+0000 7f7d0ffff700 1 -- 172.21.15.32:0/3197756579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d08007390 con 0x7f7cfc003e70 2022-08-01T22:55:53.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3197756579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc003e70 msgr2=0x7f7cfc004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3197756579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc003e70 0x7f7cfc004500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7d08001110 tx=0x7f7d08007a00).stop 2022-08-01T22:55:53.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 -- 172.21.15.32:0/3197756579 reap_dead start 2022-08-01T22:55:53.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3197756579 shutdown_connections 2022-08-01T22:55:53.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3197756579 >> 172.21.15.32:0/3197756579 conn(0x7f7d180b2b00 msgr2=0x7f7d180b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3197756579 shutdown_connections 2022-08-01T22:55:53.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3197756579 wait complete. 2022-08-01T22:55:53.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 Processor -- start 2022-08-01T22:55:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- start start 2022-08-01T22:55:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d181404a0 0x7f7d18139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18060930 0x7f7d18139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 0x7f7d180ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18152d00 con 0x7f7d180ae0a0 2022-08-01T22:55:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180618c0 con 0x7f7d181404a0 2022-08-01T22:55:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18140880 con 0x7f7d18060930 2022-08-01T22:55:53.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 0x7f7d180ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 0x7f7d180ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40500/0 (socket says 172.21.15.32:40500) 2022-08-01T22:55:53.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1cfa8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18060930 0x7f7d18139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 -- 172.21.15.32:0/3480059540 learned_addr learned my addr 172.21.15.32:0/3480059540 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1d7a9700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d181404a0 0x7f7d18139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 -- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18060930 msgr2=0x7f7d18139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18060930 0x7f7d18139880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 -- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d181404a0 msgr2=0x7f7d18139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d181404a0 0x7f7d18139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 -- 172.21.15.32:0/3480059540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180b0b70 con 0x7f7d180ae0a0 2022-08-01T22:55:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1dfaa700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 0x7f7d180ae550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f7d10000e90 tx=0x7f7d180b27e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d10014430 con 0x7f7d180ae0a0 2022-08-01T22:55:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10014590 con 0x7f7d180ae0a0 2022-08-01T22:55:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d18060e40 con 0x7f7d180ae0a0 2022-08-01T22:55:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d100148f0 con 0x7f7d180ae0a0 2022-08-01T22:55:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.381+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d18061380 con 0x7f7d180ae0a0 2022-08-01T22:55:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.385+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18139f10 con 0x7f7d180ae0a0 2022-08-01T22:55:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.385+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d10014a50 con 0x7f7d180ae0a0 2022-08-01T22:55:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.385+0000 7f7d0e7fc700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053b00 0x7f7d00055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.385+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7d1006b9d0 con 0x7f7d180ae0a0 2022-08-01T22:55:53.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.385+0000 7f7d1d7a9700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053b00 0x7f7d00055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.385+0000 7f7d1d7a9700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053b00 0x7f7d00055fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7d1400b000 tx=0x7f7d140091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.393+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d10042790 con 0x7f7d180ae0a0 2022-08-01T22:55:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.517+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7d180add90 con 0x7f7d180ae0a0 2022-08-01T22:55:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d0e7fc700 1 -- 172.21.15.32:0/3480059540 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7d10045db0 con 0x7f7d180ae0a0 2022-08-01T22:55:53.520 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:55:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053b00 msgr2=0x7f7d00055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053b00 0x7f7d00055fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7d1400b000 tx=0x7f7d140091d0).stop 2022-08-01T22:55:53.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 msgr2=0x7f7d180ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 0x7f7d180ae550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f7d10000e90 tx=0x7f7d180b27e0).stop 2022-08-01T22:55:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 shutdown_connections 2022-08-01T22:55:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053b00 0x7f7d00055fc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d180ae0a0 0x7f7d180ae550 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d18060930 0x7f7d18139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 --2- 172.21.15.32:0/3480059540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d181404a0 0x7f7d18139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 >> 172.21.15.32:0/3480059540 conn(0x7f7d180b2b00 msgr2=0x7f7d1813e740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 shutdown_connections 2022-08-01T22:55:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.521+0000 7f7d1fa0d700 1 -- 172.21.15.32:0/3480059540 wait complete. 2022-08-01T22:55:53.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:55:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 Processor -- start 2022-08-01T22:55:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 -- start start 2022-08-01T22:55:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6dbc0af550 0x7f6dbc0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dbc060330 0x7f6dbc1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc144ad0 0x7f6dbc0a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b15e0 con 0x7f6dbc060330 2022-08-01T22:55:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0aa630 con 0x7f6dbc0af550 2022-08-01T22:55:53.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc343d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc05f840 con 0x7f6dbc144ad0 2022-08-01T22:55:53.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dbc060330 0x7f6dbc1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dbc060330 0x7f6dbc1445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40506/0 (socket says 172.21.15.32:40506) 2022-08-01T22:55:53.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 -- 172.21.15.32:0/80306961 learned_addr learned my addr 172.21.15.32:0/80306961 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc2c3c700 1 --2- 172.21.15.32:0/80306961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc144ad0 0x7f6dbc0a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc243b700 1 --2- 172.21.15.32:0/80306961 >> v2:172.21.15.55:3300/0 conn(0x7f6dbc0af550 0x7f6dbc0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 -- 172.21.15.32:0/80306961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc144ad0 msgr2=0x7f6dbc0a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 --2- 172.21.15.32:0/80306961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc144ad0 0x7f6dbc0a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 -- 172.21.15.32:0/80306961 >> v2:172.21.15.55:3300/0 conn(0x7f6dbc0af550 msgr2=0x7f6dbc0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 --2- 172.21.15.32:0/80306961 >> v2:172.21.15.55:3300/0 conn(0x7f6dbc0af550 0x7f6dbc0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 -- 172.21.15.32:0/80306961 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc05fd10 con 0x7f6dbc060330 2022-08-01T22:55:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.649+0000 7f6dc1c3a700 1 --2- 172.21.15.32:0/80306961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc060330 0x7f6dbc1445e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f6dac002f70 tx=0x7f6dac006eb0).ready entity=mon.0 client_cookie=b008caf2e267c500 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc1439700 1 -- 172.21.15.32:0/80306961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dac00d570 con 0x7f6dbc060330 2022-08-01T22:55:53.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc1439700 1 -- 172.21.15.32:0/80306961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac000e90 con 0x7f6dbc060330 2022-08-01T22:55:53.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc1439700 1 -- 172.21.15.32:0/80306961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dac00d900 con 0x7f6dbc060330 2022-08-01T22:55:53.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 -- 172.21.15.32:0/80306961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc060330 msgr2=0x7f6dbc1445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 --2- 172.21.15.32:0/80306961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc060330 0x7f6dbc1445e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f6dac002f70 tx=0x7f6dac006eb0).stop 2022-08-01T22:55:53.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 -- 172.21.15.32:0/80306961 shutdown_connections 2022-08-01T22:55:53.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 --2- 172.21.15.32:0/80306961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc060330 0x7f6dbc1445e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 --2- 172.21.15.32:0/80306961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc144ad0 0x7f6dbc0a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 --2- 172.21.15.32:0/80306961 >> v2:172.21.15.55:3300/0 conn(0x7f6dbc0af550 0x7f6dbc0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 -- 172.21.15.32:0/80306961 >> 172.21.15.32:0/80306961 conn(0x7f6dbc0612d0 msgr2=0x7f6dbc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 -- 172.21.15.32:0/80306961 shutdown_connections 2022-08-01T22:55:53.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 -- 172.21.15.32:0/80306961 wait complete. 2022-08-01T22:55:53.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.653+0000 7f6dc343d700 1 Processor -- start 2022-08-01T22:55:53.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 -- start start 2022-08-01T22:55:53.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dbc060330 0x7f6dbc0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc0af550 0x7f6dbc0ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 0x7f6dbc09ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0a0b80 con 0x7f6dbc144ad0 2022-08-01T22:55:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0a0e20 con 0x7f6dbc060330 2022-08-01T22:55:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0a10c0 con 0x7f6dbc0af550 2022-08-01T22:55:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 0x7f6dbc09ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 0x7f6dbc09ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40512/0 (socket says 172.21.15.32:40512) 2022-08-01T22:55:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 -- 172.21.15.32:0/3632258868 learned_addr learned my addr 172.21.15.32:0/3632258868 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc1c3a700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc0af550 0x7f6dbc0ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc243b700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dbc060330 0x7f6dbc0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 -- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc0af550 msgr2=0x7f6dbc0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc0af550 0x7f6dbc0ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 -- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dbc060330 msgr2=0x7f6dbc0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dbc060330 0x7f6dbc0ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 -- 172.21.15.32:0/3632258868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac00b1e0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc1c3a700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc0af550 0x7f6dbc0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc2c3c700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 0x7f6dbc09ea40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6db4006e00 tx=0x7f6db40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db400c850 con 0x7f6dbc144ad0 2022-08-01T22:55:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db400c9b0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc0a1360 con 0x7f6dbc144ad0 2022-08-01T22:55:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db400cd10 con 0x7f6dbc144ad0 2022-08-01T22:55:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.657+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc0abe30 con 0x7f6dbc144ad0 2022-08-01T22:55:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.661+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6db40123f0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.661+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc0a14f0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.661+0000 7f6dab7fe700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d9c0539e0 0x7f6d9c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.661+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6db406c450 con 0x7f6dbc144ad0 2022-08-01T22:55:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.661+0000 7f6dc243b700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d9c0539e0 0x7f6d9c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.661+0000 7f6dc243b700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d9c0539e0 0x7f6d9c055ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f6db800b000 tx=0x7f6db8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.665+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6db40431b0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.793+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dbc0574e0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.793+0000 7f6dab7fe700 1 -- 172.21.15.32:0/3632258868 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6db40467d0 con 0x7f6dbc144ad0 2022-08-01T22:55:53.793 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:55:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d9c0539e0 msgr2=0x7f6d9c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d9c0539e0 0x7f6d9c055ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f6db800b000 tx=0x7f6db8008040).stop 2022-08-01T22:55:53.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 msgr2=0x7f6dbc09ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 0x7f6dbc09ea40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6db4006e00 tx=0x7f6db40091d0).stop 2022-08-01T22:55:53.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 shutdown_connections 2022-08-01T22:55:53.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d9c0539e0 0x7f6d9c055ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dbc144ad0 0x7f6dbc09ea40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6dbc0af550 0x7f6dbc0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 --2- 172.21.15.32:0/3632258868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dbc060330 0x7f6dbc0ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 >> 172.21.15.32:0/3632258868 conn(0x7f6dbc0612d0 msgr2=0x7f6dbc0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 shutdown_connections 2022-08-01T22:55:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.797+0000 7f6dc343d700 1 -- 172.21.15.32:0/3632258868 wait complete. 2022-08-01T22:55:53.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:55:53.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.925+0000 7fe33c03a700 1 Processor -- start 2022-08-01T22:55:53.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.925+0000 7fe33c03a700 1 -- start start 2022-08-01T22:55:53.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe334146ae0 0x7fe3341403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3340b4820 0x7fe3340b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe334143cf0 con 0x7fe3340b4310 2022-08-01T22:55:53.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3341476a0 con 0x7fe3340b4820 2022-08-01T22:55:53.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe334143f90 con 0x7fe334146ae0 2022-08-01T22:55:53.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3340b4820 0x7fe3340b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3340b4820 0x7fe3340b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40522/0 (socket says 172.21.15.32:40522) 2022-08-01T22:55:53.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 -- 172.21.15.32:0/369957795 learned_addr learned my addr 172.21.15.32:0/369957795 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe339dd6700 1 --2- 172.21.15.32:0/369957795 >> v2:172.21.15.55:3300/0 conn(0x7fe334146ae0 0x7fe3341403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 -- 172.21.15.32:0/369957795 >> v2:172.21.15.55:3300/0 conn(0x7fe334146ae0 msgr2=0x7fe3341403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 --2- 172.21.15.32:0/369957795 >> v2:172.21.15.55:3300/0 conn(0x7fe334146ae0 0x7fe3341403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 -- 172.21.15.32:0/369957795 >> v1:172.21.15.55:6789/0 conn(0x7fe3340b4310 legacy=0x7fe3341408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 -- 172.21.15.32:0/369957795 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340b0e20 con 0x7fe3340b4820 2022-08-01T22:55:53.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33a5d7700 1 --2- 172.21.15.32:0/369957795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe3340b5060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe32c0032f0 tx=0x7fe32c009290).ready entity=mon.0 client_cookie=5196bd1cb8599a06 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe338dd4700 1 -- 172.21.15.32:0/369957795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe32c00b870 con 0x7fe3340b4820 2022-08-01T22:55:53.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe338dd4700 1 -- 172.21.15.32:0/369957795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c00b9d0 con 0x7fe3340b4820 2022-08-01T22:55:53.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe338dd4700 1 -- 172.21.15.32:0/369957795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe32c00bb40 con 0x7fe3340b4820 2022-08-01T22:55:53.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- 172.21.15.32:0/369957795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 msgr2=0x7fe3340b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 --2- 172.21.15.32:0/369957795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe3340b5060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe32c0032f0 tx=0x7fe32c009290).stop 2022-08-01T22:55:53.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- 172.21.15.32:0/369957795 shutdown_connections 2022-08-01T22:55:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 --2- 172.21.15.32:0/369957795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe3340b5060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 --2- 172.21.15.32:0/369957795 >> v2:172.21.15.55:3300/0 conn(0x7fe334146ae0 0x7fe3341403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- 172.21.15.32:0/369957795 >> 172.21.15.32:0/369957795 conn(0x7fe33405c620 msgr2=0x7fe33405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- 172.21.15.32:0/369957795 shutdown_connections 2022-08-01T22:55:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.929+0000 7fe33c03a700 1 -- 172.21.15.32:0/369957795 wait complete. 2022-08-01T22:55:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 Processor -- start 2022-08-01T22:55:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 -- start start 2022-08-01T22:55:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 0x7fe3340ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe334152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe334146ae0 0x7fe3340afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340b7a70 con 0x7fe3340b4820 2022-08-01T22:55:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340b87d0 con 0x7fe334146ae0 2022-08-01T22:55:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.933+0000 7fe33c03a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340b8a70 con 0x7fe3340b4310 2022-08-01T22:55:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 0x7fe3340ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe3395d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe334152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 0x7fe3340ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52262/0 (socket says 172.21.15.32:52262) 2022-08-01T22:55:53.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe3395d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe334152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40528/0 (socket says 172.21.15.32:40528) 2022-08-01T22:55:53.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 -- 172.21.15.32:0/3449594244 learned_addr learned my addr 172.21.15.32:0/3449594244 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:53.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe33a5d7700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe334146ae0 0x7fe3340afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 -- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe334146ae0 msgr2=0x7fe3340afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe334146ae0 0x7fe3340afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 -- 172.21.15.32:0/3449594244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 msgr2=0x7fe334152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe334152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 -- 172.21.15.32:0/3449594244 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c009530 con 0x7fe3340b4310 2022-08-01T22:55:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe339dd6700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 0x7fe3340ade50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe3300030b0 tx=0x7fe330008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe330011780 con 0x7fe3340b4310 2022-08-01T22:55:53.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3300118e0 con 0x7fe3340b4310 2022-08-01T22:55:53.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3340b8d10 con 0x7fe3340b4310 2022-08-01T22:55:53.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe330011c40 con 0x7fe3340b4310 2022-08-01T22:55:53.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe334153360 con 0x7fe3340b4310 2022-08-01T22:55:53.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340574e0 con 0x7fe3340b4310 2022-08-01T22:55:53.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.937+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe33001c460 con 0x7fe3340b4310 2022-08-01T22:55:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.941+0000 7fe326ffd700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3140539e0 0x7fe314055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.941+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe33006c440 con 0x7fe3340b4310 2022-08-01T22:55:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.941+0000 7fe3395d5700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3140539e0 0x7fe314055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.941+0000 7fe3395d5700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3140539e0 0x7fe314055ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe32800d000 tx=0x7fe328007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:53.949+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe330043120 con 0x7fe3340b4310 2022-08-01T22:55:54.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.077+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3340b9230 con 0x7fe3340b4310 2022-08-01T22:55:54.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.077+0000 7fe326ffd700 1 -- 172.21.15.32:0/3449594244 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe330046740 con 0x7fe3340b4310 2022-08-01T22:55:54.079 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:55:54.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3140539e0 msgr2=0x7fe314055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3140539e0 0x7fe314055ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe32800d000 tx=0x7fe328007b70).stop 2022-08-01T22:55:54.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 msgr2=0x7fe3340ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 0x7fe3340ade50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe3300030b0 tx=0x7fe330008040).stop 2022-08-01T22:55:54.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 shutdown_connections 2022-08-01T22:55:54.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3140539e0 0x7fe314055ea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe334146ae0 0x7fe3340afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3340b4820 0x7fe334152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 --2- 172.21.15.32:0/3449594244 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3340b4310 0x7fe3340ade50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 >> 172.21.15.32:0/3449594244 conn(0x7fe33405c620 msgr2=0x7fe33413e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 shutdown_connections 2022-08-01T22:55:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.081+0000 7fe33c03a700 1 -- 172.21.15.32:0/3449594244 wait complete. 2022-08-01T22:55:54.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:55:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381d504700 1 Processor -- start 2022-08-01T22:55:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381d504700 1 -- start start 2022-08-01T22:55:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381d504700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38180a9da0 0x7f3818148540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381d504700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180b3570 con 0x7f38180b1a80 2022-08-01T22:55:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381d504700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180b0e00 con 0x7f38180b3190 2022-08-01T22:55:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381d504700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38180b10a0 con 0x7f38180a9da0 2022-08-01T22:55:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f38177fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38180b1a80 0x7f3818147e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56306/0 (socket says 172.21.15.32:56306) 2022-08-01T22:55:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f38177fe700 1 -- 172.21.15.32:0/2550607067 learned_addr learned my addr 172.21.15.32:0/2550607067 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f381cd03700 1 --2- 172.21.15.32:0/2550607067 >> v2:172.21.15.55:3300/0 conn(0x7f38180a9da0 0x7f3818148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351168859 0 0) 0x7f38180b3570 con 0x7f38180b1a80 2022-08-01T22:55:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3804003260 con 0x7f38180b1a80 2022-08-01T22:55:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246582156 0 0) 0x7f38180b0e00 con 0x7f38180b3190 2022-08-01T22:55:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.201+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38180b3570 con 0x7f38180b3190 2022-08-01T22:55:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381cd03700 1 -- 172.21.15.32:0/2550607067 >> v1:172.21.15.55:6789/0 conn(0x7f38180b3190 legacy=0x7f38180656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381cd03700 1 -- 172.21.15.32:0/2550607067 >> v1:172.21.15.32:6789/0 conn(0x7f38180b1a80 legacy=0x7f3818147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2859796481 0 0) 0x7f3804003260 con 0x7f38180b1a80 2022-08-01T22:55:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381cd03700 1 -- 172.21.15.32:0/2550607067 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180b1340 con 0x7f38180a9da0 2022-08-01T22:55:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381cd03700 1 --2- 172.21.15.32:0/2550607067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180a9da0 0x7f3818148540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f3808001110 tx=0x7f38100091d0).ready entity=mon.1 client_cookie=9b1f1135bccf5bc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3808001660 con 0x7f38180a9da0 2022-08-01T22:55:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381000f510 con 0x7f38180a9da0 2022-08-01T22:55:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f3816ffd700 1 -- 172.21.15.32:0/2550607067 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f381000f680 con 0x7f38180a9da0 2022-08-01T22:55:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 -- 172.21.15.32:0/2550607067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180a9da0 msgr2=0x7f3818148540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 --2- 172.21.15.32:0/2550607067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180a9da0 0x7f3818148540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f3808001110 tx=0x7f38100091d0).stop 2022-08-01T22:55:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 -- 172.21.15.32:0/2550607067 shutdown_connections 2022-08-01T22:55:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 --2- 172.21.15.32:0/2550607067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180a9da0 0x7f3818148540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 -- 172.21.15.32:0/2550607067 >> 172.21.15.32:0/2550607067 conn(0x7f3818063870 msgr2=0x7f3818063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 -- 172.21.15.32:0/2550607067 shutdown_connections 2022-08-01T22:55:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 -- 172.21.15.32:0/2550607067 wait complete. 2022-08-01T22:55:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 Processor -- start 2022-08-01T22:55:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.205+0000 7f381d504700 1 -- start start 2022-08-01T22:55:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38180a9da0 0x7f3818146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180b1a80 0x7f3818157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 0x7f38181458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818147380 con 0x7f38180b3190 2022-08-01T22:55:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818147610 con 0x7f38180b1a80 2022-08-01T22:55:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3818156bc0 con 0x7f38180a9da0 2022-08-01T22:55:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 0x7f38181458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 0x7f38181458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40542/0 (socket says 172.21.15.32:40542) 2022-08-01T22:55:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 -- 172.21.15.32:0/1258854567 learned_addr learned my addr 172.21.15.32:0/1258854567 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f38177fe700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180b1a80 0x7f3818157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f3817fff700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38180a9da0 0x7f3818146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 -- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38180a9da0 msgr2=0x7f3818146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38180a9da0 0x7f3818146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 -- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180b1a80 msgr2=0x7f3818157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180b1a80 0x7f3818157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 -- 172.21.15.32:0/1258854567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009470 con 0x7f38180b3190 2022-08-01T22:55:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381cd03700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 0x7f38181458a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f3808001110 tx=0x7f3810016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38100032d0 con 0x7f38180b3190 2022-08-01T22:55:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3810011b80 con 0x7f38180b3190 2022-08-01T22:55:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3810021440 con 0x7f38180b3190 2022-08-01T22:55:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3818156e60 con 0x7f38180b3190 2022-08-01T22:55:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3818145d90 con 0x7f38180b3190 2022-08-01T22:55:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.209+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818156ff0 con 0x7f38180b3190 2022-08-01T22:55:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.213+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3810027020 con 0x7f38180b3190 2022-08-01T22:55:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.213+0000 7f38157fa700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37f8053b30 0x7f37f8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.213+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f381006cfd0 con 0x7f38180b3190 2022-08-01T22:55:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.213+0000 7f3817fff700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37f8053b30 0x7f37f8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.213+0000 7f3817fff700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37f8053b30 0x7f37f8055ff0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f380c00d000 tx=0x7f380c007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.221+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3810043d30 con 0x7f38180b3190 2022-08-01T22:55:54.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.341+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38180574e0 con 0x7f38180b3190 2022-08-01T22:55:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.341+0000 7f38157fa700 1 -- 172.21.15.32:0/1258854567 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f38100150a0 con 0x7f38180b3190 2022-08-01T22:55:54.343 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:55:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37f8053b30 msgr2=0x7f37f8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37f8053b30 0x7f37f8055ff0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f380c00d000 tx=0x7f380c007a20).stop 2022-08-01T22:55:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 msgr2=0x7f38181458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 0x7f38181458a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f3808001110 tx=0x7f3810016000).stop 2022-08-01T22:55:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 shutdown_connections 2022-08-01T22:55:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37f8053b30 0x7f37f8055ff0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38180b3190 0x7f38181458a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38180b1a80 0x7f3818157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.345+0000 7f381d504700 1 --2- 172.21.15.32:0/1258854567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38180a9da0 0x7f3818146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.349+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 >> 172.21.15.32:0/1258854567 conn(0x7f3818063870 msgr2=0x7f3818066290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.349+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 shutdown_connections 2022-08-01T22:55:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.349+0000 7f381d504700 1 -- 172.21.15.32:0/1258854567 wait complete. 2022-08-01T22:55:54.361 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2022-08-01T22:55:54.362 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2022-08-01T22:55:54.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2022-08-01T22:55:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 Processor -- start 2022-08-01T22:55:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 -- start start 2022-08-01T22:55:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29240b01a0 0x7f29240a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29240afd60 0x7f29240a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29240a7250 con 0x7f29240b2710 2022-08-01T22:55:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240a7480 con 0x7f29240b01a0 2022-08-01T22:55:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f292950e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240a7720 con 0x7f29240afd60 2022-08-01T22:55:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f2923fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29240b2710 0x7f292414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56320/0 (socket says 172.21.15.32:56320) 2022-08-01T22:55:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f2923fff700 1 -- 172.21.15.32:0/3106124535 learned_addr learned my addr 172.21.15.32:0/3106124535 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 --2- 172.21.15.32:0/3106124535 >> v2:172.21.15.55:3300/0 conn(0x7f29240b01a0 0x7f29240a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682957634 0 0) 0x7f29240a7250 con 0x7f29240b2710 2022-08-01T22:55:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f2928d0d700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29240afd60 0x7f29240a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2904003260 con 0x7f29240b2710 2022-08-01T22:55:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482437598 0 0) 0x7f2904003260 con 0x7f29240b2710 2022-08-01T22:55:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29240afd60 msgr2=0x7f29240a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29240afd60 0x7f29240a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 >> v2:172.21.15.55:3300/0 conn(0x7f29240b01a0 msgr2=0x7f29240a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:54.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 --2- 172.21.15.32:0/3106124535 >> v2:172.21.15.55:3300/0 conn(0x7f29240b01a0 0x7f29240a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240a79c0 con 0x7f29240b2710 2022-08-01T22:55:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f2914001660 con 0x7f29240b2710 2022-08-01T22:55:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 >> v1:172.21.15.32:6789/0 conn(0x7f29240b2710 legacy=0x7f292414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904003750 0x7f2904003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2904004500 0x7f2904004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2904004e70 0x7f290400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240a7480 con 0x7f2904004500 2022-08-01T22:55:54.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240a7720 con 0x7f2904004e70 2022-08-01T22:55:54.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290400bc40 con 0x7f2904003750 2022-08-01T22:55:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2904004500 0x7f2904004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f2928d0d700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29240afd60 0x7f29240a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f2923fff700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904003750 0x7f2904003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 -- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904003750 msgr2=0x7f2904003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:54.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904003750 0x7f2904003b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 -- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2904004e70 msgr2=0x7f290400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2904004e70 0x7f290400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T22:55:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.469+0000 7f291b7fe700 1 -- 172.21.15.32:0/3106124535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904003260 con 0x7f2904004500 2022-08-01T22:55:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f291b7fe700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2904004500 0x7f2904004980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f290c00c000 tx=0x7f290c006df0).ready entity=mon.0 client_cookie=a9a71b050d2419a8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f290c00e800 con 0x7f2904004500 2022-08-01T22:55:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c00e960 con 0x7f2904004500 2022-08-01T22:55:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f29237fe700 1 -- 172.21.15.32:0/3106124535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f290c00ecc0 con 0x7f2904004500 2022-08-01T22:55:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f292950e700 1 -- 172.21.15.32:0/3106124535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2904004500 msgr2=0x7f2904004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f292950e700 1 --2- 172.21.15.32:0/3106124535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2904004500 0x7f2904004980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f290c00c000 tx=0x7f290c006df0).stop 2022-08-01T22:55:54.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f2928d0d700 1 -- 172.21.15.32:0/3106124535 reap_dead start 2022-08-01T22:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f292950e700 1 -- 172.21.15.32:0/3106124535 shutdown_connections 2022-08-01T22:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f292950e700 1 -- 172.21.15.32:0/3106124535 >> 172.21.15.32:0/3106124535 conn(0x7f2924060310 msgr2=0x7f29240b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f292950e700 1 -- 172.21.15.32:0/3106124535 shutdown_connections 2022-08-01T22:55:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.473+0000 7f292950e700 1 -- 172.21.15.32:0/3106124535 wait complete. 2022-08-01T22:55:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 Processor -- start 2022-08-01T22:55:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 -- start start 2022-08-01T22:55:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904004500 0x7f2924065f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 0x7f2924159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29240668f0 0x7f292415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240ac350 con 0x7f2924066470 2022-08-01T22:55:54.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240ac7e0 con 0x7f29240668f0 2022-08-01T22:55:54.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240aadf0 con 0x7f2904004500 2022-08-01T22:55:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 0x7f2924159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 0x7f2924159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40560/0 (socket says 172.21.15.32:40560) 2022-08-01T22:55:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 -- 172.21.15.32:0/239159684 learned_addr learned my addr 172.21.15.32:0/239159684 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f2923fff700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904004500 0x7f2924065f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f2928d0d700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29240668f0 0x7f292415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 -- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904004500 msgr2=0x7f2924065f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904004500 0x7f2924065f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 -- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29240668f0 msgr2=0x7f292415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29240668f0 0x7f292415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 -- 172.21.15.32:0/239159684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240abcd0 con 0x7f2924066470 2022-08-01T22:55:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f291b7fe700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 0x7f2924159c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2924157130 tx=0x7f290c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f290c01b820 con 0x7f2924066470 2022-08-01T22:55:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c000e90 con 0x7f2924066470 2022-08-01T22:55:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29240ac540 con 0x7f2924066470 2022-08-01T22:55:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f290c01bb70 con 0x7f2924066470 2022-08-01T22:55:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2924157410 con 0x7f2924066470 2022-08-01T22:55:54.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.477+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f290c01bcd0 con 0x7f2924066470 2022-08-01T22:55:54.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.481+0000 7f2921ffb700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2908053a10 0x7f2908055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.481+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f290c06d490 con 0x7f2924066470 2022-08-01T22:55:54.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.481+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29240abe60 con 0x7f2924066470 2022-08-01T22:55:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.481+0000 7f2923fff700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2908053a10 0x7f2908055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.481+0000 7f2923fff700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2908053a10 0x7f2908055ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f291400e000 tx=0x7f2914009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.485+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f290c044140 con 0x7f2924066470 2022-08-01T22:55:54.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.641+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f29240574e0 con 0x7f2924066470 2022-08-01T22:55:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.645+0000 7f2921ffb700 1 -- 172.21.15.32:0/239159684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+131773 (secure 0 0 0) 0x7f290c047760 con 0x7f2924066470 2022-08-01T22:55:54.649 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:54.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2908053a10 msgr2=0x7f2908055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2908053a10 0x7f2908055ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f291400e000 tx=0x7f2914009040).stop 2022-08-01T22:55:54.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 msgr2=0x7f2924159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 0x7f2924159c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2924157130 tx=0x7f290c019000).stop 2022-08-01T22:55:54.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 shutdown_connections 2022-08-01T22:55:54.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2908053a10 0x7f2908055ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29240668f0 0x7f292415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2924066470 0x7f2924159c50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 --2- 172.21.15.32:0/239159684 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2904004500 0x7f2924065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 >> 172.21.15.32:0/239159684 conn(0x7f2924060310 msgr2=0x7f29240678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 shutdown_connections 2022-08-01T22:55:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.649+0000 7f292950e700 1 -- 172.21.15.32:0/239159684 wait complete. 2022-08-01T22:55:54.662 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":22889},{"type":"v1","addr":"172.21.15.55:6801","nonce":22889}]},"active_addr":"172.21.15.55:6801/22889","active_change":"2022-08-01T22:55:34.201973+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":816641280}]},{"addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":675249355}]},{"addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":321641444}]}]}} 2022-08-01T22:55:54.672 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2022-08-01T22:55:54.673 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2022-08-01T22:55:54.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0837e2e700 1 Processor -- start 2022-08-01T22:55:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0837e2e700 1 -- start start 2022-08-01T22:55:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0837e2e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f083013d590 0x7f0830146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0837e2e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0830147570 con 0x7f08300b68f0 2022-08-01T22:55:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0837e2e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08300ac6e0 con 0x7f08300aa210 2022-08-01T22:55:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0837e2e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08300ac980 con 0x7f083013d590 2022-08-01T22:55:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f08353c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f083013d590 0x7f0830146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0835bca700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f08300b68f0 0x7f083013ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56340/0 (socket says 172.21.15.32:56340) 2022-08-01T22:55:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0835bca700 1 -- 172.21.15.32:0/272151621 learned_addr learned my addr 172.21.15.32:0/272151621 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f08353c9700 1 -- 172.21.15.32:0/272151621 >> v1:172.21.15.55:6789/0 conn(0x7f08300aa210 legacy=0x7f0830146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f0834bc8700 1 -- 172.21.15.32:0/272151621 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233161186 0 0) 0x7f08300ac6e0 con 0x7f08300aa210 2022-08-01T22:55:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f08353c9700 1 -- 172.21.15.32:0/272151621 >> v1:172.21.15.32:6789/0 conn(0x7f08300b68f0 legacy=0x7f083013ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.777+0000 7f08353c9700 1 -- 172.21.15.32:0/272151621 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300acc20 con 0x7f083013d590 2022-08-01T22:55:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f08353c9700 1 --2- 172.21.15.32:0/272151621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083013d590 0x7f0830146970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f082c001110 tx=0x7f0820006df0).ready entity=mon.0 client_cookie=7e90fd3e4781f9a3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0834bc8700 1 -- 172.21.15.32:0/272151621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082000d5c0 con 0x7f083013d590 2022-08-01T22:55:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0834bc8700 1 -- 172.21.15.32:0/272151621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0820000e90 con 0x7f083013d590 2022-08-01T22:55:54.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0834bc8700 1 -- 172.21.15.32:0/272151621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082000d910 con 0x7f083013d590 2022-08-01T22:55:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- 172.21.15.32:0/272151621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083013d590 msgr2=0x7f0830146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 --2- 172.21.15.32:0/272151621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083013d590 0x7f0830146970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f082c001110 tx=0x7f0820006df0).stop 2022-08-01T22:55:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- 172.21.15.32:0/272151621 shutdown_connections 2022-08-01T22:55:54.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 --2- 172.21.15.32:0/272151621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083013d590 0x7f0830146970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- 172.21.15.32:0/272151621 >> 172.21.15.32:0/272151621 conn(0x7f08300b3610 msgr2=0x7f08300b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- 172.21.15.32:0/272151621 shutdown_connections 2022-08-01T22:55:54.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- 172.21.15.32:0/272151621 wait complete. 2022-08-01T22:55:54.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 Processor -- start 2022-08-01T22:55:54.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- start start 2022-08-01T22:55:54.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08300aa210 0x7f0830153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 0x7f08301458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f083013d590 0x7f0830155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08301527b0 con 0x7f08300b68f0 2022-08-01T22:55:54.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830152a50 con 0x7f08300aa210 2022-08-01T22:55:54.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f0837e2e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0830152cf0 con 0x7f083013d590 2022-08-01T22:55:54.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f08353c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 0x7f08301458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f08353c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 0x7f08301458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40574/0 (socket says 172.21.15.32:40574) 2022-08-01T22:55:54.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.781+0000 7f08353c9700 1 -- 172.21.15.32:0/3780419320 learned_addr learned my addr 172.21.15.32:0/3780419320 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:54.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f0835bca700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08300aa210 0x7f0830153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08363cb700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f083013d590 0x7f0830155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08353c9700 1 -- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f083013d590 msgr2=0x7f0830155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08353c9700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f083013d590 0x7f0830155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08353c9700 1 -- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08300aa210 msgr2=0x7f0830153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08353c9700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08300aa210 0x7f0830153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08353c9700 1 -- 172.21.15.32:0/3780419320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082000b180 con 0x7f08300b68f0 2022-08-01T22:55:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f08353c9700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 0x7f08301458d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f082c001110 tx=0x7f0820014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0820016070 con 0x7f08300b68f0 2022-08-01T22:55:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0830063540 con 0x7f08300b68f0 2022-08-01T22:55:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082000de40 con 0x7f08300b68f0 2022-08-01T22:55:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0820013da0 con 0x7f08300b68f0 2022-08-01T22:55:54.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08300631c0 con 0x7f08300b68f0 2022-08-01T22:55:54.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0830152eb0 con 0x7f08300b68f0 2022-08-01T22:55:54.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.785+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f082000f030 con 0x7f08300b68f0 2022-08-01T22:55:54.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.789+0000 7f081effd700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0810053aa0 0x7f0810055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:54.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.789+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f082006cd30 con 0x7f08300b68f0 2022-08-01T22:55:54.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.789+0000 7f0835bca700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0810053aa0 0x7f0810055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:54.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.789+0000 7f0835bca700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0810053aa0 0x7f0810055f60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f082c00d000 tx=0x7f082c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:54.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.793+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0820043a50 con 0x7f08300b68f0 2022-08-01T22:55:54.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.917+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08300acec0 con 0x7f08300b68f0 2022-08-01T22:55:54.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.917+0000 7f081effd700 1 -- 172.21.15.32:0/3780419320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f0820023070 con 0x7f08300b68f0 2022-08-01T22:55:54.920 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:54.920 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:54.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0810053aa0 msgr2=0x7f0810055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0810053aa0 0x7f0810055f60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f082c00d000 tx=0x7f082c009040).stop 2022-08-01T22:55:54.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 msgr2=0x7f08301458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:54.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 0x7f08301458d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f082c001110 tx=0x7f0820014000).stop 2022-08-01T22:55:54.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 shutdown_connections 2022-08-01T22:55:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0810053aa0 0x7f0810055f60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f083013d590 0x7f0830155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300b68f0 0x7f08301458d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 --2- 172.21.15.32:0/3780419320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08300aa210 0x7f0830153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 >> 172.21.15.32:0/3780419320 conn(0x7f08300b3610 msgr2=0x7f0830139270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:54.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 shutdown_connections 2022-08-01T22:55:54.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:54.921+0000 7f0837e2e700 1 -- 172.21.15.32:0/3780419320 wait complete. 2022-08-01T22:55:54.936 INFO:tasks.ceph.ceph_manager.ceph:all up! 2022-08-01T22:55:54.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:55:55.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fdeeb700 1 Processor -- start 2022-08-01T22:55:55.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fdeeb700 1 -- start start 2022-08-01T22:55:55.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fdeeb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f80b2730 0x7f04f80666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fdeeb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f80654c0 con 0x7f04f80b3190 2022-08-01T22:55:55.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fdeeb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f80b3570 con 0x7f04f8066ba0 2022-08-01T22:55:55.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fdeeb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f8062190 con 0x7f04f80b2730 2022-08-01T22:55:55.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fcee9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04f80b3190 0x7f04f8060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56354/0 (socket says 172.21.15.32:56354) 2022-08-01T22:55:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04fcee9700 1 -- 172.21.15.32:0/937443190 learned_addr learned my addr 172.21.15.32:0/937443190 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f7fff700 1 --2- 172.21.15.32:0/937443190 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f80b2730 0x7f04f80666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882664067 0 0) 0x7f04f80654c0 con 0x7f04f80b3190 2022-08-01T22:55:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04dc003260 con 0x7f04f80b3190 2022-08-01T22:55:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376872177 0 0) 0x7f04f80b3570 con 0x7f04f8066ba0 2022-08-01T22:55:55.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f80654c0 con 0x7f04f8066ba0 2022-08-01T22:55:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204571502 0 0) 0x7f04dc003260 con 0x7f04f80b3190 2022-08-01T22:55:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f7fff700 1 -- 172.21.15.32:0/937443190 >> v1:172.21.15.55:6789/0 conn(0x7f04f8066ba0 legacy=0x7f04f814dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f7fff700 1 -- 172.21.15.32:0/937443190 >> v1:172.21.15.32:6789/0 conn(0x7f04f80b3190 legacy=0x7f04f8060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f7fff700 1 -- 172.21.15.32:0/937443190 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f805fbf0 con 0x7f04f80b2730 2022-08-01T22:55:55.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f7fff700 1 --2- 172.21.15.32:0/937443190 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f80b2730 0x7f04f80666b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f04ec001110 tx=0x7f04e4006df0).ready entity=mon.2 client_cookie=21d0656dea250022 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04ec001660 con 0x7f04f80b2730 2022-08-01T22:55:55.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4000e90 con 0x7f04f80b2730 2022-08-01T22:55:55.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.040+0000 7f04f77fe700 1 -- 172.21.15.32:0/937443190 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04e4012460 con 0x7f04f80b2730 2022-08-01T22:55:55.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- 172.21.15.32:0/937443190 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f80b2730 msgr2=0x7f04f80666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/937443190 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f80b2730 0x7f04f80666b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f04ec001110 tx=0x7f04e4006df0).stop 2022-08-01T22:55:55.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- 172.21.15.32:0/937443190 shutdown_connections 2022-08-01T22:55:55.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/937443190 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f80b2730 0x7f04f80666b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- 172.21.15.32:0/937443190 >> 172.21.15.32:0/937443190 conn(0x7f04f8063870 msgr2=0x7f04f8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- 172.21.15.32:0/937443190 shutdown_connections 2022-08-01T22:55:55.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- 172.21.15.32:0/937443190 wait complete. 2022-08-01T22:55:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 Processor -- start 2022-08-01T22:55:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- start start 2022-08-01T22:55:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f8066ba0 0x7f04f814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04f80b2730 0x7f04f80b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 0x7f04f80b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f80accf0 con 0x7f04f80b3190 2022-08-01T22:55:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f80acf20 con 0x7f04f80b2730 2022-08-01T22:55:55.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fdeeb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f80ad1c0 con 0x7f04f8066ba0 2022-08-01T22:55:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 0x7f04f80b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 0x7f04f80b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40588/0 (socket says 172.21.15.32:40588) 2022-08-01T22:55:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 -- 172.21.15.32:0/88079563 learned_addr learned my addr 172.21.15.32:0/88079563 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fcee9700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f8066ba0 0x7f04f814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04f7fff700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04f80b2730 0x7f04f80b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 -- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f8066ba0 msgr2=0x7f04f814a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f8066ba0 0x7f04f814a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 -- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04f80b2730 msgr2=0x7f04f80b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04f80b2730 0x7f04f80b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04fd6ea700 1 -- 172.21.15.32:0/88079563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e400b180 con 0x7f04f80b3190 2022-08-01T22:55:55.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.044+0000 7f04f7fff700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04f80b2730 0x7f04f80b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:55:55.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04fd6ea700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 0x7f04f80b0550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f04f0001490 tx=0x7f04f0007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f0002310 con 0x7f04f80b3190 2022-08-01T22:55:55.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f0002470 con 0x7f04f80b3190 2022-08-01T22:55:55.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f80ac250 con 0x7f04f80b3190 2022-08-01T22:55:55.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f0001840 con 0x7f04f80b3190 2022-08-01T22:55:55.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f8155910 con 0x7f04f80b3190 2022-08-01T22:55:55.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f04f0010030 con 0x7f04f80b3190 2022-08-01T22:55:55.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f80add60 con 0x7f04f80b3190 2022-08-01T22:55:55.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04f5ffb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f04d8053aa0 0x7f04d8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f04f006be30 con 0x7f04f80b3190 2022-08-01T22:55:55.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04fcee9700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f04d8053aa0 0x7f04d8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.048+0000 7f04fcee9700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f04d8053aa0 0x7f04d8055f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f04ec00d000 tx=0x7f04ec000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.056+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04f0042bf0 con 0x7f04f80b3190 2022-08-01T22:55:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.180+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04f805fe90 con 0x7f04f80b3190 2022-08-01T22:55:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04f5ffb700 1 -- 172.21.15.32:0/88079563 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f04f0046210 con 0x7f04f80b3190 2022-08-01T22:55:55.185 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:55:55.186 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:55:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f04d8053aa0 msgr2=0x7f04d8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f04d8053aa0 0x7f04d8055f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f04ec00d000 tx=0x7f04ec000bc0).stop 2022-08-01T22:55:55.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 msgr2=0x7f04f80b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 0x7f04f80b0550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f04f0001490 tx=0x7f04f0007dd0).stop 2022-08-01T22:55:55.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 shutdown_connections 2022-08-01T22:55:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f04d8053aa0 0x7f04d8055f60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04f80b3190 0x7f04f80b0550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04f80b2730 0x7f04f80b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 --2- 172.21.15.32:0/88079563 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04f8066ba0 0x7f04f814a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 >> 172.21.15.32:0/88079563 conn(0x7f04f8063870 msgr2=0x7f04f81461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 shutdown_connections 2022-08-01T22:55:55.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.184+0000 7f04fdeeb700 1 -- 172.21.15.32:0/88079563 wait complete. 2022-08-01T22:55:55.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2022-08-01T22:55:55.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.300+0000 7fad2eb9e700 1 Processor -- start 2022-08-01T22:55:55.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.300+0000 7fad2eb9e700 1 -- start start 2022-08-01T22:55:55.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad280b0630 0x7fad280adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad280ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad280b0a10 0x7fad281438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280b2c80 con 0x7fad280b0a10 2022-08-01T22:55:55.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280b2900 con 0x7fad280b0630 2022-08-01T22:55:55.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280b2eb0 con 0x7fad280aa2b0 2022-08-01T22:55:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad280b0a10 0x7fad281438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad280b0a10 0x7fad281438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40598/0 (socket says 172.21.15.32:40598) 2022-08-01T22:55:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 -- 172.21.15.32:0/4033934302 learned_addr learned my addr 172.21.15.32:0/4033934302 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad27fff700 1 --2- 172.21.15.32:0/4033934302 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad280ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2c93a700 1 --2- 172.21.15.32:0/4033934302 >> v2:172.21.15.55:3300/0 conn(0x7fad280b0630 0x7fad280adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 -- 172.21.15.32:0/4033934302 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 msgr2=0x7fad280ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 --2- 172.21.15.32:0/4033934302 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad280ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 -- 172.21.15.32:0/4033934302 >> v2:172.21.15.55:3300/0 conn(0x7fad280b0630 msgr2=0x7fad280adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 --2- 172.21.15.32:0/4033934302 >> v2:172.21.15.55:3300/0 conn(0x7fad280b0630 0x7fad280adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 -- 172.21.15.32:0/4033934302 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280a9d00 con 0x7fad280b0a10 2022-08-01T22:55:55.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2d13b700 1 --2- 172.21.15.32:0/4033934302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad281438a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fad280b2bd0 tx=0x7fad20009290).ready entity=mon.0 client_cookie=3f693465aa3a1bd4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad277fe700 1 -- 172.21.15.32:0/4033934302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2000b680 con 0x7fad280b0a10 2022-08-01T22:55:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad277fe700 1 -- 172.21.15.32:0/4033934302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2000b7e0 con 0x7fad280b0a10 2022-08-01T22:55:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad277fe700 1 -- 172.21.15.32:0/4033934302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2000b950 con 0x7fad280b0a10 2022-08-01T22:55:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4033934302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 msgr2=0x7fad281438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4033934302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad281438a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fad280b2bd0 tx=0x7fad20009290).stop 2022-08-01T22:55:55.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4033934302 shutdown_connections 2022-08-01T22:55:55.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4033934302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad281438a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4033934302 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad280ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4033934302 >> v2:172.21.15.55:3300/0 conn(0x7fad280b0630 0x7fad280adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4033934302 >> 172.21.15.32:0/4033934302 conn(0x7fad28060950 msgr2=0x7fad28060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4033934302 shutdown_connections 2022-08-01T22:55:55.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.304+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4033934302 wait complete. 2022-08-01T22:55:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 Processor -- start 2022-08-01T22:55:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 -- start start 2022-08-01T22:55:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad28130f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad280b0630 0x7fad28152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad28152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28137460 con 0x7fad280b0a10 2022-08-01T22:55:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28138e70 con 0x7fad280b0630 2022-08-01T22:55:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad28139110 con 0x7fad280aa2b0 2022-08-01T22:55:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad28152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad28152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40604/0 (socket says 172.21.15.32:40604) 2022-08-01T22:55:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 -- 172.21.15.32:0/4195936489 learned_addr learned my addr 172.21.15.32:0/4195936489 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2c93a700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad28130f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad27fff700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad280b0630 0x7fad28152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 -- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 msgr2=0x7fad28130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad280aa2b0 0x7fad28130f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 -- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad280b0630 msgr2=0x7fad28152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad280b0630 0x7fad28152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 -- 172.21.15.32:0/4195936489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20009530 con 0x7fad280b0a10 2022-08-01T22:55:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2d13b700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad28152b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad280b2bd0 tx=0x7fad20016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2000b680 con 0x7fad280b0a10 2022-08-01T22:55:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad200107a0 con 0x7fad280b0a10 2022-08-01T22:55:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad281393b0 con 0x7fad280b0a10 2022-08-01T22:55:55.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad20010b00 con 0x7fad280b0a10 2022-08-01T22:55:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.308+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad28140830 con 0x7fad280b0a10 2022-08-01T22:55:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad28139850 con 0x7fad280b0a10 2022-08-01T22:55:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad2000f070 con 0x7fad280b0a10 2022-08-01T22:55:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad25ffb700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fad2006d0c0 con 0x7fad280b0a10 2022-08-01T22:55:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad2c93a700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad25ffb700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7fad0c05b3f0 0x7fad0c05d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 --> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad0c05de70 con 0x7fad0c05b3f0 2022-08-01T22:55:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2006d4a0 con 0x7fad280b0a10 2022-08-01T22:55:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad27fff700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7fad0c05b3f0 0x7fad0c05d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad27fff700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7fad0c05b3f0 0x7fad0c05d890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.312+0000 7fad2c93a700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fad1800b000 tx=0x7fad18006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.316+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== osd.0 v2:172.21.15.32:6824/23003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fad0c05de70 con 0x7fad0c05b3f0 2022-08-01T22:55:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 --> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad2813a580 con 0x7fad0c05b3f0 2022-08-01T22:55:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad25ffb700 1 -- 172.21.15.32:0/4195936489 <== osd.0 v2:172.21.15.32:6824/23003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad10000e90 con 0x7fad0c05b3f0 2022-08-01T22:55:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7fad0c05b3f0 msgr2=0x7fad0c05d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7fad0c05b3f0 0x7fad0c05d890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 msgr2=0x7fad0c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fad1800b000 tx=0x7fad18006df0).stop 2022-08-01T22:55:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 msgr2=0x7fad28152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 --2- 172.21.15.32:0/4195936489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad280b0a10 0x7fad28152b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad280b2bd0 tx=0x7fad20016000).stop 2022-08-01T22:55:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2d13b700 1 -- 172.21.15.32:0/4195936489 reap_dead start 2022-08-01T22:55:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 shutdown_connections 2022-08-01T22:55:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 >> 172.21.15.32:0/4195936489 conn(0x7fad28060950 msgr2=0x7fad28060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 shutdown_connections 2022-08-01T22:55:55.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.332+0000 7fad2eb9e700 1 -- 172.21.15.32:0/4195936489 wait complete. 2022-08-01T22:55:55.344 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:55.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2022-08-01T22:55:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 Processor -- start 2022-08-01T22:55:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- start start 2022-08-01T22:55:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dac0b6710 0x7f5dac0b7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0b6af0 con 0x7f5dac0b5cd0 2022-08-01T22:55:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dac147640 con 0x7f5dac0b6710 2022-08-01T22:55:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0b6250 con 0x7f5dac1481d0 2022-08-01T22:55:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dac0b6710 0x7f5dac0b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dac0b6710 0x7f5dac0b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40610/0 (socket says 172.21.15.32:40610) 2022-08-01T22:55:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 -- 172.21.15.32:0/367806039 learned_addr learned my addr 172.21.15.32:0/367806039 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db1c5b700 1 --2- 172.21.15.32:0/367806039 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5da3fff700 1 -- 172.21.15.32:0/367806039 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554300952 0 0) 0x7f5dac0b6af0 con 0x7f5dac0b5cd0 2022-08-01T22:55:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 -- 172.21.15.32:0/367806039 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 msgr2=0x7f5dac144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 --2- 172.21.15.32:0/367806039 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 -- 172.21.15.32:0/367806039 >> v1:172.21.15.32:6789/0 conn(0x7f5dac0b5cd0 legacy=0x7f5dac0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 -- 172.21.15.32:0/367806039 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac147830 con 0x7f5dac0b6710 2022-08-01T22:55:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db145a700 1 --2- 172.21.15.32:0/367806039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac0b7010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5da8002f70 tx=0x7f5da8006eb0).ready entity=mon.0 client_cookie=9ee33254b0707a18 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5da3fff700 1 -- 172.21.15.32:0/367806039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5da800d5c0 con 0x7f5dac0b6710 2022-08-01T22:55:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5da3fff700 1 -- 172.21.15.32:0/367806039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da8000e90 con 0x7f5dac0b6710 2022-08-01T22:55:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5da3fff700 1 -- 172.21.15.32:0/367806039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5da800d910 con 0x7f5dac0b6710 2022-08-01T22:55:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- 172.21.15.32:0/367806039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 msgr2=0x7f5dac0b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 --2- 172.21.15.32:0/367806039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac0b7010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5da8002f70 tx=0x7f5da8006eb0).stop 2022-08-01T22:55:55.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- 172.21.15.32:0/367806039 shutdown_connections 2022-08-01T22:55:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 --2- 172.21.15.32:0/367806039 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 --2- 172.21.15.32:0/367806039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac0b7010 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- 172.21.15.32:0/367806039 >> 172.21.15.32:0/367806039 conn(0x7f5dac05c620 msgr2=0x7f5dac05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- 172.21.15.32:0/367806039 shutdown_connections 2022-08-01T22:55:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.452+0000 7f5db36be700 1 -- 172.21.15.32:0/367806039 wait complete. 2022-08-01T22:55:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 Processor -- start 2022-08-01T22:55:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 -- start start 2022-08-01T22:55:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dac0b5cd0 0x7f5dac145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac14a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac14af30 con 0x7f5dac0b6710 2022-08-01T22:55:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac14b3e0 con 0x7f5dac0b5cd0 2022-08-01T22:55:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac14b680 con 0x7f5dac1481d0 2022-08-01T22:55:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac14a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac14a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40618/0 (socket says 172.21.15.32:40618) 2022-08-01T22:55:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 -- 172.21.15.32:0/3824678625 learned_addr learned my addr 172.21.15.32:0/3824678625 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db145a700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dac0b5cd0 0x7f5dac145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db1c5b700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 -- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 msgr2=0x7f5dac159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5dac1481d0 0x7f5dac159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 -- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dac0b5cd0 msgr2=0x7f5dac145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dac0b5cd0 0x7f5dac145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 -- 172.21.15.32:0/3824678625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da800b1e0 con 0x7f5dac0b6710 2022-08-01T22:55:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db0c59700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac14a890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f5d9c000bc0 tx=0x7f5d9c007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d9c013450 con 0x7f5dac0b6710 2022-08-01T22:55:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0135b0 con 0x7f5dac0b6710 2022-08-01T22:55:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dac14b920 con 0x7f5dac0b6710 2022-08-01T22:55:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d9c013910 con 0x7f5dac0b6710 2022-08-01T22:55:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.456+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac0663e0 con 0x7f5dac0b6710 2022-08-01T22:55:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5dac14bab0 con 0x7f5dac0b6710 2022-08-01T22:55:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d9c01a060 con 0x7f5dac0b6710 2022-08-01T22:55:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5da27fc700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d94053a10 0x7f5d94055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5d9c06c090 con 0x7f5dac0b6710 2022-08-01T22:55:55.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5da27fc700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7f5d9405b390 0x7f5d9405d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 --> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d9405deb0 con 0x7f5d9405b390 2022-08-01T22:55:55.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9c06c470 con 0x7f5dac0b6710 2022-08-01T22:55:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5db145a700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d94053a10 0x7f5d94055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5db1c5b700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7f5d9405b390 0x7f5d9405d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5db1c5b700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7f5d9405b390 0x7f5d9405d830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.460+0000 7f5db145a700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d94053a10 0x7f5d94055ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f5da801c000 tx=0x7f5da8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.464+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== osd.1 v2:172.21.15.32:6800/23005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f5d9405deb0 con 0x7f5d9405b390 2022-08-01T22:55:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.480+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 --> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5dac15aa10 con 0x7f5d9405b390 2022-08-01T22:55:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.480+0000 7f5da27fc700 1 -- 172.21.15.32:0/3824678625 <== osd.1 v2:172.21.15.32:6800/23005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5dac15aa10 con 0x7f5d9405b390 2022-08-01T22:55:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7f5d9405b390 msgr2=0x7f5d9405d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7f5d9405b390 0x7f5d9405d830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d94053a10 msgr2=0x7f5d94055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d94053a10 0x7f5d94055ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f5da801c000 tx=0x7f5da8015000).stop 2022-08-01T22:55:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 msgr2=0x7f5dac14a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 --2- 172.21.15.32:0/3824678625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dac0b6710 0x7f5dac14a890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f5d9c000bc0 tx=0x7f5d9c007a30).stop 2022-08-01T22:55:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db1c5b700 1 -- 172.21.15.32:0/3824678625 reap_dead start 2022-08-01T22:55:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 shutdown_connections 2022-08-01T22:55:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 >> 172.21.15.32:0/3824678625 conn(0x7f5dac05c620 msgr2=0x7f5dac143eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 shutdown_connections 2022-08-01T22:55:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.484+0000 7f5db36be700 1 -- 172.21.15.32:0/3824678625 wait complete. 2022-08-01T22:55:55.500 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2022-08-01T22:55:55.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2022-08-01T22:55:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.612+0000 7f362ffff700 1 Processor -- start 2022-08-01T22:55:55.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.612+0000 7f362ffff700 1 -- start start 2022-08-01T22:55:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f36300614b0 0x7f3630061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36300ad750 0x7f36300b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36301405d0 0x7f3630140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36300aee80 con 0x7f36300ad750 2022-08-01T22:55:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36300b70b0 con 0x7f36300614b0 2022-08-01T22:55:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300af0b0 con 0x7f36301405d0 2022-08-01T22:55:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36300ad750 0x7f36300b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36300ad750 0x7f36300b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40626/0 (socket says 172.21.15.32:40626) 2022-08-01T22:55:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 -- 172.21.15.32:0/1811234668 learned_addr learned my addr 172.21.15.32:0/1811234668 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362f7fe700 1 --2- 172.21.15.32:0/1811234668 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36301405d0 0x7f3630140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362effd700 1 --2- 172.21.15.32:0/1811234668 >> v2:172.21.15.55:3300/0 conn(0x7f36300614b0 0x7f3630061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 -- 172.21.15.32:0/1811234668 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36301405d0 msgr2=0x7f3630140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 --2- 172.21.15.32:0/1811234668 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36301405d0 0x7f3630140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 -- 172.21.15.32:0/1811234668 >> v2:172.21.15.55:3300/0 conn(0x7f36300614b0 msgr2=0x7f3630061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 --2- 172.21.15.32:0/1811234668 >> v2:172.21.15.55:3300/0 conn(0x7f36300614b0 0x7f3630061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 -- 172.21.15.32:0/1811234668 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300af350 con 0x7f36300ad750 2022-08-01T22:55:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362effd700 1 --2- 172.21.15.32:0/1811234668 >> v2:172.21.15.55:3300/0 conn(0x7f36300614b0 0x7f3630061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f36267fc700 1 --2- 172.21.15.32:0/1811234668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36300ad750 0x7f36300b9ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f36300aed30 tx=0x7f3618006eb0).ready entity=mon.0 client_cookie=79b2095dc843943f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362e7fc700 1 -- 172.21.15.32:0/1811234668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361800d570 con 0x7f36300ad750 2022-08-01T22:55:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362e7fc700 1 -- 172.21.15.32:0/1811234668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618000e90 con 0x7f36300ad750 2022-08-01T22:55:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362e7fc700 1 -- 172.21.15.32:0/1811234668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361800d900 con 0x7f36300ad750 2022-08-01T22:55:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- 172.21.15.32:0/1811234668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36300ad750 msgr2=0x7f36300b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- 172.21.15.32:0/1811234668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36300ad750 0x7f36300b9ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f36300aed30 tx=0x7f3618006eb0).stop 2022-08-01T22:55:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- 172.21.15.32:0/1811234668 shutdown_connections 2022-08-01T22:55:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- 172.21.15.32:0/1811234668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36300ad750 0x7f36300b9ff0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- 172.21.15.32:0/1811234668 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36301405d0 0x7f3630140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 --2- 172.21.15.32:0/1811234668 >> v2:172.21.15.55:3300/0 conn(0x7f36300614b0 0x7f3630061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- 172.21.15.32:0/1811234668 >> 172.21.15.32:0/1811234668 conn(0x7f3630066f70 msgr2=0x7f3630067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- 172.21.15.32:0/1811234668 shutdown_connections 2022-08-01T22:55:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.616+0000 7f362ffff700 1 -- 172.21.15.32:0/1811234668 wait complete. 2022-08-01T22:55:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 Processor -- start 2022-08-01T22:55:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 -- start start 2022-08-01T22:55:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36300614b0 0x7f36300b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36300ad750 0x7f36300ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36301405d0 0x7f36300aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b18b0 con 0x7f36301405d0 2022-08-01T22:55:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363009f020 con 0x7f36300614b0 2022-08-01T22:55:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b2920 con 0x7f36300ad750 2022-08-01T22:55:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36301405d0 0x7f36300aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36301405d0 0x7f36300aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40634/0 (socket says 172.21.15.32:40634) 2022-08-01T22:55:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 -- 172.21.15.32:0/658159780 learned_addr learned my addr 172.21.15.32:0/658159780 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362effd700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36300614b0 0x7f36300b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f36267fc700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36300ad750 0x7f36300ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 -- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36300ad750 msgr2=0x7f36300ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36300ad750 0x7f36300ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 -- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36300614b0 msgr2=0x7f36300b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36300614b0 0x7f36300b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 -- 172.21.15.32:0/658159780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361800b1e0 con 0x7f36301405d0 2022-08-01T22:55:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.620+0000 7f362f7fe700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36301405d0 0x7f36300aca30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f363009f160 tx=0x7f36280091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3628012600 con 0x7f36301405d0 2022-08-01T22:55:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628012760 con 0x7f36301405d0 2022-08-01T22:55:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3630144260 con 0x7f36301405d0 2022-08-01T22:55:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3628012ac0 con 0x7f36301405d0 2022-08-01T22:55:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36300b1610 con 0x7f36301405d0 2022-08-01T22:55:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f363009ed60 con 0x7f36301405d0 2022-08-01T22:55:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3628022070 con 0x7f36301405d0 2022-08-01T22:55:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3614053a70 0x7f3614055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f362806c4c0 con 0x7f36301405d0 2022-08-01T22:55:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362effd700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3614053a70 0x7f3614055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f361405b3f0 0x7f361405d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 --> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f361405de70 con 0x7f361405b3f0 2022-08-01T22:55:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f362806eb50 con 0x7f36301405d0 2022-08-01T22:55:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f36267fc700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f361405b3f0 0x7f361405d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f36267fc700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f361405b3f0 0x7f361405d890 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.624+0000 7f362effd700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3614053a70 0x7f3614055f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f362000c000 tx=0x7f3620008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.628+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== osd.2 v2:172.21.15.32:6808/23004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f361405de70 con 0x7f361405b3f0 2022-08-01T22:55:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 --> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36300af5f0 con 0x7f361405b3f0 2022-08-01T22:55:55.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362cff9700 1 -- 172.21.15.32:0/658159780 <== osd.2 v2:172.21.15.32:6808/23004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f36300af5f0 con 0x7f361405b3f0 2022-08-01T22:55:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f361405b3f0 msgr2=0x7f361405d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f361405b3f0 0x7f361405d890 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3614053a70 msgr2=0x7f3614055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3614053a70 0x7f3614055f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f362000c000 tx=0x7f3620008040).stop 2022-08-01T22:55:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36301405d0 msgr2=0x7f36300aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 --2- 172.21.15.32:0/658159780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36301405d0 0x7f36300aca30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f363009f160 tx=0x7f36280091d0).stop 2022-08-01T22:55:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362f7fe700 1 -- 172.21.15.32:0/658159780 reap_dead start 2022-08-01T22:55:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 shutdown_connections 2022-08-01T22:55:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 >> 172.21.15.32:0/658159780 conn(0x7f3630066f70 msgr2=0x7f3630144c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 shutdown_connections 2022-08-01T22:55:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.644+0000 7f362ffff700 1 -- 172.21.15.32:0/658159780 wait complete. 2022-08-01T22:55:55.659 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:55.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2022-08-01T22:55:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.764+0000 7f4d4ffff700 1 Processor -- start 2022-08-01T22:55:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.764+0000 7f4d4ffff700 1 -- start start 2022-08-01T22:55:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d501466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d500aee80 con 0x7f4d500ad750 2022-08-01T22:55:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d500b70b0 con 0x7f4d500614b0 2022-08-01T22:55:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500af0b0 con 0x7f4d50061890 2022-08-01T22:55:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4d500ad750 0x7f4d501405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56416/0 (socket says 172.21.15.32:56416) 2022-08-01T22:55:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 -- 172.21.15.32:0/1915604288 learned_addr learned my addr 172.21.15.32:0/1915604288 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4f7fe700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d501466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652599827 0 0) 0x7f4d500aee80 con 0x7f4d500ad750 2022-08-01T22:55:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d34003260 con 0x7f4d500ad750 2022-08-01T22:55:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897486162 0 0) 0x7f4d34003260 con 0x7f4d500ad750 2022-08-01T22:55:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 msgr2=0x7f4d501466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d501466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 >> v1:172.21.15.55:6789/0 conn(0x7f4d500614b0 legacy=0x7f4d500b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500af350 con 0x7f4d500ad750 2022-08-01T22:55:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f4d40001660 con 0x7f4d500ad750 2022-08-01T22:55:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 >> v1:172.21.15.32:6789/0 conn(0x7f4d500ad750 legacy=0x7f4d501405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d34003780 0x7f4d34003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d34004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d34004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500af0b0 con 0x7f4d34004500 2022-08-01T22:55:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3400b780 con 0x7f4d34003780 2022-08-01T22:55:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3400b940 con 0x7f4d50061890 2022-08-01T22:55:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d34004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4f7fe700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d34004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4effd700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d34003780 0x7f4d34003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 -- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 msgr2=0x7f4d34004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d34004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 -- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d34003780 msgr2=0x7f4d34003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d34003780 0x7f4d34003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 -- 172.21.15.32:0/1915604288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d34003260 con 0x7f4d34004500 2022-08-01T22:55:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4f7fe700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d50061890 0x7f4d34004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4effd700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d34003780 0x7f4d34003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:55:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d34004980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f4d40001370 tx=0x7f4d500af4e0).ready entity=mon.0 client_cookie=3cd1b930e68c5f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d400070d0 con 0x7f4d34004500 2022-08-01T22:55:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d40007230 con 0x7f4d34004500 2022-08-01T22:55:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.768+0000 7f4d4dffb700 1 -- 172.21.15.32:0/1915604288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4000c8c0 con 0x7f4d34004500 2022-08-01T22:55:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- 172.21.15.32:0/1915604288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 msgr2=0x7f4d34004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 --2- 172.21.15.32:0/1915604288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d34004980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f4d40001370 tx=0x7f4d500af4e0).stop 2022-08-01T22:55:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4f7fe700 1 -- 172.21.15.32:0/1915604288 reap_dead start 2022-08-01T22:55:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- 172.21.15.32:0/1915604288 shutdown_connections 2022-08-01T22:55:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- 172.21.15.32:0/1915604288 >> 172.21.15.32:0/1915604288 conn(0x7f4d50066f70 msgr2=0x7f4d50067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- 172.21.15.32:0/1915604288 shutdown_connections 2022-08-01T22:55:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- 172.21.15.32:0/1915604288 wait complete. 2022-08-01T22:55:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 Processor -- start 2022-08-01T22:55:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- start start 2022-08-01T22:55:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d500b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d500b2930 0x7f4d50155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d501560e0 0x7f4d500abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500b1420 con 0x7f4d34004500 2022-08-01T22:55:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500b0990 con 0x7f4d500b2930 2022-08-01T22:55:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500b06f0 con 0x7f4d501560e0 2022-08-01T22:55:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d500b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d500b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40652/0 (socket says 172.21.15.32:40652) 2022-08-01T22:55:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4effd700 1 -- 172.21.15.32:0/2025485041 learned_addr learned my addr 172.21.15.32:0/2025485041 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d500b2930 0x7f4d50155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.772+0000 7f4d4f7fe700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d501560e0 0x7f4d500abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4effd700 1 -- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d501560e0 msgr2=0x7f4d500abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4effd700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4d501560e0 0x7f4d500abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4effd700 1 -- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d500b2930 msgr2=0x7f4d50155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4effd700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d500b2930 0x7f4d50155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4effd700 1 -- 172.21.15.32:0/2025485041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500b0ce0 con 0x7f4d34004500 2022-08-01T22:55:55.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4effd700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d500b7fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4d500b1560 tx=0x7f4d44009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4400de40 con 0x7f4d34004500 2022-08-01T22:55:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d440104e0 con 0x7f4d34004500 2022-08-01T22:55:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d500b0f80 con 0x7f4d34004500 2022-08-01T22:55:55.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d44010840 con 0x7f4d34004500 2022-08-01T22:55:55.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d50156590 con 0x7f4d34004500 2022-08-01T22:55:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d500b1110 con 0x7f4d34004500 2022-08-01T22:55:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4d4401a030 con 0x7f4d34004500 2022-08-01T22:55:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4d2c053a70 0x7f4d2c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4d4406ba10 con 0x7f4d34004500 2022-08-01T22:55:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7f4d2c05b3f0 0x7f4d2c05d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 --> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d2c05de70 con 0x7f4d2c05b3f0 2022-08-01T22:55:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.776+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d44070240 con 0x7f4d34004500 2022-08-01T22:55:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.780+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4d2c053a70 0x7f4d2c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.780+0000 7f4d4f7fe700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7f4d2c05b3f0 0x7f4d2c05d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.780+0000 7f4d4f7fe700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7f4d2c05b3f0 0x7f4d2c05d890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.780+0000 7f4d4e7fc700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4d2c053a70 0x7f4d2c055f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4d4001b000 tx=0x7f4d40013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.780+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== osd.3 v2:172.21.15.32:6816/23015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f4d2c05de70 con 0x7f4d2c05b3f0 2022-08-01T22:55:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 --> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d50142940 con 0x7f4d2c05b3f0 2022-08-01T22:55:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d3ffff700 1 -- 172.21.15.32:0/2025485041 <== osd.3 v2:172.21.15.32:6816/23015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4d50142940 con 0x7f4d2c05b3f0 2022-08-01T22:55:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7f4d2c05b3f0 msgr2=0x7f4d2c05d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7f4d2c05b3f0 0x7f4d2c05d890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4d2c053a70 msgr2=0x7f4d2c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4d2c053a70 0x7f4d2c055f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4d4001b000 tx=0x7f4d40013000).stop 2022-08-01T22:55:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 msgr2=0x7f4d500b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 --2- 172.21.15.32:0/2025485041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d34004500 0x7f4d500b7fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4d500b1560 tx=0x7f4d44009040).stop 2022-08-01T22:55:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4f7fe700 1 -- 172.21.15.32:0/2025485041 reap_dead start 2022-08-01T22:55:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 shutdown_connections 2022-08-01T22:55:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.800+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 >> 172.21.15.32:0/2025485041 conn(0x7f4d50066f70 msgr2=0x7f4d501469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.804+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 shutdown_connections 2022-08-01T22:55:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.804+0000 7f4d4ffff700 1 -- 172.21.15.32:0/2025485041 wait complete. 2022-08-01T22:55:55.826 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:55.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2022-08-01T22:55:55.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 Processor -- start 2022-08-01T22:55:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 -- start start 2022-08-01T22:55:55.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa2c0b1f40 0x7faa2c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa2c148530 0x7faa2c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0b04e0 con 0x7faa2c0a7250 2022-08-01T22:55:55.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c0a9250 con 0x7faa2c148530 2022-08-01T22:55:55.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa33d01700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c0b0780 con 0x7faa2c0b1f40 2022-08-01T22:55:55.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa3129c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa2c148530 0x7faa2c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa3129c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa2c148530 0x7faa2c0b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40662/0 (socket says 172.21.15.32:40662) 2022-08-01T22:55:55.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa3129c700 1 -- 172.21.15.32:0/3593068453 learned_addr learned my addr 172.21.15.32:0/3593068453 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.936+0000 7faa31a9d700 1 --2- 172.21.15.32:0/3593068453 >> v2:172.21.15.55:3300/0 conn(0x7faa2c0b1f40 0x7faa2c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa3129c700 1 -- 172.21.15.32:0/3593068453 >> v2:172.21.15.55:3300/0 conn(0x7faa2c0b1f40 msgr2=0x7faa2c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:55.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa3129c700 1 --2- 172.21.15.32:0/3593068453 >> v2:172.21.15.55:3300/0 conn(0x7faa2c0b1f40 0x7faa2c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa3129c700 1 -- 172.21.15.32:0/3593068453 >> v1:172.21.15.55:6789/0 conn(0x7faa2c0a7250 legacy=0x7faa2c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa3129c700 1 -- 172.21.15.32:0/3593068453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0b0a20 con 0x7faa2c148530 2022-08-01T22:55:55.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa3129c700 1 --2- 172.21.15.32:0/3593068453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c148530 0x7faa2c0b2320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7faa2c1478c0 tx=0x7faa1c009290).ready entity=mon.0 client_cookie=c5b13eb85857a475 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa30a9b700 1 -- 172.21.15.32:0/3593068453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1c00b860 con 0x7faa2c148530 2022-08-01T22:55:55.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa30a9b700 1 -- 172.21.15.32:0/3593068453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c00b9c0 con 0x7faa2c148530 2022-08-01T22:55:55.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa30a9b700 1 -- 172.21.15.32:0/3593068453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1c00bb30 con 0x7faa2c148530 2022-08-01T22:55:55.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 -- 172.21.15.32:0/3593068453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c148530 msgr2=0x7faa2c0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 --2- 172.21.15.32:0/3593068453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c148530 0x7faa2c0b2320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7faa2c1478c0 tx=0x7faa1c009290).stop 2022-08-01T22:55:55.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 -- 172.21.15.32:0/3593068453 shutdown_connections 2022-08-01T22:55:55.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 --2- 172.21.15.32:0/3593068453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c148530 0x7faa2c0b2320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 --2- 172.21.15.32:0/3593068453 >> v2:172.21.15.55:3300/0 conn(0x7faa2c0b1f40 0x7faa2c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 -- 172.21.15.32:0/3593068453 >> 172.21.15.32:0/3593068453 conn(0x7faa2c05f840 msgr2=0x7faa2c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 -- 172.21.15.32:0/3593068453 shutdown_connections 2022-08-01T22:55:55.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.940+0000 7faa33d01700 1 -- 172.21.15.32:0/3593068453 wait complete. 2022-08-01T22:55:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 Processor -- start 2022-08-01T22:55:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 -- start start 2022-08-01T22:55:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa2c0a7250 0x7faa2c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c0b1f40 0x7faa2c156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa2c148530 0x7faa2c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c14b470 con 0x7faa2c0b1f40 2022-08-01T22:55:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0aa340 con 0x7faa2c148530 2022-08-01T22:55:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c14ba30 con 0x7faa2c0a7250 2022-08-01T22:55:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c0b1f40 0x7faa2c156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c0b1f40 0x7faa2c156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40670/0 (socket says 172.21.15.32:40670) 2022-08-01T22:55:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 -- 172.21.15.32:0/3359185554 learned_addr learned my addr 172.21.15.32:0/3359185554 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3229e700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa2c148530 0x7faa2c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa31a9d700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa2c0a7250 0x7faa2c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 -- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa2c0a7250 msgr2=0x7faa2c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa2c0a7250 0x7faa2c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 -- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa2c148530 msgr2=0x7faa2c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa2c148530 0x7faa2c0ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 -- 172.21.15.32:0/3359185554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009530 con 0x7faa2c0b1f40 2022-08-01T22:55:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa31a9d700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa2c0a7250 0x7faa2c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa3129c700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c0b1f40 0x7faa2c156fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7faa2c1478c0 tx=0x7faa1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1c021480 con 0x7faa2c0b1f40 2022-08-01T22:55:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c0215e0 con 0x7faa2c0b1f40 2022-08-01T22:55:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c157640 con 0x7faa2c0b1f40 2022-08-01T22:55:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1c021940 con 0x7faa2c0b1f40 2022-08-01T22:55:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.944+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0aa7a0 con 0x7faa2c0b1f40 2022-08-01T22:55:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa2c0a9e20 con 0x7faa2c0b1f40 2022-08-01T22:55:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faa1c00f070 con 0x7faa2c0b1f40 2022-08-01T22:55:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa2affd700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa0c053b30 0x7faa0c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7faa1c06d500 con 0x7faa2c0b1f40 2022-08-01T22:55:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa31a9d700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa0c053b30 0x7faa0c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa2affd700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7faa0c05b4b0 0x7faa0c05d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 --> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa0c05df30 con 0x7faa0c05b4b0 2022-08-01T22:55:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa1c06fa80 con 0x7faa2c0b1f40 2022-08-01T22:55:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa3229e700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7faa0c05b4b0 0x7faa0c05d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa31a9d700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa0c053b30 0x7faa0c055ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7faa2000c000 tx=0x7faa20008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.948+0000 7faa3229e700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7faa0c05b4b0 0x7faa0c05d950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.952+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== osd.4 v2:172.21.15.55:6819/23335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7faa0c05df30 con 0x7faa0c05b4b0 2022-08-01T22:55:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 --> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa2c0aa9a0 con 0x7faa0c05b4b0 2022-08-01T22:55:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa2affd700 1 -- 172.21.15.32:0/3359185554 <== osd.4 v2:172.21.15.55:6819/23335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faa24003730 con 0x7faa0c05b4b0 2022-08-01T22:55:55.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7faa0c05b4b0 msgr2=0x7faa0c05d950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7faa0c05b4b0 0x7faa0c05d950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:55.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa0c053b30 msgr2=0x7faa0c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa0c053b30 0x7faa0c055ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7faa2000c000 tx=0x7faa20008040).stop 2022-08-01T22:55:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c0b1f40 msgr2=0x7faa2c156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 --2- 172.21.15.32:0/3359185554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2c0b1f40 0x7faa2c156fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7faa2c1478c0 tx=0x7faa1c016000).stop 2022-08-01T22:55:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa3229e700 1 -- 172.21.15.32:0/3359185554 reap_dead start 2022-08-01T22:55:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 shutdown_connections 2022-08-01T22:55:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 >> 172.21.15.32:0/3359185554 conn(0x7faa2c05f840 msgr2=0x7faa2c143f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 shutdown_connections 2022-08-01T22:55:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:55.972+0000 7faa33d01700 1 -- 172.21.15.32:0/3359185554 wait complete. 2022-08-01T22:55:55.985 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:55.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2022-08-01T22:55:56.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 Processor -- start 2022-08-01T22:55:56.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 -- start start 2022-08-01T22:55:56.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 0x7fe8bc143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8bc0aa0d0 0x7fe8bc0af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0b3a10 con 0x7fe8bc0af6f0 2022-08-01T22:55:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc1443f0 con 0x7fe8bc0aa0d0 2022-08-01T22:55:56.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c4353700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0afb80 con 0x7fe8bc0b3630 2022-08-01T22:55:56.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c18ee700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8bc0aa0d0 0x7fe8bc0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 0x7fe8bc143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 0x7fe8bc143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52416/0 (socket says 172.21.15.32:52416) 2022-08-01T22:55:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 -- 172.21.15.32:0/2279245935 learned_addr learned my addr 172.21.15.32:0/2279245935 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c28f0700 1 --1- 172.21.15.32:0/2279245935 >> v1:172.21.15.55:6789/0 conn(0x7fe8bc0af6f0 0x7fe8bc05f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:49130/0 (socket says 172.21.15.32:49130) 2022-08-01T22:55:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 -- 172.21.15.32:0/2279245935 >> v2:172.21.15.55:3300/0 conn(0x7fe8bc0aa0d0 msgr2=0x7fe8bc0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 --2- 172.21.15.32:0/2279245935 >> v2:172.21.15.55:3300/0 conn(0x7fe8bc0aa0d0 0x7fe8bc0af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 -- 172.21.15.32:0/2279245935 >> v1:172.21.15.55:6789/0 conn(0x7fe8bc0af6f0 legacy=0x7fe8bc05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 -- 172.21.15.32:0/2279245935 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc146b40 con 0x7fe8bc0b3630 2022-08-01T22:55:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c20ef700 1 --2- 172.21.15.32:0/2279245935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 0x7fe8bc143f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe8b8002f70 tx=0x7fe8b8006eb0).ready entity=mon.2 client_cookie=4504a31f52d1515a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c10ed700 1 -- 172.21.15.32:0/2279245935 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8b800d570 con 0x7fe8bc0b3630 2022-08-01T22:55:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c10ed700 1 -- 172.21.15.32:0/2279245935 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8000e90 con 0x7fe8bc0b3630 2022-08-01T22:55:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.096+0000 7fe8c10ed700 1 -- 172.21.15.32:0/2279245935 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8b800d900 con 0x7fe8bc0b3630 2022-08-01T22:55:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 -- 172.21.15.32:0/2279245935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 msgr2=0x7fe8bc143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 --2- 172.21.15.32:0/2279245935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 0x7fe8bc143f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe8b8002f70 tx=0x7fe8b8006eb0).stop 2022-08-01T22:55:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 -- 172.21.15.32:0/2279245935 shutdown_connections 2022-08-01T22:55:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 --2- 172.21.15.32:0/2279245935 >> v2:172.21.15.55:3300/0 conn(0x7fe8bc0aa0d0 0x7fe8bc0af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 --2- 172.21.15.32:0/2279245935 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0b3630 0x7fe8bc143f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 -- 172.21.15.32:0/2279245935 >> 172.21.15.32:0/2279245935 conn(0x7fe8bc060950 msgr2=0x7fe8bc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 -- 172.21.15.32:0/2279245935 shutdown_connections 2022-08-01T22:55:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 -- 172.21.15.32:0/2279245935 wait complete. 2022-08-01T22:55:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 Processor -- start 2022-08-01T22:55:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.100+0000 7fe8c4353700 1 -- start start 2022-08-01T22:55:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0aa0d0 0x7fe8bc1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8bc0af6f0 0x7fe8bc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8bc0b3630 0x7fe8bc145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b44c0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc140d60 con 0x7fe8bc0af6f0 2022-08-01T22:55:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc143010 con 0x7fe8bc0aa0d0 2022-08-01T22:55:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8bc0b3630 0x7fe8bc145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c20ef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0aa0d0 0x7fe8bc1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c18ee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8bc0af6f0 0x7fe8bc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8bc0b3630 0x7fe8bc145170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40686/0 (socket says 172.21.15.32:40686) 2022-08-01T22:55:56.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c18ee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8bc0af6f0 0x7fe8bc0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:50472/0 (socket says 172.21.15.32:50472) 2022-08-01T22:55:56.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 -- 172.21.15.32:0/3743740308 learned_addr learned my addr 172.21.15.32:0/3743740308 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 -- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0aa0d0 msgr2=0x7fe8bc1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8bc0aa0d0 0x7fe8bc1497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 -- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8bc0af6f0 msgr2=0x7fe8bc0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8bc0af6f0 0x7fe8bc0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 -- 172.21.15.32:0/3743740308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b800b1e0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c28f0700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8bc0b3630 0x7fe8bc145170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe8bc0aec80 tx=0x7fe8b40013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8b4005ae0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4005c40 con 0x7fe8bc0b3630 2022-08-01T22:55:56.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc0b46b0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8b40070d0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0b51b0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.104+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8bc0b4b50 con 0x7fe8bc0b3630 2022-08-01T22:55:56.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe8b4010070 con 0x7fe8bc0b3630 2022-08-01T22:55:56.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8ab7fe700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe89c053a10 0x7fe89c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe8b406b7e0 con 0x7fe8bc0b3630 2022-08-01T22:55:56.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8ab7fe700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7fe89c05b390 0x7fe89c05d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 --> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe89c05deb0 con 0x7fe89c05b390 2022-08-01T22:55:56.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b4070240 con 0x7fe8bc0b3630 2022-08-01T22:55:56.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8c20ef700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe89c053a10 0x7fe89c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8c18ee700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7fe89c05b390 0x7fe89c05d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8c20ef700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe89c053a10 0x7fe89c055ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe8b801c000 tx=0x7fe8b8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.108+0000 7fe8c18ee700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7fe89c05b390 0x7fe89c05d830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.112+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== osd.5 v2:172.21.15.55:6802/23336 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fe89c05deb0 con 0x7fe89c05b390 2022-08-01T22:55:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 --> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8bc146de0 con 0x7fe89c05b390 2022-08-01T22:55:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8ab7fe700 1 -- 172.21.15.32:0/3743740308 <== osd.5 v2:172.21.15.55:6802/23336 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe8ac000e90 con 0x7fe89c05b390 2022-08-01T22:55:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7fe89c05b390 msgr2=0x7fe89c05d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7fe89c05b390 0x7fe89c05d830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe89c053a10 msgr2=0x7fe89c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe89c053a10 0x7fe89c055ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe8b801c000 tx=0x7fe8b8015000).stop 2022-08-01T22:55:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8bc0b3630 msgr2=0x7fe8bc145170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 --2- 172.21.15.32:0/3743740308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8bc0b3630 0x7fe8bc145170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe8bc0aec80 tx=0x7fe8b40013a0).stop 2022-08-01T22:55:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c28f0700 1 -- 172.21.15.32:0/3743740308 reap_dead start 2022-08-01T22:55:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 shutdown_connections 2022-08-01T22:55:56.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 >> 172.21.15.32:0/3743740308 conn(0x7fe8bc060950 msgr2=0x7fe8bc13e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 shutdown_connections 2022-08-01T22:55:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.132+0000 7fe8c4353700 1 -- 172.21.15.32:0/3743740308 wait complete. 2022-08-01T22:55:56.149 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:56.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2022-08-01T22:55:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 Processor -- start 2022-08-01T22:55:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- start start 2022-08-01T22:55:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f80614b0 0x7f46f8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46f81405d0 0x7f46f8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f80aee80 con 0x7f46f80ad750 2022-08-01T22:55:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f80b70b0 con 0x7f46f81405d0 2022-08-01T22:55:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80af0b0 con 0x7f46f80614b0 2022-08-01T22:55:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46f81405d0 0x7f46f8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46f81405d0 0x7f46f8140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40698/0 (socket says 172.21.15.32:40698) 2022-08-01T22:55:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 -- 172.21.15.32:0/3796245453 learned_addr learned my addr 172.21.15.32:0/3796245453 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f659c700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f80614b0 0x7f46f8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 -- 172.21.15.32:0/3796245453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f80614b0 msgr2=0x7f46f8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f80614b0 0x7f46f8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 -- 172.21.15.32:0/3796245453 >> v1:172.21.15.55:6789/0 conn(0x7f46f80ad750 legacy=0x7f46f80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 -- 172.21.15.32:0/3796245453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80af350 con 0x7f46f81405d0 2022-08-01T22:55:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f659c700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f80614b0 0x7f46f8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f6d9d700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f81405d0 0x7f46f8140a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f46f00032f0 tx=0x7f46f0009290).ready entity=mon.0 client_cookie=d2240a59afba3269 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f559a700 1 -- 172.21.15.32:0/3796245453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46f000b8a0 con 0x7f46f81405d0 2022-08-01T22:55:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f559a700 1 -- 172.21.15.32:0/3796245453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000ba00 con 0x7f46f81405d0 2022-08-01T22:55:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f559a700 1 -- 172.21.15.32:0/3796245453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46f000bb70 con 0x7f46f81405d0 2022-08-01T22:55:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- 172.21.15.32:0/3796245453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f81405d0 msgr2=0x7f46f8140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f81405d0 0x7f46f8140a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f46f00032f0 tx=0x7f46f0009290).stop 2022-08-01T22:55:56.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- 172.21.15.32:0/3796245453 shutdown_connections 2022-08-01T22:55:56.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f81405d0 0x7f46f8140a60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 --2- 172.21.15.32:0/3796245453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f80614b0 0x7f46f8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.264+0000 7f46f759e700 1 -- 172.21.15.32:0/3796245453 >> 172.21.15.32:0/3796245453 conn(0x7f46f8066f70 msgr2=0x7f46f8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- 172.21.15.32:0/3796245453 shutdown_connections 2022-08-01T22:55:56.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- 172.21.15.32:0/3796245453 wait complete. 2022-08-01T22:55:56.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 Processor -- start 2022-08-01T22:55:56.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- start start 2022-08-01T22:55:56.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80614b0 0x7f46f80ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f80ad750 0x7f46f80a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f81405d0 0x7f46f80a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80b1160 con 0x7f46f80ad750 2022-08-01T22:55:56.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8153150 con 0x7f46f80614b0 2022-08-01T22:55:56.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80a9bf0 con 0x7f46f81405d0 2022-08-01T22:55:56.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f80ad750 0x7f46f80a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f80ad750 0x7f46f80a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40702/0 (socket says 172.21.15.32:40702) 2022-08-01T22:55:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 -- 172.21.15.32:0/1518347269 learned_addr learned my addr 172.21.15.32:0/1518347269 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f659c700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80614b0 0x7f46f80ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f6d9d700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f81405d0 0x7f46f80a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 -- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f81405d0 msgr2=0x7f46f80a9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46f81405d0 0x7f46f80a9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 -- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80614b0 msgr2=0x7f46f80ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80614b0 0x7f46f80ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 -- 172.21.15.32:0/1518347269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0009530 con 0x7f46f80ad750 2022-08-01T22:55:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f5d9b700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f80ad750 0x7f46f80a8e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f46f80b6c70 tx=0x7f46ec007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46ec007030 con 0x7f46f80ad750 2022-08-01T22:55:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec007190 con 0x7f46f80ad750 2022-08-01T22:55:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f80b9030 con 0x7f46f80ad750 2022-08-01T22:55:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46ec006510 con 0x7f46f80ad750 2022-08-01T22:55:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.268+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f80b2c50 con 0x7f46f80ad750 2022-08-01T22:55:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46f80b0fe0 con 0x7f46f80ad750 2022-08-01T22:55:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f46ec010030 con 0x7f46f80ad750 2022-08-01T22:55:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46e77fe700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46d4053950 0x7f46d4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f46ec06bba0 con 0x7f46f80ad750 2022-08-01T22:55:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46f659c700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46d4053950 0x7f46d4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46e77fe700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7f46d405b370 0x7f46d405d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 --> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46d405de90 con 0x7f46d405b370 2022-08-01T22:55:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec06e230 con 0x7f46f80ad750 2022-08-01T22:55:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46f6d9d700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7f46d405b370 0x7f46d405d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46f659c700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46d4053950 0x7f46d4055e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f46e800c000 tx=0x7f46e8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.272+0000 7f46f6d9d700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7f46d405b370 0x7f46d405d810 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.276+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== osd.6 v2:172.21.15.55:6810/23334 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f46d405de90 con 0x7f46d405b370 2022-08-01T22:55:56.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.292+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 --> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46f805c0a0 con 0x7f46d405b370 2022-08-01T22:55:56.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46e77fe700 1 -- 172.21.15.32:0/1518347269 <== osd.6 v2:172.21.15.55:6810/23334 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f46f805c0a0 con 0x7f46d405b370 2022-08-01T22:55:56.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7f46d405b370 msgr2=0x7f46d405d810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7f46d405b370 0x7f46d405d810 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46d4053950 msgr2=0x7f46d4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46d4053950 0x7f46d4055e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f46e800c000 tx=0x7f46e8008040).stop 2022-08-01T22:55:56.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f80ad750 msgr2=0x7f46f80a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 --2- 172.21.15.32:0/1518347269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f80ad750 0x7f46f80a8e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f46f80b6c70 tx=0x7f46ec007c10).stop 2022-08-01T22:55:56.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f6d9d700 1 -- 172.21.15.32:0/1518347269 reap_dead start 2022-08-01T22:55:56.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 shutdown_connections 2022-08-01T22:55:56.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 >> 172.21.15.32:0/1518347269 conn(0x7f46f8066f70 msgr2=0x7f46f805ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 shutdown_connections 2022-08-01T22:55:56.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.296+0000 7f46f759e700 1 -- 172.21.15.32:0/1518347269 wait complete. 2022-08-01T22:55:56.308 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:56.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2022-08-01T22:55:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.416+0000 7f5466e2f700 1 Processor -- start 2022-08-01T22:55:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.416+0000 7f5466e2f700 1 -- start start 2022-08-01T22:55:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54600a9320 0x7f54601466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54600b0560 0x7f5460140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600b12d0 con 0x7f54600b0560 2022-08-01T22:55:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600a9700 con 0x7f54600a9320 2022-08-01T22:55:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b1500 con 0x7f54600b0d10 2022-08-01T22:55:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54600b0560 0x7f5460140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54600b0560 0x7f5460140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40712/0 (socket says 172.21.15.32:40712) 2022-08-01T22:55:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 -- 172.21.15.32:0/3342561764 learned_addr learned my addr 172.21.15.32:0/3342561764 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5465e2d700 1 --2- 172.21.15.32:0/3342561764 >> v2:172.21.15.55:3300/0 conn(0x7f54600a9320 0x7f54601466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546562c700 1 --2- 172.21.15.32:0/3342561764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 -- 172.21.15.32:0/3342561764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 msgr2=0x7f5460146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 --2- 172.21.15.32:0/3342561764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 -- 172.21.15.32:0/3342561764 >> v2:172.21.15.55:3300/0 conn(0x7f54600a9320 msgr2=0x7f54601466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 --2- 172.21.15.32:0/3342561764 >> v2:172.21.15.55:3300/0 conn(0x7f54600a9320 0x7f54601466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 -- 172.21.15.32:0/3342561764 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600b17a0 con 0x7f54600b0560 2022-08-01T22:55:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5465e2d700 1 --2- 172.21.15.32:0/3342561764 >> v2:172.21.15.55:3300/0 conn(0x7f54600a9320 0x7f54601466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f546662e700 1 --2- 172.21.15.32:0/3342561764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460140350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f54580032f0 tx=0x7f5458009290).ready entity=mon.0 client_cookie=fdac86419c966b6c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5464e2b700 1 -- 172.21.15.32:0/3342561764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f545800b870 con 0x7f54600b0560 2022-08-01T22:55:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5464e2b700 1 -- 172.21.15.32:0/3342561764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545800b9d0 con 0x7f54600b0560 2022-08-01T22:55:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5464e2b700 1 -- 172.21.15.32:0/3342561764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f545800bb40 con 0x7f54600b0560 2022-08-01T22:55:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- 172.21.15.32:0/3342561764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 msgr2=0x7f5460140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- 172.21.15.32:0/3342561764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460140350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f54580032f0 tx=0x7f5458009290).stop 2022-08-01T22:55:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- 172.21.15.32:0/3342561764 shutdown_connections 2022-08-01T22:55:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- 172.21.15.32:0/3342561764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460140350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- 172.21.15.32:0/3342561764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 --2- 172.21.15.32:0/3342561764 >> v2:172.21.15.55:3300/0 conn(0x7f54600a9320 0x7f54601466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- 172.21.15.32:0/3342561764 >> 172.21.15.32:0/3342561764 conn(0x7f54600612d0 msgr2=0x7f5460062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- 172.21.15.32:0/3342561764 shutdown_connections 2022-08-01T22:55:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.420+0000 7f5466e2f700 1 -- 172.21.15.32:0/3342561764 wait complete. 2022-08-01T22:55:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 Processor -- start 2022-08-01T22:55:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 -- start start 2022-08-01T22:55:56.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54600a9320 0x7f5460144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601533f0 con 0x7f54600b0560 2022-08-01T22:55:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600ab010 con 0x7f54600a9320 2022-08-01T22:55:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600ab2b0 con 0x7f54600b0d10 2022-08-01T22:55:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5465e2d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54600a9320 0x7f5460144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40718/0 (socket says 172.21.15.32:40718) 2022-08-01T22:55:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5465e2d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54600a9320 0x7f5460144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:50502/0 (socket says 172.21.15.32:50502) 2022-08-01T22:55:56.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546662e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 -- 172.21.15.32:0/1339362465 learned_addr learned my addr 172.21.15.32:0/1339362465 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 -- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 msgr2=0x7f5460155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f54600b0d10 0x7f5460155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 -- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54600a9320 msgr2=0x7f5460144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54600a9320 0x7f5460144740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 -- 172.21.15.32:0/1339362465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458009530 con 0x7f54600b0560 2022-08-01T22:55:56.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f546562c700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460152f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f54500030b0 tx=0x7f5450008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5450011780 con 0x7f54600b0560 2022-08-01T22:55:56.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54500118e0 con 0x7f54600b0560 2022-08-01T22:55:56.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54600ab550 con 0x7f54600b0560 2022-08-01T22:55:56.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5450011c40 con 0x7f54600b0560 2022-08-01T22:55:56.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.424+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54600ab770 con 0x7f54600b0560 2022-08-01T22:55:56.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5460155cf0 con 0x7f54600b0560 2022-08-01T22:55:56.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f545001c460 con 0x7f54600b0560 2022-08-01T22:55:56.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f544effd700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54400539e0 0x7f5440055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f545006c160 con 0x7f54600b0560 2022-08-01T22:55:56.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f544effd700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f544005b360 0x7f544005d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f5465e2d700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54400539e0 0x7f5440055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 --> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f544005de80 con 0x7f544005b360 2022-08-01T22:55:56.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f545006e790 con 0x7f54600b0560 2022-08-01T22:55:56.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f546662e700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f544005b360 0x7f544005d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f5465e2d700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54400539e0 0x7f5440055ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f545c00c000 tx=0x7f545c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.428+0000 7f546662e700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f544005b360 0x7f544005d800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.432+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== osd.7 v2:172.21.15.55:6818/23359 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f544005de80 con 0x7f544005b360 2022-08-01T22:55:56.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.448+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 --> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54601566f0 con 0x7f544005b360 2022-08-01T22:55:56.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.448+0000 7f544effd700 1 -- 172.21.15.32:0/1339362465 <== osd.7 v2:172.21.15.55:6818/23359 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f54601566f0 con 0x7f544005b360 2022-08-01T22:55:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f544005b360 msgr2=0x7f544005d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f544005b360 0x7f544005d800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54400539e0 msgr2=0x7f5440055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54400539e0 0x7f5440055ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f545c00c000 tx=0x7f545c008040).stop 2022-08-01T22:55:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 msgr2=0x7f5460152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 --2- 172.21.15.32:0/1339362465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54600b0560 0x7f5460152f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f54500030b0 tx=0x7f5450008040).stop 2022-08-01T22:55:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f546662e700 1 -- 172.21.15.32:0/1339362465 reap_dead start 2022-08-01T22:55:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 shutdown_connections 2022-08-01T22:55:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 >> 172.21.15.32:0/1339362465 conn(0x7f54600612d0 msgr2=0x7f54600b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 shutdown_connections 2022-08-01T22:55:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.452+0000 7f5466e2f700 1 -- 172.21.15.32:0/1339362465 wait complete. 2022-08-01T22:55:56.464 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:56.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2022-08-01T22:55:56.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 Processor -- start 2022-08-01T22:55:56.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 -- start start 2022-08-01T22:55:56.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04300b2710 0x7f043014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04300b01a0 0x7f043014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04300a7250 con 0x7f04300afd60 2022-08-01T22:55:56.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04300a7480 con 0x7f04300b2710 2022-08-01T22:55:56.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04300a7720 con 0x7f04300b01a0 2022-08-01T22:55:56.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04300b2710 0x7f043014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04300afd60 0x7f04300a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56502/0 (socket says 172.21.15.32:56502) 2022-08-01T22:55:56.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042ed9d700 1 -- 172.21.15.32:0/3161922111 learned_addr learned my addr 172.21.15.32:0/3161922111 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:56.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042dd9b700 1 --2- 172.21.15.32:0/3161922111 >> v2:172.21.15.55:3300/0 conn(0x7f04300b01a0 0x7f043014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042d59a700 1 -- 172.21.15.32:0/3161922111 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871455835 0 0) 0x7f04300a7250 con 0x7f04300afd60 2022-08-01T22:55:56.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.572+0000 7f042d59a700 1 -- 172.21.15.32:0/3161922111 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0414003260 con 0x7f04300afd60 2022-08-01T22:55:56.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042d59a700 1 -- 172.21.15.32:0/3161922111 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2472705825 0 0) 0x7f0414003260 con 0x7f04300afd60 2022-08-01T22:55:56.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042e59c700 1 -- 172.21.15.32:0/3161922111 >> v2:172.21.15.55:3300/0 conn(0x7f04300b01a0 msgr2=0x7f043014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:56.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042e59c700 1 --2- 172.21.15.32:0/3161922111 >> v2:172.21.15.55:3300/0 conn(0x7f04300b01a0 0x7f043014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042e59c700 1 -- 172.21.15.32:0/3161922111 >> v1:172.21.15.32:6789/0 conn(0x7f04300afd60 legacy=0x7f04300a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042e59c700 1 -- 172.21.15.32:0/3161922111 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300a79c0 con 0x7f04300b2710 2022-08-01T22:55:56.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042dd9b700 1 --2- 172.21.15.32:0/3161922111 >> v2:172.21.15.55:3300/0 conn(0x7f04300b01a0 0x7f043014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:56.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042e59c700 1 --2- 172.21.15.32:0/3161922111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f043014afd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f0428001800 tx=0x7f0424006df0).ready entity=mon.0 client_cookie=3024aedbb01592a1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042d59a700 1 -- 172.21.15.32:0/3161922111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0428001660 con 0x7f04300b2710 2022-08-01T22:55:56.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042d59a700 1 -- 172.21.15.32:0/3161922111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424000e90 con 0x7f04300b2710 2022-08-01T22:55:56.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042d59a700 1 -- 172.21.15.32:0/3161922111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0424012460 con 0x7f04300b2710 2022-08-01T22:55:56.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- 172.21.15.32:0/3161922111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 msgr2=0x7f043014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 --2- 172.21.15.32:0/3161922111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f043014afd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f0428001800 tx=0x7f0424006df0).stop 2022-08-01T22:55:56.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- 172.21.15.32:0/3161922111 shutdown_connections 2022-08-01T22:55:56.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 --2- 172.21.15.32:0/3161922111 >> v2:172.21.15.55:3300/0 conn(0x7f04300b01a0 0x7f043014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 --2- 172.21.15.32:0/3161922111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f043014afd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- 172.21.15.32:0/3161922111 >> 172.21.15.32:0/3161922111 conn(0x7f0430060310 msgr2=0x7f04300b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- 172.21.15.32:0/3161922111 shutdown_connections 2022-08-01T22:55:56.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- 172.21.15.32:0/3161922111 wait complete. 2022-08-01T22:55:56.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 Processor -- start 2022-08-01T22:55:56.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- start start 2022-08-01T22:55:56.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04300afd60 0x7f043014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04300b01a0 0x7f0430148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f0430149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430156c60 con 0x7f04300b2710 2022-08-01T22:55:56.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430066b20 con 0x7f04300b01a0 2022-08-01T22:55:56.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04300660d0 con 0x7f04300afd60 2022-08-01T22:55:56.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f0430149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f0430149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40734/0 (socket says 172.21.15.32:40734) 2022-08-01T22:55:56.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042ed9d700 1 -- 172.21.15.32:0/841003064 learned_addr learned my addr 172.21.15.32:0/841003064 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:56.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042dd9b700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04300b01a0 0x7f0430148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.576+0000 7f042e59c700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04300afd60 0x7f043014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042ed9d700 1 -- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04300afd60 msgr2=0x7f043014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042ed9d700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04300afd60 0x7f043014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042ed9d700 1 -- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04300b01a0 msgr2=0x7f0430148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042ed9d700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04300b01a0 0x7f0430148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042ed9d700 1 -- 172.21.15.32:0/841003064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420009470 con 0x7f04300b2710 2022-08-01T22:55:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042ed9d700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f0430149040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0430066cb0 tx=0x7f0428001e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0428007170 con 0x7f04300b2710 2022-08-01T22:55:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04280072d0 con 0x7f04300b2710 2022-08-01T22:55:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042400b180 con 0x7f04300b2710 2022-08-01T22:55:56.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0428004710 con 0x7f04300b2710 2022-08-01T22:55:56.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0430065020 con 0x7f04300b2710 2022-08-01T22:55:56.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04300574e0 con 0x7f04300b2710 2022-08-01T22:55:56.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f042801b070 con 0x7f04300b2710 2022-08-01T22:55:56.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f041f7fe700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f040c053950 0x7f040c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:56.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f042806b5b0 con 0x7f04300b2710 2022-08-01T22:55:56.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.580+0000 7f042e59c700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f040c053950 0x7f040c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:56.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.584+0000 7f042e59c700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f040c053950 0x7f040c055e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f042401a000 tx=0x7f0424014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:56.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.588+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04280422f0 con 0x7f04300b2710 2022-08-01T22:55:56.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.712+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0430156da0 con 0x7f04300b2710 2022-08-01T22:55:56.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.712+0000 7f041f7fe700 1 -- 172.21.15.32:0/841003064 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0428045910 con 0x7f04300b2710 2022-08-01T22:55:56.715 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2022-08-01T22:55:56.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f040c053950 msgr2=0x7f040c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f040c053950 0x7f040c055e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f042401a000 tx=0x7f0424014000).stop 2022-08-01T22:55:56.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 msgr2=0x7f0430149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f0430149040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0430066cb0 tx=0x7f0428001e00).stop 2022-08-01T22:55:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 shutdown_connections 2022-08-01T22:55:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f040c053950 0x7f040c055e10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04300b2710 0x7f0430149040 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04300b01a0 0x7f0430148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 --2- 172.21.15.32:0/841003064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f04300afd60 0x7f043014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:56.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 >> 172.21.15.32:0/841003064 conn(0x7f0430060310 msgr2=0x7f043005c310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:56.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 shutdown_connections 2022-08-01T22:55:56.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:56.716+0000 7f042f59e700 1 -- 172.21.15.32:0/841003064 wait complete. 2022-08-01T22:55:56.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2022-08-01T22:55:57.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2022-08-01T22:55:57.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 Processor -- start 2022-08-01T22:55:57.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 -- start start 2022-08-01T22:55:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c3c0b6710 0x7f6c3c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c3c0b6af0 0x7f6c3c12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c144240 con 0x7f6c3c0b4550 2022-08-01T22:55:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c143d00 con 0x7f6c3c0b6af0 2022-08-01T22:55:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c42c9c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c143fa0 con 0x7f6c3c0b6710 2022-08-01T22:55:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c3c0b6af0 0x7f6c3c12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c3c0b6af0 0x7f6c3c12bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40744/0 (socket says 172.21.15.32:40744) 2022-08-01T22:55:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 -- 172.21.15.32:0/1821706178 learned_addr learned my addr 172.21.15.32:0/1821706178 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:57.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c40a38700 1 --2- 172.21.15.32:0/1821706178 >> v2:172.21.15.55:3300/0 conn(0x7f6c3c0b6710 0x7f6c3c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:57.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 -- 172.21.15.32:0/1821706178 >> v2:172.21.15.55:3300/0 conn(0x7f6c3c0b6710 msgr2=0x7f6c3c0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:57.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 --2- 172.21.15.32:0/1821706178 >> v2:172.21.15.55:3300/0 conn(0x7f6c3c0b6710 0x7f6c3c0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 -- 172.21.15.32:0/1821706178 >> v1:172.21.15.55:6789/0 conn(0x7f6c3c0b4550 legacy=0x7f6c3c061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 -- 172.21.15.32:0/1821706178 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c061780 con 0x7f6c3c0b6af0 2022-08-01T22:55:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c40a38700 1 --2- 172.21.15.32:0/1821706178 >> v2:172.21.15.55:3300/0 conn(0x7f6c3c0b6710 0x7f6c3c0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c41239700 1 --2- 172.21.15.32:0/1821706178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6af0 0x7f6c3c12bb90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f6c3c1432b0 tx=0x7f6c34009290).ready entity=mon.0 client_cookie=e77b5d980a919584 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c3b7fe700 1 -- 172.21.15.32:0/1821706178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c3400b860 con 0x7f6c3c0b6af0 2022-08-01T22:55:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c3b7fe700 1 -- 172.21.15.32:0/1821706178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3400b9c0 con 0x7f6c3c0b6af0 2022-08-01T22:55:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.843+0000 7f6c3b7fe700 1 -- 172.21.15.32:0/1821706178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c3400bb30 con 0x7f6c3c0b6af0 2022-08-01T22:55:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- 172.21.15.32:0/1821706178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6af0 msgr2=0x7f6c3c12bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/1821706178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6af0 0x7f6c3c12bb90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f6c3c1432b0 tx=0x7f6c34009290).stop 2022-08-01T22:55:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- 172.21.15.32:0/1821706178 shutdown_connections 2022-08-01T22:55:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/1821706178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6af0 0x7f6c3c12bb90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/1821706178 >> v2:172.21.15.55:3300/0 conn(0x7f6c3c0b6710 0x7f6c3c0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- 172.21.15.32:0/1821706178 >> 172.21.15.32:0/1821706178 conn(0x7f6c3c05c620 msgr2=0x7f6c3c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- 172.21.15.32:0/1821706178 shutdown_connections 2022-08-01T22:55:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- 172.21.15.32:0/1821706178 wait complete. 2022-08-01T22:55:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 Processor -- start 2022-08-01T22:55:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- start start 2022-08-01T22:55:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c3c0b4550 0x7f6c3c12a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 0x7f6c3c0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c3c0b6af0 0x7f6c3c0af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b2f80 con 0x7f6c3c0b6710 2022-08-01T22:55:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0ae990 con 0x7f6c3c0b4550 2022-08-01T22:55:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c42c9c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b31b0 con 0x7f6c3c0b6af0 2022-08-01T22:55:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 0x7f6c3c0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 0x7f6c3c0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40748/0 (socket says 172.21.15.32:40748) 2022-08-01T22:55:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c3bfff700 1 -- 172.21.15.32:0/4281863451 learned_addr learned my addr 172.21.15.32:0/4281863451 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c40a38700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c3c0b4550 0x7f6c3c12a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.847+0000 7f6c41239700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c3c0b6af0 0x7f6c3c0af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c3bfff700 1 -- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c3c0b6af0 msgr2=0x7f6c3c0af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c3bfff700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c3c0b6af0 0x7f6c3c0af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c3bfff700 1 -- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c3c0b4550 msgr2=0x7f6c3c12a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c3bfff700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c3c0b4550 0x7f6c3c12a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:57.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c3bfff700 1 -- 172.21.15.32:0/4281863451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34009530 con 0x7f6c3c0b6710 2022-08-01T22:55:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c3bfff700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 0x7f6c3c0af0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6c3c0b1e10 tx=0x7f6c30007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c30007030 con 0x7f6c3c0b6710 2022-08-01T22:55:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30007190 con 0x7f6c3c0b6710 2022-08-01T22:55:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c30006510 con 0x7f6c3c0b6710 2022-08-01T22:55:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c0b3450 con 0x7f6c3c0b6710 2022-08-01T22:55:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c12ae20 con 0x7f6c3c0b6710 2022-08-01T22:55:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c0b3d50 con 0x7f6c3c0b6710 2022-08-01T22:55:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c30010030 con 0x7f6c3c0b6710 2022-08-01T22:55:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c39ffb700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c1c053aa0 0x7f6c1c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6c3006bb50 con 0x7f6c3c0b6710 2022-08-01T22:55:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.851+0000 7f6c40a38700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c1c053aa0 0x7f6c1c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.855+0000 7f6c40a38700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c1c053aa0 0x7f6c1c055f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f6c2c00c000 tx=0x7f6c2c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.859+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c30042910 con 0x7f6c3c0b6710 2022-08-01T22:55:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.987+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c3c061a20 con 0x7f6c3c0b6710 2022-08-01T22:55:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.987+0000 7f6c39ffb700 1 -- 172.21.15.32:0/4281863451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c30045f30 con 0x7f6c3c0b6710 2022-08-01T22:55:57.993 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c1c053aa0 msgr2=0x7f6c1c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c1c053aa0 0x7f6c1c055f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f6c2c00c000 tx=0x7f6c2c008040).stop 2022-08-01T22:55:58.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 msgr2=0x7f6c3c0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 0x7f6c3c0af0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6c3c0b1e10 tx=0x7f6c30007be0).stop 2022-08-01T22:55:58.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 shutdown_connections 2022-08-01T22:55:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c1c053aa0 0x7f6c1c055f60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c3c0b6af0 0x7f6c3c0af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3c0b6710 0x7f6c3c0af0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 --2- 172.21.15.32:0/4281863451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c3c0b4550 0x7f6c3c12a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 >> 172.21.15.32:0/4281863451 conn(0x7f6c3c05c620 msgr2=0x7f6c3c127890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 shutdown_connections 2022-08-01T22:55:58.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:57.991+0000 7f6c42c9c700 1 -- 172.21.15.32:0/4281863451 wait complete. 2022-08-01T22:55:58.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2022-08-01T22:55:58.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2022-08-01T22:55:58.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.123+0000 7fdffebef700 1 Processor -- start 2022-08-01T22:55:58.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.123+0000 7fdffebef700 1 -- start start 2022-08-01T22:55:58.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff80b0470 0x7fdff814b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80adf10 con 0x7fdff80b2710 2022-08-01T22:55:58.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80ae1b0 con 0x7fdff80b13f0 2022-08-01T22:55:58.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff80a9520 con 0x7fdff80b0470 2022-08-01T22:55:58.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff80b0470 0x7fdff814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff80b0470 0x7fdff814b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40758/0 (socket says 172.21.15.32:40758) 2022-08-01T22:55:58.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 -- 172.21.15.32:0/3182726395 learned_addr learned my addr 172.21.15.32:0/3182726395 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 -- 172.21.15.32:0/3182726395 >> v1:172.21.15.55:6789/0 conn(0x7fdff80b13f0 legacy=0x7fdff80afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 -- 172.21.15.32:0/3182726395 >> v1:172.21.15.32:6789/0 conn(0x7fdff80b2710 legacy=0x7fdff80abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 -- 172.21.15.32:0/3182726395 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80a99c0 con 0x7fdff80b0470 2022-08-01T22:55:58.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffe3ee700 1 --2- 172.21.15.32:0/3182726395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff814b640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fdff00032f0 tx=0x7fdff000b530).ready entity=mon.0 client_cookie=468748601d255ab1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffcbeb700 1 -- 172.21.15.32:0/3182726395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdff0010660 con 0x7fdff80b0470 2022-08-01T22:55:58.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffcbeb700 1 -- 172.21.15.32:0/3182726395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff00107c0 con 0x7fdff80b0470 2022-08-01T22:55:58.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffcbeb700 1 -- 172.21.15.32:0/3182726395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdff0010b20 con 0x7fdff80b0470 2022-08-01T22:55:58.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- 172.21.15.32:0/3182726395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 msgr2=0x7fdff814b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 --2- 172.21.15.32:0/3182726395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff814b640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fdff00032f0 tx=0x7fdff000b530).stop 2022-08-01T22:55:58.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- 172.21.15.32:0/3182726395 shutdown_connections 2022-08-01T22:55:58.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 --2- 172.21.15.32:0/3182726395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff814b640 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- 172.21.15.32:0/3182726395 >> 172.21.15.32:0/3182726395 conn(0x7fdff8060310 msgr2=0x7fdff80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- 172.21.15.32:0/3182726395 shutdown_connections 2022-08-01T22:55:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.127+0000 7fdffebef700 1 -- 172.21.15.32:0/3182726395 wait complete. 2022-08-01T22:55:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 Processor -- start 2022-08-01T22:55:58.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 -- start start 2022-08-01T22:55:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff8066b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80b13f0 0x7fdff8063a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b2710 0x7fdff8063f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80ac610 con 0x7fdff80b0470 2022-08-01T22:55:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff814be40 con 0x7fdff80b2710 2022-08-01T22:55:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffebef700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80675b0 con 0x7fdff80b13f0 2022-08-01T22:55:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff8066b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff8066b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40760/0 (socket says 172.21.15.32:40760) 2022-08-01T22:55:58.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 -- 172.21.15.32:0/4001565915 learned_addr learned my addr 172.21.15.32:0/4001565915 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffd3ec700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80b13f0 0x7fdff8063a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffe3ee700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b2710 0x7fdff8063f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 -- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80b13f0 msgr2=0x7fdff8063a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80b13f0 0x7fdff8063a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 -- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b2710 msgr2=0x7fdff8063f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b2710 0x7fdff8063f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 -- 172.21.15.32:0/4001565915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff000b7d0 con 0x7fdff80b0470 2022-08-01T22:55:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdffdbed700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff8066b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fdff8065b50 tx=0x7fdfec007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfec00ce20 con 0x7fdff80b0470 2022-08-01T22:55:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.131+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec00f470 con 0x7fdff80b0470 2022-08-01T22:55:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff8063190 con 0x7fdff80b0470 2022-08-01T22:55:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfec00f7d0 con 0x7fdff80b0470 2022-08-01T22:55:58.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff8152580 con 0x7fdff80b0470 2022-08-01T22:55:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff80574e0 con 0x7fdff80b0470 2022-08-01T22:55:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdfec019030 con 0x7fdff80b0470 2022-08-01T22:55:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdfeaffd700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfd8053a70 0x7fdfd8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdfec06b3e0 con 0x7fdff80b0470 2022-08-01T22:55:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdffd3ec700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfd8053a70 0x7fdfd8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.135+0000 7fdffd3ec700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfd8053a70 0x7fdfd8055f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fdff400b000 tx=0x7fdff4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.143+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdfec0421a0 con 0x7fdff80b0470 2022-08-01T22:55:58.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.267+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdff8069a70 con 0x7fdff80b0470 2022-08-01T22:55:58.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.267+0000 7fdfeaffd700 1 -- 172.21.15.32:0/4001565915 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdfec0457c0 con 0x7fdff80b0470 2022-08-01T22:55:58.271 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2022-08-01T22:55:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfd8053a70 msgr2=0x7fdfd8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfd8053a70 0x7fdfd8055f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fdff400b000 tx=0x7fdff4009040).stop 2022-08-01T22:55:58.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 msgr2=0x7fdff8066b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff8066b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fdff8065b50 tx=0x7fdfec007c50).stop 2022-08-01T22:55:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 shutdown_connections 2022-08-01T22:55:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfd8053a70 0x7fdfd8055f30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b2710 0x7fdff8063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80b13f0 0x7fdff8063a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 --2- 172.21.15.32:0/4001565915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b0470 0x7fdff8066b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 >> 172.21.15.32:0/4001565915 conn(0x7fdff8060310 msgr2=0x7fdff80b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 shutdown_connections 2022-08-01T22:55:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.271+0000 7fdffebef700 1 -- 172.21.15.32:0/4001565915 wait complete. 2022-08-01T22:55:58.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2022-08-01T22:55:58.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2022-08-01T22:55:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 Processor -- start 2022-08-01T22:55:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 -- start start 2022-08-01T22:55:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa57c0b2720 0x7fa57c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c0b1ed0 0x7fa57c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c147470 con 0x7fa57c05c6d0 2022-08-01T22:55:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57c144150 con 0x7fa57c0b2720 2022-08-01T22:55:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa580f1e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c143110 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa572ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa57c05c6d0 0x7fa57c143a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56544/0 (socket says 172.21.15.32:56544) 2022-08-01T22:55:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa572ffd700 1 -- 172.21.15.32:0/2739247889 learned_addr learned my addr 172.21.15.32:0/2739247889 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa57b7fe700 1 --2- 172.21.15.32:0/2739247889 >> v2:172.21.15.32:3300/0 conn(0x7fa57c0b2720 0x7fa57c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa57bfff700 1 --2- 172.21.15.32:0/2739247889 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c0b1ed0 0x7fa57c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa57affd700 1 -- 172.21.15.32:0/2739247889 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208796474 0 0) 0x7fa57c147470 con 0x7fa57c05c6d0 2022-08-01T22:55:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.411+0000 7fa57affd700 1 -- 172.21.15.32:0/2739247889 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa560003260 con 0x7fa57c05c6d0 2022-08-01T22:55:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57b7fe700 1 -- 172.21.15.32:0/2739247889 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c0b1ed0 msgr2=0x7fa57c0b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:58.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57b7fe700 1 --2- 172.21.15.32:0/2739247889 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c0b1ed0 0x7fa57c0b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57b7fe700 1 -- 172.21.15.32:0/2739247889 >> v1:172.21.15.32:6789/0 conn(0x7fa57c05c6d0 legacy=0x7fa57c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57b7fe700 1 -- 172.21.15.32:0/2739247889 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c143300 con 0x7fa57c0b2720 2022-08-01T22:55:58.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57b7fe700 1 --2- 172.21.15.32:0/2739247889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b2720 0x7fa57c0b0910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fa56c002f70 tx=0x7fa56c006df0).ready entity=mon.0 client_cookie=bdea50604cdf6d64 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57affd700 1 -- 172.21.15.32:0/2739247889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa56c00d4a0 con 0x7fa57c0b2720 2022-08-01T22:55:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57affd700 1 -- 172.21.15.32:0/2739247889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c000e90 con 0x7fa57c0b2720 2022-08-01T22:55:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa57affd700 1 -- 172.21.15.32:0/2739247889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa56c00d7f0 con 0x7fa57c0b2720 2022-08-01T22:55:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 -- 172.21.15.32:0/2739247889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b2720 msgr2=0x7fa57c0b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 --2- 172.21.15.32:0/2739247889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b2720 0x7fa57c0b0910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fa56c002f70 tx=0x7fa56c006df0).stop 2022-08-01T22:55:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 -- 172.21.15.32:0/2739247889 shutdown_connections 2022-08-01T22:55:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 --2- 172.21.15.32:0/2739247889 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c0b1ed0 0x7fa57c0b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 --2- 172.21.15.32:0/2739247889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b2720 0x7fa57c0b0910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 -- 172.21.15.32:0/2739247889 >> 172.21.15.32:0/2739247889 conn(0x7fa57c061360 msgr2=0x7fa57c062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 -- 172.21.15.32:0/2739247889 shutdown_connections 2022-08-01T22:55:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.415+0000 7fa580f1e700 1 -- 172.21.15.32:0/2739247889 wait complete. 2022-08-01T22:55:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 Processor -- start 2022-08-01T22:55:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 -- start start 2022-08-01T22:55:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 0x7fa57c0ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57c0b2720 0x7fa57c13ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c13b2d0 0x7fa57c13d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0ab510 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0aef70 con 0x7fa57c0b2720 2022-08-01T22:55:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa580f1e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0af1a0 con 0x7fa57c13b2d0 2022-08-01T22:55:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 0x7fa57c0ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 0x7fa57c0ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40774/0 (socket says 172.21.15.32:40774) 2022-08-01T22:55:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 -- 172.21.15.32:0/442977961 learned_addr learned my addr 172.21.15.32:0/442977961 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa572ffd700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57c0b2720 0x7fa57c13ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57bfff700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c13b2d0 0x7fa57c13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 -- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c13b2d0 msgr2=0x7fa57c13d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c13b2d0 0x7fa57c13d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 -- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57c0b2720 msgr2=0x7fa57c13ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57c0b2720 0x7fa57c13ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 -- 172.21.15.32:0/442977961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c00b180 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa572ffd700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57c0b2720 0x7fa57c13ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:55:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.419+0000 7fa57b7fe700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 0x7fa57c0ac310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa56c002f70 tx=0x7fa56c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa56c012860 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c0129c0 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0af440 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa56c012d20 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c0af980 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa56c01c070 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa5797fa700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa554053b30 0x7fa554055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c0574e0 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa56c06cba0 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa572ffd700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa554053b30 0x7fa554055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.423+0000 7fa572ffd700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa554053b30 0x7fa554055ff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa56400b000 tx=0x7fa564009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.427+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa56c043900 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.555+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa57c069a70 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.555+0000 7fa5797fa700 1 -- 172.21.15.32:0/442977961 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa56c02c070 con 0x7fa57c0b1ed0 2022-08-01T22:55:58.560 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa554053b30 msgr2=0x7fa554055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa554053b30 0x7fa554055ff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa56400b000 tx=0x7fa564009040).stop 2022-08-01T22:55:58.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 msgr2=0x7fa57c0ac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 0x7fa57c0ac310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa56c002f70 tx=0x7fa56c014000).stop 2022-08-01T22:55:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 shutdown_connections 2022-08-01T22:55:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa554053b30 0x7fa554055ff0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa57c13b2d0 0x7fa57c13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57c0b2720 0x7fa57c13ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 --2- 172.21.15.32:0/442977961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1ed0 0x7fa57c0ac310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 >> 172.21.15.32:0/442977961 conn(0x7fa57c061360 msgr2=0x7fa57c142880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 shutdown_connections 2022-08-01T22:55:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.559+0000 7fa580f1e700 1 -- 172.21.15.32:0/442977961 wait complete. 2022-08-01T22:55:58.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2022-08-01T22:55:58.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2022-08-01T22:55:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 Processor -- start 2022-08-01T22:55:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 -- start start 2022-08-01T22:55:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd3b80b6710 0x7fd3b80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3b80b6af0 0x7fd3b812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b8144240 con 0x7fd3b80b4550 2022-08-01T22:55:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b8143d00 con 0x7fd3b80b6af0 2022-08-01T22:55:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3be203700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b8143fa0 con 0x7fd3b80b6710 2022-08-01T22:55:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3b7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3b80b6af0 0x7fd3b812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3b7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3b80b6af0 0x7fd3b812bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40786/0 (socket says 172.21.15.32:40786) 2022-08-01T22:55:58.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3b7fff700 1 -- 172.21.15.32:0/3819221929 learned_addr learned my addr 172.21.15.32:0/3819221929 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.667+0000 7fd3b77fe700 1 --2- 172.21.15.32:0/3819221929 >> v2:172.21.15.55:3300/0 conn(0x7fd3b80b6710 0x7fd3b80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b67fc700 1 -- 172.21.15.32:0/3819221929 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451451142 0 0) 0x7fd3b8144240 con 0x7fd3b80b4550 2022-08-01T22:55:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b67fc700 1 -- 172.21.15.32:0/3819221929 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a4003260 con 0x7fd3b80b4550 2022-08-01T22:55:58.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b7fff700 1 -- 172.21.15.32:0/3819221929 >> v2:172.21.15.55:3300/0 conn(0x7fd3b80b6710 msgr2=0x7fd3b80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:58.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b7fff700 1 --2- 172.21.15.32:0/3819221929 >> v2:172.21.15.55:3300/0 conn(0x7fd3b80b6710 0x7fd3b80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b7fff700 1 -- 172.21.15.32:0/3819221929 >> v1:172.21.15.55:6789/0 conn(0x7fd3b80b4550 legacy=0x7fd3b8061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b7fff700 1 -- 172.21.15.32:0/3819221929 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8061780 con 0x7fd3b80b6af0 2022-08-01T22:55:58.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b7fff700 1 --2- 172.21.15.32:0/3819221929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b812bb90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fd3b81432b0 tx=0x7fd3b0006eb0).ready entity=mon.0 client_cookie=1c98e35e008b3228 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b67fc700 1 -- 172.21.15.32:0/3819221929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b000e6b0 con 0x7fd3b80b6af0 2022-08-01T22:55:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b67fc700 1 -- 172.21.15.32:0/3819221929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0000e90 con 0x7fd3b80b6af0 2022-08-01T22:55:58.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3b67fc700 1 -- 172.21.15.32:0/3819221929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b000ea40 con 0x7fd3b80b6af0 2022-08-01T22:55:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 -- 172.21.15.32:0/3819221929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 msgr2=0x7fd3b812bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 --2- 172.21.15.32:0/3819221929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b812bb90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fd3b81432b0 tx=0x7fd3b0006eb0).stop 2022-08-01T22:55:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 -- 172.21.15.32:0/3819221929 shutdown_connections 2022-08-01T22:55:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 --2- 172.21.15.32:0/3819221929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b812bb90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 --2- 172.21.15.32:0/3819221929 >> v2:172.21.15.55:3300/0 conn(0x7fd3b80b6710 0x7fd3b80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 -- 172.21.15.32:0/3819221929 >> 172.21.15.32:0/3819221929 conn(0x7fd3b805c620 msgr2=0x7fd3b805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 -- 172.21.15.32:0/3819221929 shutdown_connections 2022-08-01T22:55:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.671+0000 7fd3be203700 1 -- 172.21.15.32:0/3819221929 wait complete. 2022-08-01T22:55:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 Processor -- start 2022-08-01T22:55:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 -- start start 2022-08-01T22:55:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 0x7fd3b81277c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3b80b6710 0x7fd3b8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8148e30 con 0x7fd3b80b6af0 2022-08-01T22:55:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8148450 con 0x7fd3b80b6710 2022-08-01T22:55:58.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3be203700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b81486f0 con 0x7fd3b80b4550 2022-08-01T22:55:58.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3b77fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 0x7fd3b81277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3b7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3b77fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 0x7fd3b81277c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52526/0 (socket says 172.21.15.32:52526) 2022-08-01T22:55:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3b77fe700 1 -- 172.21.15.32:0/1899927688 learned_addr learned my addr 172.21.15.32:0/1899927688 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.675+0000 7fd3b6ffd700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3b80b6710 0x7fd3b8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b77fe700 1 -- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3b80b6710 msgr2=0x7fd3b8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b77fe700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3b80b6710 0x7fd3b8155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b77fe700 1 -- 172.21.15.32:0/1899927688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 msgr2=0x7fd3b8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b77fe700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b77fe700 1 -- 172.21.15.32:0/1899927688 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b000b1e0 con 0x7fd3b80b4550 2022-08-01T22:55:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b7fff700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:55:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b77fe700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 0x7fd3b81277c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd3b8148f70 tx=0x7fd3ac006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3ac010780 con 0x7fd3b80b4550 2022-08-01T22:55:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac0108e0 con 0x7fd3b80b4550 2022-08-01T22:55:58.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b8148990 con 0x7fd3b80b4550 2022-08-01T22:55:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3ac010c40 con 0x7fd3b80b4550 2022-08-01T22:55:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b81495d0 con 0x7fd3b80b4550 2022-08-01T22:55:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.679+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b8061a20 con 0x7fd3b80b4550 2022-08-01T22:55:58.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.683+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd3ac01c460 con 0x7fd3b80b4550 2022-08-01T22:55:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.683+0000 7fd3b4ff9700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd398053b30 0x7fd398055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.683+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd3ac06c480 con 0x7fd3b80b4550 2022-08-01T22:55:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.683+0000 7fd3b6ffd700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd398053b30 0x7fd398055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.683+0000 7fd3b6ffd700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd398053b30 0x7fd398055ff0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd3a800d000 tx=0x7fd3a8007d80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.695+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3ac043140 con 0x7fd3b80b4550 2022-08-01T22:55:58.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd3b80574e0 con 0x7fd3b80b4550 2022-08-01T22:55:58.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3b4ff9700 1 -- 172.21.15.32:0/1899927688 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd3ac046760 con 0x7fd3b80b4550 2022-08-01T22:55:58.826 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:58.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd398053b30 msgr2=0x7fd398055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd398053b30 0x7fd398055ff0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd3a800d000 tx=0x7fd3a8007d80).stop 2022-08-01T22:55:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 msgr2=0x7fd3b81277c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 0x7fd3b81277c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd3b8148f70 tx=0x7fd3ac006df0).stop 2022-08-01T22:55:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 shutdown_connections 2022-08-01T22:55:58.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd398053b30 0x7fd398055ff0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b80b6af0 0x7fd3b8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3b80b6710 0x7fd3b8155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 --2- 172.21.15.32:0/1899927688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3b80b4550 0x7fd3b81277c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 >> 172.21.15.32:0/1899927688 conn(0x7fd3b805c620 msgr2=0x7fd3b81432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 shutdown_connections 2022-08-01T22:55:58.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.823+0000 7fd3be203700 1 -- 172.21.15.32:0/1899927688 wait complete. 2022-08-01T22:55:58.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2022-08-01T22:55:58.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2022-08-01T22:55:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 Processor -- start 2022-08-01T22:55:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 -- start start 2022-08-01T22:55:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91580b6a90 0x7f91580acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9158060aa0 0x7f91580aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580b6e70 con 0x7f915805c620 2022-08-01T22:55:58.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9158061370 con 0x7f9158060aa0 2022-08-01T22:55:58.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ff64700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9158061610 con 0x7f91580b6a90 2022-08-01T22:55:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915d4ff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9158060aa0 0x7f91580aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915e501700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f915805c620 0x7f91580aa540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56570/0 (socket says 172.21.15.32:56570) 2022-08-01T22:55:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915e501700 1 -- 172.21.15.32:0/1753641977 learned_addr learned my addr 172.21.15.32:0/1753641977 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915dd00700 1 --2- 172.21.15.32:0/1753641977 >> v2:172.21.15.55:3300/0 conn(0x7f91580b6a90 0x7f91580acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ccfe700 1 -- 172.21.15.32:0/1753641977 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670014902 0 0) 0x7f91580b6e70 con 0x7f915805c620 2022-08-01T22:55:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ccfe700 1 -- 172.21.15.32:0/1753641977 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9140003260 con 0x7f915805c620 2022-08-01T22:55:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915d4ff700 1 -- 172.21.15.32:0/1753641977 >> v2:172.21.15.55:3300/0 conn(0x7f91580b6a90 msgr2=0x7f91580acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915d4ff700 1 --2- 172.21.15.32:0/1753641977 >> v2:172.21.15.55:3300/0 conn(0x7f91580b6a90 0x7f91580acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915d4ff700 1 -- 172.21.15.32:0/1753641977 >> v1:172.21.15.32:6789/0 conn(0x7f915805c620 legacy=0x7f91580aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915d4ff700 1 -- 172.21.15.32:0/1753641977 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580618b0 con 0x7f9158060aa0 2022-08-01T22:55:58.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915d4ff700 1 --2- 172.21.15.32:0/1753641977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580aa050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f9148002f70 tx=0x7f9148006df0).ready entity=mon.0 client_cookie=18b0bff3955171dd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ccfe700 1 -- 172.21.15.32:0/1753641977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f914800d4f0 con 0x7f9158060aa0 2022-08-01T22:55:58.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ccfe700 1 -- 172.21.15.32:0/1753641977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9148000e90 con 0x7f9158060aa0 2022-08-01T22:55:58.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.947+0000 7f915ccfe700 1 -- 172.21.15.32:0/1753641977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f914800d840 con 0x7f9158060aa0 2022-08-01T22:55:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- 172.21.15.32:0/1753641977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 msgr2=0x7f91580aa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 --2- 172.21.15.32:0/1753641977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580aa050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f9148002f70 tx=0x7f9148006df0).stop 2022-08-01T22:55:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- 172.21.15.32:0/1753641977 shutdown_connections 2022-08-01T22:55:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 --2- 172.21.15.32:0/1753641977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580aa050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 --2- 172.21.15.32:0/1753641977 >> v2:172.21.15.55:3300/0 conn(0x7f91580b6a90 0x7f91580acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- 172.21.15.32:0/1753641977 >> 172.21.15.32:0/1753641977 conn(0x7f91580b3610 msgr2=0x7f91580b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- 172.21.15.32:0/1753641977 shutdown_connections 2022-08-01T22:55:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- 172.21.15.32:0/1753641977 wait complete. 2022-08-01T22:55:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 Processor -- start 2022-08-01T22:55:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- start start 2022-08-01T22:55:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f915805c620 0x7f91581502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91580b6a90 0x7f9158142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915813dda0 con 0x7f9158060aa0 2022-08-01T22:55:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915813e040 con 0x7f915805c620 2022-08-01T22:55:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915ff64700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915813e2e0 con 0x7f91580b6a90 2022-08-01T22:55:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915d4ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915d4ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40804/0 (socket says 172.21.15.32:40804) 2022-08-01T22:55:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.951+0000 7f915d4ff700 1 -- 172.21.15.32:0/3199249908 learned_addr learned my addr 172.21.15.32:0/3199249908 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915dd00700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f915805c620 0x7f91581502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915e501700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91580b6a90 0x7f9158142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915d4ff700 1 -- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91580b6a90 msgr2=0x7f9158142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915d4ff700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91580b6a90 0x7f9158142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915d4ff700 1 -- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f915805c620 msgr2=0x7f91581502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915d4ff700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f915805c620 0x7f91581502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915d4ff700 1 -- 172.21.15.32:0/3199249908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914800b180 con 0x7f9158060aa0 2022-08-01T22:55:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915d4ff700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580b0930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f9148002f70 tx=0x7f9148014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9148013e90 con 0x7f9158060aa0 2022-08-01T22:55:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800de60 con 0x7f9158060aa0 2022-08-01T22:55:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915813e580 con 0x7f9158060aa0 2022-08-01T22:55:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f914801e5f0 con 0x7f9158060aa0 2022-08-01T22:55:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580b28d0 con 0x7f9158060aa0 2022-08-01T22:55:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9158061b50 con 0x7f9158060aa0 2022-08-01T22:55:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9148025030 con 0x7f9158060aa0 2022-08-01T22:55:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f9146ffd700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9138053a10 0x7f9138055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f914806cc70 con 0x7f9158060aa0 2022-08-01T22:55:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.955+0000 7f915dd00700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9138053a10 0x7f9138055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.959+0000 7f915dd00700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9138053a10 0x7f9138055ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f915400b000 tx=0x7f9154006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:58.959+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9148043a70 con 0x7f9158060aa0 2022-08-01T22:55:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.086+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f91580574e0 con 0x7f9158060aa0 2022-08-01T22:55:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.086+0000 7f9146ffd700 1 -- 172.21.15.32:0/3199249908 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9148023020 con 0x7f9158060aa0 2022-08-01T22:55:59.092 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9138053a10 msgr2=0x7f9138055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9138053a10 0x7f9138055ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f915400b000 tx=0x7f9154006df0).stop 2022-08-01T22:55:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 msgr2=0x7f91580b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580b0930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f9148002f70 tx=0x7f9148014000).stop 2022-08-01T22:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 shutdown_connections 2022-08-01T22:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9138053a10 0x7f9138055ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91580b6a90 0x7f9158142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9158060aa0 0x7f91580b0930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 --2- 172.21.15.32:0/3199249908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f915805c620 0x7f91581502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 >> 172.21.15.32:0/3199249908 conn(0x7f91580b3610 msgr2=0x7f91580a9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 shutdown_connections 2022-08-01T22:55:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.090+0000 7f915ff64700 1 -- 172.21.15.32:0/3199249908 wait complete. 2022-08-01T22:55:59.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2022-08-01T22:55:59.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2022-08-01T22:55:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 Processor -- start 2022-08-01T22:55:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 -- start start 2022-08-01T22:55:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58580b2710 0x7f585814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58580b01a0 0x7f585814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58580a7250 con 0x7f58580b01a0 2022-08-01T22:55:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58580a7480 con 0x7f58580b2710 2022-08-01T22:55:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e81d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580a7720 con 0x7f58580afd60 2022-08-01T22:55:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58580b01a0 0x7f585814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58580b01a0 0x7f585814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40810/0 (socket says 172.21.15.32:40810) 2022-08-01T22:55:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 -- 172.21.15.32:0/3039971826 learned_addr learned my addr 172.21.15.32:0/3039971826 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d81b700 1 --2- 172.21.15.32:0/3039971826 >> v2:172.21.15.55:3300/0 conn(0x7f58580b2710 0x7f585814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585e01c700 1 --2- 172.21.15.32:0/3039971826 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 -- 172.21.15.32:0/3039971826 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 msgr2=0x7f58580a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 --2- 172.21.15.32:0/3039971826 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 -- 172.21.15.32:0/3039971826 >> v2:172.21.15.55:3300/0 conn(0x7f58580b2710 msgr2=0x7f585814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 --2- 172.21.15.32:0/3039971826 >> v2:172.21.15.55:3300/0 conn(0x7f58580b2710 0x7f585814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 -- 172.21.15.32:0/3039971826 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580a79c0 con 0x7f58580b01a0 2022-08-01T22:55:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.214+0000 7f585d01a700 1 --2- 172.21.15.32:0/3039971826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b01a0 0x7f585814b4c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5854002f70 tx=0x7f5854006eb0).ready entity=mon.0 client_cookie=91c075569daf8f8d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585c819700 1 -- 172.21.15.32:0/3039971826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f585400d570 con 0x7f58580b01a0 2022-08-01T22:55:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585c819700 1 -- 172.21.15.32:0/3039971826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5854000e90 con 0x7f58580b01a0 2022-08-01T22:55:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585c819700 1 -- 172.21.15.32:0/3039971826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f585400d900 con 0x7f58580b01a0 2022-08-01T22:55:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 -- 172.21.15.32:0/3039971826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b01a0 msgr2=0x7f585814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 --2- 172.21.15.32:0/3039971826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b01a0 0x7f585814b4c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5854002f70 tx=0x7f5854006eb0).stop 2022-08-01T22:55:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 -- 172.21.15.32:0/3039971826 shutdown_connections 2022-08-01T22:55:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 --2- 172.21.15.32:0/3039971826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b01a0 0x7f585814b4c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 --2- 172.21.15.32:0/3039971826 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 --2- 172.21.15.32:0/3039971826 >> v2:172.21.15.55:3300/0 conn(0x7f58580b2710 0x7f585814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 -- 172.21.15.32:0/3039971826 >> 172.21.15.32:0/3039971826 conn(0x7f5858060310 msgr2=0x7f58580b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 -- 172.21.15.32:0/3039971826 shutdown_connections 2022-08-01T22:55:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 -- 172.21.15.32:0/3039971826 wait complete. 2022-08-01T22:55:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 Processor -- start 2022-08-01T22:55:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.218+0000 7f585e81d700 1 -- start start 2022-08-01T22:55:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58580b01a0 0x7f58580aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 0x7f58580aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580a9d50 con 0x7f58580b2710 2022-08-01T22:55:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858067ea0 con 0x7f58580b01a0 2022-08-01T22:55:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580b5550 con 0x7f58580afd60 2022-08-01T22:55:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 0x7f58580aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 0x7f58580aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40818/0 (socket says 172.21.15.32:40818) 2022-08-01T22:55:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 -- 172.21.15.32:0/2059154067 learned_addr learned my addr 172.21.15.32:0/2059154067 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585d81b700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585d01a700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58580b01a0 0x7f58580aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 -- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 msgr2=0x7f58580aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 -- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58580b01a0 msgr2=0x7f58580aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58580b01a0 0x7f58580aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 -- 172.21.15.32:0/2059154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585400b1e0 con 0x7f58580b2710 2022-08-01T22:55:59.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e01c700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 0x7f58580aa9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5850006e00 tx=0x7f58500091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f585000eeb0 con 0x7f58580b2710 2022-08-01T22:55:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585000c850 con 0x7f58580b2710 2022-08-01T22:55:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58580b35b0 con 0x7f58580b2710 2022-08-01T22:55:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f585000cbb0 con 0x7f58580b2710 2022-08-01T22:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58580b3d90 con 0x7f58580b2710 2022-08-01T22:55:59.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f585000cd10 con 0x7f58580b2710 2022-08-01T22:55:59.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58580574e0 con 0x7f58580b2710 2022-08-01T22:55:59.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f5846ffd700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053a10 0x7f5838055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f585006c2e0 con 0x7f58580b2710 2022-08-01T22:55:59.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.222+0000 7f585d81b700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053a10 0x7f5838055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.226+0000 7f585d81b700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053a10 0x7f5838055ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f584800b000 tx=0x7f5848006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.230+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58500430a0 con 0x7f58580b2710 2022-08-01T22:55:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.354+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5858069a70 con 0x7f58580b2710 2022-08-01T22:55:59.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.354+0000 7f5846ffd700 1 -- 172.21.15.32:0/2059154067 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58500466c0 con 0x7f58580b2710 2022-08-01T22:55:59.364 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:59.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053a10 msgr2=0x7f5838055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053a10 0x7f5838055ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f584800b000 tx=0x7f5848006df0).stop 2022-08-01T22:55:59.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 msgr2=0x7f58580aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 0x7f58580aa9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5850006e00 tx=0x7f58500091d0).stop 2022-08-01T22:55:59.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 shutdown_connections 2022-08-01T22:55:59.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053a10 0x7f5838055ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58580b2710 0x7f58580aa9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58580b01a0 0x7f58580aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 --2- 172.21.15.32:0/2059154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58580afd60 0x7f58580aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 >> 172.21.15.32:0/2059154067 conn(0x7f5858060310 msgr2=0x7f5858146070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 shutdown_connections 2022-08-01T22:55:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.358+0000 7f585e81d700 1 -- 172.21.15.32:0/2059154067 wait complete. 2022-08-01T22:55:59.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2022-08-01T22:55:59.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2022-08-01T22:55:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 Processor -- start 2022-08-01T22:55:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 -- start start 2022-08-01T22:55:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e40b1f40 0x7f91e40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91e40a7250 0x7f91e41448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e40b04e0 con 0x7f91e4148530 2022-08-01T22:55:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e40a9250 con 0x7f91e40a7250 2022-08-01T22:55:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ec0a8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40b0780 con 0x7f91e40b1f40 2022-08-01T22:55:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91e9643700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91e4148530 0x7f91e40b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56598/0 (socket says 172.21.15.32:56598) 2022-08-01T22:55:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91e9643700 1 -- 172.21.15.32:0/2490191494 learned_addr learned my addr 172.21.15.32:0/2490191494 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:59.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ea645700 1 --2- 172.21.15.32:0/2490191494 >> v2:172.21.15.55:3300/0 conn(0x7f91e40a7250 0x7f91e41448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91e9e44700 1 --2- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e40b1f40 0x7f91e40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91e8e42700 1 -- 172.21.15.32:0/2490191494 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414945901 0 0) 0x7f91e40b04e0 con 0x7f91e4148530 2022-08-01T22:55:59.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91e8e42700 1 -- 172.21.15.32:0/2490191494 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d0003260 con 0x7f91e4148530 2022-08-01T22:55:59.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ea645700 1 -- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e40b1f40 msgr2=0x7f91e40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:55:59.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ea645700 1 --2- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e40b1f40 0x7f91e40b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ea645700 1 -- 172.21.15.32:0/2490191494 >> v1:172.21.15.32:6789/0 conn(0x7f91e4148530 legacy=0x7f91e40b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.474+0000 7f91ea645700 1 -- 172.21.15.32:0/2490191494 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40b0a20 con 0x7f91e40a7250 2022-08-01T22:55:59.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ea645700 1 --2- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40a7250 0x7f91e41448b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f91dc002f70 tx=0x7f91dc006df0).ready entity=mon.1 client_cookie=a5d5ad5ec77e301c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91e8e42700 1 -- 172.21.15.32:0/2490191494 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91dc00d4a0 con 0x7f91e40a7250 2022-08-01T22:55:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91e8e42700 1 -- 172.21.15.32:0/2490191494 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc000e90 con 0x7f91e40a7250 2022-08-01T22:55:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91e8e42700 1 -- 172.21.15.32:0/2490191494 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91dc00d7f0 con 0x7f91e40a7250 2022-08-01T22:55:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 -- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40a7250 msgr2=0x7f91e41448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40a7250 0x7f91e41448b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f91dc002f70 tx=0x7f91dc006df0).stop 2022-08-01T22:55:59.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 -- 172.21.15.32:0/2490191494 shutdown_connections 2022-08-01T22:55:59.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40a7250 0x7f91e41448b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/2490191494 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e40b1f40 0x7f91e40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 -- 172.21.15.32:0/2490191494 >> 172.21.15.32:0/2490191494 conn(0x7f91e405f840 msgr2=0x7f91e405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 -- 172.21.15.32:0/2490191494 shutdown_connections 2022-08-01T22:55:59.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.478+0000 7f91ec0a8700 1 -- 172.21.15.32:0/2490191494 wait complete. 2022-08-01T22:55:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 Processor -- start 2022-08-01T22:55:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 -- start start 2022-08-01T22:55:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 0x7f91e40ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40b1f40 0x7f91e4156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e4148530 0x7f91e41571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e414bba0 con 0x7f91e40a7250 2022-08-01T22:55:59.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e413f2d0 con 0x7f91e40b1f40 2022-08-01T22:55:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e413f570 con 0x7f91e4148530 2022-08-01T22:55:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 0x7f91e40ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 0x7f91e40ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40830/0 (socket says 172.21.15.32:40830) 2022-08-01T22:55:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 -- 172.21.15.32:0/923715588 learned_addr learned my addr 172.21.15.32:0/923715588 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9643700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40b1f40 0x7f91e4156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ea645700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e4148530 0x7f91e41571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 -- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e4148530 msgr2=0x7f91e41571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e4148530 0x7f91e41571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 -- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40b1f40 msgr2=0x7f91e4156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40b1f40 0x7f91e4156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 -- 172.21.15.32:0/923715588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc00b180 con 0x7f91e40a7250 2022-08-01T22:55:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e9e44700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 0x7f91e40ad370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f91d800c000 tx=0x7f91d8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d8010780 con 0x7f91e40a7250 2022-08-01T22:55:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d80108e0 con 0x7f91e40a7250 2022-08-01T22:55:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d8010c40 con 0x7f91e40a7250 2022-08-01T22:55:59.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e413f810 con 0x7f91e40a7250 2022-08-01T22:55:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.482+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e41405c0 con 0x7f91e40a7250 2022-08-01T22:55:59.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.486+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e4156ad0 con 0x7f91e40a7250 2022-08-01T22:55:59.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.486+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f91d801c460 con 0x7f91e40a7250 2022-08-01T22:55:59.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.486+0000 7f91e2ffd700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91c4053a70 0x7f91c4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.486+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f91d806c400 con 0x7f91e40a7250 2022-08-01T22:55:59.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.486+0000 7f91e9643700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91c4053a70 0x7f91c4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.486+0000 7f91e9643700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91c4053a70 0x7f91c4055f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f91d400d000 tx=0x7f91d4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.494+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f91d8043160 con 0x7f91e40a7250 2022-08-01T22:55:59.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.618+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f91e40574e0 con 0x7f91e40a7250 2022-08-01T22:55:59.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.618+0000 7f91e2ffd700 1 -- 172.21.15.32:0/923715588 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91d8046780 con 0x7f91e40a7250 2022-08-01T22:55:59.626 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91c4053a70 msgr2=0x7f91c4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91c4053a70 0x7f91c4055f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f91d400d000 tx=0x7f91d4009040).stop 2022-08-01T22:55:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 msgr2=0x7f91e40ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 0x7f91e40ad370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f91d800c000 tx=0x7f91d8006df0).stop 2022-08-01T22:55:59.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 shutdown_connections 2022-08-01T22:55:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91c4053a70 0x7f91c4055f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e4148530 0x7f91e41571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e40b1f40 0x7f91e4156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 --2- 172.21.15.32:0/923715588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e40a7250 0x7f91e40ad370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 >> 172.21.15.32:0/923715588 conn(0x7f91e405f840 msgr2=0x7f91e40af860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 shutdown_connections 2022-08-01T22:55:59.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.622+0000 7f91ec0a8700 1 -- 172.21.15.32:0/923715588 wait complete. 2022-08-01T22:55:59.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2022-08-01T22:55:59.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2022-08-01T22:55:59.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b359e700 1 Processor -- start 2022-08-01T22:55:59.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b359e700 1 -- start start 2022-08-01T22:55:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f82b40a8c50 0x7f82b4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b40a7250 con 0x7f82b40b2710 2022-08-01T22:55:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b40a7480 con 0x7f82b40b01a0 2022-08-01T22:55:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b40a7720 con 0x7f82b40a8c50 2022-08-01T22:55:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f82b40b2710 0x7f82b414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56614/0 (socket says 172.21.15.32:56614) 2022-08-01T22:55:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b259c700 1 -- 172.21.15.32:0/2391749311 learned_addr learned my addr 172.21.15.32:0/2391749311 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:55:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.742+0000 7f82b2d9d700 1 --2- 172.21.15.32:0/2391749311 >> v2:172.21.15.55:3300/0 conn(0x7f82b40a8c50 0x7f82b4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b159a700 1 -- 172.21.15.32:0/2391749311 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446463802 0 0) 0x7f82b40a7250 con 0x7f82b40b2710 2022-08-01T22:55:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b159a700 1 -- 172.21.15.32:0/2391749311 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8298003260 con 0x7f82b40b2710 2022-08-01T22:55:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b2d9d700 1 -- 172.21.15.32:0/2391749311 >> v1:172.21.15.55:6789/0 conn(0x7f82b40b01a0 legacy=0x7f82b40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b2d9d700 1 -- 172.21.15.32:0/2391749311 >> v1:172.21.15.32:6789/0 conn(0x7f82b40b2710 legacy=0x7f82b414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b2d9d700 1 -- 172.21.15.32:0/2391749311 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b40a79c0 con 0x7f82b40a8c50 2022-08-01T22:55:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b2d9d700 1 --2- 172.21.15.32:0/2391749311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40a8c50 0x7f82b4152d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f82ac002f70 tx=0x7f82ac006df0).ready entity=mon.1 client_cookie=e2d0a4ddd4b0bb51 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b159a700 1 -- 172.21.15.32:0/2391749311 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac00d610 con 0x7f82b40a8c50 2022-08-01T22:55:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b159a700 1 -- 172.21.15.32:0/2391749311 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac000e90 con 0x7f82b40a8c50 2022-08-01T22:55:59.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b159a700 1 -- 172.21.15.32:0/2391749311 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac00d960 con 0x7f82b40a8c50 2022-08-01T22:55:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 -- 172.21.15.32:0/2391749311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40a8c50 msgr2=0x7f82b4152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 --2- 172.21.15.32:0/2391749311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40a8c50 0x7f82b4152d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f82ac002f70 tx=0x7f82ac006df0).stop 2022-08-01T22:55:59.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 -- 172.21.15.32:0/2391749311 shutdown_connections 2022-08-01T22:55:59.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 --2- 172.21.15.32:0/2391749311 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40a8c50 0x7f82b4152d70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 -- 172.21.15.32:0/2391749311 >> 172.21.15.32:0/2391749311 conn(0x7f82b4060310 msgr2=0x7f82b40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 -- 172.21.15.32:0/2391749311 shutdown_connections 2022-08-01T22:55:59.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 -- 172.21.15.32:0/2391749311 wait complete. 2022-08-01T22:55:59.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 Processor -- start 2022-08-01T22:55:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.746+0000 7f82b359e700 1 -- start start 2022-08-01T22:55:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82b40a8c50 0x7f82b40af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82b40b01a0 0x7f82b40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40b2710 0x7f82b40b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40b2f60 con 0x7f82b40b01a0 2022-08-01T22:55:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40aa7d0 con 0x7f82b40b2710 2022-08-01T22:55:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40b3f00 con 0x7f82b40a8c50 2022-08-01T22:55:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82b40b01a0 0x7f82b40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82b40b01a0 0x7f82b40b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40846/0 (socket says 172.21.15.32:40846) 2022-08-01T22:55:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b1d9b700 1 -- 172.21.15.32:0/308141457 learned_addr learned my addr 172.21.15.32:0/308141457 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:55:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40b2710 0x7f82b40b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b259c700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82b40a8c50 0x7f82b40af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 -- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82b40a8c50 msgr2=0x7f82b40af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82b40a8c50 0x7f82b40af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 -- 172.21.15.32:0/308141457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82b40b01a0 msgr2=0x7f82b40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82b40b01a0 0x7f82b40b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 -- 172.21.15.32:0/308141457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac00b180 con 0x7f82b40b2710 2022-08-01T22:55:59.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b2d9d700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40b2710 0x7f82b40b4b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f82b40a9870 tx=0x7f82ac014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac016070 con 0x7f82b40b2710 2022-08-01T22:55:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac00d290 con 0x7f82b40b2710 2022-08-01T22:55:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac013bb0 con 0x7f82b40b2710 2022-08-01T22:55:59.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b40b3190 con 0x7f82b40b2710 2022-08-01T22:55:59.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b414cff0 con 0x7f82b40b2710 2022-08-01T22:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.750+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b40b3630 con 0x7f82b40b2710 2022-08-01T22:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.754+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f82ac00f030 con 0x7f82b40b2710 2022-08-01T22:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.754+0000 7f82a37fe700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82900539e0 0x7f8290055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.754+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f82ac06cc70 con 0x7f82b40b2710 2022-08-01T22:55:59.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.754+0000 7f82b259c700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82900539e0 0x7f8290055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.754+0000 7f82b259c700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82900539e0 0x7f8290055ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f82a800d000 tx=0x7f82a8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.762+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f82ac043990 con 0x7f82b40b2710 2022-08-01T22:55:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.886+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f82b40574e0 con 0x7f82b40b2710 2022-08-01T22:55:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.886+0000 7f82a37fe700 1 -- 172.21.15.32:0/308141457 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82ac023090 con 0x7f82b40b2710 2022-08-01T22:55:59.892 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2022-08-01T22:55:59.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82900539e0 msgr2=0x7f8290055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82900539e0 0x7f8290055ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f82a800d000 tx=0x7f82a8009040).stop 2022-08-01T22:55:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40b2710 msgr2=0x7f82b40b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:55:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40b2710 0x7f82b40b4b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f82b40a9870 tx=0x7f82ac014000).stop 2022-08-01T22:55:59.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 shutdown_connections 2022-08-01T22:55:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82900539e0 0x7f8290055ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b40b2710 0x7f82b40b4b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82b40b01a0 0x7f82b40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 --2- 172.21.15.32:0/308141457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82b40a8c50 0x7f82b40af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:55:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 >> 172.21.15.32:0/308141457 conn(0x7f82b4060310 msgr2=0x7f82b4144cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:55:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 shutdown_connections 2022-08-01T22:55:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:55:59.890+0000 7f82b359e700 1 -- 172.21.15.32:0/308141457 wait complete. 2022-08-01T22:55:59.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2022-08-01T22:55:59.904 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2022-08-01T22:55:59.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T22:56:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.006+0000 7f939bbd6700 1 Processor -- start 2022-08-01T22:56:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.006+0000 7f939bbd6700 1 -- start start 2022-08-01T22:56:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939bbd6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f939414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939bbd6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9394060670 con 0x7f93940617e0 2022-08-01T22:56:00.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939bbd6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9394060e00 con 0x7f9394060290 2022-08-01T22:56:00.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939bbd6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93940672c0 con 0x7f9394066e70 2022-08-01T22:56:00.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93940617e0 0x7f9394066760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56628/0 (socket says 172.21.15.32:56628) 2022-08-01T22:56:00.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 -- 172.21.15.32:0/3629766904 learned_addr learned my addr 172.21.15.32:0/3629766904 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:00.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939a173700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f939414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058584008 0 0) 0x7f9394060670 con 0x7f93940617e0 2022-08-01T22:56:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f937c003260 con 0x7f93940617e0 2022-08-01T22:56:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869133540 0 0) 0x7f9394060e00 con 0x7f9394060290 2022-08-01T22:56:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9394060670 con 0x7f9394060290 2022-08-01T22:56:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262754283 0 0) 0x7f937c003260 con 0x7f93940617e0 2022-08-01T22:56:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 msgr2=0x7f939414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f939414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 >> v1:172.21.15.55:6789/0 conn(0x7f9394060290 legacy=0x7f9394065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940610a0 con 0x7f93940617e0 2022-08-01T22:56:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f9384001680 con 0x7f93940617e0 2022-08-01T22:56:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 >> v1:172.21.15.32:6789/0 conn(0x7f93940617e0 legacy=0x7f9394066760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f937c003910 0x7f937c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937c004500 0x7f937c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f937c004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93940672c0 con 0x7f937c004500 2022-08-01T22:56:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c00b800 con 0x7f937c003910 2022-08-01T22:56:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c00b9c0 con 0x7f9394066e70 2022-08-01T22:56:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937c004500 0x7f937c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399972700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f937c003910 0x7f937c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939a173700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f937c004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 -- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 msgr2=0x7f937c004e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f937c004e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 -- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f937c003910 msgr2=0x7f937c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f937c003910 0x7f937c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 -- 172.21.15.32:0/3629766904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c003260 con 0x7f937c004500 2022-08-01T22:56:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939a173700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9394066e70 0x7f937c004e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399972700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f937c003910 0x7f937c003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9399171700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937c004500 0x7f937c004980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9384001450 tx=0x7f9394064d40).ready entity=mon.0 client_cookie=6db917946e938738 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9384007170 con 0x7f937c004500 2022-08-01T22:56:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93840072d0 con 0x7f937c004500 2022-08-01T22:56:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f9398970700 1 -- 172.21.15.32:0/3629766904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938400c8c0 con 0x7f937c004500 2022-08-01T22:56:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939bbd6700 1 -- 172.21.15.32:0/3629766904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937c004500 msgr2=0x7f937c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939bbd6700 1 --2- 172.21.15.32:0/3629766904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937c004500 0x7f937c004980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9384001450 tx=0x7f9394064d40).stop 2022-08-01T22:56:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.010+0000 7f939a173700 1 -- 172.21.15.32:0/3629766904 reap_dead start 2022-08-01T22:56:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- 172.21.15.32:0/3629766904 shutdown_connections 2022-08-01T22:56:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- 172.21.15.32:0/3629766904 >> 172.21.15.32:0/3629766904 conn(0x7f939405d0c0 msgr2=0x7f939405f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- 172.21.15.32:0/3629766904 shutdown_connections 2022-08-01T22:56:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- 172.21.15.32:0/3629766904 wait complete. 2022-08-01T22:56:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 Processor -- start 2022-08-01T22:56:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- start start 2022-08-01T22:56:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f937c004500 0x7f93940a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 0x7f9394146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394147340 0x7f93941529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9394155f80 con 0x7f9394156c00 2022-08-01T22:56:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9394156220 con 0x7f9394147340 2022-08-01T22:56:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939bbd6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93941564c0 con 0x7f937c004500 2022-08-01T22:56:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 0x7f9394146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 0x7f9394146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40866/0 (socket says 172.21.15.32:40866) 2022-08-01T22:56:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 -- 172.21.15.32:0/1027737576 learned_addr learned my addr 172.21.15.32:0/1027737576 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399972700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f937c004500 0x7f93940a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939a173700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394147340 0x7f93941529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 -- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f937c004500 msgr2=0x7f93940a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f937c004500 0x7f93940a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 -- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394147340 msgr2=0x7f93941529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394147340 0x7f93941529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 -- 172.21.15.32:0/1027737576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394156760 con 0x7f9394156c00 2022-08-01T22:56:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f939a173700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394147340 0x7f93941529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.014+0000 7f9399171700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 0x7f9394146e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9384001450 tx=0x7f9384013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9384007170 con 0x7f9394156c00 2022-08-01T22:56:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384000ed0 con 0x7f9394156c00 2022-08-01T22:56:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93940a9620 con 0x7f9394156c00 2022-08-01T22:56:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93941530c0 con 0x7f9394156c00 2022-08-01T22:56:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9384004730 con 0x7f9394156c00 2022-08-01T22:56:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93941568f0 con 0x7f9394156c00 2022-08-01T22:56:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9384027070 con 0x7f9394156c00 2022-08-01T22:56:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9382ffd700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9374053a70 0x7f9374055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f938406c430 con 0x7f9394156c00 2022-08-01T22:56:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9399972700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9374053a70 0x7f9374055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.018+0000 7f9399972700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9374053a70 0x7f9374055f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f938c00d000 tx=0x7f938c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.022+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93840431f0 con 0x7f9394156c00 2022-08-01T22:56:00.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.146+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9394061340 con 0x7f9374053a70 2022-08-01T22:56:00.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.154+0000 7f9382ffd700 1 -- 172.21.15.32:0/1027737576 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82367 (secure 0 0 0) 0x7f9394061340 con 0x7f9374053a70 2022-08-01T22:56:00.159 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:00.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9374053a70 msgr2=0x7f9374055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9374053a70 0x7f9374055f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f938c00d000 tx=0x7f938c009040).stop 2022-08-01T22:56:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 msgr2=0x7f9394146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 0x7f9394146e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9384001450 tx=0x7f9384013000).stop 2022-08-01T22:56:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 shutdown_connections 2022-08-01T22:56:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9374053a70 0x7f9374055f30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394147340 0x7f93941529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9394156c00 0x7f9394146e50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 --2- 172.21.15.32:0/1027737576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f937c004500 0x7f93940a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 >> 172.21.15.32:0/1027737576 conn(0x7f939405d0c0 msgr2=0x7f93940b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 shutdown_connections 2022-08-01T22:56:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.158+0000 7f939bbd6700 1 -- 172.21.15.32:0/1027737576 wait complete. 2022-08-01T22:56:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2022-08-01T22:56:00.171 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2022-08-01T22:55:58.225717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":817128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002627"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.420404+0000","last_change":"2022-08-01T22:55:50.416870+0000","last_active":"2022-08-01T22:55:50.420404+0000","last_peered":"2022-08-01T22:55:50.420404+0000","last_clean":"2022-08-01T22:55:50.420404+0000","last_became_active":"2022-08-01T22:55:50.415830+0000","last_became_peered":"2022-08-01T22:55:50.415830+0000","last_unstale":"2022-08-01T22:55:50.420404+0000","last_undegraded":"2022-08-01T22:55:50.420404+0000","last_fullsized":"2022-08-01T22:55:50.420404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.412829+0000","last_change":"2022-08-01T22:55:40.426793+0000","last_active":"2022-08-01T22:55:50.412829+0000","last_peered":"2022-08-01T22:55:50.412829+0000","last_clean":"2022-08-01T22:55:50.412829+0000","last_became_active":"2022-08-01T22:55:40.425307+0000","last_became_peered":"2022-08-01T22:55:40.425307+0000","last_unstale":"2022-08-01T22:55:50.412829+0000","last_undegraded":"2022-08-01T22:55:50.412829+0000","last_fullsized":"2022-08-01T22:55:50.412829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_clean_scrub_stamp":"2022-08-01T22:55:34.279477+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.363126+0000","last_change":"2022-08-01T22:55:44.188728+0000","last_active":"2022-08-01T22:55:50.363126+0000","last_peered":"2022-08-01T22:55:50.363126+0000","last_clean":"2022-08-01T22:55:50.363126+0000","last_became_active":"2022-08-01T22:55:44.188196+0000","last_became_peered":"2022-08-01T22:55:44.188196+0000","last_unstale":"2022-08-01T22:55:50.363126+0000","last_undegraded":"2022-08-01T22:55:50.363126+0000","last_fullsized":"2022-08-01T22:55:50.363126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.690548+0000","last_change":"2022-08-01T22:55:47.319526+0000","last_active":"2022-08-01T22:55:50.690548+0000","last_peered":"2022-08-01T22:55:50.690548+0000","last_clean":"2022-08-01T22:55:50.690548+0000","last_became_active":"2022-08-01T22:55:47.319054+0000","last_became_peered":"2022-08-01T22:55:47.319054+0000","last_unstale":"2022-08-01T22:55:50.690548+0000","last_undegraded":"2022-08-01T22:55:50.690548+0000","last_fullsized":"2022-08-01T22:55:50.690548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.421657+0000","last_change":"2022-08-01T22:55:50.420361+0000","last_active":"2022-08-01T22:55:50.421657+0000","last_peered":"2022-08-01T22:55:50.421657+0000","last_clean":"2022-08-01T22:55:50.421657+0000","last_became_active":"2022-08-01T22:55:50.419823+0000","last_became_peered":"2022-08-01T22:55:50.419823+0000","last_unstale":"2022-08-01T22:55:50.421657+0000","last_undegraded":"2022-08-01T22:55:50.421657+0000","last_fullsized":"2022-08-01T22:55:50.421657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.412678+0000","last_change":"2022-08-01T22:55:44.188357+0000","last_active":"2022-08-01T22:55:50.412678+0000","last_peered":"2022-08-01T22:55:50.412678+0000","last_clean":"2022-08-01T22:55:50.412678+0000","last_became_active":"2022-08-01T22:55:44.187581+0000","last_became_peered":"2022-08-01T22:55:44.187581+0000","last_unstale":"2022-08-01T22:55:50.412678+0000","last_undegraded":"2022-08-01T22:55:50.412678+0000","last_fullsized":"2022-08-01T22:55:50.412678+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.364764+0000","last_change":"2022-08-01T22:55:47.319852+0000","last_active":"2022-08-01T22:55:50.364764+0000","last_peered":"2022-08-01T22:55:50.364764+0000","last_clean":"2022-08-01T22:55:50.364764+0000","last_became_active":"2022-08-01T22:55:47.319441+0000","last_became_peered":"2022-08-01T22:55:47.319441+0000","last_unstale":"2022-08-01T22:55:50.364764+0000","last_undegraded":"2022-08-01T22:55:50.364764+0000","last_fullsized":"2022-08-01T22:55:50.364764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.372349+0000","last_change":"2022-08-01T22:55:50.371166+0000","last_active":"2022-08-01T22:55:50.372349+0000","last_peered":"2022-08-01T22:55:50.372349+0000","last_clean":"2022-08-01T22:55:50.372349+0000","last_became_active":"2022-08-01T22:55:50.370504+0000","last_became_peered":"2022-08-01T22:55:50.370504+0000","last_unstale":"2022-08-01T22:55:50.372349+0000","last_undegraded":"2022-08-01T22:55:50.372349+0000","last_fullsized":"2022-08-01T22:55:50.372349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.413100+0000","last_change":"2022-08-01T22:55:44.555310+0000","last_active":"2022-08-01T22:55:50.413100+0000","last_peered":"2022-08-01T22:55:50.413100+0000","last_clean":"2022-08-01T22:55:50.413100+0000","last_became_active":"2022-08-01T22:55:44.554353+0000","last_became_peered":"2022-08-01T22:55:44.554353+0000","last_unstale":"2022-08-01T22:55:50.413100+0000","last_undegraded":"2022-08-01T22:55:50.413100+0000","last_fullsized":"2022-08-01T22:55:50.413100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.706120+0000","last_change":"2022-08-01T22:55:47.494419+0000","last_active":"2022-08-01T22:55:50.706120+0000","last_peered":"2022-08-01T22:55:50.706120+0000","last_clean":"2022-08-01T22:55:50.706120+0000","last_became_active":"2022-08-01T22:55:47.493623+0000","last_became_peered":"2022-08-01T22:55:47.493623+0000","last_unstale":"2022-08-01T22:55:50.706120+0000","last_undegraded":"2022-08-01T22:55:50.706120+0000","last_fullsized":"2022-08-01T22:55:50.706120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.417449+0000","last_change":"2022-08-01T22:55:50.416360+0000","last_active":"2022-08-01T22:55:50.417449+0000","last_peered":"2022-08-01T22:55:50.417449+0000","last_clean":"2022-08-01T22:55:50.417449+0000","last_became_active":"2022-08-01T22:55:50.414865+0000","last_became_peered":"2022-08-01T22:55:50.414865+0000","last_unstale":"2022-08-01T22:55:50.417449+0000","last_undegraded":"2022-08-01T22:55:50.417449+0000","last_fullsized":"2022-08-01T22:55:50.417449+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.363011+0000","last_change":"2022-08-01T22:55:44.188785+0000","last_active":"2022-08-01T22:55:50.363011+0000","last_peered":"2022-08-01T22:55:50.363011+0000","last_clean":"2022-08-01T22:55:50.363011+0000","last_became_active":"2022-08-01T22:55:44.188345+0000","last_became_peered":"2022-08-01T22:55:44.188345+0000","last_unstale":"2022-08-01T22:55:50.363011+0000","last_undegraded":"2022-08-01T22:55:50.363011+0000","last_fullsized":"2022-08-01T22:55:50.363011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.706456+0000","last_change":"2022-08-01T22:55:47.321815+0000","last_active":"2022-08-01T22:55:50.706456+0000","last_peered":"2022-08-01T22:55:50.706456+0000","last_clean":"2022-08-01T22:55:50.706456+0000","last_became_active":"2022-08-01T22:55:47.321450+0000","last_became_peered":"2022-08-01T22:55:47.321450+0000","last_unstale":"2022-08-01T22:55:50.706456+0000","last_undegraded":"2022-08-01T22:55:50.706456+0000","last_fullsized":"2022-08-01T22:55:50.706456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.374938+0000","last_change":"2022-08-01T22:55:50.374202+0000","last_active":"2022-08-01T22:55:50.374938+0000","last_peered":"2022-08-01T22:55:50.374938+0000","last_clean":"2022-08-01T22:55:50.374938+0000","last_became_active":"2022-08-01T22:55:50.373589+0000","last_became_peered":"2022-08-01T22:55:50.373589+0000","last_unstale":"2022-08-01T22:55:50.374938+0000","last_undegraded":"2022-08-01T22:55:50.374938+0000","last_fullsized":"2022-08-01T22:55:50.374938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.708711+0000","last_change":"2022-08-01T22:55:47.322349+0000","last_active":"2022-08-01T22:55:50.708711+0000","last_peered":"2022-08-01T22:55:50.708711+0000","last_clean":"2022-08-01T22:55:50.708711+0000","last_became_active":"2022-08-01T22:55:47.321659+0000","last_became_peered":"2022-08-01T22:55:47.321659+0000","last_unstale":"2022-08-01T22:55:50.708711+0000","last_undegraded":"2022-08-01T22:55:50.708711+0000","last_fullsized":"2022-08-01T22:55:50.708711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.351095+0000","last_change":"2022-08-01T22:55:44.414856+0000","last_active":"2022-08-01T22:55:50.351095+0000","last_peered":"2022-08-01T22:55:50.351095+0000","last_clean":"2022-08-01T22:55:50.351095+0000","last_became_active":"2022-08-01T22:55:44.413727+0000","last_became_peered":"2022-08-01T22:55:44.413727+0000","last_unstale":"2022-08-01T22:55:50.351095+0000","last_undegraded":"2022-08-01T22:55:50.351095+0000","last_fullsized":"2022-08-01T22:55:50.351095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.421089+0000","last_change":"2022-08-01T22:55:50.417907+0000","last_active":"2022-08-01T22:55:50.421089+0000","last_peered":"2022-08-01T22:55:50.421089+0000","last_clean":"2022-08-01T22:55:50.421089+0000","last_became_active":"2022-08-01T22:55:50.416804+0000","last_became_peered":"2022-08-01T22:55:50.416804+0000","last_unstale":"2022-08-01T22:55:50.421089+0000","last_undegraded":"2022-08-01T22:55:50.421089+0000","last_fullsized":"2022-08-01T22:55:50.421089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.706380+0000","last_change":"2022-08-01T22:55:47.493727+0000","last_active":"2022-08-01T22:55:50.706380+0000","last_peered":"2022-08-01T22:55:50.706380+0000","last_clean":"2022-08-01T22:55:50.706380+0000","last_became_active":"2022-08-01T22:55:47.493016+0000","last_became_peered":"2022-08-01T22:55:47.493016+0000","last_unstale":"2022-08-01T22:55:50.706380+0000","last_undegraded":"2022-08-01T22:55:50.706380+0000","last_fullsized":"2022-08-01T22:55:50.706380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.412622+0000","last_change":"2022-08-01T22:55:44.415068+0000","last_active":"2022-08-01T22:55:50.412622+0000","last_peered":"2022-08-01T22:55:50.412622+0000","last_clean":"2022-08-01T22:55:50.412622+0000","last_became_active":"2022-08-01T22:55:44.414349+0000","last_became_peered":"2022-08-01T22:55:44.414349+0000","last_unstale":"2022-08-01T22:55:50.412622+0000","last_undegraded":"2022-08-01T22:55:50.412622+0000","last_fullsized":"2022-08-01T22:55:50.412622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.420614+0000","last_change":"2022-08-01T22:55:50.418458+0000","last_active":"2022-08-01T22:55:50.420614+0000","last_peered":"2022-08-01T22:55:50.420614+0000","last_clean":"2022-08-01T22:55:50.420614+0000","last_became_active":"2022-08-01T22:55:50.416823+0000","last_became_peered":"2022-08-01T22:55:50.416823+0000","last_unstale":"2022-08-01T22:55:50.420614+0000","last_undegraded":"2022-08-01T22:55:50.420614+0000","last_fullsized":"2022-08-01T22:55:50.420614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"24'7","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.707678+0000","last_change":"2022-08-01T22:55:47.444349+0000","last_active":"2022-08-01T22:55:50.707678+0000","last_peered":"2022-08-01T22:55:50.707678+0000","last_clean":"2022-08-01T22:55:50.707678+0000","last_became_active":"2022-08-01T22:55:47.443749+0000","last_became_peered":"2022-08-01T22:55:47.443749+0000","last_unstale":"2022-08-01T22:55:50.707678+0000","last_undegraded":"2022-08-01T22:55:50.707678+0000","last_fullsized":"2022-08-01T22:55:50.707678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.350894+0000","last_change":"2022-08-01T22:55:44.190143+0000","last_active":"2022-08-01T22:55:50.350894+0000","last_peered":"2022-08-01T22:55:50.350894+0000","last_clean":"2022-08-01T22:55:50.350894+0000","last_became_active":"2022-08-01T22:55:44.189745+0000","last_became_peered":"2022-08-01T22:55:44.189745+0000","last_unstale":"2022-08-01T22:55:50.350894+0000","last_undegraded":"2022-08-01T22:55:50.350894+0000","last_fullsized":"2022-08-01T22:55:50.350894+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.421458+0000","last_change":"2022-08-01T22:55:50.419300+0000","last_active":"2022-08-01T22:55:50.421458+0000","last_peered":"2022-08-01T22:55:50.421458+0000","last_clean":"2022-08-01T22:55:50.421458+0000","last_became_active":"2022-08-01T22:55:50.418583+0000","last_became_peered":"2022-08-01T22:55:50.418583+0000","last_unstale":"2022-08-01T22:55:50.421458+0000","last_undegraded":"2022-08-01T22:55:50.421458+0000","last_fullsized":"2022-08-01T22:55:50.421458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.705103+0000","last_change":"2022-08-01T22:55:47.319248+0000","last_active":"2022-08-01T22:55:50.705103+0000","last_peered":"2022-08-01T22:55:50.705103+0000","last_clean":"2022-08-01T22:55:50.705103+0000","last_became_active":"2022-08-01T22:55:47.318884+0000","last_became_peered":"2022-08-01T22:55:47.318884+0000","last_unstale":"2022-08-01T22:55:50.705103+0000","last_undegraded":"2022-08-01T22:55:50.705103+0000","last_fullsized":"2022-08-01T22:55:50.705103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.367009+0000","last_change":"2022-08-01T22:55:44.191477+0000","last_active":"2022-08-01T22:55:50.367009+0000","last_peered":"2022-08-01T22:55:50.367009+0000","last_clean":"2022-08-01T22:55:50.367009+0000","last_became_active":"2022-08-01T22:55:44.191167+0000","last_became_peered":"2022-08-01T22:55:44.191167+0000","last_unstale":"2022-08-01T22:55:50.367009+0000","last_undegraded":"2022-08-01T22:55:50.367009+0000","last_fullsized":"2022-08-01T22:55:50.367009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":102140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":101565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":101725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":101599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":102300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":101599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]}]}} 2022-08-01T22:56:00.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T22:56:00.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0025ad6700 1 Processor -- start 2022-08-01T22:56:00.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0025ad6700 1 -- start start 2022-08-01T22:56:00.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0025ad6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 0x7f00201448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0025ad6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00200b04e0 con 0x7f0020148530 2022-08-01T22:56:00.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0025ad6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00200b0780 con 0x7f00200b1f40 2022-08-01T22:56:00.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0025ad6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200a9250 con 0x7f00200b1160 2022-08-01T22:56:00.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0016ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0020148530 0x7f00200a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56648/0 (socket says 172.21.15.32:56648) 2022-08-01T22:56:00.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0016ffd700 1 -- 172.21.15.32:0/2320993687 learned_addr learned my addr 172.21.15.32:0/2320993687 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:00.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 0x7f00201448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044509463 0 0) 0x7f00200b04e0 con 0x7f0020148530 2022-08-01T22:56:00.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0000003260 con 0x7f0020148530 2022-08-01T22:56:00.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3739776504 0 0) 0x7f0000003260 con 0x7f0020148530 2022-08-01T22:56:00.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 msgr2=0x7f00201448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 0x7f00201448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 >> v1:172.21.15.55:6789/0 conn(0x7f00200b1f40 legacy=0x7f00200b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200b0a20 con 0x7f0020148530 2022-08-01T22:56:00.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f0008001660 con 0x7f0020148530 2022-08-01T22:56:00.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 >> v1:172.21.15.32:6789/0 conn(0x7f0020148530 legacy=0x7f00200a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 0x7f0000003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0000003cb0 0x7f0000004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000049f0 0x7f0000004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000000b800 con 0x7f00000049f0 2022-08-01T22:56:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000000b940 con 0x7f0000003cb0 2022-08-01T22:56:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000000bb00 con 0x7f00200b1160 2022-08-01T22:56:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000049f0 0x7f0000004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001f7fe700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 0x7f0000003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f0016ffd700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0000003cb0 0x7f0000004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 -- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 msgr2=0x7f0000003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f00200b1160 0x7f0000003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 -- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0000003cb0 msgr2=0x7f0000004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0000003cb0 0x7f0000004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 -- 172.21.15.32:0/2320993687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0000003260 con 0x7f00000049f0 2022-08-01T22:56:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001ffff700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000049f0 0x7f0000004e70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f001800c000 tx=0x7f00180091d0).ready entity=mon.0 client_cookie=47df63c596a237e9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001800e7e0 con 0x7f00000049f0 2022-08-01T22:56:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001800e940 con 0x7f00000049f0 2022-08-01T22:56:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.278+0000 7f001effd700 1 -- 172.21.15.32:0/2320993687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001800eca0 con 0x7f00000049f0 2022-08-01T22:56:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- 172.21.15.32:0/2320993687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000049f0 msgr2=0x7f0000004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 --2- 172.21.15.32:0/2320993687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000049f0 0x7f0000004e70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f001800c000 tx=0x7f00180091d0).stop 2022-08-01T22:56:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f001ffff700 1 -- 172.21.15.32:0/2320993687 reap_dead start 2022-08-01T22:56:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- 172.21.15.32:0/2320993687 shutdown_connections 2022-08-01T22:56:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- 172.21.15.32:0/2320993687 >> 172.21.15.32:0/2320993687 conn(0x7f002005f840 msgr2=0x7f002005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- 172.21.15.32:0/2320993687 shutdown_connections 2022-08-01T22:56:00.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- 172.21.15.32:0/2320993687 wait complete. 2022-08-01T22:56:00.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 Processor -- start 2022-08-01T22:56:00.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- start start 2022-08-01T22:56:00.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0020143330 0x7f00200b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 0x7f002014ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f002015a140 0x7f002015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002014a8b0 con 0x7f00200b6360 2022-08-01T22:56:00.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020143710 con 0x7f002015a140 2022-08-01T22:56:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.282+0000 7f0025ad6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002014a610 con 0x7f0020143330 2022-08-01T22:56:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 0x7f002014ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0020143330 0x7f00200b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 0x7f002014ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40886/0 (socket says 172.21.15.32:40886) 2022-08-01T22:56:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 -- 172.21.15.32:0/214315846 learned_addr learned my addr 172.21.15.32:0/214315846 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:00.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001ffff700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f002015a140 0x7f002015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 -- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0020143330 msgr2=0x7f00200b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0020143330 0x7f00200b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 -- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f002015a140 msgr2=0x7f002015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f002015a140 0x7f002015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 -- 172.21.15.32:0/214315846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200b4050 con 0x7f00200b6360 2022-08-01T22:56:00.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0016ffd700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 0x7f002014ac20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f000800b000 tx=0x7f00200b0670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0008015630 con 0x7f00200b6360 2022-08-01T22:56:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008010040 con 0x7f00200b6360 2022-08-01T22:56:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002014b9c0 con 0x7f00200b6360 2022-08-01T22:56:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0008015980 con 0x7f00200b6360 2022-08-01T22:56:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002014a230 con 0x7f00200b6360 2022-08-01T22:56:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00200ac810 con 0x7f00200b6360 2022-08-01T22:56:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f000801e3f0 con 0x7f00200b6360 2022-08-01T22:56:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001d7fa700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00040539e0 0x7f0004055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f000806bd50 con 0x7f00200b6360 2022-08-01T22:56:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.286+0000 7f001f7fe700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00040539e0 0x7f0004055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.290+0000 7f001f7fe700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00040539e0 0x7f0004055ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0010003000 tx=0x7f0010007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.294+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00080429a0 con 0x7f00200b6360 2022-08-01T22:56:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.418+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00200574e0 con 0x7f00040539e0 2022-08-01T22:56:00.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f001d7fa700 1 -- 172.21.15.32:0/214315846 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82342 (secure 0 0 0) 0x7f00200574e0 con 0x7f00040539e0 2022-08-01T22:56:00.429 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:00.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00040539e0 msgr2=0x7f0004055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00040539e0 0x7f0004055ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0010003000 tx=0x7f0010007a00).stop 2022-08-01T22:56:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 msgr2=0x7f002014ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 0x7f002014ac20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f000800b000 tx=0x7f00200b0670).stop 2022-08-01T22:56:00.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 shutdown_connections 2022-08-01T22:56:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00040539e0 0x7f0004055ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f002015a140 0x7f002015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00200b6360 0x7f002014ac20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 --2- 172.21.15.32:0/214315846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0020143330 0x7f00200b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 >> 172.21.15.32:0/214315846 conn(0x7f002005f840 msgr2=0x7f0020146ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 shutdown_connections 2022-08-01T22:56:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.426+0000 7f0025ad6700 1 -- 172.21.15.32:0/214315846 wait complete. 2022-08-01T22:56:00.443 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2022-08-01T22:56:00.444 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2022-08-01T22:56:00.226415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":817128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.871595"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.420404+0000","last_change":"2022-08-01T22:55:50.416870+0000","last_active":"2022-08-01T22:55:50.420404+0000","last_peered":"2022-08-01T22:55:50.420404+0000","last_clean":"2022-08-01T22:55:50.420404+0000","last_became_active":"2022-08-01T22:55:50.415830+0000","last_became_peered":"2022-08-01T22:55:50.415830+0000","last_unstale":"2022-08-01T22:55:50.420404+0000","last_undegraded":"2022-08-01T22:55:50.420404+0000","last_fullsized":"2022-08-01T22:55:50.420404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.412829+0000","last_change":"2022-08-01T22:55:40.426793+0000","last_active":"2022-08-01T22:55:50.412829+0000","last_peered":"2022-08-01T22:55:50.412829+0000","last_clean":"2022-08-01T22:55:50.412829+0000","last_became_active":"2022-08-01T22:55:40.425307+0000","last_became_peered":"2022-08-01T22:55:40.425307+0000","last_unstale":"2022-08-01T22:55:50.412829+0000","last_undegraded":"2022-08-01T22:55:50.412829+0000","last_fullsized":"2022-08-01T22:55:50.412829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_clean_scrub_stamp":"2022-08-01T22:55:34.279477+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.363126+0000","last_change":"2022-08-01T22:55:44.188728+0000","last_active":"2022-08-01T22:55:50.363126+0000","last_peered":"2022-08-01T22:55:50.363126+0000","last_clean":"2022-08-01T22:55:50.363126+0000","last_became_active":"2022-08-01T22:55:44.188196+0000","last_became_peered":"2022-08-01T22:55:44.188196+0000","last_unstale":"2022-08-01T22:55:50.363126+0000","last_undegraded":"2022-08-01T22:55:50.363126+0000","last_fullsized":"2022-08-01T22:55:50.363126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.690548+0000","last_change":"2022-08-01T22:55:47.319526+0000","last_active":"2022-08-01T22:55:50.690548+0000","last_peered":"2022-08-01T22:55:50.690548+0000","last_clean":"2022-08-01T22:55:50.690548+0000","last_became_active":"2022-08-01T22:55:47.319054+0000","last_became_peered":"2022-08-01T22:55:47.319054+0000","last_unstale":"2022-08-01T22:55:50.690548+0000","last_undegraded":"2022-08-01T22:55:50.690548+0000","last_fullsized":"2022-08-01T22:55:50.690548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.421657+0000","last_change":"2022-08-01T22:55:50.420361+0000","last_active":"2022-08-01T22:55:50.421657+0000","last_peered":"2022-08-01T22:55:50.421657+0000","last_clean":"2022-08-01T22:55:50.421657+0000","last_became_active":"2022-08-01T22:55:50.419823+0000","last_became_peered":"2022-08-01T22:55:50.419823+0000","last_unstale":"2022-08-01T22:55:50.421657+0000","last_undegraded":"2022-08-01T22:55:50.421657+0000","last_fullsized":"2022-08-01T22:55:50.421657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.412678+0000","last_change":"2022-08-01T22:55:44.188357+0000","last_active":"2022-08-01T22:55:50.412678+0000","last_peered":"2022-08-01T22:55:50.412678+0000","last_clean":"2022-08-01T22:55:50.412678+0000","last_became_active":"2022-08-01T22:55:44.187581+0000","last_became_peered":"2022-08-01T22:55:44.187581+0000","last_unstale":"2022-08-01T22:55:50.412678+0000","last_undegraded":"2022-08-01T22:55:50.412678+0000","last_fullsized":"2022-08-01T22:55:50.412678+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.364764+0000","last_change":"2022-08-01T22:55:47.319852+0000","last_active":"2022-08-01T22:55:50.364764+0000","last_peered":"2022-08-01T22:55:50.364764+0000","last_clean":"2022-08-01T22:55:50.364764+0000","last_became_active":"2022-08-01T22:55:47.319441+0000","last_became_peered":"2022-08-01T22:55:47.319441+0000","last_unstale":"2022-08-01T22:55:50.364764+0000","last_undegraded":"2022-08-01T22:55:50.364764+0000","last_fullsized":"2022-08-01T22:55:50.364764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.372349+0000","last_change":"2022-08-01T22:55:50.371166+0000","last_active":"2022-08-01T22:55:50.372349+0000","last_peered":"2022-08-01T22:55:50.372349+0000","last_clean":"2022-08-01T22:55:50.372349+0000","last_became_active":"2022-08-01T22:55:50.370504+0000","last_became_peered":"2022-08-01T22:55:50.370504+0000","last_unstale":"2022-08-01T22:55:50.372349+0000","last_undegraded":"2022-08-01T22:55:50.372349+0000","last_fullsized":"2022-08-01T22:55:50.372349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.413100+0000","last_change":"2022-08-01T22:55:44.555310+0000","last_active":"2022-08-01T22:55:50.413100+0000","last_peered":"2022-08-01T22:55:50.413100+0000","last_clean":"2022-08-01T22:55:50.413100+0000","last_became_active":"2022-08-01T22:55:44.554353+0000","last_became_peered":"2022-08-01T22:55:44.554353+0000","last_unstale":"2022-08-01T22:55:50.413100+0000","last_undegraded":"2022-08-01T22:55:50.413100+0000","last_fullsized":"2022-08-01T22:55:50.413100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.706120+0000","last_change":"2022-08-01T22:55:47.494419+0000","last_active":"2022-08-01T22:55:50.706120+0000","last_peered":"2022-08-01T22:55:50.706120+0000","last_clean":"2022-08-01T22:55:50.706120+0000","last_became_active":"2022-08-01T22:55:47.493623+0000","last_became_peered":"2022-08-01T22:55:47.493623+0000","last_unstale":"2022-08-01T22:55:50.706120+0000","last_undegraded":"2022-08-01T22:55:50.706120+0000","last_fullsized":"2022-08-01T22:55:50.706120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.417449+0000","last_change":"2022-08-01T22:55:50.416360+0000","last_active":"2022-08-01T22:55:50.417449+0000","last_peered":"2022-08-01T22:55:50.417449+0000","last_clean":"2022-08-01T22:55:50.417449+0000","last_became_active":"2022-08-01T22:55:50.414865+0000","last_became_peered":"2022-08-01T22:55:50.414865+0000","last_unstale":"2022-08-01T22:55:50.417449+0000","last_undegraded":"2022-08-01T22:55:50.417449+0000","last_fullsized":"2022-08-01T22:55:50.417449+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.363011+0000","last_change":"2022-08-01T22:55:44.188785+0000","last_active":"2022-08-01T22:55:50.363011+0000","last_peered":"2022-08-01T22:55:50.363011+0000","last_clean":"2022-08-01T22:55:50.363011+0000","last_became_active":"2022-08-01T22:55:44.188345+0000","last_became_peered":"2022-08-01T22:55:44.188345+0000","last_unstale":"2022-08-01T22:55:50.363011+0000","last_undegraded":"2022-08-01T22:55:50.363011+0000","last_fullsized":"2022-08-01T22:55:50.363011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.706456+0000","last_change":"2022-08-01T22:55:47.321815+0000","last_active":"2022-08-01T22:55:50.706456+0000","last_peered":"2022-08-01T22:55:50.706456+0000","last_clean":"2022-08-01T22:55:50.706456+0000","last_became_active":"2022-08-01T22:55:47.321450+0000","last_became_peered":"2022-08-01T22:55:47.321450+0000","last_unstale":"2022-08-01T22:55:50.706456+0000","last_undegraded":"2022-08-01T22:55:50.706456+0000","last_fullsized":"2022-08-01T22:55:50.706456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.374938+0000","last_change":"2022-08-01T22:55:50.374202+0000","last_active":"2022-08-01T22:55:50.374938+0000","last_peered":"2022-08-01T22:55:50.374938+0000","last_clean":"2022-08-01T22:55:50.374938+0000","last_became_active":"2022-08-01T22:55:50.373589+0000","last_became_peered":"2022-08-01T22:55:50.373589+0000","last_unstale":"2022-08-01T22:55:50.374938+0000","last_undegraded":"2022-08-01T22:55:50.374938+0000","last_fullsized":"2022-08-01T22:55:50.374938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.708711+0000","last_change":"2022-08-01T22:55:47.322349+0000","last_active":"2022-08-01T22:55:50.708711+0000","last_peered":"2022-08-01T22:55:50.708711+0000","last_clean":"2022-08-01T22:55:50.708711+0000","last_became_active":"2022-08-01T22:55:47.321659+0000","last_became_peered":"2022-08-01T22:55:47.321659+0000","last_unstale":"2022-08-01T22:55:50.708711+0000","last_undegraded":"2022-08-01T22:55:50.708711+0000","last_fullsized":"2022-08-01T22:55:50.708711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.351095+0000","last_change":"2022-08-01T22:55:44.414856+0000","last_active":"2022-08-01T22:55:50.351095+0000","last_peered":"2022-08-01T22:55:50.351095+0000","last_clean":"2022-08-01T22:55:50.351095+0000","last_became_active":"2022-08-01T22:55:44.413727+0000","last_became_peered":"2022-08-01T22:55:44.413727+0000","last_unstale":"2022-08-01T22:55:50.351095+0000","last_undegraded":"2022-08-01T22:55:50.351095+0000","last_fullsized":"2022-08-01T22:55:50.351095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.421089+0000","last_change":"2022-08-01T22:55:50.417907+0000","last_active":"2022-08-01T22:55:50.421089+0000","last_peered":"2022-08-01T22:55:50.421089+0000","last_clean":"2022-08-01T22:55:50.421089+0000","last_became_active":"2022-08-01T22:55:50.416804+0000","last_became_peered":"2022-08-01T22:55:50.416804+0000","last_unstale":"2022-08-01T22:55:50.421089+0000","last_undegraded":"2022-08-01T22:55:50.421089+0000","last_fullsized":"2022-08-01T22:55:50.421089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.706380+0000","last_change":"2022-08-01T22:55:47.493727+0000","last_active":"2022-08-01T22:55:50.706380+0000","last_peered":"2022-08-01T22:55:50.706380+0000","last_clean":"2022-08-01T22:55:50.706380+0000","last_became_active":"2022-08-01T22:55:47.493016+0000","last_became_peered":"2022-08-01T22:55:47.493016+0000","last_unstale":"2022-08-01T22:55:50.706380+0000","last_undegraded":"2022-08-01T22:55:50.706380+0000","last_fullsized":"2022-08-01T22:55:50.706380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.412622+0000","last_change":"2022-08-01T22:55:44.415068+0000","last_active":"2022-08-01T22:55:50.412622+0000","last_peered":"2022-08-01T22:55:50.412622+0000","last_clean":"2022-08-01T22:55:50.412622+0000","last_became_active":"2022-08-01T22:55:44.414349+0000","last_became_peered":"2022-08-01T22:55:44.414349+0000","last_unstale":"2022-08-01T22:55:50.412622+0000","last_undegraded":"2022-08-01T22:55:50.412622+0000","last_fullsized":"2022-08-01T22:55:50.412622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.420614+0000","last_change":"2022-08-01T22:55:50.418458+0000","last_active":"2022-08-01T22:55:50.420614+0000","last_peered":"2022-08-01T22:55:50.420614+0000","last_clean":"2022-08-01T22:55:50.420614+0000","last_became_active":"2022-08-01T22:55:50.416823+0000","last_became_peered":"2022-08-01T22:55:50.416823+0000","last_unstale":"2022-08-01T22:55:50.420614+0000","last_undegraded":"2022-08-01T22:55:50.420614+0000","last_fullsized":"2022-08-01T22:55:50.420614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"24'7","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.707678+0000","last_change":"2022-08-01T22:55:47.444349+0000","last_active":"2022-08-01T22:55:50.707678+0000","last_peered":"2022-08-01T22:55:50.707678+0000","last_clean":"2022-08-01T22:55:50.707678+0000","last_became_active":"2022-08-01T22:55:47.443749+0000","last_became_peered":"2022-08-01T22:55:47.443749+0000","last_unstale":"2022-08-01T22:55:50.707678+0000","last_undegraded":"2022-08-01T22:55:50.707678+0000","last_fullsized":"2022-08-01T22:55:50.707678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.350894+0000","last_change":"2022-08-01T22:55:44.190143+0000","last_active":"2022-08-01T22:55:50.350894+0000","last_peered":"2022-08-01T22:55:50.350894+0000","last_clean":"2022-08-01T22:55:50.350894+0000","last_became_active":"2022-08-01T22:55:44.189745+0000","last_became_peered":"2022-08-01T22:55:44.189745+0000","last_unstale":"2022-08-01T22:55:50.350894+0000","last_undegraded":"2022-08-01T22:55:50.350894+0000","last_fullsized":"2022-08-01T22:55:50.350894+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.421458+0000","last_change":"2022-08-01T22:55:50.419300+0000","last_active":"2022-08-01T22:55:50.421458+0000","last_peered":"2022-08-01T22:55:50.421458+0000","last_clean":"2022-08-01T22:55:50.421458+0000","last_became_active":"2022-08-01T22:55:50.418583+0000","last_became_peered":"2022-08-01T22:55:50.418583+0000","last_unstale":"2022-08-01T22:55:50.421458+0000","last_undegraded":"2022-08-01T22:55:50.421458+0000","last_fullsized":"2022-08-01T22:55:50.421458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:49.324608+0000","last_clean_scrub_stamp":"2022-08-01T22:55:49.324608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.705103+0000","last_change":"2022-08-01T22:55:47.319248+0000","last_active":"2022-08-01T22:55:50.705103+0000","last_peered":"2022-08-01T22:55:50.705103+0000","last_clean":"2022-08-01T22:55:50.705103+0000","last_became_active":"2022-08-01T22:55:47.318884+0000","last_became_peered":"2022-08-01T22:55:47.318884+0000","last_unstale":"2022-08-01T22:55:50.705103+0000","last_undegraded":"2022-08-01T22:55:50.705103+0000","last_fullsized":"2022-08-01T22:55:50.705103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:46.294311+0000","last_clean_scrub_stamp":"2022-08-01T22:55:46.294311+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2022-08-01T22:55:50.367009+0000","last_change":"2022-08-01T22:55:44.191477+0000","last_active":"2022-08-01T22:55:50.367009+0000","last_peered":"2022-08-01T22:55:50.367009+0000","last_clean":"2022-08-01T22:55:50.367009+0000","last_became_active":"2022-08-01T22:55:44.191167+0000","last_became_peered":"2022-08-01T22:55:44.191167+0000","last_unstale":"2022-08-01T22:55:50.367009+0000","last_undegraded":"2022-08-01T22:55:50.367009+0000","last_fullsized":"2022-08-01T22:55:50.367009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":102140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":101565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":101725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":101599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":102300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":101599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]}]}} 2022-08-01T22:56:00.448 INFO:tasks.ceph.ceph_manager.ceph:clean! 2022-08-01T22:56:00.449 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2022-08-01T22:56:00.449 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2022-08-01T22:56:00.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2022-08-01T22:56:00.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.550+0000 7f4160bfb700 1 Processor -- start 2022-08-01T22:56:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.550+0000 7f4160bfb700 1 -- start start 2022-08-01T22:56:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.550+0000 7f4160bfb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.550+0000 7f4160bfb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f415c1405d0 0x7f415c140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c0aee80 con 0x7f415c0614b0 2022-08-01T22:56:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c0b70b0 con 0x7f415c1405d0 2022-08-01T22:56:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0af0b0 con 0x7f415c0ad750 2022-08-01T22:56:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f415c0614b0 0x7f415c0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56670/0 (socket says 172.21.15.32:56670) 2022-08-01T22:56:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415b7fe700 1 -- 172.21.15.32:0/3369421580 learned_addr learned my addr 172.21.15.32:0/3369421580 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4153fff700 1 --2- 172.21.15.32:0/3369421580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415bfff700 1 --2- 172.21.15.32:0/3369421580 >> v2:172.21.15.55:3300/0 conn(0x7f415c1405d0 0x7f415c140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4153fff700 1 -- 172.21.15.32:0/3369421580 >> v2:172.21.15.55:3300/0 conn(0x7f415c1405d0 msgr2=0x7f415c140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4153fff700 1 --2- 172.21.15.32:0/3369421580 >> v2:172.21.15.55:3300/0 conn(0x7f415c1405d0 0x7f415c140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4153fff700 1 -- 172.21.15.32:0/3369421580 >> v1:172.21.15.32:6789/0 conn(0x7f415c0614b0 legacy=0x7f415c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415affd700 1 -- 172.21.15.32:0/3369421580 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008284787 0 0) 0x7f415c0aee80 con 0x7f415c0614b0 2022-08-01T22:56:00.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4153fff700 1 -- 172.21.15.32:0/3369421580 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0af350 con 0x7f415c0ad750 2022-08-01T22:56:00.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4153fff700 1 --2- 172.21.15.32:0/3369421580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c061890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4144002f70 tx=0x7f4144006df0).ready entity=mon.2 client_cookie=e07cc2e0bbcdf57 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415affd700 1 -- 172.21.15.32:0/3369421580 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414400d570 con 0x7f415c0ad750 2022-08-01T22:56:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415affd700 1 -- 172.21.15.32:0/3369421580 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144000e90 con 0x7f415c0ad750 2022-08-01T22:56:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f415affd700 1 -- 172.21.15.32:0/3369421580 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414400d8c0 con 0x7f415c0ad750 2022-08-01T22:56:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- 172.21.15.32:0/3369421580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 msgr2=0x7f415c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 --2- 172.21.15.32:0/3369421580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c061890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4144002f70 tx=0x7f4144006df0).stop 2022-08-01T22:56:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- 172.21.15.32:0/3369421580 shutdown_connections 2022-08-01T22:56:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 --2- 172.21.15.32:0/3369421580 >> v2:172.21.15.55:3300/0 conn(0x7f415c1405d0 0x7f415c140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 --2- 172.21.15.32:0/3369421580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c061890 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- 172.21.15.32:0/3369421580 >> 172.21.15.32:0/3369421580 conn(0x7f415c066f70 msgr2=0x7f415c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- 172.21.15.32:0/3369421580 shutdown_connections 2022-08-01T22:56:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.554+0000 7f4160bfb700 1 -- 172.21.15.32:0/3369421580 wait complete. 2022-08-01T22:56:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 Processor -- start 2022-08-01T22:56:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 -- start start 2022-08-01T22:56:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 0x7f415c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f415c1405d0 0x7f415c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c149880 con 0x7f415c0614b0 2022-08-01T22:56:00.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c149b00 con 0x7f415c1405d0 2022-08-01T22:56:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f4160bfb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c149da0 con 0x7f415c0ad750 2022-08-01T22:56:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.558+0000 7f415b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 0x7f415c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 0x7f415c155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40898/0 (socket says 172.21.15.32:40898) 2022-08-01T22:56:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 -- 172.21.15.32:0/119372788 learned_addr learned my addr 172.21.15.32:0/119372788 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415bfff700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f415c1405d0 0x7f415c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f4153fff700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 -- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 msgr2=0x7f415c156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 -- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f415c1405d0 msgr2=0x7f415c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f415c1405d0 0x7f415c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 -- 172.21.15.32:0/119372788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414400b180 con 0x7f415c0614b0 2022-08-01T22:56:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415bfff700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f415c1405d0 0x7f415c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f415b7fe700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 0x7f415c155e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f414c000ed0 tx=0x7f414c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414c0070d0 con 0x7f415c0614b0 2022-08-01T22:56:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c007230 con 0x7f415c0614b0 2022-08-01T22:56:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c14a040 con 0x7f415c0614b0 2022-08-01T22:56:00.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414c005610 con 0x7f415c0614b0 2022-08-01T22:56:00.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c150120 con 0x7f415c0614b0 2022-08-01T22:56:00.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c0af5f0 con 0x7f415c0614b0 2022-08-01T22:56:00.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.562+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f414c010070 con 0x7f415c0614b0 2022-08-01T22:56:00.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.566+0000 7f41597fa700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f41400539e0 0x7f4140055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.566+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f414c06b680 con 0x7f415c0614b0 2022-08-01T22:56:00.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.566+0000 7f4153fff700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f41400539e0 0x7f4140055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.566+0000 7f4153fff700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f41400539e0 0x7f4140055ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f414401a000 tx=0x7f4144014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.570+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f414c042440 con 0x7f415c0614b0 2022-08-01T22:56:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.738+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f415c0574e0 con 0x7f415c0614b0 2022-08-01T22:56:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.738+0000 7f41597fa700 1 -- 172.21.15.32:0/119372788 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f414c045a60 con 0x7f415c0614b0 2022-08-01T22:56:00.741 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f41400539e0 msgr2=0x7f4140055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f41400539e0 0x7f4140055ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f414401a000 tx=0x7f4144014000).stop 2022-08-01T22:56:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 msgr2=0x7f415c155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 0x7f415c155e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f414c000ed0 tx=0x7f414c007ac0).stop 2022-08-01T22:56:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 shutdown_connections 2022-08-01T22:56:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f41400539e0 0x7f4140055ea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f415c1405d0 0x7f415c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f415c0ad750 0x7f415c156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 --2- 172.21.15.32:0/119372788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0614b0 0x7f415c155e60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 >> 172.21.15.32:0/119372788 conn(0x7f415c066f70 msgr2=0x7f415c0b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 shutdown_connections 2022-08-01T22:56:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.742+0000 7f4160bfb700 1 -- 172.21.15.32:0/119372788 wait complete. 2022-08-01T22:56:00.758 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-08-01T22:56:00.758 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2022-08-01T22:56:00.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:00.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f935499f700 1 Processor -- start 2022-08-01T22:56:00.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f935499f700 1 -- start start 2022-08-01T22:56:00.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f935499f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9350061d10 0x7f9350144630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f935499f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500b2100 con 0x7f93500b41f0 2022-08-01T22:56:00.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f935499f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500b69a0 con 0x7f93500b65c0 2022-08-01T22:56:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f935499f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b1630 con 0x7f9350061d10 2022-08-01T22:56:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934f59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93500b41f0 0x7f9350144b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56684/0 (socket says 172.21.15.32:56684) 2022-08-01T22:56:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934f59e700 1 -- 172.21.15.32:0/2140391078 learned_addr learned my addr 172.21.15.32:0/2140391078 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934e59c700 1 --2- 172.21.15.32:0/2140391078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9350061d10 0x7f9350144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456290071 0 0) 0x7f93500b2100 con 0x7f93500b41f0 2022-08-01T22:56:00.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9330003260 con 0x7f93500b41f0 2022-08-01T22:56:00.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451417395 0 0) 0x7f93500b69a0 con 0x7f93500b65c0 2022-08-01T22:56:00.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93500b2100 con 0x7f93500b65c0 2022-08-01T22:56:00.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934e59c700 1 -- 172.21.15.32:0/2140391078 >> v1:172.21.15.55:6789/0 conn(0x7f93500b65c0 legacy=0x7f9350061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934e59c700 1 -- 172.21.15.32:0/2140391078 >> v1:172.21.15.32:6789/0 conn(0x7f93500b41f0 legacy=0x7f9350144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934e59c700 1 -- 172.21.15.32:0/2140391078 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93501482a0 con 0x7f9350061d10 2022-08-01T22:56:00.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934e59c700 1 --2- 172.21.15.32:0/2140391078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9350061d10 0x7f9350144630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f93400032f0 tx=0x7f93400091d0).ready entity=mon.2 client_cookie=71d0e6604c47b663 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934000b640 con 0x7f9350061d10 2022-08-01T22:56:00.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934000b7a0 con 0x7f9350061d10 2022-08-01T22:56:00.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.862+0000 7f934dd9b700 1 -- 172.21.15.32:0/2140391078 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934000b910 con 0x7f9350061d10 2022-08-01T22:56:00.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- 172.21.15.32:0/2140391078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9350061d10 msgr2=0x7f9350144630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 --2- 172.21.15.32:0/2140391078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9350061d10 0x7f9350144630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f93400032f0 tx=0x7f93400091d0).stop 2022-08-01T22:56:00.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- 172.21.15.32:0/2140391078 shutdown_connections 2022-08-01T22:56:00.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 --2- 172.21.15.32:0/2140391078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9350061d10 0x7f9350144630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- 172.21.15.32:0/2140391078 >> 172.21.15.32:0/2140391078 conn(0x7f935005c620 msgr2=0x7f935005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:00.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- 172.21.15.32:0/2140391078 shutdown_connections 2022-08-01T22:56:00.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- 172.21.15.32:0/2140391078 wait complete. 2022-08-01T22:56:00.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 Processor -- start 2022-08-01T22:56:00.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- start start 2022-08-01T22:56:00.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9350061d10 0x7f93500b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 0x7f9350149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93500b65c0 0x7f9350149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935014b820 con 0x7f93500b41f0 2022-08-01T22:56:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935014b320 con 0x7f9350061d10 2022-08-01T22:56:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f935499f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935014b580 con 0x7f93500b65c0 2022-08-01T22:56:00.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 0x7f9350149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 0x7f9350149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40914/0 (socket says 172.21.15.32:40914) 2022-08-01T22:56:00.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 -- 172.21.15.32:0/3300875617 learned_addr learned my addr 172.21.15.32:0/3300875617 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934f59e700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93500b65c0 0x7f9350149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934ed9d700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9350061d10 0x7f93500b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 -- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93500b65c0 msgr2=0x7f9350149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93500b65c0 0x7f9350149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 -- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9350061d10 msgr2=0x7f93500b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9350061d10 0x7f93500b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 -- 172.21.15.32:0/3300875617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009470 con 0x7f93500b41f0 2022-08-01T22:56:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.866+0000 7f934e59c700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 0x7f9350149650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f93400032f0 tx=0x7f9340016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934000b910 con 0x7f93500b41f0 2022-08-01T22:56:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93501491b0 con 0x7f93500b41f0 2022-08-01T22:56:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934001f3f0 con 0x7f93500b41f0 2022-08-01T22:56:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934000b640 con 0x7f93500b41f0 2022-08-01T22:56:00.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935014a760 con 0x7f93500b41f0 2022-08-01T22:56:00.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93500b2330 con 0x7f93500b41f0 2022-08-01T22:56:00.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f934000f070 con 0x7f93500b41f0 2022-08-01T22:56:00.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f9347fff700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9334053a70 0x7f9334055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f934006cb20 con 0x7f93500b41f0 2022-08-01T22:56:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f934ed9d700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9334053a70 0x7f9334055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.870+0000 7f934ed9d700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9334053a70 0x7f9334055f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f933800d000 tx=0x7f9338007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:00.878+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9340043840 con 0x7f93500b41f0 2022-08-01T22:56:01.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.005+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93500574e0 con 0x7f93500b41f0 2022-08-01T22:56:01.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f9347fff700 1 -- 172.21.15.32:0/3300875617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f9340030fc0 con 0x7f93500b41f0 2022-08-01T22:56:01.011 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:01.011 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9334053a70 msgr2=0x7f9334055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9334053a70 0x7f9334055f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f933800d000 tx=0x7f9338007a20).stop 2022-08-01T22:56:01.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 msgr2=0x7f9350149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 0x7f9350149650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f93400032f0 tx=0x7f9340016000).stop 2022-08-01T22:56:01.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 shutdown_connections 2022-08-01T22:56:01.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9334053a70 0x7f9334055f30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93500b65c0 0x7f9350149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93500b41f0 0x7f9350149650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 --2- 172.21.15.32:0/3300875617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9350061d10 0x7f93500b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 >> 172.21.15.32:0/3300875617 conn(0x7f935005c620 msgr2=0x7f93500b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 shutdown_connections 2022-08-01T22:56:01.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.009+0000 7f935499f700 1 -- 172.21.15.32:0/3300875617 wait complete. 2022-08-01T22:56:01.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:01.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:01.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.125+0000 7f8f6df19700 1 Processor -- start 2022-08-01T22:56:01.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.125+0000 7f8f6df19700 1 -- start start 2022-08-01T22:56:01.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8f68064b10 0x7f8f68144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f68064ef0 con 0x7f8f680b36e0 2022-08-01T22:56:01.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f68066690 con 0x7f8f68141a80 2022-08-01T22:56:01.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6813ebf0 con 0x7f8f68064b10 2022-08-01T22:56:01.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f67fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f680b36e0 0x7f8f68141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56698/0 (socket says 172.21.15.32:56698) 2022-08-01T22:56:01.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f67fff700 1 -- 172.21.15.32:0/2279992122 learned_addr learned my addr 172.21.15.32:0/2279992122 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:01.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6cf17700 1 --2- 172.21.15.32:0/2279992122 >> v2:172.21.15.55:3300/0 conn(0x7f8f68064b10 0x7f8f68144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f677fe700 1 -- 172.21.15.32:0/2279992122 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339198649 0 0) 0x7f8f68064ef0 con 0x7f8f680b36e0 2022-08-01T22:56:01.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f677fe700 1 -- 172.21.15.32:0/2279992122 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f48003260 con 0x7f8f680b36e0 2022-08-01T22:56:01.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6cf17700 1 -- 172.21.15.32:0/2279992122 >> v1:172.21.15.55:6789/0 conn(0x7f8f68141a80 legacy=0x7f8f68066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6cf17700 1 -- 172.21.15.32:0/2279992122 >> v1:172.21.15.32:6789/0 conn(0x7f8f680b36e0 legacy=0x7f8f68141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f677fe700 1 -- 172.21.15.32:0/2279992122 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848700151 0 0) 0x7f8f48003260 con 0x7f8f680b36e0 2022-08-01T22:56:01.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6cf17700 1 -- 172.21.15.32:0/2279992122 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f680b4850 con 0x7f8f68064b10 2022-08-01T22:56:01.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6cf17700 1 --2- 172.21.15.32:0/2279992122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f68064b10 0x7f8f68144e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f8f50001800 tx=0x7f8f580091d0).ready entity=mon.1 client_cookie=fd2e9ffa8b5b8565 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f677fe700 1 -- 172.21.15.32:0/2279992122 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f50001660 con 0x7f8f68064b10 2022-08-01T22:56:01.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f677fe700 1 -- 172.21.15.32:0/2279992122 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800b580 con 0x7f8f68064b10 2022-08-01T22:56:01.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f677fe700 1 -- 172.21.15.32:0/2279992122 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f5800b6f0 con 0x7f8f68064b10 2022-08-01T22:56:01.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- 172.21.15.32:0/2279992122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f68064b10 msgr2=0x7f8f68144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 --2- 172.21.15.32:0/2279992122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f68064b10 0x7f8f68144e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f8f50001800 tx=0x7f8f580091d0).stop 2022-08-01T22:56:01.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- 172.21.15.32:0/2279992122 shutdown_connections 2022-08-01T22:56:01.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 --2- 172.21.15.32:0/2279992122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f68064b10 0x7f8f68144e60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- 172.21.15.32:0/2279992122 >> 172.21.15.32:0/2279992122 conn(0x7f8f6805f840 msgr2=0x7f8f6805fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- 172.21.15.32:0/2279992122 shutdown_connections 2022-08-01T22:56:01.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.129+0000 7f8f6df19700 1 -- 172.21.15.32:0/2279992122 wait complete. 2022-08-01T22:56:01.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 Processor -- start 2022-08-01T22:56:01.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 -- start start 2022-08-01T22:56:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 0x7f8f680accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f680b36e0 0x7f8f68139250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f68141a80 0x7f8f680b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680ad820 con 0x7f8f68064b10 2022-08-01T22:56:01.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680ad580 con 0x7f8f680b36e0 2022-08-01T22:56:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680b54d0 con 0x7f8f68141a80 2022-08-01T22:56:01.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 0x7f8f680accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 0x7f8f680accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40926/0 (socket says 172.21.15.32:40926) 2022-08-01T22:56:01.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 -- 172.21.15.32:0/3509608830 learned_addr learned my addr 172.21.15.32:0/3509608830 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:01.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f67fff700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f680b36e0 0x7f8f68139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6d718700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f68141a80 0x7f8f680b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 -- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f68141a80 msgr2=0x7f8f680b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f68141a80 0x7f8f680b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 -- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f680b36e0 msgr2=0x7f8f68139250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f680b36e0 0x7f8f68139250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 -- 172.21.15.32:0/3509608830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009470 con 0x7f8f68064b10 2022-08-01T22:56:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6d718700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f68141a80 0x7f8f680b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:01.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f67fff700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f680b36e0 0x7f8f68139250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:01.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6cf17700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 0x7f8f680accd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8f50001800 tx=0x7f8f58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f5800b6f0 con 0x7f8f68064b10 2022-08-01T22:56:01.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f580032f0 con 0x7f8f68064b10 2022-08-01T22:56:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6813e7f0 con 0x7f8f68064b10 2022-08-01T22:56:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f580105b0 con 0x7f8f68064b10 2022-08-01T22:56:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.133+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f68139ab0 con 0x7f8f68064b10 2022-08-01T22:56:01.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.137+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f680b4af0 con 0x7f8f68064b10 2022-08-01T22:56:01.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.137+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8f5800f070 con 0x7f8f68064b10 2022-08-01T22:56:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.137+0000 7f8f65ffb700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f4c053b00 0x7f8f4c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.137+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8f5806d5e0 con 0x7f8f68064b10 2022-08-01T22:56:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.137+0000 7f8f67fff700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f4c053b00 0x7f8f4c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.137+0000 7f8f67fff700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f4c053b00 0x7f8f4c055fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8f5000b000 tx=0x7f8f50000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.145+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f58044320 con 0x7f8f68064b10 2022-08-01T22:56:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.269+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8f68069d90 con 0x7f8f68064b10 2022-08-01T22:56:01.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.269+0000 7f8f65ffb700 1 -- 172.21.15.32:0/3509608830 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8f58015020 con 0x7f8f68064b10 2022-08-01T22:56:01.272 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f4c053b00 msgr2=0x7f8f4c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f4c053b00 0x7f8f4c055fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8f5000b000 tx=0x7f8f50000bc0).stop 2022-08-01T22:56:01.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 msgr2=0x7f8f680accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 0x7f8f680accd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8f50001800 tx=0x7f8f58016000).stop 2022-08-01T22:56:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 shutdown_connections 2022-08-01T22:56:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f4c053b00 0x7f8f4c055fc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f68141a80 0x7f8f680b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f680b36e0 0x7f8f68139250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 --2- 172.21.15.32:0/3509608830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f68064b10 0x7f8f680accd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 >> 172.21.15.32:0/3509608830 conn(0x7f8f6805f840 msgr2=0x7f8f6813ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 shutdown_connections 2022-08-01T22:56:01.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.273+0000 7f8f6df19700 1 -- 172.21.15.32:0/3509608830 wait complete. 2022-08-01T22:56:01.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 Processor -- start 2022-08-01T22:56:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 -- start start 2022-08-01T22:56:01.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3ec0a9320 0x7fc3ec1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3ec0b0d10 0x7fc3ec146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0b0560 0x7fc3ec140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0b12d0 con 0x7fc3ec0a9320 2022-08-01T22:56:01.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0a9700 con 0x7fc3ec0b0d10 2022-08-01T22:56:01.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3f1655700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0b1500 con 0x7fc3ec0b0560 2022-08-01T22:56:01.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3ec0a9320 0x7fc3ec1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3ec0a9320 0x7fc3ec1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40934/0 (socket says 172.21.15.32:40934) 2022-08-01T22:56:01.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.389+0000 7fc3ebfff700 1 -- 172.21.15.32:0/3983754480 learned_addr learned my addr 172.21.15.32:0/3983754480 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:01.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f0e54700 1 --2- 172.21.15.32:0/3983754480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0b0560 0x7fc3ec140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3eb7fe700 1 --2- 172.21.15.32:0/3983754480 >> v2:172.21.15.55:3300/0 conn(0x7fc3ec0b0d10 0x7fc3ec146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3ebfff700 1 -- 172.21.15.32:0/3983754480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0b0560 msgr2=0x7fc3ec140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:01.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/3983754480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0b0560 0x7fc3ec140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3ebfff700 1 -- 172.21.15.32:0/3983754480 >> v2:172.21.15.55:3300/0 conn(0x7fc3ec0b0d10 msgr2=0x7fc3ec146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:01.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/3983754480 >> v2:172.21.15.55:3300/0 conn(0x7fc3ec0b0d10 0x7fc3ec146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3ebfff700 1 -- 172.21.15.32:0/3983754480 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0b17a0 con 0x7fc3ec0a9320 2022-08-01T22:56:01.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3eb7fe700 1 --2- 172.21.15.32:0/3983754480 >> v2:172.21.15.55:3300/0 conn(0x7fc3ec0b0d10 0x7fc3ec146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:01.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/3983754480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0a9320 0x7fc3ec1466e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc3e0002f70 tx=0x7fc3e0006eb0).ready entity=mon.0 client_cookie=3a9286a8e944273b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3eaffd700 1 -- 172.21.15.32:0/3983754480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3e000d570 con 0x7fc3ec0a9320 2022-08-01T22:56:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3eaffd700 1 -- 172.21.15.32:0/3983754480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e0000e90 con 0x7fc3ec0a9320 2022-08-01T22:56:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3eaffd700 1 -- 172.21.15.32:0/3983754480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3e000d900 con 0x7fc3ec0a9320 2022-08-01T22:56:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 -- 172.21.15.32:0/3983754480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0a9320 msgr2=0x7fc3ec1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 --2- 172.21.15.32:0/3983754480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0a9320 0x7fc3ec1466e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc3e0002f70 tx=0x7fc3e0006eb0).stop 2022-08-01T22:56:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 -- 172.21.15.32:0/3983754480 shutdown_connections 2022-08-01T22:56:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 --2- 172.21.15.32:0/3983754480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0b0560 0x7fc3ec140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 --2- 172.21.15.32:0/3983754480 >> v2:172.21.15.55:3300/0 conn(0x7fc3ec0b0d10 0x7fc3ec146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 --2- 172.21.15.32:0/3983754480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0a9320 0x7fc3ec1466e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 -- 172.21.15.32:0/3983754480 >> 172.21.15.32:0/3983754480 conn(0x7fc3ec0612d0 msgr2=0x7fc3ec062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 -- 172.21.15.32:0/3983754480 shutdown_connections 2022-08-01T22:56:01.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.393+0000 7fc3f1655700 1 -- 172.21.15.32:0/3983754480 wait complete. 2022-08-01T22:56:01.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 Processor -- start 2022-08-01T22:56:01.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 -- start start 2022-08-01T22:56:01.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0a9320 0x7fc3ec0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 0x7fc3ec0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ec0b0d10 0x7fc3ec1429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec1412d0 con 0x7fc3ec0b0560 2022-08-01T22:56:01.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0abbd0 con 0x7fc3ec0b0d10 2022-08-01T22:56:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f1655700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec141030 con 0x7fc3ec0a9320 2022-08-01T22:56:01.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 0x7fc3ec0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 0x7fc3ec0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40942/0 (socket says 172.21.15.32:40942) 2022-08-01T22:56:01.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 -- 172.21.15.32:0/4246677110 learned_addr learned my addr 172.21.15.32:0/4246677110 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0a9320 0x7fc3ec0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f0e54700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ec0b0d10 0x7fc3ec1429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 -- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0a9320 msgr2=0x7fc3ec0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0a9320 0x7fc3ec0b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 -- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ec0b0d10 msgr2=0x7fc3ec1429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ec0b0d10 0x7fc3ec1429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 -- 172.21.15.32:0/4246677110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e000b1e0 con 0x7fc3ec0b0560 2022-08-01T22:56:01.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3f0e54700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ec0b0d10 0x7fc3ec1429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:01.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0a9320 0x7fc3ec0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:01.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.397+0000 7fc3eb7fe700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 0x7fc3ec0aafa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc3dc006940 tx=0x7fc3dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3dc011780 con 0x7fc3ec0b0560 2022-08-01T22:56:01.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0118e0 con 0x7fc3ec0b0560 2022-08-01T22:56:01.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec0b50b0 con 0x7fc3ec0b0560 2022-08-01T22:56:01.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3dc011c40 con 0x7fc3ec0b0560 2022-08-01T22:56:01.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec06a5f0 con 0x7fc3ec0b0560 2022-08-01T22:56:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec0b5240 con 0x7fc3ec0b0560 2022-08-01T22:56:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc3dc01c460 con 0x7fc3ec0b0560 2022-08-01T22:56:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3e97fa700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc3cc053950 0x7fc3cc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc3dc06c660 con 0x7fc3ec0b0560 2022-08-01T22:56:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.401+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc3cc053950 0x7fc3cc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.405+0000 7fc3ebfff700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc3cc053950 0x7fc3cc055e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc3e001a000 tx=0x7fc3e0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.409+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3dc043320 con 0x7fc3ec0b0560 2022-08-01T22:56:01.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3ec0574e0 con 0x7fc3ec0b0560 2022-08-01T22:56:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3e97fa700 1 -- 172.21.15.32:0/4246677110 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc3dc046940 con 0x7fc3ec0b0560 2022-08-01T22:56:01.575 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc3cc053950 msgr2=0x7fc3cc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc3cc053950 0x7fc3cc055e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc3e001a000 tx=0x7fc3e0015000).stop 2022-08-01T22:56:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 msgr2=0x7fc3ec0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 0x7fc3ec0aafa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc3dc006940 tx=0x7fc3dc008040).stop 2022-08-01T22:56:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 shutdown_connections 2022-08-01T22:56:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc3cc053950 0x7fc3cc055e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ec0b0d10 0x7fc3ec1429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0b0560 0x7fc3ec0aafa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 --2- 172.21.15.32:0/4246677110 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3ec0a9320 0x7fc3ec0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 >> 172.21.15.32:0/4246677110 conn(0x7fc3ec0612d0 msgr2=0x7fc3ec13e060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 shutdown_connections 2022-08-01T22:56:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.537+0000 7fc3f1655700 1 -- 172.21.15.32:0/4246677110 wait complete. 2022-08-01T22:56:01.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3e4c94700 1 Processor -- start 2022-08-01T22:56:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3e4c94700 1 -- start start 2022-08-01T22:56:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3e4c94700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3e00b4310 0x7fe3e0140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3e4c94700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e0143cf0 con 0x7fe3e00b4820 2022-08-01T22:56:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3e4c94700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e0143f90 con 0x7fe3e0146ae0 2022-08-01T22:56:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3e4c94700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e01476a0 con 0x7fe3e00b4310 2022-08-01T22:56:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ded9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3e00b4820 0x7fe3e00b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56724/0 (socket says 172.21.15.32:56724) 2022-08-01T22:56:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ded9d700 1 -- 172.21.15.32:0/4159942695 learned_addr learned my addr 172.21.15.32:0/4159942695 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ddd9b700 1 --2- 172.21.15.32:0/4159942695 >> v2:172.21.15.32:3300/0 conn(0x7fe3e00b4310 0x7fe3e0140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3dd59a700 1 -- 172.21.15.32:0/4159942695 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460566500 0 0) 0x7fe3e0143cf0 con 0x7fe3e00b4820 2022-08-01T22:56:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/4159942695 >> v1:172.21.15.55:6789/0 conn(0x7fe3e0146ae0 legacy=0x7fe3e01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/4159942695 >> v1:172.21.15.32:6789/0 conn(0x7fe3e00b4820 legacy=0x7fe3e00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/4159942695 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00b0e20 con 0x7fe3e00b4310 2022-08-01T22:56:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3ddd9b700 1 --2- 172.21.15.32:0/4159942695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4310 0x7fe3e0140ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fe3c8002f70 tx=0x7fe3c8006df0).ready entity=mon.0 client_cookie=89792c15be981745 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3dd59a700 1 -- 172.21.15.32:0/4159942695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3c800d610 con 0x7fe3e00b4310 2022-08-01T22:56:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3dd59a700 1 -- 172.21.15.32:0/4159942695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c8000e90 con 0x7fe3e00b4310 2022-08-01T22:56:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.697+0000 7fe3dd59a700 1 -- 172.21.15.32:0/4159942695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3c800d960 con 0x7fe3e00b4310 2022-08-01T22:56:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- 172.21.15.32:0/4159942695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4310 msgr2=0x7fe3e0140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/4159942695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4310 0x7fe3e0140ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fe3c8002f70 tx=0x7fe3c8006df0).stop 2022-08-01T22:56:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- 172.21.15.32:0/4159942695 shutdown_connections 2022-08-01T22:56:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/4159942695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4310 0x7fe3e0140ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- 172.21.15.32:0/4159942695 >> 172.21.15.32:0/4159942695 conn(0x7fe3e005c620 msgr2=0x7fe3e005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- 172.21.15.32:0/4159942695 shutdown_connections 2022-08-01T22:56:01.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- 172.21.15.32:0/4159942695 wait complete. 2022-08-01T22:56:01.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 Processor -- start 2022-08-01T22:56:01.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- start start 2022-08-01T22:56:01.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3e00b4310 0x7fe3e00b9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 0x7fe3e00b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3e0146ae0 0x7fe3e00af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00b88e0 con 0x7fe3e00b4820 2022-08-01T22:56:01.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01533f0 con 0x7fe3e00b4310 2022-08-01T22:56:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3e4c94700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e0152f90 con 0x7fe3e0146ae0 2022-08-01T22:56:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3ddd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 0x7fe3e00b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3ddd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 0x7fe3e00b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40956/0 (socket says 172.21.15.32:40956) 2022-08-01T22:56:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/218042339 learned_addr learned my addr 172.21.15.32:0/218042339 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3de59c700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3e00b4310 0x7fe3e00b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.701+0000 7fe3ded9d700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3e0146ae0 0x7fe3e00af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3e0146ae0 msgr2=0x7fe3e00af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ddd9b700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3e0146ae0 0x7fe3e00af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3e00b4310 msgr2=0x7fe3e00b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ddd9b700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3e00b4310 0x7fe3e00b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ddd9b700 1 -- 172.21.15.32:0/218042339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c800b180 con 0x7fe3e00b4820 2022-08-01T22:56:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ded9d700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3e0146ae0 0x7fe3e00af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3ddd9b700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 0x7fe3e00b7700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe3c8002f70 tx=0x7fe3c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3c800d290 con 0x7fe3e00b4820 2022-08-01T22:56:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c8011b80 con 0x7fe3e00b4820 2022-08-01T22:56:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e00b0840 con 0x7fe3e00b4820 2022-08-01T22:56:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3c801e460 con 0x7fe3e00b4820 2022-08-01T22:56:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e00b7bf0 con 0x7fe3e00b4820 2022-08-01T22:56:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e00b86b0 con 0x7fe3e00b4820 2022-08-01T22:56:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.705+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe3c8025030 con 0x7fe3e00b4820 2022-08-01T22:56:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.709+0000 7fe3d77fe700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3c4053b00 0x7fe3c4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.709+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe3c806d1b0 con 0x7fe3e00b4820 2022-08-01T22:56:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.709+0000 7fe3de59c700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3c4053b00 0x7fe3c4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.709+0000 7fe3de59c700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3c4053b00 0x7fe3c4055fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe3d000d000 tx=0x7fe3d0007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.713+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe3c8047e60 con 0x7fe3e00b4820 2022-08-01T22:56:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.837+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3e0152db0 con 0x7fe3e00b4820 2022-08-01T22:56:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3d77fe700 1 -- 172.21.15.32:0/218042339 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe3c8043f70 con 0x7fe3e00b4820 2022-08-01T22:56:01.843 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3c4053b00 msgr2=0x7fe3c4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3c4053b00 0x7fe3c4055fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe3d000d000 tx=0x7fe3d0007b90).stop 2022-08-01T22:56:01.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 msgr2=0x7fe3e00b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 0x7fe3e00b7700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe3c8002f70 tx=0x7fe3c8015000).stop 2022-08-01T22:56:01.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 shutdown_connections 2022-08-01T22:56:01.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3c4053b00 0x7fe3c4055fc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3e0146ae0 0x7fe3e00af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e00b4820 0x7fe3e00b7700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 --2- 172.21.15.32:0/218042339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3e00b4310 0x7fe3e00b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 >> 172.21.15.32:0/218042339 conn(0x7fe3e005c620 msgr2=0x7fe3e013f580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 shutdown_connections 2022-08-01T22:56:01.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.841+0000 7fe3e4c94700 1 -- 172.21.15.32:0/218042339 wait complete. 2022-08-01T22:56:01.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:01.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.965+0000 7fd0a95bd700 1 Processor -- start 2022-08-01T22:56:01.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.965+0000 7fd0a95bd700 1 -- start start 2022-08-01T22:56:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0a40ad750 0x7fd0a4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a40aee80 con 0x7fd0a41405d0 2022-08-01T22:56:01.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a40b70b0 con 0x7fd0a40614b0 2022-08-01T22:56:01.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a40af0b0 con 0x7fd0a40ad750 2022-08-01T22:56:01.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0a40ad750 0x7fd0a4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a8dbc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0a41405d0 0x7fd0a41466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56740/0 (socket says 172.21.15.32:56740) 2022-08-01T22:56:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a8dbc700 1 -- 172.21.15.32:0/4247809497 learned_addr learned my addr 172.21.15.32:0/4247809497 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175194590 0 0) 0x7fd0a40aee80 con 0x7fd0a41405d0 2022-08-01T22:56:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd084003260 con 0x7fd0a41405d0 2022-08-01T22:56:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208641932 0 0) 0x7fd0a40b70b0 con 0x7fd0a40614b0 2022-08-01T22:56:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a40aee80 con 0x7fd0a40614b0 2022-08-01T22:56:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a37fe700 1 -- 172.21.15.32:0/4247809497 >> v1:172.21.15.55:6789/0 conn(0x7fd0a40614b0 legacy=0x7fd0a40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a37fe700 1 -- 172.21.15.32:0/4247809497 >> v1:172.21.15.32:6789/0 conn(0x7fd0a41405d0 legacy=0x7fd0a41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a37fe700 1 -- 172.21.15.32:0/4247809497 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a40af350 con 0x7fd0a40ad750 2022-08-01T22:56:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a37fe700 1 --2- 172.21.15.32:0/4247809497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a40ad750 0x7fd0a4061890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fd08c002f70 tx=0x7fd08c006df0).ready entity=mon.0 client_cookie=d8f27b22cb1df03 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c00d540 con 0x7fd0a40ad750 2022-08-01T22:56:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c000e90 con 0x7fd0a40ad750 2022-08-01T22:56:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a2ffd700 1 -- 172.21.15.32:0/4247809497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c00d890 con 0x7fd0a40ad750 2022-08-01T22:56:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- 172.21.15.32:0/4247809497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a40ad750 msgr2=0x7fd0a4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/4247809497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a40ad750 0x7fd0a4061890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fd08c002f70 tx=0x7fd08c006df0).stop 2022-08-01T22:56:01.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- 172.21.15.32:0/4247809497 shutdown_connections 2022-08-01T22:56:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/4247809497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a40ad750 0x7fd0a4061890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- 172.21.15.32:0/4247809497 >> 172.21.15.32:0/4247809497 conn(0x7fd0a4066f70 msgr2=0x7fd0a4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- 172.21.15.32:0/4247809497 shutdown_connections 2022-08-01T22:56:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.969+0000 7fd0a95bd700 1 -- 172.21.15.32:0/4247809497 wait complete. 2022-08-01T22:56:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 Processor -- start 2022-08-01T22:56:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 -- start start 2022-08-01T22:56:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a40614b0 0x7fd0a4152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0a40ad750 0x7fd0a4152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 0x7fd0a40aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4153080 con 0x7fd0a41405d0 2022-08-01T22:56:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4153320 con 0x7fd0a40614b0 2022-08-01T22:56:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40aa6f0 con 0x7fd0a40ad750 2022-08-01T22:56:01.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 0x7fd0a40aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 0x7fd0a40aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40970/0 (socket says 172.21.15.32:40970) 2022-08-01T22:56:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 -- 172.21.15.32:0/1152217678 learned_addr learned my addr 172.21.15.32:0/1152217678 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a37fe700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0a40ad750 0x7fd0a4152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a3fff700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a40614b0 0x7fd0a4152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 -- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0a40ad750 msgr2=0x7fd0a4152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0a40ad750 0x7fd0a4152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:01.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 -- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a40614b0 msgr2=0x7fd0a4152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:01.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a40614b0 0x7fd0a4152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 -- 172.21.15.32:0/1152217678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c00b180 con 0x7fd0a41405d0 2022-08-01T22:56:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a37fe700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0a40ad750 0x7fd0a4152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a8dbc700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 0x7fd0a40aba50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd09c001490 tx=0x7fd09c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd09c005ba0 con 0x7fd0a41405d0 2022-08-01T22:56:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c005d00 con 0x7fd0a41405d0 2022-08-01T22:56:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a40aab90 con 0x7fd0a41405d0 2022-08-01T22:56:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd09c0072c0 con 0x7fd0a41405d0 2022-08-01T22:56:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.973+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a40ad1e0 con 0x7fd0a41405d0 2022-08-01T22:56:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.977+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a40ab1c0 con 0x7fd0a41405d0 2022-08-01T22:56:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.977+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd09c010070 con 0x7fd0a41405d0 2022-08-01T22:56:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.977+0000 7fd0a17fa700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd088053950 0x7fd088055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.977+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd09c06bc20 con 0x7fd0a41405d0 2022-08-01T22:56:02.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.977+0000 7fd0a3fff700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd088053950 0x7fd088055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.977+0000 7fd0a3fff700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd088053950 0x7fd088055e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd09400b000 tx=0x7fd094009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:02.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:01.985+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0a40ab1c0 con 0x7fd0a41405d0 2022-08-01T22:56:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.109+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0a40ab1c0 con 0x7fd0a41405d0 2022-08-01T22:56:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.109+0000 7fd0a17fa700 1 -- 172.21.15.32:0/1152217678 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd0a40ab1c0 con 0x7fd0a41405d0 2022-08-01T22:56:02.115 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd088053950 msgr2=0x7fd088055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd088053950 0x7fd088055e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd09400b000 tx=0x7fd094009040).stop 2022-08-01T22:56:02.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 msgr2=0x7fd0a40aba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 0x7fd0a40aba50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd09c001490 tx=0x7fd09c007e00).stop 2022-08-01T22:56:02.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 shutdown_connections 2022-08-01T22:56:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd088053950 0x7fd088055e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0a41405d0 0x7fd0a40aba50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0a40ad750 0x7fd0a4152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 --2- 172.21.15.32:0/1152217678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0a40614b0 0x7fd0a4152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 >> 172.21.15.32:0/1152217678 conn(0x7fd0a4066f70 msgr2=0x7fd0a40b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 shutdown_connections 2022-08-01T22:56:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.113+0000 7fd0a95bd700 1 -- 172.21.15.32:0/1152217678 wait complete. 2022-08-01T22:56:02.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:02.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 Processor -- start 2022-08-01T22:56:02.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 -- start start 2022-08-01T22:56:02.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe540061890 0x7fe5401466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5400aee80 con 0x7fe5400ad750 2022-08-01T22:56:02.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5400b70b0 con 0x7fe5400614b0 2022-08-01T22:56:02.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400af0b0 con 0x7fe540061890 2022-08-01T22:56:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe540061890 0x7fe5401466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5400ad750 0x7fe5401405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56750/0 (socket says 172.21.15.32:56750) 2022-08-01T22:56:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53dd9b700 1 -- 172.21.15.32:0/3276383350 learned_addr learned my addr 172.21.15.32:0/3276383350 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3276383350 >> v1:172.21.15.55:6789/0 conn(0x7fe5400614b0 legacy=0x7fe5400b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3276383350 >> v1:172.21.15.32:6789/0 conn(0x7fe5400ad750 legacy=0x7fe5401405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3276383350 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400af350 con 0x7fe540061890 2022-08-01T22:56:02.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53ed9d700 1 --2- 172.21.15.32:0/3276383350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe540061890 0x7fe5401466b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fe538002f70 tx=0x7fe538006df0).ready entity=mon.0 client_cookie=2c68a32661bccf3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:02.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53d59a700 1 -- 172.21.15.32:0/3276383350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53800d610 con 0x7fe540061890 2022-08-01T22:56:02.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53d59a700 1 -- 172.21.15.32:0/3276383350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538000e90 con 0x7fe540061890 2022-08-01T22:56:02.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53d59a700 1 -- 172.21.15.32:0/3276383350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53800d960 con 0x7fe540061890 2022-08-01T22:56:02.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 -- 172.21.15.32:0/3276383350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe540061890 msgr2=0x7fe5401466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.225+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3276383350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe540061890 0x7fe5401466b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fe538002f70 tx=0x7fe538006df0).stop 2022-08-01T22:56:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- 172.21.15.32:0/3276383350 shutdown_connections 2022-08-01T22:56:02.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3276383350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe540061890 0x7fe5401466b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- 172.21.15.32:0/3276383350 >> 172.21.15.32:0/3276383350 conn(0x7fe540066f70 msgr2=0x7fe540067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:02.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- 172.21.15.32:0/3276383350 shutdown_connections 2022-08-01T22:56:02.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- 172.21.15.32:0/3276383350 wait complete. 2022-08-01T22:56:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 Processor -- start 2022-08-01T22:56:02.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- start start 2022-08-01T22:56:02.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5400614b0 0x7fe5400b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe540061890 0x7fe5400b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 0x7fe54013e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400b2900 con 0x7fe5400ad750 2022-08-01T22:56:02.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400b37a0 con 0x7fe540061890 2022-08-01T22:56:02.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400b2b60 con 0x7fe5400614b0 2022-08-01T22:56:02.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 0x7fe54013e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe540061890 0x7fe5400b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5400614b0 0x7fe5400b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 0x7fe54013e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40986/0 (socket says 172.21.15.32:40986) 2022-08-01T22:56:02.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.229+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3268373612 learned_addr learned my addr 172.21.15.32:0/3268373612 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:02.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5400614b0 msgr2=0x7fe5400b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53ed9d700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5400614b0 0x7fe5400b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe540061890 msgr2=0x7fe5400b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53ed9d700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe540061890 0x7fe5400b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53ed9d700 1 -- 172.21.15.32:0/3268373612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53800b180 con 0x7fe5400ad750 2022-08-01T22:56:02.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53e59c700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5400614b0 0x7fe5400b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:02.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53ed9d700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 0x7fe54013e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fe538002f70 tx=0x7fe538015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53800bed0 con 0x7fe5400ad750 2022-08-01T22:56:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538011b80 con 0x7fe5400ad750 2022-08-01T22:56:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5400b2e00 con 0x7fe5400ad750 2022-08-01T22:56:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe53801e460 con 0x7fe5400ad750 2022-08-01T22:56:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5400b1c30 con 0x7fe5400ad750 2022-08-01T22:56:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5400af5f0 con 0x7fe5400ad750 2022-08-01T22:56:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.233+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe538025030 con 0x7fe5400ad750 2022-08-01T22:56:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.237+0000 7fe52f7fe700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe51c053950 0x7fe51c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.237+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe53806d420 con 0x7fe5400ad750 2022-08-01T22:56:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.237+0000 7fe53e59c700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe51c053950 0x7fe51c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.237+0000 7fe53e59c700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe51c053950 0x7fe51c055e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe53400d000 tx=0x7fe534000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.241+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5380441e0 con 0x7fe5400ad750 2022-08-01T22:56:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.401+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5400574e0 con 0x7fe5400ad750 2022-08-01T22:56:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.401+0000 7fe52f7fe700 1 -- 172.21.15.32:0/3268373612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2486 (secure 0 0 0) 0x7fe538023090 con 0x7fe5400ad750 2022-08-01T22:56:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:02.410 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3750794992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3750794992},{"type":"v1","addr":"172.21.15.32:6835","nonce":3750794992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3123630923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3123630923},{"type":"v1","addr":"172.21.15.55:6835","nonce":3123630923}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:55:50.680148+0000","modified":"2022-08-01T22:55:52.352879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/2632088620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2632088620},{"type":"v1","addr":"172.21.15.32:6833","nonce":2632088620}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T22:56:02.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe51c053950 msgr2=0x7fe51c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe51c053950 0x7fe51c055e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe53400d000 tx=0x7fe534000bc0).stop 2022-08-01T22:56:02.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 msgr2=0x7fe54013e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 0x7fe54013e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fe538002f70 tx=0x7fe538015000).stop 2022-08-01T22:56:02.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 shutdown_connections 2022-08-01T22:56:02.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe51c053950 0x7fe51c055e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5400ad750 0x7fe54013e9f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe540061890 0x7fe5400b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 --2- 172.21.15.32:0/3268373612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5400614b0 0x7fe5400b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 >> 172.21.15.32:0/3268373612 conn(0x7fe540066f70 msgr2=0x7fe54013fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:02.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 shutdown_connections 2022-08-01T22:56:02.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.405+0000 7fe53f59e700 1 -- 172.21.15.32:0/3268373612 wait complete. 2022-08-01T22:56:02.416 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2022-08-01T22:56:02.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:55:50.680148+0000', 'modified': '2022-08-01T22:55:52.352879+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6833/2632088620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6832', 'nonce': 2632088620}, {'type': 'v1', 'addr': '172.21.15.32:6833', 'nonce': 2632088620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:56:02.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:56:02.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T22:56:02.842 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":4264,"want_state":"up:active","state":"up:active","rank_uptime":12.145585279000001,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":18.556837697999999} 2022-08-01T22:56:02.844 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 4264, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 12.145585279, "state": "up:active", "uptime": 18.556837698, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:56:02.845 INFO:teuthology.run_tasks:Running task ceph-fuse... 2022-08-01T22:56:02.866 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2022-08-01T22:56:02.867 INFO:tasks.ceph_fuse:config is None 2022-08-01T22:56:02.868 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None} 2022-08-01T22:56:02.868 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2022-08-01T22:56:02.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 Processor -- start 2022-08-01T22:56:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 -- start start 2022-08-01T22:56:02.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c380614b0 0x7f5c38061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c38140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:02.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380aee80 con 0x7f5c380ad750 2022-08-01T22:56:02.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380b70b0 con 0x7f5c380614b0 2022-08-01T22:56:02.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c3cb6b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380af0b0 con 0x7f5c381405d0 2022-08-01T22:56:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c36ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c380ad750 0x7f5c380b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56766/0 (socket says 172.21.15.32:56766) 2022-08-01T22:56:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c36ffd700 1 -- 172.21.15.32:0/3325770716 learned_addr learned my addr 172.21.15.32:0/3325770716 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c377fe700 1 --2- 172.21.15.32:0/3325770716 >> v2:172.21.15.55:3300/0 conn(0x7f5c380614b0 0x7f5c38061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c37fff700 1 --2- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c38140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c367fc700 1 -- 172.21.15.32:0/3325770716 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283989935 0 0) 0x7f5c380aee80 con 0x7f5c380ad750 2022-08-01T22:56:02.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c367fc700 1 -- 172.21.15.32:0/3325770716 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c24003260 con 0x7f5c380ad750 2022-08-01T22:56:02.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c377fe700 1 -- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 msgr2=0x7f5c38140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c377fe700 1 --2- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c38140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c377fe700 1 -- 172.21.15.32:0/3325770716 >> v1:172.21.15.32:6789/0 conn(0x7f5c380ad750 legacy=0x7f5c380b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c377fe700 1 -- 172.21.15.32:0/3325770716 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380af350 con 0x7f5c380614b0 2022-08-01T22:56:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c377fe700 1 --2- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380614b0 0x7f5c38061890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f5c380aed30 tx=0x7f5c28006df0).ready entity=mon.1 client_cookie=e3ce26c7d77307f9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.973+0000 7f5c367fc700 1 -- 172.21.15.32:0/3325770716 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2800d570 con 0x7f5c380614b0 2022-08-01T22:56:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c367fc700 1 -- 172.21.15.32:0/3325770716 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c28000e90 con 0x7f5c380614b0 2022-08-01T22:56:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c367fc700 1 -- 172.21.15.32:0/3325770716 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2800d8c0 con 0x7f5c380614b0 2022-08-01T22:56:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380614b0 msgr2=0x7f5c38061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380614b0 0x7f5c38061890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f5c380aed30 tx=0x7f5c28006df0).stop 2022-08-01T22:56:02.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/3325770716 shutdown_connections 2022-08-01T22:56:02.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c38140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/3325770716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380614b0 0x7f5c38061890 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/3325770716 >> 172.21.15.32:0/3325770716 conn(0x7f5c38066f70 msgr2=0x7f5c38067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/3325770716 shutdown_connections 2022-08-01T22:56:02.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/3325770716 wait complete. 2022-08-01T22:56:02.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 Processor -- start 2022-08-01T22:56:03.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- start start 2022-08-01T22:56:03.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 0x7f5c380ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380ad750 0x7f5c380abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c380b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38152dd0 con 0x7f5c380614b0 2022-08-01T22:56:03.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380aba10 con 0x7f5c380ad750 2022-08-01T22:56:03.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c3cb6b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380aae80 con 0x7f5c381405d0 2022-08-01T22:56:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 0x7f5c380ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 0x7f5c380ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40996/0 (socket says 172.21.15.32:40996) 2022-08-01T22:56:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.977+0000 7f5c377fe700 1 -- 172.21.15.32:0/687841586 learned_addr learned my addr 172.21.15.32:0/687841586 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c37fff700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c380b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c36ffd700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380ad750 0x7f5c380abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c377fe700 1 -- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 msgr2=0x7f5c380b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c377fe700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c380b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c377fe700 1 -- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380ad750 msgr2=0x7f5c380abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c377fe700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380ad750 0x7f5c380abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c377fe700 1 -- 172.21.15.32:0/687841586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2800b180 con 0x7f5c380614b0 2022-08-01T22:56:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c377fe700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 0x7f5c380ab520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f5c380aed30 tx=0x7f5c28014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c28013de0 con 0x7f5c380614b0 2022-08-01T22:56:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c28002f70 con 0x7f5c380614b0 2022-08-01T22:56:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2801e5f0 con 0x7f5c380614b0 2022-08-01T22:56:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c38153000 con 0x7f5c380614b0 2022-08-01T22:56:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c380ac350 con 0x7f5c380614b0 2022-08-01T22:56:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c380af5f0 con 0x7f5c380614b0 2022-08-01T22:56:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.981+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c28025030 con 0x7f5c380614b0 2022-08-01T22:56:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.985+0000 7f5c34ff9700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c18053b00 0x7f5c18055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.985+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5c2806ceb0 con 0x7f5c380614b0 2022-08-01T22:56:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.985+0000 7f5c36ffd700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c18053b00 0x7f5c18055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.985+0000 7f5c36ffd700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c18053b00 0x7f5c18055fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5c2c00d000 tx=0x7f5c2c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:02.989+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c28043c70 con 0x7f5c380614b0 2022-08-01T22:56:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.112+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c380574e0 con 0x7f5c380614b0 2022-08-01T22:56:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c34ff9700 1 -- 172.21.15.32:0/687841586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5c28023020 con 0x7f5c380614b0 2022-08-01T22:56:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:03.122 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c18053b00 msgr2=0x7f5c18055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c18053b00 0x7f5c18055fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5c2c00d000 tx=0x7f5c2c009040).stop 2022-08-01T22:56:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 msgr2=0x7f5c380ab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 0x7f5c380ab520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f5c380aed30 tx=0x7f5c28014000).stop 2022-08-01T22:56:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 shutdown_connections 2022-08-01T22:56:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c18053b00 0x7f5c18055fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c381405d0 0x7f5c380b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c380ad750 0x7f5c380abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.116+0000 7f5c3cb6b700 1 --2- 172.21.15.32:0/687841586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c380614b0 0x7f5c380ab520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.120+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 >> 172.21.15.32:0/687841586 conn(0x7f5c38066f70 msgr2=0x7f5c380b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.120+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 shutdown_connections 2022-08-01T22:56:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.120+0000 7f5c3cb6b700 1 -- 172.21.15.32:0/687841586 wait complete. 2022-08-01T22:56:03.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:03.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.236+0000 7fcd8484d700 1 Processor -- start 2022-08-01T22:56:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.236+0000 7fcd8484d700 1 -- start start 2022-08-01T22:56:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd800b1460 0x7fcd800b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 0x7fcd801413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd800a7960 con 0x7fcd800b2050 2022-08-01T22:56:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800b18d0 con 0x7fcd800b1460 2022-08-01T22:56:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800b07e0 con 0x7fcd800b2430 2022-08-01T22:56:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 0x7fcd801413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7dd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd800b1460 0x7fcd800b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 0x7fcd801413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:52744/0 (socket says 172.21.15.32:52744) 2022-08-01T22:56:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/2950221903 learned_addr learned my addr 172.21.15.32:0/2950221903 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/2950221903 >> v2:172.21.15.55:3300/0 conn(0x7fcd800b1460 msgr2=0x7fcd800b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/2950221903 >> v2:172.21.15.55:3300/0 conn(0x7fcd800b1460 0x7fcd800b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/2950221903 >> v1:172.21.15.55:6789/0 conn(0x7fcd800b2050 legacy=0x7fcd800a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/2950221903 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800b0a80 con 0x7fcd800b2430 2022-08-01T22:56:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/2950221903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 0x7fcd801413e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fcd70002f70 tx=0x7fcd70006eb0).ready entity=mon.2 client_cookie=b0948e760fda526d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7d59a700 1 -- 172.21.15.32:0/2950221903 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd7000d570 con 0x7fcd800b2430 2022-08-01T22:56:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7d59a700 1 -- 172.21.15.32:0/2950221903 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd70000e90 con 0x7fcd800b2430 2022-08-01T22:56:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd7d59a700 1 -- 172.21.15.32:0/2950221903 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd7000d900 con 0x7fcd800b2430 2022-08-01T22:56:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 -- 172.21.15.32:0/2950221903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 msgr2=0x7fcd801413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.240+0000 7fcd8484d700 1 --2- 172.21.15.32:0/2950221903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 0x7fcd801413e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fcd70002f70 tx=0x7fcd70006eb0).stop 2022-08-01T22:56:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 -- 172.21.15.32:0/2950221903 shutdown_connections 2022-08-01T22:56:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 --2- 172.21.15.32:0/2950221903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2430 0x7fcd801413e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 --2- 172.21.15.32:0/2950221903 >> v2:172.21.15.55:3300/0 conn(0x7fcd800b1460 0x7fcd800b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 -- 172.21.15.32:0/2950221903 >> 172.21.15.32:0/2950221903 conn(0x7fcd8005f840 msgr2=0x7fcd8005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 -- 172.21.15.32:0/2950221903 shutdown_connections 2022-08-01T22:56:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 -- 172.21.15.32:0/2950221903 wait complete. 2022-08-01T22:56:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 Processor -- start 2022-08-01T22:56:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.244+0000 7fcd8484d700 1 -- start start 2022-08-01T22:56:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd800b1460 0x7fcd80149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2050 0x7fcd8014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 0x7fcd80156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800afd30 con 0x7fcd800b2430 2022-08-01T22:56:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800b00b0 con 0x7fcd800b1460 2022-08-01T22:56:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800b0350 con 0x7fcd800b2050 2022-08-01T22:56:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 0x7fcd80156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 0x7fcd80156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41014/0 (socket says 172.21.15.32:41014) 2022-08-01T22:56:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/4262077627 learned_addr learned my addr 172.21.15.32:0/4262077627 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7e59c700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd800b1460 0x7fcd80149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7dd9b700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2050 0x7fcd8014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2050 msgr2=0x7fcd8014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2050 0x7fcd8014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd800b1460 msgr2=0x7fcd80149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd800b1460 0x7fcd80149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/4262077627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7000b1e0 con 0x7fcd800b2430 2022-08-01T22:56:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 0x7fcd80156d30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fcd70002f70 tx=0x7fcd70015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd700068b0 con 0x7fcd800b2430 2022-08-01T22:56:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000de30 con 0x7fcd800b2430 2022-08-01T22:56:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd80149f70 con 0x7fcd800b2430 2022-08-01T22:56:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd70011e20 con 0x7fcd800b2430 2022-08-01T22:56:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.248+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8014aba0 con 0x7fcd800b2430 2022-08-01T22:56:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.252+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd800574e0 con 0x7fcd800b2430 2022-08-01T22:56:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.252+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd70025030 con 0x7fcd800b2430 2022-08-01T22:56:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.252+0000 7fcd777fe700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd64053a70 0x7fcd64055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.252+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcd7006d1f0 con 0x7fcd800b2430 2022-08-01T22:56:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.252+0000 7fcd7e59c700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd64053a70 0x7fcd64055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.252+0000 7fcd7e59c700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd64053a70 0x7fcd64055f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fcd7800d000 tx=0x7fcd78007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.256+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd70043fb0 con 0x7fcd800b2430 2022-08-01T22:56:03.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.384+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcd801491f0 con 0x7fcd800b2430 2022-08-01T22:56:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.384+0000 7fcd777fe700 1 -- 172.21.15.32:0/4262077627 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcd700230b0 con 0x7fcd800b2430 2022-08-01T22:56:03.391 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd64053a70 msgr2=0x7fcd64055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd64053a70 0x7fcd64055f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fcd7800d000 tx=0x7fcd78007f10).stop 2022-08-01T22:56:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 msgr2=0x7fcd80156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 0x7fcd80156d30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fcd70002f70 tx=0x7fcd70015000).stop 2022-08-01T22:56:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 shutdown_connections 2022-08-01T22:56:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd64053a70 0x7fcd64055f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd800b2430 0x7fcd80156d30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd800b2050 0x7fcd8014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 --2- 172.21.15.32:0/4262077627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd800b1460 0x7fcd80149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 >> 172.21.15.32:0/4262077627 conn(0x7fcd8005f840 msgr2=0x7fcd800aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 shutdown_connections 2022-08-01T22:56:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.388+0000 7fcd8484d700 1 -- 172.21.15.32:0/4262077627 wait complete. 2022-08-01T22:56:03.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:03.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 Processor -- start 2022-08-01T22:56:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 -- start start 2022-08-01T22:56:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feee80b08a0 0x7feee8142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 0x7feee813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee8143ab0 con 0x7feee80b3260 2022-08-01T22:56:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee80b0c80 con 0x7feee80b08a0 2022-08-01T22:56:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee8bb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80ae810 con 0x7feee80ad000 2022-08-01T22:56:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feee80b3260 0x7feee81433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56792/0 (socket says 172.21.15.32:56792) 2022-08-01T22:56:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 -- 172.21.15.32:0/2628309918 learned_addr learned my addr 172.21.15.32:0/2628309918 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed8b9700 1 --2- 172.21.15.32:0/2628309918 >> v2:172.21.15.55:3300/0 conn(0x7feee80b08a0 0x7feee8142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee0ba700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 0x7feee813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355576993 0 0) 0x7feee8143ab0 con 0x7feee80b3260 2022-08-01T22:56:03.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feecc005470 con 0x7feee80b3260 2022-08-01T22:56:03.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945733684 0 0) 0x7feecc005470 con 0x7feee80b3260 2022-08-01T22:56:03.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 msgr2=0x7feee813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 0x7feee813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 >> v2:172.21.15.55:3300/0 conn(0x7feee80b08a0 msgr2=0x7feee8142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 --2- 172.21.15.32:0/2628309918 >> v2:172.21.15.55:3300/0 conn(0x7feee80b08a0 0x7feee8142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80aeab0 con 0x7feee80b3260 2022-08-01T22:56:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7feee4001660 con 0x7feee80b3260 2022-08-01T22:56:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 >> v1:172.21.15.32:6789/0 conn(0x7feee80b3260 legacy=0x7feee81433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 0x7feecc005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feecc005e90 0x7feecc006710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feecc006c00 0x7feecc007080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80b0c80 con 0x7feecc005e90 2022-08-01T22:56:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80ae810 con 0x7feecc006c00 2022-08-01T22:56:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc003260 con 0x7feee80ad000 2022-08-01T22:56:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feecc005e90 0x7feecc006710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee0ba700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feecc006c00 0x7feecc007080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed8b9700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 0x7feecc005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 -- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 msgr2=0x7feecc005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feee80ad000 0x7feecc005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 -- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feecc006c00 msgr2=0x7feecc007080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feecc006c00 0x7feecc007080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 -- 172.21.15.32:0/2628309918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc005470 con 0x7feecc005e90 2022-08-01T22:56:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeee0ba700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feecc006c00 0x7feecc007080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeed0b8700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feecc005e90 0x7feecc006710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7feee4001370 tx=0x7feee80a95b0).ready entity=mon.0 client_cookie=8a6bfea1c68698b6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feee40070d0 con 0x7feecc005e90 2022-08-01T22:56:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4007230 con 0x7feecc005e90 2022-08-01T22:56:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.508+0000 7feeec8b7700 1 -- 172.21.15.32:0/2628309918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feee400c8f0 con 0x7feecc005e90 2022-08-01T22:56:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 -- 172.21.15.32:0/2628309918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feecc005e90 msgr2=0x7feecc006710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 --2- 172.21.15.32:0/2628309918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feecc005e90 0x7feecc006710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7feee4001370 tx=0x7feee80a95b0).stop 2022-08-01T22:56:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee0ba700 1 -- 172.21.15.32:0/2628309918 reap_dead start 2022-08-01T22:56:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 -- 172.21.15.32:0/2628309918 shutdown_connections 2022-08-01T22:56:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 -- 172.21.15.32:0/2628309918 >> 172.21.15.32:0/2628309918 conn(0x7feee80612d0 msgr2=0x7feee8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 -- 172.21.15.32:0/2628309918 shutdown_connections 2022-08-01T22:56:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 -- 172.21.15.32:0/2628309918 wait complete. 2022-08-01T22:56:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 Processor -- start 2022-08-01T22:56:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.512+0000 7feeee8bb700 1 -- start start 2022-08-01T22:56:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feecc005e90 0x7feee813f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feee8063ec0 0x7feee81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 0x7feee813daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8067850 con 0x7feee814e110 2022-08-01T22:56:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee809ea40 con 0x7feee8063ec0 2022-08-01T22:56:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee8067a80 con 0x7feecc005e90 2022-08-01T22:56:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 0x7feee813daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeed8b9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feecc005e90 0x7feee813f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 0x7feee813daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41034/0 (socket says 172.21.15.32:41034) 2022-08-01T22:56:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 -- 172.21.15.32:0/4035683549 learned_addr learned my addr 172.21.15.32:0/4035683549 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeed0b8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feee8063ec0 0x7feee81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 -- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feecc005e90 msgr2=0x7feee813f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feecc005e90 0x7feee813f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 -- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feee8063ec0 msgr2=0x7feee81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feee8063ec0 0x7feee81520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 -- 172.21.15.32:0/4035683549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee809eca0 con 0x7feee814e110 2022-08-01T22:56:03.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee0ba700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 0x7feee813daf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7feee000b000 tx=0x7feee0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feee0011e60 con 0x7feee814e110 2022-08-01T22:56:03.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee00145f0 con 0x7feee814e110 2022-08-01T22:56:03.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee8064670 con 0x7feee814e110 2022-08-01T22:56:03.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feee0014950 con 0x7feee814e110 2022-08-01T22:56:03.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.516+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee8063b50 con 0x7feee814e110 2022-08-01T22:56:03.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.520+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feee0012040 con 0x7feee814e110 2022-08-01T22:56:03.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.520+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee8064860 con 0x7feee814e110 2022-08-01T22:56:03.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.520+0000 7feedeffd700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feed0053980 0x7feed0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.520+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feee006c580 con 0x7feee814e110 2022-08-01T22:56:03.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.520+0000 7feeed8b9700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feed0053980 0x7feed0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.520+0000 7feeed8b9700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feed0053980 0x7feed0055e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7feed800b000 tx=0x7feed8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.524+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feee8064860 con 0x7feee814e110 2022-08-01T22:56:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feee8064860 con 0x7feee814e110 2022-08-01T22:56:03.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feedeffd700 1 -- 172.21.15.32:0/4035683549 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7feee8064860 con 0x7feee814e110 2022-08-01T22:56:03.658 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:03.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feed0053980 msgr2=0x7feed0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feed0053980 0x7feed0055e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7feed800b000 tx=0x7feed8008040).stop 2022-08-01T22:56:03.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 msgr2=0x7feee813daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 0x7feee813daf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7feee000b000 tx=0x7feee0008040).stop 2022-08-01T22:56:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 shutdown_connections 2022-08-01T22:56:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feed0053980 0x7feed0055e40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee814e110 0x7feee813daf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feee8063ec0 0x7feee81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 --2- 172.21.15.32:0/4035683549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feecc005e90 0x7feee813f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 >> 172.21.15.32:0/4035683549 conn(0x7feee80612d0 msgr2=0x7feee8062ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 shutdown_connections 2022-08-01T22:56:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.652+0000 7feeee8bb700 1 -- 172.21.15.32:0/4035683549 wait complete. 2022-08-01T22:56:03.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.776+0000 7fca3e2dc700 1 Processor -- start 2022-08-01T22:56:03.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.776+0000 7fca3e2dc700 1 -- start start 2022-08-01T22:56:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca38146ae0 0x7fca381403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fca380b4310 0x7fca381408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca38143cf0 con 0x7fca380b4820 2022-08-01T22:56:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca381476a0 con 0x7fca380b4310 2022-08-01T22:56:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca38143f90 con 0x7fca38146ae0 2022-08-01T22:56:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3c879700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca380b4820 0x7fca380b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56816/0 (socket says 172.21.15.32:56816) 2022-08-01T22:56:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3c879700 1 -- 172.21.15.32:0/328841655 learned_addr learned my addr 172.21.15.32:0/328841655 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:03.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca37fff700 1 --2- 172.21.15.32:0/328841655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca38146ae0 0x7fca381403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca377fe700 1 --2- 172.21.15.32:0/328841655 >> v2:172.21.15.55:3300/0 conn(0x7fca380b4310 0x7fca381408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca36ffd700 1 -- 172.21.15.32:0/328841655 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650728751 0 0) 0x7fca38143cf0 con 0x7fca380b4820 2022-08-01T22:56:03.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca36ffd700 1 -- 172.21.15.32:0/328841655 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca18003260 con 0x7fca380b4820 2022-08-01T22:56:03.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca37fff700 1 -- 172.21.15.32:0/328841655 >> v2:172.21.15.55:3300/0 conn(0x7fca380b4310 msgr2=0x7fca381408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:03.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca37fff700 1 --2- 172.21.15.32:0/328841655 >> v2:172.21.15.55:3300/0 conn(0x7fca380b4310 0x7fca381408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca37fff700 1 -- 172.21.15.32:0/328841655 >> v1:172.21.15.32:6789/0 conn(0x7fca380b4820 legacy=0x7fca380b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca37fff700 1 -- 172.21.15.32:0/328841655 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380b0e20 con 0x7fca38146ae0 2022-08-01T22:56:03.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca37fff700 1 --2- 172.21.15.32:0/328841655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca38146ae0 0x7fca381403c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fca20002f70 tx=0x7fca20006df0).ready entity=mon.2 client_cookie=7466582004af98e4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca36ffd700 1 -- 172.21.15.32:0/328841655 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca2000d4a0 con 0x7fca38146ae0 2022-08-01T22:56:03.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca36ffd700 1 -- 172.21.15.32:0/328841655 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20000e90 con 0x7fca38146ae0 2022-08-01T22:56:03.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca36ffd700 1 -- 172.21.15.32:0/328841655 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca2000d7f0 con 0x7fca38146ae0 2022-08-01T22:56:03.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 -- 172.21.15.32:0/328841655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca38146ae0 msgr2=0x7fca381403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/328841655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca38146ae0 0x7fca381403c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fca20002f70 tx=0x7fca20006df0).stop 2022-08-01T22:56:03.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.780+0000 7fca3e2dc700 1 -- 172.21.15.32:0/328841655 shutdown_connections 2022-08-01T22:56:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/328841655 >> v2:172.21.15.55:3300/0 conn(0x7fca380b4310 0x7fca381408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/328841655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca38146ae0 0x7fca381403c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- 172.21.15.32:0/328841655 >> 172.21.15.32:0/328841655 conn(0x7fca3805c620 msgr2=0x7fca3805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- 172.21.15.32:0/328841655 shutdown_connections 2022-08-01T22:56:03.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- 172.21.15.32:0/328841655 wait complete. 2022-08-01T22:56:03.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 Processor -- start 2022-08-01T22:56:03.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- start start 2022-08-01T22:56:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca380b4310 0x7fca380b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 0x7fca380ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca38146ae0 0x7fca380adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380b0950 con 0x7fca380b4820 2022-08-01T22:56:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca38153480 con 0x7fca38146ae0 2022-08-01T22:56:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3e2dc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca380b96b0 con 0x7fca380b4310 2022-08-01T22:56:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 0x7fca380ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 0x7fca380ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41046/0 (socket says 172.21.15.32:41046) 2022-08-01T22:56:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 -- 172.21.15.32:0/2504421886 learned_addr learned my addr 172.21.15.32:0/2504421886 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca37fff700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca380b4310 0x7fca380b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca3c879700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca38146ae0 0x7fca380adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 -- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca380b4310 msgr2=0x7fca380b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca380b4310 0x7fca380b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 -- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca38146ae0 msgr2=0x7fca380adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca38146ae0 0x7fca380adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.784+0000 7fca377fe700 1 -- 172.21.15.32:0/2504421886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2000b180 con 0x7fca380b4820 2022-08-01T22:56:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca377fe700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 0x7fca380ad780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fca2800c000 tx=0x7fca28006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca28010780 con 0x7fca380b4820 2022-08-01T22:56:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280108e0 con 0x7fca380b4820 2022-08-01T22:56:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca28010c40 con 0x7fca380b4820 2022-08-01T22:56:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca380b8a10 con 0x7fca380b4820 2022-08-01T22:56:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca38152800 con 0x7fca380b4820 2022-08-01T22:56:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca38152b90 con 0x7fca380b4820 2022-08-01T22:56:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fca2801c460 con 0x7fca380b4820 2022-08-01T22:56:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca357fa700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca1c0539e0 0x7fca1c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fca2806c4a0 con 0x7fca380b4820 2022-08-01T22:56:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca37fff700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca1c0539e0 0x7fca1c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:03.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.788+0000 7fca37fff700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca1c0539e0 0x7fca1c055ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fca2001a000 tx=0x7fca20014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:03.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.796+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca28043160 con 0x7fca380b4820 2022-08-01T22:56:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.916+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca380574e0 con 0x7fca380b4820 2022-08-01T22:56:03.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.916+0000 7fca357fa700 1 -- 172.21.15.32:0/2504421886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fca28046780 con 0x7fca380b4820 2022-08-01T22:56:03.923 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:03.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca1c0539e0 msgr2=0x7fca1c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca1c0539e0 0x7fca1c055ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fca2001a000 tx=0x7fca20014000).stop 2022-08-01T22:56:03.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 msgr2=0x7fca380ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:03.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 0x7fca380ad780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fca2800c000 tx=0x7fca28006df0).stop 2022-08-01T22:56:03.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 shutdown_connections 2022-08-01T22:56:03.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca1c0539e0 0x7fca1c055ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca38146ae0 0x7fca380adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca380b4820 0x7fca380ad780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 --2- 172.21.15.32:0/2504421886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca380b4310 0x7fca380b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:03.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 >> 172.21.15.32:0/2504421886 conn(0x7fca3805c620 msgr2=0x7fca381434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:03.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 shutdown_connections 2022-08-01T22:56:03.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:03.920+0000 7fca3e2dc700 1 -- 172.21.15.32:0/2504421886 wait complete. 2022-08-01T22:56:03.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 Processor -- start 2022-08-01T22:56:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 -- start start 2022-08-01T22:56:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79440b2050 0x7f79440b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f79441413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79440b05b0 con 0x7f79440b0140 2022-08-01T22:56:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79440a7960 con 0x7f79440b2050 2022-08-01T22:56:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7949a7c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79440b07e0 con 0x7f79440b2430 2022-08-01T22:56:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f79437fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79440b2050 0x7f79440b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f79437fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79440b2050 0x7f79440b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41054/0 (socket says 172.21.15.32:41054) 2022-08-01T22:56:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f7943fff700 1 --2- 172.21.15.32:0/2216822887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f79441413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.040+0000 7f79437fe700 1 -- 172.21.15.32:0/2216822887 learned_addr learned my addr 172.21.15.32:0/2216822887 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79437fe700 1 -- 172.21.15.32:0/2216822887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 msgr2=0x7f79441413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79437fe700 1 --2- 172.21.15.32:0/2216822887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f79441413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79437fe700 1 -- 172.21.15.32:0/2216822887 >> v1:172.21.15.55:6789/0 conn(0x7f79440b0140 legacy=0x7f79440a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79427fc700 1 -- 172.21.15.32:0/2216822887 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529035071 0 0) 0x7f79440b05b0 con 0x7f79440b0140 2022-08-01T22:56:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79437fe700 1 -- 172.21.15.32:0/2216822887 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440b0a80 con 0x7f79440b2050 2022-08-01T22:56:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79437fe700 1 --2- 172.21.15.32:0/2216822887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f79440b1460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f7938002f70 tx=0x7f7938006eb0).ready entity=mon.0 client_cookie=265fa54f893e2224 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79427fc700 1 -- 172.21.15.32:0/2216822887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f793800d570 con 0x7f79440b2050 2022-08-01T22:56:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79427fc700 1 -- 172.21.15.32:0/2216822887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938000e90 con 0x7f79440b2050 2022-08-01T22:56:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f79427fc700 1 -- 172.21.15.32:0/2216822887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f793800d8c0 con 0x7f79440b2050 2022-08-01T22:56:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 -- 172.21.15.32:0/2216822887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 msgr2=0x7f79440b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 --2- 172.21.15.32:0/2216822887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f79440b1460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f7938002f70 tx=0x7f7938006eb0).stop 2022-08-01T22:56:04.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 -- 172.21.15.32:0/2216822887 shutdown_connections 2022-08-01T22:56:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 --2- 172.21.15.32:0/2216822887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f79441413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 --2- 172.21.15.32:0/2216822887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f79440b1460 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 -- 172.21.15.32:0/2216822887 >> 172.21.15.32:0/2216822887 conn(0x7f794405f840 msgr2=0x7f794405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:04.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 -- 172.21.15.32:0/2216822887 shutdown_connections 2022-08-01T22:56:04.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 -- 172.21.15.32:0/2216822887 wait complete. 2022-08-01T22:56:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 Processor -- start 2022-08-01T22:56:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.044+0000 7f7949a7c700 1 -- start start 2022-08-01T22:56:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79440b0140 0x7f794414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f7944156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f7944157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441446e0 con 0x7f79440b2050 2022-08-01T22:56:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79440a9c70 con 0x7f79440b0140 2022-08-01T22:56:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794415b030 con 0x7f79440b2430 2022-08-01T22:56:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f7944156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f7944156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41060/0 (socket says 172.21.15.32:41060) 2022-08-01T22:56:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 -- 172.21.15.32:0/579279628 learned_addr learned my addr 172.21.15.32:0/579279628 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f79437fe700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79440b0140 0x7f794414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7943fff700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f7944157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 -- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 msgr2=0x7f7944157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f7944157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 -- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79440b0140 msgr2=0x7f794414a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79440b0140 0x7f794414a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 -- 172.21.15.32:0/579279628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793800b1e0 con 0x7f79440b2050 2022-08-01T22:56:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f79437fe700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79440b0140 0x7f794414a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7942ffd700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f7944156b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f79440aa3a0 tx=0x7f7934007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f793400ce10 con 0x7f79440b2050 2022-08-01T22:56:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793400f470 con 0x7f79440b2050 2022-08-01T22:56:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944144fb0 con 0x7f79440b2050 2022-08-01T22:56:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f793400f7d0 con 0x7f79440b2050 2022-08-01T22:56:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79440ac060 con 0x7f79440b2050 2022-08-01T22:56:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.048+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79440b0d20 con 0x7f79440b2050 2022-08-01T22:56:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.052+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7934019030 con 0x7f79440b2050 2022-08-01T22:56:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.052+0000 7f7940ff9700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79240539e0 0x7f7924055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.052+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f793406b560 con 0x7f79440b2050 2022-08-01T22:56:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.052+0000 7f79437fe700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79240539e0 0x7f7924055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.052+0000 7f79437fe700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79240539e0 0x7f7924055ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f793801a000 tx=0x7f7938015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.056+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79340421e0 con 0x7f79440b2050 2022-08-01T22:56:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79440574e0 con 0x7f79440b2050 2022-08-01T22:56:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7940ff9700 1 -- 172.21.15.32:0/579279628 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7934045800 con 0x7f79440b2050 2022-08-01T22:56:04.193 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:04.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79240539e0 msgr2=0x7f7924055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79240539e0 0x7f7924055ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f793801a000 tx=0x7f7938015000).stop 2022-08-01T22:56:04.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 msgr2=0x7f7944156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f7944156b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f79440aa3a0 tx=0x7f7934007ac0).stop 2022-08-01T22:56:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 shutdown_connections 2022-08-01T22:56:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79240539e0 0x7f7924055ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79440b2430 0x7f7944157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79440b2050 0x7f7944156b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 --2- 172.21.15.32:0/579279628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79440b0140 0x7f794414a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 >> 172.21.15.32:0/579279628 conn(0x7f794405f840 msgr2=0x7f7944145210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 shutdown_connections 2022-08-01T22:56:04.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.188+0000 7f7949a7c700 1 -- 172.21.15.32:0/579279628 wait complete. 2022-08-01T22:56:04.206 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:56:04.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 Processor -- start 2022-08-01T22:56:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 -- start start 2022-08-01T22:56:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff26c0aa040 0x7ff26c0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff26c144610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0aa420 con 0x7ff26c0b32c0 2022-08-01T22:56:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0ae490 con 0x7ff26c0aa040 2022-08-01T22:56:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff270a2a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c148300 con 0x7ff26c0af200 2022-08-01T22:56:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff269d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff26c0b32c0 0x7ff26c14b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56840/0 (socket says 172.21.15.32:56840) 2022-08-01T22:56:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff269d9b700 1 -- 172.21.15.32:0/3378523402 learned_addr learned my addr 172.21.15.32:0/3378523402 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff26c144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26a59c700 1 --2- 172.21.15.32:0/3378523402 >> v2:172.21.15.55:3300/0 conn(0x7ff26c0aa040 0x7ff26c0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881732777 0 0) 0x7ff26c0aa420 con 0x7ff26c0b32c0 2022-08-01T22:56:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff250003260 con 0x7ff26c0b32c0 2022-08-01T22:56:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1962760107 0 0) 0x7ff250003260 con 0x7ff26c0b32c0 2022-08-01T22:56:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 msgr2=0x7ff26c144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff26c144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 >> v2:172.21.15.55:3300/0 conn(0x7ff26c0aa040 msgr2=0x7ff26c0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:04.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 --2- 172.21.15.32:0/3378523402 >> v2:172.21.15.55:3300/0 conn(0x7ff26c0aa040 0x7ff26c0b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c1485a0 con 0x7ff26c0b32c0 2022-08-01T22:56:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7ff25c001660 con 0x7ff26c0b32c0 2022-08-01T22:56:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 >> v1:172.21.15.32:6789/0 conn(0x7ff26c0b32c0 legacy=0x7ff26c14b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff250003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff250003c80 0x7ff250004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2500049f0 0x7ff250004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0ae490 con 0x7ff2500049f0 2022-08-01T22:56:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c148300 con 0x7ff250003c80 2022-08-01T22:56:04.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25000b750 con 0x7ff26c0af200 2022-08-01T22:56:04.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2500049f0 0x7ff250004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26a59c700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff250003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff269d9b700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff250003c80 0x7ff250004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 -- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 msgr2=0x7ff250003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff250003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 -- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff250003c80 msgr2=0x7ff250004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff250003c80 0x7ff250004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 -- 172.21.15.32:0/3378523402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250003260 con 0x7ff2500049f0 2022-08-01T22:56:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26a59c700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0af200 0x7ff250003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff269d9b700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff250003c80 0x7ff250004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2500049f0 0x7ff250004e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7ff2500054d0 tx=0x7ff2640091d0).ready entity=mon.0 client_cookie=b5e81faedd527d5e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26400e7e0 con 0x7ff2500049f0 2022-08-01T22:56:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.308+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26400e940 con 0x7ff2500049f0 2022-08-01T22:56:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff26959a700 1 -- 172.21.15.32:0/3378523402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26400eca0 con 0x7ff2500049f0 2022-08-01T22:56:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- 172.21.15.32:0/3378523402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2500049f0 msgr2=0x7ff250004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 --2- 172.21.15.32:0/3378523402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2500049f0 0x7ff250004e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7ff2500054d0 tx=0x7ff2640091d0).stop 2022-08-01T22:56:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff26ad9d700 1 -- 172.21.15.32:0/3378523402 reap_dead start 2022-08-01T22:56:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- 172.21.15.32:0/3378523402 shutdown_connections 2022-08-01T22:56:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- 172.21.15.32:0/3378523402 >> 172.21.15.32:0/3378523402 conn(0x7ff26c060950 msgr2=0x7ff26c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- 172.21.15.32:0/3378523402 shutdown_connections 2022-08-01T22:56:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- 172.21.15.32:0/3378523402 wait complete. 2022-08-01T22:56:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 Processor -- start 2022-08-01T22:56:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- start start 2022-08-01T22:56:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0b2610 0x7ff26c146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff26c1410f0 0x7ff26c146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 0x7ff26c148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c13fef0 con 0x7ff26c146c20 2022-08-01T22:56:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c14a960 con 0x7ff26c1410f0 2022-08-01T22:56:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff270a2a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c1458a0 con 0x7ff26c0b2610 2022-08-01T22:56:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff26ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 0x7ff26c148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff269d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff26c1410f0 0x7ff26c146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff26ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 0x7ff26c148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41082/0 (socket says 172.21.15.32:41082) 2022-08-01T22:56:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.312+0000 7ff269d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff26c1410f0 0x7ff26c146730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:50866/0 (socket says 172.21.15.32:50866) 2022-08-01T22:56:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 -- 172.21.15.32:0/859017572 learned_addr learned my addr 172.21.15.32:0/859017572 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26a59c700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0b2610 0x7ff26c146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 -- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0b2610 msgr2=0x7ff26c146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0b2610 0x7ff26c146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 -- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff26c1410f0 msgr2=0x7ff26c146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff26c1410f0 0x7ff26c146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 -- 172.21.15.32:0/859017572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c13efe0 con 0x7ff26c146c20 2022-08-01T22:56:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff26ad9d700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 0x7ff26c148c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ff25c001110 tx=0x7ff26401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26400e7e0 con 0x7ff26c146c20 2022-08-01T22:56:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26400eed0 con 0x7ff26c146c20 2022-08-01T22:56:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c1452c0 con 0x7ff26c146c20 2022-08-01T22:56:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff264009600 con 0x7ff26c146c20 2022-08-01T22:56:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c0b2b90 con 0x7ff26c146c20 2022-08-01T22:56:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c14afb0 con 0x7ff26c146c20 2022-08-01T22:56:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.316+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff26402d070 con 0x7ff26c146c20 2022-08-01T22:56:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.320+0000 7ff25b7fe700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff248053b00 0x7ff248055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.320+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff26406da30 con 0x7ff26c146c20 2022-08-01T22:56:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.320+0000 7ff26a59c700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff248053b00 0x7ff248055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.320+0000 7ff26a59c700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff248053b00 0x7ff248055fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff26000b000 tx=0x7ff260008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.324+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2640447f0 con 0x7ff26c146c20 2022-08-01T22:56:04.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.492+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff26c0574e0 con 0x7ff26c146c20 2022-08-01T22:56:04.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.492+0000 7ff25b7fe700 1 -- 172.21.15.32:0/859017572 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2486 (secure 0 0 0) 0x7ff264016020 con 0x7ff26c146c20 2022-08-01T22:56:04.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:04.500 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3750794992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3750794992},{"type":"v1","addr":"172.21.15.32:6835","nonce":3750794992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3123630923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3123630923},{"type":"v1","addr":"172.21.15.55:6835","nonce":3123630923}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:55:50.680148+0000","modified":"2022-08-01T22:55:52.352879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/2632088620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2632088620},{"type":"v1","addr":"172.21.15.32:6833","nonce":2632088620}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T22:56:04.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff248053b00 msgr2=0x7ff248055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff248053b00 0x7ff248055fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff26000b000 tx=0x7ff260008040).stop 2022-08-01T22:56:04.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 msgr2=0x7ff26c148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:04.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 0x7ff26c148c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ff25c001110 tx=0x7ff26401a000).stop 2022-08-01T22:56:04.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 shutdown_connections 2022-08-01T22:56:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff248053b00 0x7ff248055fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff26c146c20 0x7ff26c148c20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff26c1410f0 0x7ff26c146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 --2- 172.21.15.32:0/859017572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff26c0b2610 0x7ff26c146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 >> 172.21.15.32:0/859017572 conn(0x7ff26c060950 msgr2=0x7ff26c13e330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 shutdown_connections 2022-08-01T22:56:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:04.496+0000 7ff270a2a700 1 -- 172.21.15.32:0/859017572 wait complete. 2022-08-01T22:56:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2022-08-01T22:56:04.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:55:50.680148+0000', 'modified': '2022-08-01T22:55:52.352879+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6833/2632088620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6832', 'nonce': 2632088620}, {'type': 'v1', 'addr': '172.21.15.32:6833', 'nonce': 2632088620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:56:04.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:56:04.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T22:56:04.933 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":4264,"want_state":"up:active","state":"up:active","rank_uptime":14.237328485000001,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":20.648570803999998} 2022-08-01T22:56:04.934 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 4264, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 14.237328485, "state": "up:active", "uptime": 20.648570804, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:56:04.935 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:56:04.935 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T22:56:04.936 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T22:56:04.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:04.937 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:04.950 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:56:04.969 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:04.970 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:05.032 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:05.033 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:05.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:05.034 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:56:05.043 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:56:05.044 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:05.093 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:56:05.116 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:05.117 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:05.173 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:05.175 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:05.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:05.178 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:56:05.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T22:56:05.185+0000 7f030015f080 -1 init, newargv = 0x556258c3f750 newargc=9ceph-fuse[22343 2022-08-01T22:56:05.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:]: starting ceph client 2022-08-01T22:56:05.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22343]: starting fuse 2022-08-01T22:56:06.289 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:56:06.311 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:06.311 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:06.372 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:06.374 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:06.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:06.382 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:56:06.390 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T22:56:06.392 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:56:06.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:06.393 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T22:56:06.393 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T22:56:06.393 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T22:56:06.394 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:56:06.394 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T22:56:06.395 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:06.395 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T22:56:06.396 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T22:56:06.396 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T22:56:06.396 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:06.397 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T22:56:06.397 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T22:56:06.397 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T22:56:06.398 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T22:56:06.398 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:06.398 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T22:56:06.399 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:56:06.399 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:56:06.399 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T22:56:06.400 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:56:06.400 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:06.400 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T22:56:06.400 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:56:06.482 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22343.asok 2022-08-01T22:56:06.490 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22343.asok 2022-08-01T22:56:06.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:06.491 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22343.asok status 2022-08-01T22:56:06.586 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T22:56:06.587 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T22:56:06.587 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:56:06.588 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:56:06.588 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T22:56:06.588 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T22:56:06.589 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T22:56:06.592 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22343", 2022-08-01T22:56:06.592 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T22:56:06.593 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:06.593 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T22:56:06.597 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T22:56:06.597 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 4611, 2022-08-01T22:56:06.597 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T22:56:06.598 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T22:56:06.600 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T22:56:06.600 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 4611 2022-08-01T22:56:06.600 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:06.601 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:56:06.601 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:56:06.601 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:56:06.601 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2845768400 2022-08-01T22:56:06.602 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T22:56:06.602 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:06.602 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:56:06.602 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:56:06.603 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:56:06.603 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2845768400 2022-08-01T22:56:06.604 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:06.605 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.4611 172.21.15.130:0/2845768400", 2022-08-01T22:56:06.605 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/2845768400", 2022-08-01T22:56:06.605 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T22:56:06.606 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 6, 2022-08-01T22:56:06.606 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 24, 2022-08-01T22:56:06.606 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:56:06.606 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T22:56:06.607 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T22:56:06.607 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T22:56:06.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:06.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f5625d08700 1 Processor -- start 2022-08-01T22:56:06.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f5625d08700 1 -- start start 2022-08-01T22:56:06.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f5625d08700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b4550 0x7f56200b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:06.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f5625d08700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5620144240 con 0x7f56200b6710 2022-08-01T22:56:06.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f5625d08700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5620143d00 con 0x7f56200b6af0 2022-08-01T22:56:06.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f5625d08700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5620143fa0 con 0x7f56200b4550 2022-08-01T22:56:06.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56200b6710 0x7f5620061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56860/0 (socket says 172.21.15.32:56860) 2022-08-01T22:56:06.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561f7fe700 1 -- 172.21.15.32:0/4001119972 learned_addr learned my addr 172.21.15.32:0/4001119972 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:06.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561effd700 1 --2- 172.21.15.32:0/4001119972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b4550 0x7f56200b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561e7fc700 1 -- 172.21.15.32:0/4001119972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163117531 0 0) 0x7f5620144240 con 0x7f56200b6710 2022-08-01T22:56:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561e7fc700 1 -- 172.21.15.32:0/4001119972 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5604003260 con 0x7f56200b6710 2022-08-01T22:56:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561effd700 1 -- 172.21.15.32:0/4001119972 >> v1:172.21.15.55:6789/0 conn(0x7f56200b6af0 legacy=0x7f562012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561effd700 1 -- 172.21.15.32:0/4001119972 >> v1:172.21.15.32:6789/0 conn(0x7f56200b6710 legacy=0x7f5620061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561effd700 1 -- 172.21.15.32:0/4001119972 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620061780 con 0x7f56200b4550 2022-08-01T22:56:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561effd700 1 --2- 172.21.15.32:0/4001119972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b4550 0x7f56200b5d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f56180032f0 tx=0x7f56180091d0).ready entity=mon.2 client_cookie=66a32d522b5dc13a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561e7fc700 1 -- 172.21.15.32:0/4001119972 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561800b6c0 con 0x7f56200b4550 2022-08-01T22:56:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561e7fc700 1 -- 172.21.15.32:0/4001119972 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561800b820 con 0x7f56200b4550 2022-08-01T22:56:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.719+0000 7f561e7fc700 1 -- 172.21.15.32:0/4001119972 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561800b990 con 0x7f56200b4550 2022-08-01T22:56:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 -- 172.21.15.32:0/4001119972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b4550 msgr2=0x7f56200b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 --2- 172.21.15.32:0/4001119972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b4550 0x7f56200b5d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f56180032f0 tx=0x7f56180091d0).stop 2022-08-01T22:56:06.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 -- 172.21.15.32:0/4001119972 shutdown_connections 2022-08-01T22:56:06.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 --2- 172.21.15.32:0/4001119972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b4550 0x7f56200b5d80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 -- 172.21.15.32:0/4001119972 >> 172.21.15.32:0/4001119972 conn(0x7f562005c620 msgr2=0x7f562005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:06.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 -- 172.21.15.32:0/4001119972 shutdown_connections 2022-08-01T22:56:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 -- 172.21.15.32:0/4001119972 wait complete. 2022-08-01T22:56:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 Processor -- start 2022-08-01T22:56:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.723+0000 7f5625d08700 1 -- start start 2022-08-01T22:56:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 0x7f5620126800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56200b6710 0x7f5620152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b6af0 0x7f5620152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5620149940 con 0x7f56200b4550 2022-08-01T22:56:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5620148a20 con 0x7f56200b6710 2022-08-01T22:56:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5620148cc0 con 0x7f56200b6af0 2022-08-01T22:56:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 0x7f5620126800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 0x7f5620126800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41092/0 (socket says 172.21.15.32:41092) 2022-08-01T22:56:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 -- 172.21.15.32:0/2780572715 learned_addr learned my addr 172.21.15.32:0/2780572715 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561effd700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56200b6710 0x7f5620152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561ffff700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b6af0 0x7f5620152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 -- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b6af0 msgr2=0x7f5620152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b6af0 0x7f5620152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 -- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56200b6710 msgr2=0x7f5620152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56200b6710 0x7f5620152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 -- 172.21.15.32:0/2780572715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618009470 con 0x7f56200b4550 2022-08-01T22:56:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561f7fe700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 0x7f5620126800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5610001110 tx=0x7f5610007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561ffff700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b6af0 0x7f5620152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56100070d0 con 0x7f56200b4550 2022-08-01T22:56:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610007230 con 0x7f56200b4550 2022-08-01T22:56:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5620148f60 con 0x7f56200b4550 2022-08-01T22:56:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5610005610 con 0x7f56200b4550 2022-08-01T22:56:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.727+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5620126cf0 con 0x7f56200b4550 2022-08-01T22:56:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.731+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5620152e90 con 0x7f56200b4550 2022-08-01T22:56:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.731+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5610010070 con 0x7f56200b4550 2022-08-01T22:56:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.731+0000 7f561cff9700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053950 0x7f55f8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.731+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f561006b730 con 0x7f56200b4550 2022-08-01T22:56:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.731+0000 7f561effd700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053950 0x7f55f8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.731+0000 7f561effd700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053950 0x7f55f8055e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f561801b000 tx=0x7f5618016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.739+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f56100423e0 con 0x7f56200b4550 2022-08-01T22:56:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.863+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5620061a20 con 0x7f56200b4550 2022-08-01T22:56:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f561cff9700 1 -- 172.21.15.32:0/2780572715 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5610045a00 con 0x7f56200b4550 2022-08-01T22:56:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:06.874 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:06.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053950 msgr2=0x7f55f8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053950 0x7f55f8055e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f561801b000 tx=0x7f5618016000).stop 2022-08-01T22:56:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 msgr2=0x7f5620126800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 0x7f5620126800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5610001110 tx=0x7f5610007aa0).stop 2022-08-01T22:56:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 shutdown_connections 2022-08-01T22:56:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053950 0x7f55f8055e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56200b6af0 0x7f5620152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56200b6710 0x7f5620152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 --2- 172.21.15.32:0/2780572715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56200b4550 0x7f5620126800 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:06.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 >> 172.21.15.32:0/2780572715 conn(0x7f562005c620 msgr2=0x7f5620144450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:06.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 shutdown_connections 2022-08-01T22:56:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:06.867+0000 7f5625d08700 1 -- 172.21.15.32:0/2780572715 wait complete. 2022-08-01T22:56:06.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:06.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.003+0000 7f37bf2bc700 1 Processor -- start 2022-08-01T22:56:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.003+0000 7f37bf2bc700 1 -- start start 2022-08-01T22:56:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f37b8066780 0x7f37b8066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37b8140e40 0x7f37b813e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b805f8e0 con 0x7f37b80b1bc0 2022-08-01T22:56:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b80b1630 con 0x7f37b8140e40 2022-08-01T22:56:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b805fb10 con 0x7f37b8066780 2022-08-01T22:56:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37b8140e40 0x7f37b813e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37b8140e40 0x7f37b813e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41104/0 (socket says 172.21.15.32:41104) 2022-08-01T22:56:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 -- 172.21.15.32:0/1955515099 learned_addr learned my addr 172.21.15.32:0/1955515099 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37be2ba700 1 --2- 172.21.15.32:0/1955515099 >> v2:172.21.15.55:3300/0 conn(0x7f37b8066780 0x7f37b8066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 -- 172.21.15.32:0/1955515099 >> v2:172.21.15.55:3300/0 conn(0x7f37b8066780 msgr2=0x7f37b8066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 --2- 172.21.15.32:0/1955515099 >> v2:172.21.15.55:3300/0 conn(0x7f37b8066780 0x7f37b8066b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 -- 172.21.15.32:0/1955515099 >> v1:172.21.15.55:6789/0 conn(0x7f37b80b1bc0 legacy=0x7f37b805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 -- 172.21.15.32:0/1955515099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b80b18d0 con 0x7f37b8140e40 2022-08-01T22:56:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37be2ba700 1 --2- 172.21.15.32:0/1955515099 >> v2:172.21.15.55:3300/0 conn(0x7f37b8066780 0x7f37b8066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37beabb700 1 --2- 172.21.15.32:0/1955515099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b813e2b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f37b00032f0 tx=0x7f37b0009290).ready entity=mon.0 client_cookie=6b1a1d6cd29cbcf7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bd2b8700 1 -- 172.21.15.32:0/1955515099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37b000b870 con 0x7f37b8140e40 2022-08-01T22:56:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bd2b8700 1 -- 172.21.15.32:0/1955515099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b000b9d0 con 0x7f37b8140e40 2022-08-01T22:56:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bd2b8700 1 -- 172.21.15.32:0/1955515099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37b000bb40 con 0x7f37b8140e40 2022-08-01T22:56:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1955515099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 msgr2=0x7f37b813e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1955515099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b813e2b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f37b00032f0 tx=0x7f37b0009290).stop 2022-08-01T22:56:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1955515099 shutdown_connections 2022-08-01T22:56:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1955515099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b813e2b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1955515099 >> v2:172.21.15.55:3300/0 conn(0x7f37b8066780 0x7f37b8066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1955515099 >> 172.21.15.32:0/1955515099 conn(0x7f37b8063870 msgr2=0x7f37b8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1955515099 shutdown_connections 2022-08-01T22:56:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.007+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1955515099 wait complete. 2022-08-01T22:56:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 Processor -- start 2022-08-01T22:56:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 -- start start 2022-08-01T22:56:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37b8066780 0x7f37b80aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37b80b1bc0 0x7f37b8142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b80ab0b0 con 0x7f37b8140e40 2022-08-01T22:56:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b8143b30 con 0x7f37b8066780 2022-08-01T22:56:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b80ab2e0 con 0x7f37b80b1bc0 2022-08-01T22:56:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b814ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41110/0 (socket says 172.21.15.32:41110) 2022-08-01T22:56:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 -- 172.21.15.32:0/1892135136 learned_addr learned my addr 172.21.15.32:0/1892135136 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37be2ba700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37b8066780 0x7f37b80aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bdab9700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37b80b1bc0 0x7f37b8142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 -- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37b80b1bc0 msgr2=0x7f37b8142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37b80b1bc0 0x7f37b8142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 -- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37b8066780 msgr2=0x7f37b80aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37b8066780 0x7f37b80aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 -- 172.21.15.32:0/1892135136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0009530 con 0x7f37b8140e40 2022-08-01T22:56:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37beabb700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b814ff60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f37b00032f0 tx=0x7f37b0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37b000b870 con 0x7f37b8140e40 2022-08-01T22:56:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b000bdb0 con 0x7f37b8140e40 2022-08-01T22:56:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37b001f610 con 0x7f37b8140e40 2022-08-01T22:56:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b8143d60 con 0x7f37b8140e40 2022-08-01T22:56:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.011+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b80aeca0 con 0x7f37b8140e40 2022-08-01T22:56:07.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.015+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b80b12b0 con 0x7f37b8140e40 2022-08-01T22:56:07.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.015+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f37b000f070 con 0x7f37b8140e40 2022-08-01T22:56:07.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.015+0000 7f37a77fe700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3798053b00 0x7f3798055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.015+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f37b006cb50 con 0x7f37b8140e40 2022-08-01T22:56:07.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.015+0000 7f37be2ba700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3798053b00 0x7f3798055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.015+0000 7f37be2ba700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3798053b00 0x7f3798055fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f37b400c000 tx=0x7f37b4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.023+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f37b0043800 con 0x7f37b8140e40 2022-08-01T22:56:07.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.147+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f37b80574e0 con 0x7f37b8140e40 2022-08-01T22:56:07.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37a77fe700 1 -- 172.21.15.32:0/1892135136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f37b0015030 con 0x7f37b8140e40 2022-08-01T22:56:07.159 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:07.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3798053b00 msgr2=0x7f3798055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3798053b00 0x7f3798055fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f37b400c000 tx=0x7f37b4008040).stop 2022-08-01T22:56:07.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 msgr2=0x7f37b814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b814ff60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f37b00032f0 tx=0x7f37b0016000).stop 2022-08-01T22:56:07.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 shutdown_connections 2022-08-01T22:56:07.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3798053b00 0x7f3798055fc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37b8140e40 0x7f37b814ff60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37b80b1bc0 0x7f37b8142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 --2- 172.21.15.32:0/1892135136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37b8066780 0x7f37b80aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 >> 172.21.15.32:0/1892135136 conn(0x7f37b8063870 msgr2=0x7f37b8140490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 shutdown_connections 2022-08-01T22:56:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.151+0000 7f37bf2bc700 1 -- 172.21.15.32:0/1892135136 wait complete. 2022-08-01T22:56:07.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:07.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.279+0000 7f5e6b2f2700 1 Processor -- start 2022-08-01T22:56:07.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.279+0000 7f5e6b2f2700 1 -- start start 2022-08-01T22:56:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b2710 0x7f5e6414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 0x7f5e6414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5e640afd60 0x7f5e640a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e640a7250 con 0x7f5e640b01a0 2022-08-01T22:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e640a7480 con 0x7f5e640afd60 2022-08-01T22:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640a7720 con 0x7f5e640b2710 2022-08-01T22:56:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e69aef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 0x7f5e6414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e69aef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 0x7f5e6414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41114/0 (socket says 172.21.15.32:41114) 2022-08-01T22:56:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e69aef700 1 -- 172.21.15.32:0/1690383379 learned_addr learned my addr 172.21.15.32:0/1690383379 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1690383379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b2710 0x7f5e6414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6aaf1700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.55:3300/0 conn(0x7f5e640afd60 0x7f5e640a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1690383379 >> v2:172.21.15.55:3300/0 conn(0x7f5e640afd60 msgr2=0x7f5e640a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.55:3300/0 conn(0x7f5e640afd60 0x7f5e640a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1690383379 >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 msgr2=0x7f5e6414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 0x7f5e6414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1690383379 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640a79c0 con 0x7f5e640b2710 2022-08-01T22:56:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e69aef700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 0x7f5e6414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6aaf1700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.55:3300/0 conn(0x7f5e640afd60 0x7f5e640a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1690383379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b2710 0x7f5e6414afd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5e54002f70 tx=0x7f5e54006df0).ready entity=mon.2 client_cookie=fe27c3fa50cbacbf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e692ee700 1 -- 172.21.15.32:0/1690383379 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5400d4a0 con 0x7f5e640b2710 2022-08-01T22:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e692ee700 1 -- 172.21.15.32:0/1690383379 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54000e90 con 0x7f5e640b2710 2022-08-01T22:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e692ee700 1 -- 172.21.15.32:0/1690383379 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5400d7f0 con 0x7f5e640b2710 2022-08-01T22:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1690383379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b2710 msgr2=0x7f5e6414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1690383379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b2710 0x7f5e6414afd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5e54002f70 tx=0x7f5e54006df0).stop 2022-08-01T22:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1690383379 shutdown_connections 2022-08-01T22:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.32:3300/0 conn(0x7f5e640b01a0 0x7f5e6414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1690383379 >> v2:172.21.15.55:3300/0 conn(0x7f5e640afd60 0x7f5e640a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1690383379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b2710 0x7f5e6414afd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1690383379 >> 172.21.15.32:0/1690383379 conn(0x7f5e64060310 msgr2=0x7f5e640b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1690383379 shutdown_connections 2022-08-01T22:56:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.283+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1690383379 wait complete. 2022-08-01T22:56:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 Processor -- start 2022-08-01T22:56:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 -- start start 2022-08-01T22:56:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 0x7f5e64148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b01a0 0x7f5e640abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e640b2710 0x7f5e640ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64156450 con 0x7f5e640afd60 2022-08-01T22:56:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64067e60 con 0x7f5e640b2710 2022-08-01T22:56:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640b4db0 con 0x7f5e640b01a0 2022-08-01T22:56:07.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 0x7f5e64148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 0x7f5e64148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41118/0 (socket says 172.21.15.32:41118) 2022-08-01T22:56:07.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1825264651 learned_addr learned my addr 172.21.15.32:0/1825264651 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e69aef700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b01a0 0x7f5e640abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6aaf1700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e640b2710 0x7f5e640ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b01a0 msgr2=0x7f5e640abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b01a0 0x7f5e640abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e640b2710 msgr2=0x7f5e640ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e640b2710 0x7f5e640ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 -- 172.21.15.32:0/1825264651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c009470 con 0x7f5e640afd60 2022-08-01T22:56:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6a2f0700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 0x7f5e64148e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f5e54002f70 tx=0x7f5e54014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e54013d50 con 0x7f5e640afd60 2022-08-01T22:56:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5401e400 con 0x7f5e640afd60 2022-08-01T22:56:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6000b1e0 con 0x7f5e640afd60 2022-08-01T22:56:07.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5401e760 con 0x7f5e640afd60 2022-08-01T22:56:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.287+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5400b180 con 0x7f5e640afd60 2022-08-01T22:56:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.291+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e641565a0 con 0x7f5e640afd60 2022-08-01T22:56:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.291+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e54025030 con 0x7f5e640afd60 2022-08-01T22:56:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.291+0000 7f5e537fe700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e44053a70 0x7f5e44055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.291+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e5406cdb0 con 0x7f5e640afd60 2022-08-01T22:56:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.291+0000 7f5e69aef700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e44053a70 0x7f5e44055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.291+0000 7f5e69aef700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e44053a70 0x7f5e44055f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5e6000b000 tx=0x7f5e60006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.299+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e641565a0 con 0x7f5e640afd60 2022-08-01T22:56:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.423+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e641565a0 con 0x7f5e640afd60 2022-08-01T22:56:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.423+0000 7f5e537fe700 1 -- 172.21.15.32:0/1825264651 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5e641565a0 con 0x7f5e640afd60 2022-08-01T22:56:07.431 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e44053a70 msgr2=0x7f5e44055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e44053a70 0x7f5e44055f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5e6000b000 tx=0x7f5e60006eb0).stop 2022-08-01T22:56:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 msgr2=0x7f5e64148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 0x7f5e64148e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f5e54002f70 tx=0x7f5e54014000).stop 2022-08-01T22:56:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 shutdown_connections 2022-08-01T22:56:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e44053a70 0x7f5e44055f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e640b2710 0x7f5e640ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e640b01a0 0x7f5e640abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 --2- 172.21.15.32:0/1825264651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640afd60 0x7f5e64148e50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 >> 172.21.15.32:0/1825264651 conn(0x7f5e64060310 msgr2=0x7f5e64149b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 shutdown_connections 2022-08-01T22:56:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.427+0000 7f5e6b2f2700 1 -- 172.21.15.32:0/1825264651 wait complete. 2022-08-01T22:56:07.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 Processor -- start 2022-08-01T22:56:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 -- start start 2022-08-01T22:56:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f091c0b2790 0x7f091c05c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c143870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c1471b0 con 0x7f091c143d60 2022-08-01T22:56:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c0ad5d0 con 0x7f091c0b2790 2022-08-01T22:56:07.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09233dc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c147450 con 0x7f091c0b1fb0 2022-08-01T22:56:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.547+0000 7f09223da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f091c0b2790 0x7f091c05c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f091c0b2790 0x7f091c05c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41130/0 (socket says 172.21.15.32:41130) 2022-08-01T22:56:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 -- 172.21.15.32:0/1364277111 learned_addr learned my addr 172.21.15.32:0/1364277111 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f0921bd9700 1 --2- 172.21.15.32:0/1364277111 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 -- 172.21.15.32:0/1364277111 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 msgr2=0x7f091c143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 --2- 172.21.15.32:0/1364277111 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 -- 172.21.15.32:0/1364277111 >> v1:172.21.15.55:6789/0 conn(0x7f091c143d60 legacy=0x7f091c13cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 -- 172.21.15.32:0/1364277111 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c143130 con 0x7f091c0b2790 2022-08-01T22:56:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09223da700 1 --2- 172.21.15.32:0/1364277111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c0b2790 0x7f091c05c620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f0918002f70 tx=0x7f0918006eb0).ready entity=mon.0 client_cookie=64d815aa398df0d3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09213d8700 1 -- 172.21.15.32:0/1364277111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f091800d570 con 0x7f091c0b2790 2022-08-01T22:56:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09213d8700 1 -- 172.21.15.32:0/1364277111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918000e90 con 0x7f091c0b2790 2022-08-01T22:56:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09213d8700 1 -- 172.21.15.32:0/1364277111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f091800d900 con 0x7f091c0b2790 2022-08-01T22:56:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 -- 172.21.15.32:0/1364277111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c0b2790 msgr2=0x7f091c05c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 --2- 172.21.15.32:0/1364277111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c0b2790 0x7f091c05c620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f0918002f70 tx=0x7f0918006eb0).stop 2022-08-01T22:56:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 -- 172.21.15.32:0/1364277111 shutdown_connections 2022-08-01T22:56:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 --2- 172.21.15.32:0/1364277111 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 --2- 172.21.15.32:0/1364277111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c0b2790 0x7f091c05c620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 -- 172.21.15.32:0/1364277111 >> 172.21.15.32:0/1364277111 conn(0x7f091c061360 msgr2=0x7f091c062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 -- 172.21.15.32:0/1364277111 shutdown_connections 2022-08-01T22:56:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.551+0000 7f09233dc700 1 -- 172.21.15.32:0/1364277111 wait complete. 2022-08-01T22:56:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 Processor -- start 2022-08-01T22:56:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 -- start start 2022-08-01T22:56:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c0b2790 0x7f091c13db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 0x7f091c13e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c144860 con 0x7f091c143d60 2022-08-01T22:56:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c13fde0 con 0x7f091c0b2790 2022-08-01T22:56:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09233dc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c140080 con 0x7f091c0b1fb0 2022-08-01T22:56:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 0x7f091c13e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 0x7f091c13e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41136/0 (socket says 172.21.15.32:41136) 2022-08-01T22:56:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 -- 172.21.15.32:0/634628491 learned_addr learned my addr 172.21.15.32:0/634628491 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f09223da700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0921bd9700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c0b2790 0x7f091c13db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 -- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 msgr2=0x7f091c1415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c1415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 -- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c0b2790 msgr2=0x7f091c13db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c0b2790 0x7f091c13db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 -- 172.21.15.32:0/634628491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091800b1e0 con 0x7f091c143d60 2022-08-01T22:56:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.555+0000 7f0922bdb700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 0x7f091c13e040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f091c0b23e0 tx=0x7f09140013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0914006aa0 con 0x7f091c143d60 2022-08-01T22:56:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914006c00 con 0x7f091c143d60 2022-08-01T22:56:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c140320 con 0x7f091c143d60 2022-08-01T22:56:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0914007030 con 0x7f091c143d60 2022-08-01T22:56:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c0b01a0 con 0x7f091c143d60 2022-08-01T22:56:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c1433d0 con 0x7f091c143d60 2022-08-01T22:56:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0914010030 con 0x7f091c143d60 2022-08-01T22:56:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f090f7fe700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08fc0539e0 0x7f08fc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f091406c050 con 0x7f091c143d60 2022-08-01T22:56:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.559+0000 7f09223da700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08fc0539e0 0x7f08fc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.563+0000 7f09223da700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08fc0539e0 0x7f08fc055ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f091801a000 tx=0x7f0918015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.567+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0914042d00 con 0x7f091c143d60 2022-08-01T22:56:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.691+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f091c0574e0 con 0x7f091c143d60 2022-08-01T22:56:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f090f7fe700 1 -- 172.21.15.32:0/634628491 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0914046320 con 0x7f091c143d60 2022-08-01T22:56:07.700 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08fc0539e0 msgr2=0x7f08fc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08fc0539e0 0x7f08fc055ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f091801a000 tx=0x7f0918015000).stop 2022-08-01T22:56:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 msgr2=0x7f091c13e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 0x7f091c13e040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f091c0b23e0 tx=0x7f09140013a0).stop 2022-08-01T22:56:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 shutdown_connections 2022-08-01T22:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08fc0539e0 0x7f08fc055ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f091c143d60 0x7f091c13e040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c0b2790 0x7f091c13db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 --2- 172.21.15.32:0/634628491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f091c0b1fb0 0x7f091c1415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 >> 172.21.15.32:0/634628491 conn(0x7f091c061360 msgr2=0x7f091c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 shutdown_connections 2022-08-01T22:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.695+0000 7f09233dc700 1 -- 172.21.15.32:0/634628491 wait complete. 2022-08-01T22:56:07.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 Processor -- start 2022-08-01T22:56:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- start start 2022-08-01T22:56:07.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f870c14b040 0x7f870c14b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c0b2c20 con 0x7f870c0b3070 2022-08-01T22:56:07.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c061830 con 0x7f870c14b040 2022-08-01T22:56:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c065c10 con 0x7f870c0b7840 2022-08-01T22:56:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f870c14b040 0x7f870c14b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f870c14b040 0x7f870c14b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41142/0 (socket says 172.21.15.32:41142) 2022-08-01T22:56:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 -- 172.21.15.32:0/1350092733 learned_addr learned my addr 172.21.15.32:0/1350092733 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870a7fc700 1 --1- 172.21.15.32:0/1350092733 >> v1:172.21.15.32:6789/0 conn(0x7f870c0b3070 0x7f870c061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56916/0 (socket says 172.21.15.32:56916) 2022-08-01T22:56:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870affd700 1 --2- 172.21.15.32:0/1350092733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 -- 172.21.15.32:0/1350092733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 msgr2=0x7f870c065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 --2- 172.21.15.32:0/1350092733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 -- 172.21.15.32:0/1350092733 >> v1:172.21.15.32:6789/0 conn(0x7f870c0b3070 legacy=0x7f870c061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 -- 172.21.15.32:0/1350092733 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c0b7c20 con 0x7f870c14b040 2022-08-01T22:56:07.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870b7fe700 1 --2- 172.21.15.32:0/1350092733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14b4f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f87040032f0 tx=0x7f8704009290).ready entity=mon.0 client_cookie=7a7050085d6d41cb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f8709ffb700 1 -- 172.21.15.32:0/1350092733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870400b870 con 0x7f870c14b040 2022-08-01T22:56:07.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f8709ffb700 1 -- 172.21.15.32:0/1350092733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870400b9d0 con 0x7f870c14b040 2022-08-01T22:56:07.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f8709ffb700 1 -- 172.21.15.32:0/1350092733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870400bb80 con 0x7f870c14b040 2022-08-01T22:56:07.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- 172.21.15.32:0/1350092733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 msgr2=0x7f870c14b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 --2- 172.21.15.32:0/1350092733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14b4f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f87040032f0 tx=0x7f8704009290).stop 2022-08-01T22:56:07.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- 172.21.15.32:0/1350092733 shutdown_connections 2022-08-01T22:56:07.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 --2- 172.21.15.32:0/1350092733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14b4f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 --2- 172.21.15.32:0/1350092733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c065720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- 172.21.15.32:0/1350092733 >> 172.21.15.32:0/1350092733 conn(0x7f870c05f840 msgr2=0x7f870c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- 172.21.15.32:0/1350092733 shutdown_connections 2022-08-01T22:56:07.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.819+0000 7f870bfff700 1 -- 172.21.15.32:0/1350092733 wait complete. 2022-08-01T22:56:07.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 Processor -- start 2022-08-01T22:56:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 -- start start 2022-08-01T22:56:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f870c0b3070 0x7f870c0acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c14a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c14aa90 con 0x7f870c14b040 2022-08-01T22:56:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c156ce0 con 0x7f870c0b3070 2022-08-01T22:56:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c156f80 con 0x7f870c0b7840 2022-08-01T22:56:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41148/0 (socket says 172.21.15.32:41148) 2022-08-01T22:56:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 -- 172.21.15.32:0/4151436920 learned_addr learned my addr 172.21.15.32:0/4151436920 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870affd700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f870c0b3070 0x7f870c0acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870a7fc700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c14a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 -- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 msgr2=0x7f870c14a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c14a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 -- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f870c0b3070 msgr2=0x7f870c0acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f870c0b3070 0x7f870c0acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.823+0000 7f870b7fe700 1 -- 172.21.15.32:0/4151436920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8704009530 con 0x7f870c14b040 2022-08-01T22:56:07.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f870b7fe700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14a5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f87040032f0 tx=0x7f8704016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8704021430 con 0x7f870c14b040 2022-08-01T22:56:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8704021590 con 0x7f870c14b040 2022-08-01T22:56:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87040218f0 con 0x7f870c14b040 2022-08-01T22:56:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c157220 con 0x7f870c14b040 2022-08-01T22:56:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c0ad2f0 con 0x7f870c14b040 2022-08-01T22:56:07.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c0b2980 con 0x7f870c14b040 2022-08-01T22:56:07.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f870400f070 con 0x7f870c14b040 2022-08-01T22:56:07.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f86fbfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86e8053b00 0x7f86e8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:07.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f870406cc20 con 0x7f870c14b040 2022-08-01T22:56:07.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.827+0000 7f870affd700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86e8053b00 0x7f86e8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.831+0000 7f870affd700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86e8053b00 0x7f86e8055fc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f86fc00c000 tx=0x7f86fc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.835+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87040439e0 con 0x7f870c14b040 2022-08-01T22:56:07.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f870c0574e0 con 0x7f870c14b040 2022-08-01T22:56:07.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f86fbfff700 1 -- 172.21.15.32:0/4151436920 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8704015030 con 0x7f870c14b040 2022-08-01T22:56:07.969 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:07.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86e8053b00 msgr2=0x7f86e8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86e8053b00 0x7f86e8055fc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f86fc00c000 tx=0x7f86fc008040).stop 2022-08-01T22:56:07.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 msgr2=0x7f870c14a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:07.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14a5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f87040032f0 tx=0x7f8704016000).stop 2022-08-01T22:56:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 shutdown_connections 2022-08-01T22:56:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86e8053b00 0x7f86e8055fc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f870c14b040 0x7f870c14a5a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f870c0b7840 0x7f870c14a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 --2- 172.21.15.32:0/4151436920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f870c0b3070 0x7f870c0acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:07.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 >> 172.21.15.32:0/4151436920 conn(0x7f870c05f840 msgr2=0x7f870c0b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:07.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 shutdown_connections 2022-08-01T22:56:07.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:07.959+0000 7f870bfff700 1 -- 172.21.15.32:0/4151436920 wait complete. 2022-08-01T22:56:07.979 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:56:07.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.075+0000 7f89a7fff700 1 Processor -- start 2022-08-01T22:56:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.075+0000 7f89a7fff700 1 -- start start 2022-08-01T22:56:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f89a80614b0 0x7f89a8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a80ad750 0x7f89a80b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89a81405d0 0x7f89a8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a80aee80 con 0x7f89a81405d0 2022-08-01T22:56:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a80b70b0 con 0x7f89a80614b0 2022-08-01T22:56:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80af0b0 con 0x7f89a80ad750 2022-08-01T22:56:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89a81405d0 0x7f89a8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89a81405d0 0x7f89a8140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41156/0 (socket says 172.21.15.32:41156) 2022-08-01T22:56:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 -- 172.21.15.32:0/336977753 learned_addr learned my addr 172.21.15.32:0/336977753 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a67fc700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a80ad750 0x7f89a80b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a6ffd700 1 --2- 172.21.15.32:0/336977753 >> v2:172.21.15.55:3300/0 conn(0x7f89a80614b0 0x7f89a8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 -- 172.21.15.32:0/336977753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a80ad750 msgr2=0x7f89a80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a80ad750 0x7f89a80b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 -- 172.21.15.32:0/336977753 >> v2:172.21.15.55:3300/0 conn(0x7f89a80614b0 msgr2=0x7f89a8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 --2- 172.21.15.32:0/336977753 >> v2:172.21.15.55:3300/0 conn(0x7f89a80614b0 0x7f89a8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 -- 172.21.15.32:0/336977753 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80af350 con 0x7f89a81405d0 2022-08-01T22:56:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a67fc700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a80ad750 0x7f89a80b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a77fe700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a81405d0 0x7f89a8140a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f89a00032f0 tx=0x7f89a0009290).ready entity=mon.0 client_cookie=9c69babfd122a77c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a5ffb700 1 -- 172.21.15.32:0/336977753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89a000b680 con 0x7f89a81405d0 2022-08-01T22:56:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a5ffb700 1 -- 172.21.15.32:0/336977753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a000b7e0 con 0x7f89a81405d0 2022-08-01T22:56:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a5ffb700 1 -- 172.21.15.32:0/336977753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89a000b950 con 0x7f89a81405d0 2022-08-01T22:56:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- 172.21.15.32:0/336977753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a81405d0 msgr2=0x7f89a8140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a81405d0 0x7f89a8140a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f89a00032f0 tx=0x7f89a0009290).stop 2022-08-01T22:56:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- 172.21.15.32:0/336977753 shutdown_connections 2022-08-01T22:56:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a81405d0 0x7f89a8140a60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- 172.21.15.32:0/336977753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a80ad750 0x7f89a80b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 --2- 172.21.15.32:0/336977753 >> v2:172.21.15.55:3300/0 conn(0x7f89a80614b0 0x7f89a8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- 172.21.15.32:0/336977753 >> 172.21.15.32:0/336977753 conn(0x7f89a8066f70 msgr2=0x7f89a8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- 172.21.15.32:0/336977753 shutdown_connections 2022-08-01T22:56:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.079+0000 7f89a7fff700 1 -- 172.21.15.32:0/336977753 wait complete. 2022-08-01T22:56:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 Processor -- start 2022-08-01T22:56:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 -- start start 2022-08-01T22:56:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89a80614b0 0x7f89a80b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 0x7f89a80b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a81405d0 0x7f89a80ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80b32b0 con 0x7f89a80ad750 2022-08-01T22:56:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8156380 con 0x7f89a80614b0 2022-08-01T22:56:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a7fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8142c70 con 0x7f89a81405d0 2022-08-01T22:56:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 0x7f89a80b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 0x7f89a80b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41160/0 (socket says 172.21.15.32:41160) 2022-08-01T22:56:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 -- 172.21.15.32:0/251068080 learned_addr learned my addr 172.21.15.32:0/251068080 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a6ffd700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89a80614b0 0x7f89a80b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a77fe700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a81405d0 0x7f89a80ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 -- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a81405d0 msgr2=0x7f89a80ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a81405d0 0x7f89a80ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 -- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89a80614b0 msgr2=0x7f89a80b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89a80614b0 0x7f89a80b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 -- 172.21.15.32:0/251068080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899800b180 con 0x7f89a80ad750 2022-08-01T22:56:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.083+0000 7f89a67fc700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 0x7f89a80b0b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f899800c000 tx=0x7f8998006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f899800c890 con 0x7f89a80ad750 2022-08-01T22:56:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899800c9f0 con 0x7f89a80ad750 2022-08-01T22:56:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a0009530 con 0x7f89a80ad750 2022-08-01T22:56:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f899800cd50 con 0x7f89a80ad750 2022-08-01T22:56:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a81565b0 con 0x7f89a80ad750 2022-08-01T22:56:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a8156770 con 0x7f89a80ad750 2022-08-01T22:56:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f899801c460 con 0x7f89a80ad750 2022-08-01T22:56:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f8997fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8984053950 0x7f8984055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f899806c720 con 0x7f89a80ad750 2022-08-01T22:56:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.087+0000 7f89a6ffd700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8984053950 0x7f8984055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.091+0000 7f89a6ffd700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8984053950 0x7f8984055e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f899c00b000 tx=0x7f899c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.099+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89a8156770 con 0x7f89a80ad750 2022-08-01T22:56:08.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89a8156770 con 0x7f89a80ad750 2022-08-01T22:56:08.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f8997fff700 1 -- 172.21.15.32:0/251068080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f89a8156770 con 0x7f89a80ad750 2022-08-01T22:56:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:08.270 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3750794992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3750794992},{"type":"v1","addr":"172.21.15.32:6835","nonce":3750794992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3123630923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3123630923},{"type":"v1","addr":"172.21.15.55:6835","nonce":3123630923}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:55:50.680148+0000","modified":"2022-08-01T22:55:52.352879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/2632088620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2632088620},{"type":"v1","addr":"172.21.15.32:6833","nonce":2632088620}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T22:56:08.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8984053950 msgr2=0x7f8984055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8984053950 0x7f8984055e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f899c00b000 tx=0x7f899c006df0).stop 2022-08-01T22:56:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 msgr2=0x7f89a80b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 0x7f89a80b0b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f899800c000 tx=0x7f8998006df0).stop 2022-08-01T22:56:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 shutdown_connections 2022-08-01T22:56:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8984053950 0x7f8984055e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f89a81405d0 0x7f89a80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ad750 0x7f89a80b0b70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 --2- 172.21.15.32:0/251068080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89a80614b0 0x7f89a80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 >> 172.21.15.32:0/251068080 conn(0x7f89a8066f70 msgr2=0x7f89a80b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.263+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 shutdown_connections 2022-08-01T22:56:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:08.267+0000 7f89a7fff700 1 -- 172.21.15.32:0/251068080 wait complete. 2022-08-01T22:56:08.275 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2022-08-01T22:56:08.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:55:50.680148+0000', 'modified': '2022-08-01T22:55:52.352879+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6833/2632088620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6832', 'nonce': 2632088620}, {'type': 'v1', 'addr': '172.21.15.32:6833', 'nonce': 2632088620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:56:08.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:56:08.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T22:56:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":4264,"want_state":"up:active","state":"up:active","rank_uptime":18.010852072999999,"mdsmap_epoch":6,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":24.422100689000001} 2022-08-01T22:56:08.708 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 4264, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 18.010852073, "state": "up:active", "uptime": 24.422100689, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:56:08.709 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:56:08.712 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T22:56:08.713 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T22:56:08.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:08.714 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:08.725 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T22:56:08.748 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:08.749 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:08.769 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:08.770 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:08.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:08.773 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T22:56:08.830 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:56:08.831 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:08.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T22:56:08.906 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:08.910 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:08.922 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:08.924 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:08.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:08.926 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T22:56:08.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T22:56:08.966+0000 7f5b30089080 -1 init, newargv = 0x56399b9549f0 newargc=9ceph-fuse[ 2022-08-01T22:56:08.979 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:22386]: starting ceph client 2022-08-01T22:56:08.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22386]: starting fuse 2022-08-01T22:56:10.081 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T22:56:10.108 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:10.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:10.169 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:10.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.172 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T22:56:10.196 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T22:56:10.196 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:56:10.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.197 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T22:56:10.197 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T22:56:10.198 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T22:56:10.198 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T22:56:10.198 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T22:56:10.199 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T22:56:10.199 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T22:56:10.199 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T22:56:10.200 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T22:56:10.200 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T22:56:10.200 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T22:56:10.201 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T22:56:10.201 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T22:56:10.202 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T22:56:10.202 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T22:56:10.203 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T22:56:10.204 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:56:10.204 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:56:10.205 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T22:56:10.214 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:56:10.215 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T22:56:10.215 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T22:56:10.216 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T22:56:10.291 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.22386.asok 2022-08-01T22:56:10.293 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22386.asok 2022-08-01T22:56:10.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22386.asok status 2022-08-01T22:56:10.391 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T22:56:10.392 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T22:56:10.392 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:56:10.393 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:56:10.393 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T22:56:10.394 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T22:56:10.395 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T22:56:10.395 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "22386", 2022-08-01T22:56:10.395 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T22:56:10.396 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T22:56:10.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T22:56:10.397 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T22:56:10.397 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4663, 2022-08-01T22:56:10.398 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T22:56:10.398 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T22:56:10.399 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T22:56:10.399 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4663 2022-08-01T22:56:10.400 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T22:56:10.400 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T22:56:10.400 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T22:56:10.401 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T22:56:10.401 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 948229036 2022-08-01T22:56:10.402 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T22:56:10.402 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T22:56:10.402 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T22:56:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T22:56:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T22:56:10.404 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 948229036 2022-08-01T22:56:10.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T22:56:10.404 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4663 172.21.15.157:0/948229036", 2022-08-01T22:56:10.404 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/948229036", 2022-08-01T22:56:10.405 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T22:56:10.405 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 6, 2022-08-01T22:56:10.405 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 24, 2022-08-01T22:56:10.405 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:56:10.405 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T22:56:10.406 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T22:56:10.406 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T22:56:10.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.408 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:10.408 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:10.424 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:56:10.424 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:56:10.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.426 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:10.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.452 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:10.453 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:10.464 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T22:56:10.465 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T22:56:10.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:10.466 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:10.493 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2022-08-01T22:56:10.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 Processor -- start 2022-08-01T22:56:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 -- start start 2022-08-01T22:56:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef40614b0 0x7f3ef4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ef41405d0 0x7f3ef4140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef40aee80 con 0x7f3ef40ad750 2022-08-01T22:56:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef40b70b0 con 0x7f3ef41405d0 2022-08-01T22:56:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efb2c5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40af0b0 con 0x7f3ef40614b0 2022-08-01T22:56:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ef41405d0 0x7f3ef4140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ef41405d0 0x7f3ef4140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41166/0 (socket says 172.21.15.32:41166) 2022-08-01T22:56:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 -- 172.21.15.32:0/4056422569 learned_addr learned my addr 172.21.15.32:0/4056422569 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efa2c3700 1 --2- 172.21.15.32:0/4056422569 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef40614b0 0x7f3ef4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 -- 172.21.15.32:0/4056422569 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef40614b0 msgr2=0x7f3ef4061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 --2- 172.21.15.32:0/4056422569 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef40614b0 0x7f3ef4061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 -- 172.21.15.32:0/4056422569 >> v1:172.21.15.55:6789/0 conn(0x7f3ef40ad750 legacy=0x7f3ef40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 -- 172.21.15.32:0/4056422569 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef40af350 con 0x7f3ef41405d0 2022-08-01T22:56:10.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3efaac4700 1 --2- 172.21.15.32:0/4056422569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef41405d0 0x7f3ef4140a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f3eec002f70 tx=0x7f3eec006eb0).ready entity=mon.0 client_cookie=a8f90aad48765785 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:10.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3ef92c1700 1 -- 172.21.15.32:0/4056422569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3eec00d570 con 0x7f3ef41405d0 2022-08-01T22:56:10.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3ef92c1700 1 -- 172.21.15.32:0/4056422569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec000e90 con 0x7f3ef41405d0 2022-08-01T22:56:10.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.639+0000 7f3ef92c1700 1 -- 172.21.15.32:0/4056422569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3eec00d900 con 0x7f3ef41405d0 2022-08-01T22:56:10.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- 172.21.15.32:0/4056422569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef41405d0 msgr2=0x7f3ef4140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/4056422569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef41405d0 0x7f3ef4140a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f3eec002f70 tx=0x7f3eec006eb0).stop 2022-08-01T22:56:10.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- 172.21.15.32:0/4056422569 shutdown_connections 2022-08-01T22:56:10.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/4056422569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef41405d0 0x7f3ef4140a60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/4056422569 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef40614b0 0x7f3ef4061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- 172.21.15.32:0/4056422569 >> 172.21.15.32:0/4056422569 conn(0x7f3ef4066f70 msgr2=0x7f3ef4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- 172.21.15.32:0/4056422569 shutdown_connections 2022-08-01T22:56:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- 172.21.15.32:0/4056422569 wait complete. 2022-08-01T22:56:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 Processor -- start 2022-08-01T22:56:10.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- start start 2022-08-01T22:56:10.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 0x7f3ef40ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef40ad750 0x7f3ef405cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef41405d0 0x7f3ef40abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40ab220 con 0x7f3ef40614b0 2022-08-01T22:56:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40aa520 con 0x7f3ef40ad750 2022-08-01T22:56:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efb2c5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40aa7c0 con 0x7f3ef41405d0 2022-08-01T22:56:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efa2c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 0x7f3ef40ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efa2c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 0x7f3ef40ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41170/0 (socket says 172.21.15.32:41170) 2022-08-01T22:56:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.643+0000 7f3efa2c3700 1 -- 172.21.15.32:0/2546975989 learned_addr learned my addr 172.21.15.32:0/2546975989 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ef9ac2700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef40ad750 0x7f3ef405cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efaac4700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef41405d0 0x7f3ef40abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efa2c3700 1 -- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef41405d0 msgr2=0x7f3ef40abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efa2c3700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef41405d0 0x7f3ef40abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efa2c3700 1 -- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef40ad750 msgr2=0x7f3ef405cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efa2c3700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef40ad750 0x7f3ef405cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efa2c3700 1 -- 172.21.15.32:0/2546975989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec00b1e0 con 0x7f3ef40614b0 2022-08-01T22:56:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efaac4700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef41405d0 0x7f3ef40abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efa2c3700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 0x7f3ef40ab880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3ef000c000 tx=0x7f3ef00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef000c850 con 0x7f3ef40614b0 2022-08-01T22:56:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef000c9b0 con 0x7f3ef40614b0 2022-08-01T22:56:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef40b1470 con 0x7f3ef40614b0 2022-08-01T22:56:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef000cd10 con 0x7f3ef40614b0 2022-08-01T22:56:10.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef40ad260 con 0x7f3ef40614b0 2022-08-01T22:56:10.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef40af5f0 con 0x7f3ef40614b0 2022-08-01T22:56:10.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ef0022070 con 0x7f3ef40614b0 2022-08-01T22:56:10.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ee37fe700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ed4053950 0x7f3ed4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.647+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3ef006c790 con 0x7f3ef40614b0 2022-08-01T22:56:10.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.651+0000 7f3ef9ac2700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ed4053950 0x7f3ed4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.651+0000 7f3ef9ac2700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ed4053950 0x7f3ed4055e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3ee400d000 tx=0x7f3ee4007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:10.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.655+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ef0043550 con 0x7f3ef40614b0 2022-08-01T22:56:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.783+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ef40574e0 con 0x7f3ef40614b0 2022-08-01T22:56:10.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.787+0000 7f3ee37fe700 1 -- 172.21.15.32:0/2546975989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f3ef0046b70 con 0x7f3ef40614b0 2022-08-01T22:56:10.801 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:10.802 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:10.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ed4053950 msgr2=0x7f3ed4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ed4053950 0x7f3ed4055e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3ee400d000 tx=0x7f3ee4007b90).stop 2022-08-01T22:56:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 msgr2=0x7f3ef40ab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 0x7f3ef40ab880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3ef000c000 tx=0x7f3ef00091d0).stop 2022-08-01T22:56:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 shutdown_connections 2022-08-01T22:56:10.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ed4053950 0x7f3ed4055e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ef41405d0 0x7f3ef40abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef40ad750 0x7f3ef405cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 --2- 172.21.15.32:0/2546975989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ef40614b0 0x7f3ef40ab880 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 >> 172.21.15.32:0/2546975989 conn(0x7f3ef4066f70 msgr2=0x7f3ef40b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:10.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 shutdown_connections 2022-08-01T22:56:10.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.791+0000 7f3efb2c5700 1 -- 172.21.15.32:0/2546975989 wait complete. 2022-08-01T22:56:10.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:10.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.923+0000 7f3aa2167700 1 Processor -- start 2022-08-01T22:56:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.923+0000 7f3aa2167700 1 -- start start 2022-08-01T22:56:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c0ad750 0x7f3a9c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0aee80 con 0x7f3a9c1405d0 2022-08-01T22:56:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0b70b0 con 0x7f3a9c0614b0 2022-08-01T22:56:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0af0b0 con 0x7f3a9c0ad750 2022-08-01T22:56:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa1966700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a9c1405d0 0x7f3a9c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56954/0 (socket says 172.21.15.32:56954) 2022-08-01T22:56:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa1966700 1 -- 172.21.15.32:0/4262884724 learned_addr learned my addr 172.21.15.32:0/4262884724 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa0964700 1 --2- 172.21.15.32:0/4262884724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c0ad750 0x7f3a9c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771870558 0 0) 0x7f3a9c0aee80 con 0x7f3a9c1405d0 2022-08-01T22:56:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a80003260 con 0x7f3a9c1405d0 2022-08-01T22:56:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259710545 0 0) 0x7f3a9c0b70b0 con 0x7f3a9c0614b0 2022-08-01T22:56:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a9c0aee80 con 0x7f3a9c0614b0 2022-08-01T22:56:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa0964700 1 -- 172.21.15.32:0/4262884724 >> v1:172.21.15.55:6789/0 conn(0x7f3a9c0614b0 legacy=0x7f3a9c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa0964700 1 -- 172.21.15.32:0/4262884724 >> v1:172.21.15.32:6789/0 conn(0x7f3a9c1405d0 legacy=0x7f3a9c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa0964700 1 -- 172.21.15.32:0/4262884724 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0af350 con 0x7f3a9c0ad750 2022-08-01T22:56:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa0964700 1 --2- 172.21.15.32:0/4262884724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c0ad750 0x7f3a9c061890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3a98002f70 tx=0x7f3a98006df0).ready entity=mon.2 client_cookie=783250c7fde37559 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a9800d5c0 con 0x7f3a9c0ad750 2022-08-01T22:56:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98000e90 con 0x7f3a9c0ad750 2022-08-01T22:56:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3a93fff700 1 -- 172.21.15.32:0/4262884724 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a9800d910 con 0x7f3a9c0ad750 2022-08-01T22:56:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- 172.21.15.32:0/4262884724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c0ad750 msgr2=0x7f3a9c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 --2- 172.21.15.32:0/4262884724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c0ad750 0x7f3a9c061890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3a98002f70 tx=0x7f3a98006df0).stop 2022-08-01T22:56:10.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- 172.21.15.32:0/4262884724 shutdown_connections 2022-08-01T22:56:10.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 --2- 172.21.15.32:0/4262884724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c0ad750 0x7f3a9c061890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- 172.21.15.32:0/4262884724 >> 172.21.15.32:0/4262884724 conn(0x7f3a9c066f70 msgr2=0x7f3a9c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:10.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- 172.21.15.32:0/4262884724 shutdown_connections 2022-08-01T22:56:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.927+0000 7f3aa2167700 1 -- 172.21.15.32:0/4262884724 wait complete. 2022-08-01T22:56:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 Processor -- start 2022-08-01T22:56:10.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 -- start start 2022-08-01T22:56:10.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 0x7f3a9c0b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 0x7f3a9c0b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c1405d0 0x7f3a9c13f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0b2d70 con 0x7f3a9c0614b0 2022-08-01T22:56:10.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c142bf0 con 0x7f3a9c0ad750 2022-08-01T22:56:10.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa2167700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0b3b40 con 0x7f3a9c1405d0 2022-08-01T22:56:10.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 0x7f3a9c0b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa0964700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 0x7f3a9c0b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 0x7f3a9c0b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41184/0 (socket says 172.21.15.32:41184) 2022-08-01T22:56:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1966700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c1405d0 0x7f3a9c13f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa0964700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 0x7f3a9c0b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:50968/0 (socket says 172.21.15.32:50968) 2022-08-01T22:56:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 -- 172.21.15.32:0/2295806130 learned_addr learned my addr 172.21.15.32:0/2295806130 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 -- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c1405d0 msgr2=0x7f3a9c13f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c1405d0 0x7f3a9c13f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 -- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 msgr2=0x7f3a9c0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:10.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 0x7f3a9c0b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:10.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 -- 172.21.15.32:0/2295806130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9800b180 con 0x7f3a9c0614b0 2022-08-01T22:56:10.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa0964700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 0x7f3a9c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.931+0000 7f3aa1165700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 0x7f3a9c0b1f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f3a9c145f50 tx=0x7f3a8c007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8c012400 con 0x7f3a9c0614b0 2022-08-01T22:56:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c012560 con 0x7f3a9c0614b0 2022-08-01T22:56:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c152f80 con 0x7f3a9c0614b0 2022-08-01T22:56:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8c0128c0 con 0x7f3a9c0614b0 2022-08-01T22:56:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0b3830 con 0x7f3a9c0614b0 2022-08-01T22:56:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c0b0ce0 con 0x7f3a9c0614b0 2022-08-01T22:56:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a8c019030 con 0x7f3a9c0614b0 2022-08-01T22:56:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3a927fc700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a84053950 0x7f3a84055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3a8c06b390 con 0x7f3a9c0614b0 2022-08-01T22:56:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.935+0000 7f3aa0964700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a84053950 0x7f3a84055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.939+0000 7f3aa0964700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a84053950 0x7f3a84055e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3a98018000 tx=0x7f3a98014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:10.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:10.943+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a8c042150 con 0x7f3a9c0614b0 2022-08-01T22:56:11.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.071+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3a9c0af5f0 con 0x7f3a9c0614b0 2022-08-01T22:56:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3a927fc700 1 -- 172.21.15.32:0/2295806130 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3a8c045770 con 0x7f3a9c0614b0 2022-08-01T22:56:11.080 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:11.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a84053950 msgr2=0x7f3a84055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a84053950 0x7f3a84055e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3a98018000 tx=0x7f3a98014000).stop 2022-08-01T22:56:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 msgr2=0x7f3a9c0b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 0x7f3a9c0b1f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f3a9c145f50 tx=0x7f3a8c007a90).stop 2022-08-01T22:56:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 shutdown_connections 2022-08-01T22:56:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a84053950 0x7f3a84055e10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a9c1405d0 0x7f3a9c13f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9c0ad750 0x7f3a9c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 --2- 172.21.15.32:0/2295806130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a9c0614b0 0x7f3a9c0b1f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 >> 172.21.15.32:0/2295806130 conn(0x7f3a9c066f70 msgr2=0x7f3a9c146a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 shutdown_connections 2022-08-01T22:56:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.075+0000 7f3aa2167700 1 -- 172.21.15.32:0/2295806130 wait complete. 2022-08-01T22:56:11.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:11.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.199+0000 7fc94995f700 1 Processor -- start 2022-08-01T22:56:11.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.199+0000 7fc94995f700 1 -- start start 2022-08-01T22:56:11.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9440b0740 0x7fc9440b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9440b38a0 0x7fc94413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc94413dbd0 0x7fc9440b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc944143ab0 con 0x7fc9440b0740 2022-08-01T22:56:11.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc944143d50 con 0x7fc9440b38a0 2022-08-01T22:56:11.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc944143ff0 con 0x7fc94413dbd0 2022-08-01T22:56:11.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9440b0740 0x7fc9440b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc9440b0740 0x7fc9440b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41190/0 (socket says 172.21.15.32:41190) 2022-08-01T22:56:11.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 -- 172.21.15.32:0/3776283527 learned_addr learned my addr 172.21.15.32:0/3776283527 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:11.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc9437fe700 1 --2- 172.21.15.32:0/3776283527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc94413dbd0 0x7fc9440b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc9427fc700 1 --2- 172.21.15.32:0/3776283527 >> v2:172.21.15.55:3300/0 conn(0x7fc9440b38a0 0x7fc94413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 -- 172.21.15.32:0/3776283527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc94413dbd0 msgr2=0x7fc9440b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:11.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 --2- 172.21.15.32:0/3776283527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc94413dbd0 0x7fc9440b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 -- 172.21.15.32:0/3776283527 >> v2:172.21.15.55:3300/0 conn(0x7fc9440b38a0 msgr2=0x7fc94413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:11.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 --2- 172.21.15.32:0/3776283527 >> v2:172.21.15.55:3300/0 conn(0x7fc9440b38a0 0x7fc94413d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 -- 172.21.15.32:0/3776283527 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944144290 con 0x7fc9440b0740 2022-08-01T22:56:11.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc9427fc700 1 --2- 172.21.15.32:0/3776283527 >> v2:172.21.15.55:3300/0 conn(0x7fc9440b38a0 0x7fc94413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:11.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc942ffd700 1 --2- 172.21.15.32:0/3776283527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9440b0740 0x7fc9440b0b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc92c0032f0 tx=0x7fc92c00b530).ready entity=mon.0 client_cookie=3fc73f46e4fa643e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc941ffb700 1 -- 172.21.15.32:0/3776283527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc92c00be70 con 0x7fc9440b0740 2022-08-01T22:56:11.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc941ffb700 1 -- 172.21.15.32:0/3776283527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c010470 con 0x7fc9440b0740 2022-08-01T22:56:11.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc941ffb700 1 -- 172.21.15.32:0/3776283527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc92c0107d0 con 0x7fc9440b0740 2022-08-01T22:56:11.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- 172.21.15.32:0/3776283527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9440b0740 msgr2=0x7fc9440b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- 172.21.15.32:0/3776283527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9440b0740 0x7fc9440b0b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc92c0032f0 tx=0x7fc92c00b530).stop 2022-08-01T22:56:11.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- 172.21.15.32:0/3776283527 shutdown_connections 2022-08-01T22:56:11.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- 172.21.15.32:0/3776283527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc94413dbd0 0x7fc9440b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- 172.21.15.32:0/3776283527 >> v2:172.21.15.55:3300/0 conn(0x7fc9440b38a0 0x7fc94413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 --2- 172.21.15.32:0/3776283527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9440b0740 0x7fc9440b0b20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- 172.21.15.32:0/3776283527 >> 172.21.15.32:0/3776283527 conn(0x7fc944060950 msgr2=0x7fc944060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- 172.21.15.32:0/3776283527 shutdown_connections 2022-08-01T22:56:11.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.203+0000 7fc94995f700 1 -- 172.21.15.32:0/3776283527 wait complete. 2022-08-01T22:56:11.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 Processor -- start 2022-08-01T22:56:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 -- start start 2022-08-01T22:56:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc9440b0740 0x7fc944142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9440b38a0 0x7fc9441522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 0x7fc94413aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94413f700 con 0x7fc94413dbd0 2022-08-01T22:56:11.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944140800 con 0x7fc9440b38a0 2022-08-01T22:56:11.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9441527c0 con 0x7fc9440b0740 2022-08-01T22:56:11.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 0x7fc94413aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 0x7fc94413aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41200/0 (socket says 172.21.15.32:41200) 2022-08-01T22:56:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 -- 172.21.15.32:0/1159978219 learned_addr learned my addr 172.21.15.32:0/1159978219 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc942ffd700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc9440b0740 0x7fc944142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9427fc700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9440b38a0 0x7fc9441522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 -- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc9440b0740 msgr2=0x7fc944142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc9440b0740 0x7fc944142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 -- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9440b38a0 msgr2=0x7fc9441522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9440b38a0 0x7fc9441522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 -- 172.21.15.32:0/1159978219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c00b7d0 con 0x7fc94413dbd0 2022-08-01T22:56:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9427fc700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9440b38a0 0x7fc9441522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc9437fe700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 0x7fc94413aa80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc93c00a000 tx=0x7fc93c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c011780 con 0x7fc94413dbd0 2022-08-01T22:56:11.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0118e0 con 0x7fc94413dbd0 2022-08-01T22:56:11.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc944140560 con 0x7fc94413dbd0 2022-08-01T22:56:11.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c011c40 con 0x7fc94413dbd0 2022-08-01T22:56:11.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.207+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc944140dc0 con 0x7fc94413dbd0 2022-08-01T22:56:11.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.211+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc944140f80 con 0x7fc94413dbd0 2022-08-01T22:56:11.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.211+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc93c01c460 con 0x7fc94413dbd0 2022-08-01T22:56:11.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.211+0000 7fc93bfff700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc91c053950 0x7fc91c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.211+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc93c06c3d0 con 0x7fc94413dbd0 2022-08-01T22:56:11.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.211+0000 7fc942ffd700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc91c053950 0x7fc91c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.211+0000 7fc942ffd700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc91c053950 0x7fc91c055e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fc92c01e000 tx=0x7fc9440b0660).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.219+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc93c043130 con 0x7fc94413dbd0 2022-08-01T22:56:11.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.343+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc944143730 con 0x7fc94413dbd0 2022-08-01T22:56:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.343+0000 7fc93bfff700 1 -- 172.21.15.32:0/1159978219 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc93c046750 con 0x7fc94413dbd0 2022-08-01T22:56:11.352 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc91c053950 msgr2=0x7fc91c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc91c053950 0x7fc91c055e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fc92c01e000 tx=0x7fc9440b0660).stop 2022-08-01T22:56:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 msgr2=0x7fc94413aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 0x7fc94413aa80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc93c00a000 tx=0x7fc93c008040).stop 2022-08-01T22:56:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 shutdown_connections 2022-08-01T22:56:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc91c053950 0x7fc91c055e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc94413dbd0 0x7fc94413aa80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9440b38a0 0x7fc9441522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 --2- 172.21.15.32:0/1159978219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc9440b0740 0x7fc944142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 >> 172.21.15.32:0/1159978219 conn(0x7fc944060950 msgr2=0x7fc944141980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 shutdown_connections 2022-08-01T22:56:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.347+0000 7fc94995f700 1 -- 172.21.15.32:0/1159978219 wait complete. 2022-08-01T22:56:11.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 Processor -- start 2022-08-01T22:56:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 -- start start 2022-08-01T22:56:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b0066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50b01402d0 0x7f50b0140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b0067030 con 0x7f50b00a9da0 2022-08-01T22:56:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b0064a70 con 0x7f50b01402d0 2022-08-01T22:56:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.467+0000 7f50b6bcd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0061f60 con 0x7f50b00b31d0 2022-08-01T22:56:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50b01402d0 0x7f50b0140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50b01402d0 0x7f50b0140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41208/0 (socket says 172.21.15.32:41208) 2022-08-01T22:56:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 -- 172.21.15.32:0/1946024351 learned_addr learned my addr 172.21.15.32:0/1946024351 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b53ca700 1 --1- 172.21.15.32:0/1946024351 >> v1:172.21.15.55:6789/0 conn(0x7f50b00a9da0 0x7f50b0060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:49652/0 (socket says 172.21.15.32:49652) 2022-08-01T22:56:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b5bcb700 1 --2- 172.21.15.32:0/1946024351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b0066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b4bc9700 1 -- 172.21.15.32:0/1946024351 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247143281 0 0) 0x7f50b0067030 con 0x7f50b00a9da0 2022-08-01T22:56:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 -- 172.21.15.32:0/1946024351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 msgr2=0x7f50b0066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 --2- 172.21.15.32:0/1946024351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b0066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 -- 172.21.15.32:0/1946024351 >> v1:172.21.15.55:6789/0 conn(0x7f50b00a9da0 legacy=0x7f50b0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 -- 172.21.15.32:0/1946024351 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00b35b0 con 0x7f50b01402d0 2022-08-01T22:56:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b63cc700 1 --2- 172.21.15.32:0/1946024351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b01402d0 0x7f50b0140780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f50a80032f0 tx=0x7f50a8009290).ready entity=mon.0 client_cookie=f54ada2fb54d6f9f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b4bc9700 1 -- 172.21.15.32:0/1946024351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a800b8a0 con 0x7f50b01402d0 2022-08-01T22:56:11.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b4bc9700 1 -- 172.21.15.32:0/1946024351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a800ba00 con 0x7f50b01402d0 2022-08-01T22:56:11.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b4bc9700 1 -- 172.21.15.32:0/1946024351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50a800bb70 con 0x7f50b01402d0 2022-08-01T22:56:11.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 -- 172.21.15.32:0/1946024351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b01402d0 msgr2=0x7f50b0140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/1946024351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b01402d0 0x7f50b0140780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f50a80032f0 tx=0x7f50a8009290).stop 2022-08-01T22:56:11.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 -- 172.21.15.32:0/1946024351 shutdown_connections 2022-08-01T22:56:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/1946024351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b01402d0 0x7f50b0140780 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/1946024351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b0066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 -- 172.21.15.32:0/1946024351 >> 172.21.15.32:0/1946024351 conn(0x7f50b0063870 msgr2=0x7f50b0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 -- 172.21.15.32:0/1946024351 shutdown_connections 2022-08-01T22:56:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.471+0000 7f50b6bcd700 1 -- 172.21.15.32:0/1946024351 wait complete. 2022-08-01T22:56:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 Processor -- start 2022-08-01T22:56:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 -- start start 2022-08-01T22:56:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 0x7f50b0145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50b01402d0 0x7f50b014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00acb80 con 0x7f50b00a9da0 2022-08-01T22:56:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00ace20 con 0x7f50b01402d0 2022-08-01T22:56:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b014efc0 con 0x7f50b00b31d0 2022-08-01T22:56:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 0x7f50b0145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 0x7f50b0145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41210/0 (socket says 172.21.15.32:41210) 2022-08-01T22:56:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 -- 172.21.15.32:0/2583645098 learned_addr learned my addr 172.21.15.32:0/2583645098 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b53ca700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b63cc700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50b01402d0 0x7f50b014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 -- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 msgr2=0x7f50b014f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b014f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 -- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50b01402d0 msgr2=0x7f50b014f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50b01402d0 0x7f50b014f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 -- 172.21.15.32:0/2583645098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a8009530 con 0x7f50b00a9da0 2022-08-01T22:56:11.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b5bcb700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 0x7f50b0145800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f50ac00c000 tx=0x7f50ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50ac010780 con 0x7f50b00a9da0 2022-08-01T22:56:11.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac0108e0 con 0x7f50b00a9da0 2022-08-01T22:56:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b0146090 con 0x7f50b00a9da0 2022-08-01T22:56:11.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50ac010c40 con 0x7f50b00a9da0 2022-08-01T22:56:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.475+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b00aa350 con 0x7f50b00a9da0 2022-08-01T22:56:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.479+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b0066af0 con 0x7f50b00a9da0 2022-08-01T22:56:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.479+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f50ac01c460 con 0x7f50b00a9da0 2022-08-01T22:56:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.479+0000 7f50a2ffd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5090053950 0x7f5090055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.479+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f50ac06c410 con 0x7f50b00a9da0 2022-08-01T22:56:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.479+0000 7f50b53ca700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5090053950 0x7f5090055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.479+0000 7f50b53ca700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5090053950 0x7f5090055e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f50a400d000 tx=0x7f50a4007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.491+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f50ac043170 con 0x7f50b00a9da0 2022-08-01T22:56:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.615+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50b00574e0 con 0x7f50b00a9da0 2022-08-01T22:56:11.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50a2ffd700 1 -- 172.21.15.32:0/2583645098 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f50ac046790 con 0x7f50b00a9da0 2022-08-01T22:56:11.636 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5090053950 msgr2=0x7f5090055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5090053950 0x7f5090055e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f50a400d000 tx=0x7f50a4007a00).stop 2022-08-01T22:56:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 msgr2=0x7f50b0145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 0x7f50b0145800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f50ac00c000 tx=0x7f50ac006df0).stop 2022-08-01T22:56:11.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 shutdown_connections 2022-08-01T22:56:11.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5090053950 0x7f5090055e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50b01402d0 0x7f50b014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50b00b31d0 0x7f50b014f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 --2- 172.21.15.32:0/2583645098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50b00a9da0 0x7f50b0145800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 >> 172.21.15.32:0/2583645098 conn(0x7f50b0063870 msgr2=0x7f50b00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 shutdown_connections 2022-08-01T22:56:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.619+0000 7f50b6bcd700 1 -- 172.21.15.32:0/2583645098 wait complete. 2022-08-01T22:56:11.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 Processor -- start 2022-08-01T22:56:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 -- start start 2022-08-01T22:56:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b28b0 0x7f5bac13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bac141920 0x7f5bac13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac0aac20 con 0x7f5bac0b1180 2022-08-01T22:56:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac141db0 con 0x7f5bac141920 2022-08-01T22:56:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb2190700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac140e20 con 0x7f5bac0b28b0 2022-08-01T22:56:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb198f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bac141920 0x7f5bac13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb098d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bac0b1180 0x7f5bac141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56992/0 (socket says 172.21.15.32:56992) 2022-08-01T22:56:11.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb098d700 1 -- 172.21.15.32:0/1185574231 learned_addr learned my addr 172.21.15.32:0/1185574231 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.755+0000 7f5bb118e700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b28b0 0x7f5bac13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb198f700 1 -- 172.21.15.32:0/1185574231 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b28b0 msgr2=0x7f5bac13e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb198f700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b28b0 0x7f5bac13e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5ba3fff700 1 -- 172.21.15.32:0/1185574231 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292950809 0 0) 0x7f5ba8001110 con 0x7f5bac0b1180 2022-08-01T22:56:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb198f700 1 -- 172.21.15.32:0/1185574231 >> v1:172.21.15.32:6789/0 conn(0x7f5bac0b1180 legacy=0x7f5bac141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb198f700 1 -- 172.21.15.32:0/1185574231 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0b2c90 con 0x7f5bac141920 2022-08-01T22:56:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb118e700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b28b0 0x7f5bac13e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb198f700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac141920 0x7f5bac13e6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f5ba40032f0 tx=0x7f5ba40091d0).ready entity=mon.0 client_cookie=d457e008590e8642 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5ba3fff700 1 -- 172.21.15.32:0/1185574231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba400b6c0 con 0x7f5bac141920 2022-08-01T22:56:11.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5ba3fff700 1 -- 172.21.15.32:0/1185574231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba400b820 con 0x7f5bac141920 2022-08-01T22:56:11.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5ba3fff700 1 -- 172.21.15.32:0/1185574231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba400b990 con 0x7f5bac141920 2022-08-01T22:56:11.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 -- 172.21.15.32:0/1185574231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac141920 msgr2=0x7f5bac13e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac141920 0x7f5bac13e6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f5ba40032f0 tx=0x7f5ba40091d0).stop 2022-08-01T22:56:11.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 -- 172.21.15.32:0/1185574231 shutdown_connections 2022-08-01T22:56:11.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac141920 0x7f5bac13e6b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 --2- 172.21.15.32:0/1185574231 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b28b0 0x7f5bac13e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 -- 172.21.15.32:0/1185574231 >> 172.21.15.32:0/1185574231 conn(0x7f5bac060310 msgr2=0x7f5bac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 -- 172.21.15.32:0/1185574231 shutdown_connections 2022-08-01T22:56:11.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.759+0000 7f5bb2190700 1 -- 172.21.15.32:0/1185574231 wait complete. 2022-08-01T22:56:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 Processor -- start 2022-08-01T22:56:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 -- start start 2022-08-01T22:56:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b1180 0x7f5bac05c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 0x7f5bac05c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bac141920 0x7f5bac0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac150ea0 con 0x7f5bac0b28b0 2022-08-01T22:56:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac1424e0 con 0x7f5bac141920 2022-08-01T22:56:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb2190700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac142780 con 0x7f5bac0b1180 2022-08-01T22:56:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 0x7f5bac05c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 0x7f5bac05c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41226/0 (socket says 172.21.15.32:41226) 2022-08-01T22:56:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 -- 172.21.15.32:0/2769551912 learned_addr learned my addr 172.21.15.32:0/2769551912 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb198f700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bac141920 0x7f5bac0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb118e700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b1180 0x7f5bac05c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 -- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b1180 msgr2=0x7f5bac05c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b1180 0x7f5bac05c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 -- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bac141920 msgr2=0x7f5bac0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bac141920 0x7f5bac0b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 -- 172.21.15.32:0/2769551912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c00b180 con 0x7f5bac0b28b0 2022-08-01T22:56:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.763+0000 7f5bb098d700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 0x7f5bac05c660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5ba8000ed0 tx=0x7f5ba8007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba80070d0 con 0x7f5bac0b28b0 2022-08-01T22:56:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba8007230 con 0x7f5bac0b28b0 2022-08-01T22:56:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba4009470 con 0x7f5bac0b28b0 2022-08-01T22:56:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba8005610 con 0x7f5bac0b28b0 2022-08-01T22:56:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac1509c0 con 0x7f5bac0b28b0 2022-08-01T22:56:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5ba801a060 con 0x7f5bac0b28b0 2022-08-01T22:56:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac142940 con 0x7f5bac0b28b0 2022-08-01T22:56:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5ba27fc700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b940539e0 0x7f5b94055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5ba806b6d0 con 0x7f5bac0b28b0 2022-08-01T22:56:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5bb118e700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b940539e0 0x7f5b94055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.767+0000 7f5bb118e700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b940539e0 0x7f5b94055ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f5b9c00b000 tx=0x7f5b9c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.771+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ba8042380 con 0x7f5bac0b28b0 2022-08-01T22:56:11.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5bac0574e0 con 0x7f5bac0b28b0 2022-08-01T22:56:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5ba27fc700 1 -- 172.21.15.32:0/2769551912 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5ba8018070 con 0x7f5bac0b28b0 2022-08-01T22:56:11.908 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b940539e0 msgr2=0x7f5b94055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b940539e0 0x7f5b94055ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f5b9c00b000 tx=0x7f5b9c006df0).stop 2022-08-01T22:56:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 msgr2=0x7f5bac05c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 0x7f5bac05c660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5ba8000ed0 tx=0x7f5ba8007920).stop 2022-08-01T22:56:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 shutdown_connections 2022-08-01T22:56:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b940539e0 0x7f5b94055ea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bac141920 0x7f5bac0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac0b28b0 0x7f5bac05c660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 --2- 172.21.15.32:0/2769551912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5bac0b1180 0x7f5bac05c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 >> 172.21.15.32:0/2769551912 conn(0x7f5bac060310 msgr2=0x7f5bac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 shutdown_connections 2022-08-01T22:56:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:11.899+0000 7f5bb2190700 1 -- 172.21.15.32:0/2769551912 wait complete. 2022-08-01T22:56:11.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:12.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.019+0000 7f4e675dc700 1 Processor -- start 2022-08-01T22:56:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.019+0000 7f4e675dc700 1 -- start start 2022-08-01T22:56:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e60067180 0x7f4e60143a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e60067560 con 0x7f4e600b0870 2022-08-01T22:56:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e600b0160 con 0x7f4e600ad1c0 2022-08-01T22:56:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e600b0400 con 0x7f4e60067180 2022-08-01T22:56:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65b79700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e600b0870 0x7f4e600b1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57010/0 (socket says 172.21.15.32:57010) 2022-08-01T22:56:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65b79700 1 -- 172.21.15.32:0/316432143 learned_addr learned my addr 172.21.15.32:0/316432143 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65378700 1 --2- 172.21.15.32:0/316432143 >> v2:172.21.15.55:3300/0 conn(0x7f4e60067180 0x7f4e60143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271933145 0 0) 0x7f4e60067560 con 0x7f4e600b0870 2022-08-01T22:56:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e48003260 con 0x7f4e600b0870 2022-08-01T22:56:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045286672 0 0) 0x7f4e600b0160 con 0x7f4e600ad1c0 2022-08-01T22:56:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e60067560 con 0x7f4e600ad1c0 2022-08-01T22:56:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65378700 1 -- 172.21.15.32:0/316432143 >> v1:172.21.15.55:6789/0 conn(0x7f4e600ad1c0 legacy=0x7f4e60143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1470993980 0 0) 0x7f4e48003260 con 0x7f4e600b0870 2022-08-01T22:56:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65378700 1 -- 172.21.15.32:0/316432143 >> v1:172.21.15.32:6789/0 conn(0x7f4e600b0870 legacy=0x7f4e600b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65378700 1 -- 172.21.15.32:0/316432143 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600af530 con 0x7f4e60067180 2022-08-01T22:56:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e65378700 1 --2- 172.21.15.32:0/316432143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e60143a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4e58001490 tx=0x7f4e50006df0).ready entity=mon.1 client_cookie=32a3acdde4c6496a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e58001a00 con 0x7f4e60067180 2022-08-01T22:56:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e50000e90 con 0x7f4e60067180 2022-08-01T22:56:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e4ffff700 1 -- 172.21.15.32:0/316432143 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e50012460 con 0x7f4e60067180 2022-08-01T22:56:12.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- 172.21.15.32:0/316432143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 msgr2=0x7f4e60143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 --2- 172.21.15.32:0/316432143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e60143a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4e58001490 tx=0x7f4e50006df0).stop 2022-08-01T22:56:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- 172.21.15.32:0/316432143 shutdown_connections 2022-08-01T22:56:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 --2- 172.21.15.32:0/316432143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e60143a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- 172.21.15.32:0/316432143 >> 172.21.15.32:0/316432143 conn(0x7f4e6005f840 msgr2=0x7f4e6005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- 172.21.15.32:0/316432143 shutdown_connections 2022-08-01T22:56:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.023+0000 7f4e675dc700 1 -- 172.21.15.32:0/316432143 wait complete. 2022-08-01T22:56:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 Processor -- start 2022-08-01T22:56:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 -- start start 2022-08-01T22:56:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e600a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e600b0870 0x7f4e60153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 0x7f4e600a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60153a30 con 0x7f4e600b4830 2022-08-01T22:56:12.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600a0c40 con 0x7f4e60067180 2022-08-01T22:56:12.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60152b60 con 0x7f4e600b0870 2022-08-01T22:56:12.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 0x7f4e600a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 0x7f4e600a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41238/0 (socket says 172.21.15.32:41238) 2022-08-01T22:56:12.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 -- 172.21.15.32:0/1687718827 learned_addr learned my addr 172.21.15.32:0/1687718827 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:12.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65378700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e600a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e64b77700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e600b0870 0x7f4e60153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 -- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e600b0870 msgr2=0x7f4e60153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e600b0870 0x7f4e60153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 -- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 msgr2=0x7f4e600a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e600a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 -- 172.21.15.32:0/1687718827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5000b180 con 0x7f4e600b4830 2022-08-01T22:56:12.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e65b79700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 0x7f4e600a1190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4e58001be0 tx=0x7f4e58001a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e58004ab0 con 0x7f4e600b4830 2022-08-01T22:56:12.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58004c10 con 0x7f4e600b4830 2022-08-01T22:56:12.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e60152e00 con 0x7f4e600b4830 2022-08-01T22:56:12.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e58007170 con 0x7f4e600b4830 2022-08-01T22:56:12.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.027+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e600b4cc0 con 0x7f4e600b4830 2022-08-01T22:56:12.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.031+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e600574e0 con 0x7f4e600b4830 2022-08-01T22:56:12.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.031+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e5801b070 con 0x7f4e600b4830 2022-08-01T22:56:12.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.031+0000 7f4e4e7fc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e40053a70 0x7f4e40055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.031+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4e5806b8d0 con 0x7f4e600b4830 2022-08-01T22:56:12.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.031+0000 7f4e65378700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e40053a70 0x7f4e40055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.031+0000 7f4e65378700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e40053a70 0x7f4e40055f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4e5001a000 tx=0x7f4e50014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.035+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e58042520 con 0x7f4e600b4830 2022-08-01T22:56:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.159+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e600679a0 con 0x7f4e600b4830 2022-08-01T22:56:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e4e7fc700 1 -- 172.21.15.32:0/1687718827 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f4e58045b40 con 0x7f4e600b4830 2022-08-01T22:56:12.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:12.173 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e40053a70 msgr2=0x7f4e40055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e40053a70 0x7f4e40055f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4e5001a000 tx=0x7f4e50014000).stop 2022-08-01T22:56:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 msgr2=0x7f4e600a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 0x7f4e600a1190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4e58001be0 tx=0x7f4e58001a00).stop 2022-08-01T22:56:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 shutdown_connections 2022-08-01T22:56:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e40053a70 0x7f4e40055f30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e600b4830 0x7f4e600a1190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e600b0870 0x7f4e60153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 --2- 172.21.15.32:0/1687718827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e60067180 0x7f4e600a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 >> 172.21.15.32:0/1687718827 conn(0x7f4e6005f840 msgr2=0x7f4e600b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 shutdown_connections 2022-08-01T22:56:12.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.163+0000 7f4e675dc700 1 -- 172.21.15.32:0/1687718827 wait complete. 2022-08-01T22:56:12.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:12.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 Processor -- start 2022-08-01T22:56:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 -- start start 2022-08-01T22:56:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44d00b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f44d0147450 0x7f44d0143740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d00ae7e0 con 0x7f44d00b1ef0 2022-08-01T22:56:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00b1960 con 0x7f44d0147450 2022-08-01T22:56:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d7afc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b2470 con 0x7f44d00b1580 2022-08-01T22:56:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d5097700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44d00b1ef0 0x7f44d0146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57022/0 (socket says 172.21.15.32:57022) 2022-08-01T22:56:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d5097700 1 -- 172.21.15.32:0/2701515068 learned_addr learned my addr 172.21.15.32:0/2701515068 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:12.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d5898700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44d00b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d6099700 1 --2- 172.21.15.32:0/2701515068 >> v2:172.21.15.55:3300/0 conn(0x7f44d0147450 0x7f44d0143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223065206 0 0) 0x7f44d00ae7e0 con 0x7f44d00b1ef0 2022-08-01T22:56:12.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.287+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b8003260 con 0x7f44d00b1ef0 2022-08-01T22:56:12.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1924979162 0 0) 0x7f44b8003260 con 0x7f44d00b1ef0 2022-08-01T22:56:12.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 msgr2=0x7f44d00b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44d00b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 >> v2:172.21.15.55:3300/0 conn(0x7f44d0147450 msgr2=0x7f44d0143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 --2- 172.21.15.32:0/2701515068 >> v2:172.21.15.55:3300/0 conn(0x7f44d0147450 0x7f44d0143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00b2710 con 0x7f44d00b1ef0 2022-08-01T22:56:12.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f44cc001660 con 0x7f44d00b1ef0 2022-08-01T22:56:12.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 >> v1:172.21.15.32:6789/0 conn(0x7f44d00b1ef0 legacy=0x7f44d0146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8003750 0x7f44b8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44b8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b80049f0 0x7f44b8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b1960 con 0x7f44b8003750 2022-08-01T22:56:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b2470 con 0x7f44b80049f0 2022-08-01T22:56:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b800b750 con 0x7f44d00b1580 2022-08-01T22:56:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8003750 0x7f44b8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d6099700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b80049f0 0x7f44b8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5097700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44b8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 -- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 msgr2=0x7f44b8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44b8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 -- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b80049f0 msgr2=0x7f44b8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b80049f0 0x7f44b8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 -- 172.21.15.32:0/2701515068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8003260 con 0x7f44b8003750 2022-08-01T22:56:12.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5097700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d00b1580 0x7f44b8004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:12.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d5898700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8003750 0x7f44b8003b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f44c000c000 tx=0x7f44c0006df0).ready entity=mon.0 client_cookie=72c257251ec6dda7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c000e830 con 0x7f44b8003750 2022-08-01T22:56:12.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c000e990 con 0x7f44b8003750 2022-08-01T22:56:12.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d4896700 1 -- 172.21.15.32:0/2701515068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c000ecf0 con 0x7f44b8003750 2022-08-01T22:56:12.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d7afc700 1 -- 172.21.15.32:0/2701515068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8003750 msgr2=0x7f44b8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2701515068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8003750 0x7f44b8003b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f44c000c000 tx=0x7f44c0006df0).stop 2022-08-01T22:56:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d6099700 1 -- 172.21.15.32:0/2701515068 reap_dead start 2022-08-01T22:56:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d7afc700 1 -- 172.21.15.32:0/2701515068 shutdown_connections 2022-08-01T22:56:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d7afc700 1 -- 172.21.15.32:0/2701515068 >> 172.21.15.32:0/2701515068 conn(0x7f44d0060950 msgr2=0x7f44d0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d7afc700 1 -- 172.21.15.32:0/2701515068 shutdown_connections 2022-08-01T22:56:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.291+0000 7f44d7afc700 1 -- 172.21.15.32:0/2701515068 wait complete. 2022-08-01T22:56:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 Processor -- start 2022-08-01T22:56:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 -- start start 2022-08-01T22:56:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44d0143050 0x7f44d0141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 0x7f44d0141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d0152ee0 0x7f44d0153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00acdb0 con 0x7f44d00ad8d0 2022-08-01T22:56:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0143430 con 0x7f44d0143050 2022-08-01T22:56:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00ad480 con 0x7f44d0152ee0 2022-08-01T22:56:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 0x7f44d0141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 0x7f44d0141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41260/0 (socket says 172.21.15.32:41260) 2022-08-01T22:56:12.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 -- 172.21.15.32:0/2322803885 learned_addr learned my addr 172.21.15.32:0/2322803885 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:12.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5898700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44d0143050 0x7f44d0141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d6099700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d0152ee0 0x7f44d0153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 -- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d0152ee0 msgr2=0x7f44d0153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d0152ee0 0x7f44d0153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 -- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44d0143050 msgr2=0x7f44d0141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44d0143050 0x7f44d0141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 -- 172.21.15.32:0/2322803885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0063a60 con 0x7f44d00ad8d0 2022-08-01T22:56:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d5097700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 0x7f44d0141f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f44cc00b000 tx=0x7f44d00b2c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44cc010710 con 0x7f44d00ad8d0 2022-08-01T22:56:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc010870 con 0x7f44d00ad8d0 2022-08-01T22:56:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d0063d00 con 0x7f44d00ad8d0 2022-08-01T22:56:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44cc010bd0 con 0x7f44d00ad8d0 2022-08-01T22:56:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.295+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d00acfe0 con 0x7f44d00ad8d0 2022-08-01T22:56:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.299+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d00ad1a0 con 0x7f44d00ad8d0 2022-08-01T22:56:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.299+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f44cc010d30 con 0x7f44d00ad8d0 2022-08-01T22:56:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.299+0000 7f44beffd700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44b0053b30 0x7f44b0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.299+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f44cc06c120 con 0x7f44d00ad8d0 2022-08-01T22:56:12.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.299+0000 7f44d5898700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44b0053b30 0x7f44b0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.303+0000 7f44d5898700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44b0053b30 0x7f44b0055ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f44c001a000 tx=0x7f44c0019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.307+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44cc042e80 con 0x7f44d00ad8d0 2022-08-01T22:56:12.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.427+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f44d00b29b0 con 0x7f44d00ad8d0 2022-08-01T22:56:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.427+0000 7f44beffd700 1 -- 172.21.15.32:0/2322803885 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f44cc0464a0 con 0x7f44d00ad8d0 2022-08-01T22:56:12.440 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44b0053b30 msgr2=0x7f44b0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44b0053b30 0x7f44b0055ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f44c001a000 tx=0x7f44c0019000).stop 2022-08-01T22:56:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 msgr2=0x7f44d0141f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 0x7f44d0141f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f44cc00b000 tx=0x7f44d00b2c50).stop 2022-08-01T22:56:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 shutdown_connections 2022-08-01T22:56:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44b0053b30 0x7f44b0055ff0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44d0152ee0 0x7f44d0153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d00ad8d0 0x7f44d0141f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 --2- 172.21.15.32:0/2322803885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44d0143050 0x7f44d0141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 >> 172.21.15.32:0/2322803885 conn(0x7f44d0060950 msgr2=0x7f44d00b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 shutdown_connections 2022-08-01T22:56:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.431+0000 7f44d7afc700 1 -- 172.21.15.32:0/2322803885 wait complete. 2022-08-01T22:56:12.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41ab311700 1 Processor -- start 2022-08-01T22:56:12.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41ab311700 1 -- start start 2022-08-01T22:56:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41ab311700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f41a40a8c50 0x7f41a4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41ab311700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a40a7250 con 0x7f41a40b01a0 2022-08-01T22:56:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41ab311700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a40a7480 con 0x7f41a40b2710 2022-08-01T22:56:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41ab311700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a40a7720 con 0x7f41a40a8c50 2022-08-01T22:56:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a9b0e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41a40b01a0 0x7f41a40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57042/0 (socket says 172.21.15.32:57042) 2022-08-01T22:56:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a9b0e700 1 -- 172.21.15.32:0/1411925047 learned_addr learned my addr 172.21.15.32:0/1411925047 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41aab10700 1 --2- 172.21.15.32:0/1411925047 >> v2:172.21.15.55:3300/0 conn(0x7f41a40a8c50 0x7f41a4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093503167 0 0) 0x7f41a40a7250 con 0x7f41a40b01a0 2022-08-01T22:56:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f418c003260 con 0x7f41a40b01a0 2022-08-01T22:56:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329174366 0 0) 0x7f418c003260 con 0x7f41a40b01a0 2022-08-01T22:56:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 >> v2:172.21.15.55:3300/0 conn(0x7f41a40a8c50 msgr2=0x7f41a4152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 --2- 172.21.15.32:0/1411925047 >> v2:172.21.15.55:3300/0 conn(0x7f41a40a8c50 0x7f41a4152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 >> v1:172.21.15.55:6789/0 conn(0x7f41a40b2710 legacy=0x7f41a414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a40a79c0 con 0x7f41a40b01a0 2022-08-01T22:56:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f41a00019e0 con 0x7f41a40b01a0 2022-08-01T22:56:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 >> v1:172.21.15.32:6789/0 conn(0x7f41a40b01a0 legacy=0x7f41a40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f418c003750 0x7f418c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418c004500 0x7f418c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f418c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40a7720 con 0x7f418c004500 2022-08-01T22:56:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c00bd20 con 0x7f418c003750 2022-08-01T22:56:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c00bee0 con 0x7f418c004e70 2022-08-01T22:56:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41a9b0e700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418c004500 0x7f418c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41aab10700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f418c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.567+0000 7f41aa30f700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f418c003750 0x7f418c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 -- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f418c003750 msgr2=0x7f418c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f418c003750 0x7f418c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 -- 172.21.15.32:0/1411925047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418c004500 msgr2=0x7f418c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418c004500 0x7f418c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 -- 172.21.15.32:0/1411925047 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c003260 con 0x7f418c004e70 2022-08-01T22:56:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 -- 172.21.15.32:0/1411925047 reap_dead start 2022-08-01T22:56:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41aab10700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f418c00b750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f419c00c000 tx=0x7f419c006df0).ready entity=mon.2 client_cookie=906d4f89af5043e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419c016070 con 0x7f418c004e70 2022-08-01T22:56:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c00e5f0 con 0x7f418c004e70 2022-08-01T22:56:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41a930d700 1 -- 172.21.15.32:0/1411925047 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419c00e950 con 0x7f418c004e70 2022-08-01T22:56:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 -- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 msgr2=0x7f418c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f418c00b750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f419c00c000 tx=0x7f419c006df0).stop 2022-08-01T22:56:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 -- 172.21.15.32:0/1411925047 shutdown_connections 2022-08-01T22:56:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 --2- 172.21.15.32:0/1411925047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f418c00b750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 -- 172.21.15.32:0/1411925047 >> 172.21.15.32:0/1411925047 conn(0x7f41a4060310 msgr2=0x7f41a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 -- 172.21.15.32:0/1411925047 shutdown_connections 2022-08-01T22:56:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.571+0000 7f41ab311700 1 -- 172.21.15.32:0/1411925047 wait complete. 2022-08-01T22:56:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 Processor -- start 2022-08-01T22:56:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 -- start start 2022-08-01T22:56:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f41a40b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 0x7f41a4063c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41a4156740 0x7f41a4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40abef0 con 0x7f41a40aba50 2022-08-01T22:56:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40ab0b0 con 0x7f41a4156740 2022-08-01T22:56:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40ab350 con 0x7f418c004e70 2022-08-01T22:56:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 0x7f41a4063c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 0x7f41a4063c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41280/0 (socket says 172.21.15.32:41280) 2022-08-01T22:56:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 -- 172.21.15.32:0/3167963427 learned_addr learned my addr 172.21.15.32:0/3167963427 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41aab10700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41a4156740 0x7f41a4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41aa30f700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f41a40b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 -- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 msgr2=0x7f41a40b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f41a40b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 -- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41a4156740 msgr2=0x7f41a4156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41a4156740 0x7f41a4156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 -- 172.21.15.32:0/3167963427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a40aa370 con 0x7f41a40aba50 2022-08-01T22:56:12.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41a9b0e700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 0x7f41a4063c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f41a000d000 tx=0x7f41a40a7b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41a000f880 con 0x7f41a40aba50 2022-08-01T22:56:12.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a000f9e0 con 0x7f41a40aba50 2022-08-01T22:56:12.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41a000fd40 con 0x7f41a40aba50 2022-08-01T22:56:12.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a41570e0 con 0x7f41a40aba50 2022-08-01T22:56:12.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.575+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a40a94f0 con 0x7f41a40aba50 2022-08-01T22:56:12.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.579+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a40ab510 con 0x7f41a40aba50 2022-08-01T22:56:12.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.579+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f41a001c360 con 0x7f41a40aba50 2022-08-01T22:56:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.579+0000 7f41937fe700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4184053b00 0x7f4184055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.579+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f41a006bc60 con 0x7f41a40aba50 2022-08-01T22:56:12.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.579+0000 7f41aa30f700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4184053b00 0x7f4184055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.579+0000 7f41aa30f700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4184053b00 0x7f4184055fc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4194003000 tx=0x7f4194007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.587+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41a00429c0 con 0x7f41a40aba50 2022-08-01T22:56:12.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.711+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f41a40574e0 con 0x7f41a40aba50 2022-08-01T22:56:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.711+0000 7f41937fe700 1 -- 172.21.15.32:0/3167963427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f41a0019030 con 0x7f41a40aba50 2022-08-01T22:56:12.719 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4184053b00 msgr2=0x7f4184055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4184053b00 0x7f4184055fc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4194003000 tx=0x7f4194007a00).stop 2022-08-01T22:56:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 msgr2=0x7f41a4063c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 0x7f41a4063c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f41a000d000 tx=0x7f41a40a7b50).stop 2022-08-01T22:56:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 shutdown_connections 2022-08-01T22:56:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4184053b00 0x7f4184055fc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41a4156740 0x7f41a4156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40aba50 0x7f41a4063c80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 --2- 172.21.15.32:0/3167963427 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f418c004e70 0x7f41a40b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 >> 172.21.15.32:0/3167963427 conn(0x7f41a4060310 msgr2=0x7f41a4148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 shutdown_connections 2022-08-01T22:56:12.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.715+0000 7f41ab311700 1 -- 172.21.15.32:0/3167963427 wait complete. 2022-08-01T22:56:12.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 Processor -- start 2022-08-01T22:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- start start 2022-08-01T22:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31600b1250 0x7f31600b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3160140d30 0x7f31600b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600ac950 con 0x7f31600acd00 2022-08-01T22:56:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31600ac1b0 con 0x7f3160140d30 2022-08-01T22:56:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31601411a0 con 0x7f31600b1250 2022-08-01T22:56:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f31656d6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3160140d30 0x7f31600b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f315ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31600acd00 0x7f316013d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57062/0 (socket says 172.21.15.32:57062) 2022-08-01T22:56:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f315ffff700 1 -- 172.21.15.32:0/1503468621 learned_addr learned my addr 172.21.15.32:0/1503468621 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3164ed5700 1 --2- 172.21.15.32:0/1503468621 >> v2:172.21.15.55:3300/0 conn(0x7f31600b1250 0x7f31600b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f31656d6700 1 -- 172.21.15.32:0/1503468621 >> v2:172.21.15.55:3300/0 conn(0x7f31600b1250 msgr2=0x7f31600b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f31656d6700 1 --2- 172.21.15.32:0/1503468621 >> v2:172.21.15.55:3300/0 conn(0x7f31600b1250 0x7f31600b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f31656d6700 1 -- 172.21.15.32:0/1503468621 >> v1:172.21.15.32:6789/0 conn(0x7f31600acd00 legacy=0x7f316013d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f315f7fe700 1 -- 172.21.15.32:0/1503468621 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156288928 0 0) 0x7f31600ac950 con 0x7f31600acd00 2022-08-01T22:56:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f31656d6700 1 -- 172.21.15.32:0/1503468621 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600ac410 con 0x7f3160140d30 2022-08-01T22:56:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f31656d6700 1 --2- 172.21.15.32:0/1503468621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3160140d30 0x7f31600b2670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f31580032f0 tx=0x7f31580091d0).ready entity=mon.0 client_cookie=6e39c1e8eaee01a9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f315f7fe700 1 -- 172.21.15.32:0/1503468621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f315800f8a0 con 0x7f3160140d30 2022-08-01T22:56:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f315f7fe700 1 -- 172.21.15.32:0/1503468621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315800fa00 con 0x7f3160140d30 2022-08-01T22:56:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f315f7fe700 1 -- 172.21.15.32:0/1503468621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f315800fd60 con 0x7f3160140d30 2022-08-01T22:56:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- 172.21.15.32:0/1503468621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3160140d30 msgr2=0x7f31600b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 --2- 172.21.15.32:0/1503468621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3160140d30 0x7f31600b2670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f31580032f0 tx=0x7f31580091d0).stop 2022-08-01T22:56:12.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- 172.21.15.32:0/1503468621 shutdown_connections 2022-08-01T22:56:12.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 --2- 172.21.15.32:0/1503468621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3160140d30 0x7f31600b2670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 --2- 172.21.15.32:0/1503468621 >> v2:172.21.15.55:3300/0 conn(0x7f31600b1250 0x7f31600b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- 172.21.15.32:0/1503468621 >> 172.21.15.32:0/1503468621 conn(0x7f316005f840 msgr2=0x7f316005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:12.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- 172.21.15.32:0/1503468621 shutdown_connections 2022-08-01T22:56:12.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.851+0000 7f3167139700 1 -- 172.21.15.32:0/1503468621 wait complete. 2022-08-01T22:56:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 Processor -- start 2022-08-01T22:56:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 -- start start 2022-08-01T22:56:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 0x7f316009ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31600b1250 0x7f316013eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3160140d30 0x7f316013f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3160065530 con 0x7f31600acd00 2022-08-01T22:56:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316009fc70 con 0x7f3160140d30 2022-08-01T22:56:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31600677b0 con 0x7f31600b1250 2022-08-01T22:56:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 0x7f316009ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 0x7f316009ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41294/0 (socket says 172.21.15.32:41294) 2022-08-01T22:56:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 -- 172.21.15.32:0/578997367 learned_addr learned my addr 172.21.15.32:0/578997367 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f315ffff700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31600b1250 0x7f316013eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f31656d6700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3160140d30 0x7f316013f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 -- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31600b1250 msgr2=0x7f316013eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31600b1250 0x7f316013eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 -- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3160140d30 msgr2=0x7f316013f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3160140d30 0x7f316013f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 -- 172.21.15.32:0/578997367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158009470 con 0x7f31600acd00 2022-08-01T22:56:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f31656d6700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3160140d30 0x7f316013f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3164ed5700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 0x7f316009ed80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f315400c000 tx=0x7f3154006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3154010780 con 0x7f31600acd00 2022-08-01T22:56:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31540108e0 con 0x7f31600acd00 2022-08-01T22:56:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3160064b90 con 0x7f31600acd00 2022-08-01T22:56:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3154010c40 con 0x7f31600acd00 2022-08-01T22:56:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.855+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316013f510 con 0x7f31600acd00 2022-08-01T22:56:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.859+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3160065030 con 0x7f31600acd00 2022-08-01T22:56:12.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.859+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f315401c460 con 0x7f31600acd00 2022-08-01T22:56:12.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.859+0000 7f315dffb700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31400539e0 0x7f3140055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:12.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.859+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f315406c570 con 0x7f31600acd00 2022-08-01T22:56:12.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.859+0000 7f315ffff700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31400539e0 0x7f3140055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:12.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.859+0000 7f315ffff700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31400539e0 0x7f3140055ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f314c00d000 tx=0x7f314c0011a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:12.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.863+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3160065030 con 0x7f31600acd00 2022-08-01T22:56:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.987+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3160065030 con 0x7f31600acd00 2022-08-01T22:56:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.987+0000 7f315dffb700 1 -- 172.21.15.32:0/578997367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3160065030 con 0x7f31600acd00 2022-08-01T22:56:12.997 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31400539e0 msgr2=0x7f3140055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31400539e0 0x7f3140055ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f314c00d000 tx=0x7f314c0011a0).stop 2022-08-01T22:56:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 msgr2=0x7f316009ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 0x7f316009ed80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f315400c000 tx=0x7f3154006df0).stop 2022-08-01T22:56:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 shutdown_connections 2022-08-01T22:56:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31400539e0 0x7f3140055ea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3160140d30 0x7f316013f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31600b1250 0x7f316013eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 --2- 172.21.15.32:0/578997367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31600acd00 0x7f316009ed80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 >> 172.21.15.32:0/578997367 conn(0x7f316005f840 msgr2=0x7f31600b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 shutdown_connections 2022-08-01T22:56:13.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:12.991+0000 7f3167139700 1 -- 172.21.15.32:0/578997367 wait complete. 2022-08-01T22:56:13.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:13.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.111+0000 7f9e3b3eb700 1 Processor -- start 2022-08-01T22:56:13.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.111+0000 7f9e3b3eb700 1 -- start start 2022-08-01T22:56:13.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e340614b0 0x7f9e34061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e340ad750 0x7f9e340b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e340aee80 con 0x7f9e341405d0 2022-08-01T22:56:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340b70b0 con 0x7f9e340ad750 2022-08-01T22:56:13.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340af0b0 con 0x7f9e340614b0 2022-08-01T22:56:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e340ad750 0x7f9e340b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e340ad750 0x7f9e340b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41300/0 (socket says 172.21.15.32:41300) 2022-08-01T22:56:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 -- 172.21.15.32:0/1051415793 learned_addr learned my addr 172.21.15.32:0/1051415793 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3abea700 1 --1- 172.21.15.32:0/1051415793 >> v1:172.21.15.32:6789/0 conn(0x7f9e341405d0 0x7f9e341466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57078/0 (socket says 172.21.15.32:57078) 2022-08-01T22:56:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3a3e9700 1 --2- 172.21.15.32:0/1051415793 >> v2:172.21.15.55:3300/0 conn(0x7f9e340614b0 0x7f9e34061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 -- 172.21.15.32:0/1051415793 >> v2:172.21.15.55:3300/0 conn(0x7f9e340614b0 msgr2=0x7f9e34061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 --2- 172.21.15.32:0/1051415793 >> v2:172.21.15.55:3300/0 conn(0x7f9e340614b0 0x7f9e34061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 -- 172.21.15.32:0/1051415793 >> v1:172.21.15.32:6789/0 conn(0x7f9e341405d0 legacy=0x7f9e341466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 -- 172.21.15.32:0/1051415793 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340af350 con 0x7f9e340ad750 2022-08-01T22:56:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e39be8700 1 --2- 172.21.15.32:0/1051415793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340ad750 0x7f9e340b9ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f9e340aed30 tx=0x7f9e28006eb0).ready entity=mon.0 client_cookie=8a89c17489d2e2d3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e393e7700 1 -- 172.21.15.32:0/1051415793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2800d610 con 0x7f9e340ad750 2022-08-01T22:56:13.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e393e7700 1 -- 172.21.15.32:0/1051415793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28000e90 con 0x7f9e340ad750 2022-08-01T22:56:13.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e393e7700 1 -- 172.21.15.32:0/1051415793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2800d960 con 0x7f9e340ad750 2022-08-01T22:56:13.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/1051415793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340ad750 msgr2=0x7f9e340b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/1051415793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340ad750 0x7f9e340b9ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f9e340aed30 tx=0x7f9e28006eb0).stop 2022-08-01T22:56:13.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/1051415793 shutdown_connections 2022-08-01T22:56:13.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/1051415793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340ad750 0x7f9e340b9ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/1051415793 >> v2:172.21.15.55:3300/0 conn(0x7f9e340614b0 0x7f9e34061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/1051415793 >> 172.21.15.32:0/1051415793 conn(0x7f9e34066f70 msgr2=0x7f9e34067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/1051415793 shutdown_connections 2022-08-01T22:56:13.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.115+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/1051415793 wait complete. 2022-08-01T22:56:13.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 Processor -- start 2022-08-01T22:56:13.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 -- start start 2022-08-01T22:56:13.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 0x7f9e340b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e340ad750 0x7f9e3409eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e341405d0 0x7f9e3409f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340b2540 con 0x7f9e340614b0 2022-08-01T22:56:13.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340b3360 con 0x7f9e340ad750 2022-08-01T22:56:13.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3b3eb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340b2770 con 0x7f9e341405d0 2022-08-01T22:56:13.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3a3e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 0x7f9e340b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3a3e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 0x7f9e340b7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41306/0 (socket says 172.21.15.32:41306) 2022-08-01T22:56:13.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3a3e9700 1 -- 172.21.15.32:0/2601744612 learned_addr learned my addr 172.21.15.32:0/2601744612 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e39be8700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e340ad750 0x7f9e3409eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.119+0000 7f9e3abea700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e341405d0 0x7f9e3409f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3a3e9700 1 -- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e341405d0 msgr2=0x7f9e3409f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3a3e9700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e341405d0 0x7f9e3409f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3a3e9700 1 -- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e340ad750 msgr2=0x7f9e3409eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3a3e9700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e340ad750 0x7f9e3409eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3a3e9700 1 -- 172.21.15.32:0/2601744612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2800b1e0 con 0x7f9e340614b0 2022-08-01T22:56:13.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e39be8700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e340ad750 0x7f9e3409eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:13.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3a3e9700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 0x7f9e340b7ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f9e300030b0 tx=0x7f9e30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e30011780 con 0x7f9e340614b0 2022-08-01T22:56:13.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e300118e0 con 0x7f9e340614b0 2022-08-01T22:56:13.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e340b2a10 con 0x7f9e340614b0 2022-08-01T22:56:13.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e30011c40 con 0x7f9e340614b0 2022-08-01T22:56:13.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e340b2e50 con 0x7f9e340614b0 2022-08-01T22:56:13.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e340b34d0 con 0x7f9e340614b0 2022-08-01T22:56:13.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.123+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e3001c460 con 0x7f9e340614b0 2022-08-01T22:56:13.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.127+0000 7f9e277fe700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e14053950 0x7f9e14055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.127+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9e3006c620 con 0x7f9e340614b0 2022-08-01T22:56:13.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.127+0000 7f9e39be8700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e14053950 0x7f9e14055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.127+0000 7f9e39be8700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e14053950 0x7f9e14055e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f9e2801a000 tx=0x7f9e28015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.131+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e340b34d0 con 0x7f9e340614b0 2022-08-01T22:56:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.259+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e340b34d0 con 0x7f9e340614b0 2022-08-01T22:56:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e277fe700 1 -- 172.21.15.32:0/2601744612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9e340b34d0 con 0x7f9e340614b0 2022-08-01T22:56:13.277 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e14053950 msgr2=0x7f9e14055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e14053950 0x7f9e14055e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f9e2801a000 tx=0x7f9e28015000).stop 2022-08-01T22:56:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 msgr2=0x7f9e340b7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 0x7f9e340b7ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f9e300030b0 tx=0x7f9e30008040).stop 2022-08-01T22:56:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 shutdown_connections 2022-08-01T22:56:13.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e14053950 0x7f9e14055e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e341405d0 0x7f9e3409f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e340ad750 0x7f9e3409eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 --2- 172.21.15.32:0/2601744612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e340614b0 0x7f9e340b7ea0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 >> 172.21.15.32:0/2601744612 conn(0x7f9e34066f70 msgr2=0x7f9e340b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 shutdown_connections 2022-08-01T22:56:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.263+0000 7f9e3b3eb700 1 -- 172.21.15.32:0/2601744612 wait complete. 2022-08-01T22:56:13.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:13.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.383+0000 7f5ea1290700 1 Processor -- start 2022-08-01T22:56:13.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.383+0000 7f5ea1290700 1 -- start start 2022-08-01T22:56:13.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e9c148530 0x7f5e9c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0a7250 0x7f5e9c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9c0b04e0 con 0x7f5e9c0b1f40 2022-08-01T22:56:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c0a9250 con 0x7f5e9c148530 2022-08-01T22:56:13.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0b0780 con 0x7f5e9c0a7250 2022-08-01T22:56:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e9c148530 0x7f5e9c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e9c148530 0x7f5e9c0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41314/0 (socket says 172.21.15.32:41314) 2022-08-01T22:56:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 -- 172.21.15.32:0/938317095 learned_addr learned my addr 172.21.15.32:0/938317095 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9b7fe700 1 --2- 172.21.15.32:0/938317095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0a7250 0x7f5e9c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 -- 172.21.15.32:0/938317095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0a7250 msgr2=0x7f5e9c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 --2- 172.21.15.32:0/938317095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0a7250 0x7f5e9c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 -- 172.21.15.32:0/938317095 >> v1:172.21.15.55:6789/0 conn(0x7f5e9c0b1f40 legacy=0x7f5e9c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 -- 172.21.15.32:0/938317095 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0b0a20 con 0x7f5e9c148530 2022-08-01T22:56:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e9a7fc700 1 --2- 172.21.15.32:0/938317095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c0b1160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f5e9c0b3300 tx=0x7f5e8c009290).ready entity=mon.0 client_cookie=227927ddf66cc6d5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e99ffb700 1 -- 172.21.15.32:0/938317095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8c00b820 con 0x7f5e9c148530 2022-08-01T22:56:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e99ffb700 1 -- 172.21.15.32:0/938317095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8c00b980 con 0x7f5e9c148530 2022-08-01T22:56:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5e99ffb700 1 -- 172.21.15.32:0/938317095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8c00bb30 con 0x7f5e9c148530 2022-08-01T22:56:13.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- 172.21.15.32:0/938317095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 msgr2=0x7f5e9c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 --2- 172.21.15.32:0/938317095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c0b1160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f5e9c0b3300 tx=0x7f5e8c009290).stop 2022-08-01T22:56:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- 172.21.15.32:0/938317095 shutdown_connections 2022-08-01T22:56:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 --2- 172.21.15.32:0/938317095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c0b1160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 --2- 172.21.15.32:0/938317095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0a7250 0x7f5e9c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- 172.21.15.32:0/938317095 >> 172.21.15.32:0/938317095 conn(0x7f5e9c05f840 msgr2=0x7f5e9c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- 172.21.15.32:0/938317095 shutdown_connections 2022-08-01T22:56:13.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 -- 172.21.15.32:0/938317095 wait complete. 2022-08-01T22:56:13.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.387+0000 7f5ea1290700 1 Processor -- start 2022-08-01T22:56:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 -- start start 2022-08-01T22:56:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e9c0a7250 0x7f5e9c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0b1f40 0x7f5e9c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c149a20 con 0x7f5e9c148530 2022-08-01T22:56:13.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c149cc0 con 0x7f5e9c0a7250 2022-08-01T22:56:13.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c149f60 con 0x7f5e9c0b1f40 2022-08-01T22:56:13.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c15a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41324/0 (socket says 172.21.15.32:41324) 2022-08-01T22:56:13.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/2957883755 learned_addr learned my addr 172.21.15.32:0/2957883755 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9affd700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e9c0a7250 0x7f5e9c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9a7fc700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0b1f40 0x7f5e9c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0b1f40 msgr2=0x7f5e9c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0b1f40 0x7f5e9c0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e9c0a7250 msgr2=0x7f5e9c14b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e9c0a7250 0x7f5e9c14b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/2957883755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c009530 con 0x7f5e9c148530 2022-08-01T22:56:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e9b7fe700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c15a360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f5e9400a000 tx=0x7f5e94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e94011780 con 0x7f5e9c148530 2022-08-01T22:56:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e940118e0 con 0x7f5e9c148530 2022-08-01T22:56:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c14b430 con 0x7f5e9c148530 2022-08-01T22:56:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e94011c40 con 0x7f5e9c148530 2022-08-01T22:56:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.391+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c0ae4b0 con 0x7f5e9c148530 2022-08-01T22:56:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.395+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c0574e0 con 0x7f5e9c148530 2022-08-01T22:56:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.395+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e94012420 con 0x7f5e9c148530 2022-08-01T22:56:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.395+0000 7f5e8bfff700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053a70 0x7f5e78055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.395+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e9406be50 con 0x7f5e9c148530 2022-08-01T22:56:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.395+0000 7f5e9affd700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053a70 0x7f5e78055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.395+0000 7f5e9affd700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053a70 0x7f5e78055f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f5e9000d000 tx=0x7f5e90007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.399+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e94042aa0 con 0x7f5e9c148530 2022-08-01T22:56:13.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.527+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e9c14b5c0 con 0x7f5e9c148530 2022-08-01T22:56:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.527+0000 7f5e8bfff700 1 -- 172.21.15.32:0/2957883755 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5e940460c0 con 0x7f5e9c148530 2022-08-01T22:56:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:13.536 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053a70 msgr2=0x7f5e78055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053a70 0x7f5e78055f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f5e9000d000 tx=0x7f5e90007b70).stop 2022-08-01T22:56:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 msgr2=0x7f5e9c15a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c15a360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f5e9400a000 tx=0x7f5e94008040).stop 2022-08-01T22:56:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 shutdown_connections 2022-08-01T22:56:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053a70 0x7f5e78055f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e9c148530 0x7f5e9c15a360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9c0b1f40 0x7f5e9c0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 --2- 172.21.15.32:0/2957883755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e9c0a7250 0x7f5e9c14b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 >> 172.21.15.32:0/2957883755 conn(0x7f5e9c05f840 msgr2=0x7f5e9c0b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 shutdown_connections 2022-08-01T22:56:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.531+0000 7f5ea1290700 1 -- 172.21.15.32:0/2957883755 wait complete. 2022-08-01T22:56:13.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:13.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:13.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 Processor -- start 2022-08-01T22:56:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 -- start start 2022-08-01T22:56:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7cf00ad750 0x7f7cf0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cf01405d0 0x7f7cf0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf00aee80 con 0x7f7cf00614b0 2022-08-01T22:56:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf00b70b0 con 0x7f7cf01405d0 2022-08-01T22:56:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cf4dc0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf00af0b0 con 0x7f7cf00ad750 2022-08-01T22:56:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cf01405d0 0x7f7cf0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cef7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7cf00614b0 0x7f7cf00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57102/0 (socket says 172.21.15.32:57102) 2022-08-01T22:56:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cef7fe700 1 -- 172.21.15.32:0/240726970 learned_addr learned my addr 172.21.15.32:0/240726970 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:13.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceeffd700 1 --2- 172.21.15.32:0/240726970 >> v2:172.21.15.55:3300/0 conn(0x7f7cf00ad750 0x7f7cf0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cee7fc700 1 -- 172.21.15.32:0/240726970 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366495056 0 0) 0x7f7cf00aee80 con 0x7f7cf00614b0 2022-08-01T22:56:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cee7fc700 1 -- 172.21.15.32:0/240726970 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cdc003260 con 0x7f7cf00614b0 2022-08-01T22:56:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7cee7fc700 1 -- 172.21.15.32:0/240726970 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201052685 0 0) 0x7f7cdc003260 con 0x7f7cf00614b0 2022-08-01T22:56:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceffff700 1 -- 172.21.15.32:0/240726970 >> v2:172.21.15.55:3300/0 conn(0x7f7cf00ad750 msgr2=0x7f7cf0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceffff700 1 --2- 172.21.15.32:0/240726970 >> v2:172.21.15.55:3300/0 conn(0x7f7cf00ad750 0x7f7cf0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceffff700 1 -- 172.21.15.32:0/240726970 >> v1:172.21.15.32:6789/0 conn(0x7f7cf00614b0 legacy=0x7f7cf00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceffff700 1 -- 172.21.15.32:0/240726970 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf00af350 con 0x7f7cf01405d0 2022-08-01T22:56:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.655+0000 7f7ceeffd700 1 --2- 172.21.15.32:0/240726970 >> v2:172.21.15.55:3300/0 conn(0x7f7cf00ad750 0x7f7cf0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceffff700 1 --2- 172.21.15.32:0/240726970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf01405d0 0x7f7cf0140a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7ce4001800 tx=0x7f7ce80091d0).ready entity=mon.0 client_cookie=e8d48ce3918ce539 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cee7fc700 1 -- 172.21.15.32:0/240726970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce4001660 con 0x7f7cf01405d0 2022-08-01T22:56:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cee7fc700 1 -- 172.21.15.32:0/240726970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce800f510 con 0x7f7cf01405d0 2022-08-01T22:56:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cee7fc700 1 -- 172.21.15.32:0/240726970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce800f680 con 0x7f7cf01405d0 2022-08-01T22:56:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/240726970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf01405d0 msgr2=0x7f7cf0140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/240726970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf01405d0 0x7f7cf0140a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7ce4001800 tx=0x7f7ce80091d0).stop 2022-08-01T22:56:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/240726970 shutdown_connections 2022-08-01T22:56:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/240726970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf01405d0 0x7f7cf0140a80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/240726970 >> v2:172.21.15.55:3300/0 conn(0x7f7cf00ad750 0x7f7cf0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/240726970 >> 172.21.15.32:0/240726970 conn(0x7f7cf0066f70 msgr2=0x7f7cf0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/240726970 shutdown_connections 2022-08-01T22:56:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/240726970 wait complete. 2022-08-01T22:56:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 Processor -- start 2022-08-01T22:56:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- start start 2022-08-01T22:56:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cf00614b0 0x7f7cf00b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 0x7f7cf0155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cf01405d0 0x7f7cf01560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0144020 con 0x7f7cf00ad750 2022-08-01T22:56:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0141000 con 0x7f7cf01405d0 2022-08-01T22:56:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf01534f0 con 0x7f7cf00614b0 2022-08-01T22:56:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 0x7f7cf0155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 0x7f7cf0155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41336/0 (socket says 172.21.15.32:41336) 2022-08-01T22:56:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 -- 172.21.15.32:0/2612786359 learned_addr learned my addr 172.21.15.32:0/2612786359 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cef7fe700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cf00614b0 0x7f7cf00b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceffff700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cf01405d0 0x7f7cf01560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 -- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cf00614b0 msgr2=0x7f7cf00b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cf00614b0 0x7f7cf00b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 -- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cf01405d0 msgr2=0x7f7cf01560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cf01405d0 0x7f7cf01560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 -- 172.21.15.32:0/2612786359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce000b180 con 0x7f7cf00ad750 2022-08-01T22:56:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7ceeffd700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 0x7f7cf0155be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f7cf0141750 tx=0x7f7ce0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce000ca40 con 0x7f7cf00ad750 2022-08-01T22:56:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce8009470 con 0x7f7cf00ad750 2022-08-01T22:56:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.659+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce000cba0 con 0x7f7cf00ad750 2022-08-01T22:56:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce000ee80 con 0x7f7cf00ad750 2022-08-01T22:56:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf0141230 con 0x7f7cf00ad750 2022-08-01T22:56:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf00ad070 con 0x7f7cf00ad750 2022-08-01T22:56:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ce001c460 con 0x7f7cf00ad750 2022-08-01T22:56:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cecff9700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cd00539e0 0x7f7cd0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7ce006c780 con 0x7f7cf00ad750 2022-08-01T22:56:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cef7fe700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cd00539e0 0x7f7cd0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.663+0000 7f7cef7fe700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cd00539e0 0x7f7cd0055ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f7ce400b000 tx=0x7f7ce4001340).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.671+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ce0043440 con 0x7f7cf00ad750 2022-08-01T22:56:13.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.795+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7cf00af5f0 con 0x7f7cf00ad750 2022-08-01T22:56:13.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.799+0000 7f7cecff9700 1 -- 172.21.15.32:0/2612786359 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7ce0046a60 con 0x7f7cf00ad750 2022-08-01T22:56:13.805 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.799+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cd00539e0 msgr2=0x7f7cd0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.799+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cd00539e0 0x7f7cd0055ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f7ce400b000 tx=0x7f7ce4001340).stop 2022-08-01T22:56:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.799+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 msgr2=0x7f7cf0155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.799+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 0x7f7cf0155be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f7cf0141750 tx=0x7f7ce0006df0).stop 2022-08-01T22:56:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 shutdown_connections 2022-08-01T22:56:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cd00539e0 0x7f7cd0055ea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cf01405d0 0x7f7cf01560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf00ad750 0x7f7cf0155be0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 --2- 172.21.15.32:0/2612786359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cf00614b0 0x7f7cf00b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 >> 172.21.15.32:0/2612786359 conn(0x7f7cf0066f70 msgr2=0x7f7cf00b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 shutdown_connections 2022-08-01T22:56:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.803+0000 7f7cf4dc0700 1 -- 172.21.15.32:0/2612786359 wait complete. 2022-08-01T22:56:13.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 Processor -- start 2022-08-01T22:56:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- start start 2022-08-01T22:56:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faac40614b0 0x7faac4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faac41405d0 0x7faac4140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40aee80 con 0x7faac40ad750 2022-08-01T22:56:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40b70b0 con 0x7faac40614b0 2022-08-01T22:56:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40af0b0 con 0x7faac41405d0 2022-08-01T22:56:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faac40614b0 0x7faac4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faac40614b0 0x7faac4061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41342/0 (socket says 172.21.15.32:41342) 2022-08-01T22:56:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 -- 172.21.15.32:0/3883430719 learned_addr learned my addr 172.21.15.32:0/3883430719 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacad8e700 1 --2- 172.21.15.32:0/3883430719 >> v2:172.21.15.55:3300/0 conn(0x7faac41405d0 0x7faac4140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 -- 172.21.15.32:0/3883430719 >> v2:172.21.15.55:3300/0 conn(0x7faac41405d0 msgr2=0x7faac4140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 --2- 172.21.15.32:0/3883430719 >> v2:172.21.15.55:3300/0 conn(0x7faac41405d0 0x7faac4140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 -- 172.21.15.32:0/3883430719 >> v1:172.21.15.55:6789/0 conn(0x7faac40ad750 legacy=0x7faac40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 -- 172.21.15.32:0/3883430719 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40af350 con 0x7faac40614b0 2022-08-01T22:56:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faaca58d700 1 --2- 172.21.15.32:0/3883430719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac40614b0 0x7faac4061890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7faac40aed30 tx=0x7faab4006eb0).ready entity=mon.0 client_cookie=5582fd383f727a25 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faac958b700 1 -- 172.21.15.32:0/3883430719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab400d610 con 0x7faac40614b0 2022-08-01T22:56:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faac958b700 1 -- 172.21.15.32:0/3883430719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab4000e90 con 0x7faac40614b0 2022-08-01T22:56:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faac958b700 1 -- 172.21.15.32:0/3883430719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab400d9a0 con 0x7faac40614b0 2022-08-01T22:56:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- 172.21.15.32:0/3883430719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac40614b0 msgr2=0x7faac4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 --2- 172.21.15.32:0/3883430719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac40614b0 0x7faac4061890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7faac40aed30 tx=0x7faab4006eb0).stop 2022-08-01T22:56:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- 172.21.15.32:0/3883430719 shutdown_connections 2022-08-01T22:56:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 --2- 172.21.15.32:0/3883430719 >> v2:172.21.15.55:3300/0 conn(0x7faac41405d0 0x7faac4140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 --2- 172.21.15.32:0/3883430719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac40614b0 0x7faac4061890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- 172.21.15.32:0/3883430719 >> 172.21.15.32:0/3883430719 conn(0x7faac4066f70 msgr2=0x7faac4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- 172.21.15.32:0/3883430719 shutdown_connections 2022-08-01T22:56:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.931+0000 7faacb58f700 1 -- 172.21.15.32:0/3883430719 wait complete. 2022-08-01T22:56:13.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 Processor -- start 2022-08-01T22:56:13.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 -- start start 2022-08-01T22:56:13.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faac40614b0 0x7faac4156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faac40ad750 0x7faac4143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 0x7faac40b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4156510 con 0x7faac41405d0 2022-08-01T22:56:13.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4143940 con 0x7faac40614b0 2022-08-01T22:56:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac4156740 con 0x7faac40ad750 2022-08-01T22:56:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 0x7faac40b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 0x7faac40b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41352/0 (socket says 172.21.15.32:41352) 2022-08-01T22:56:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 -- 172.21.15.32:0/1304042206 learned_addr learned my addr 172.21.15.32:0/1304042206 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faaca58d700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faac40614b0 0x7faac4156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faac9d8c700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faac40ad750 0x7faac4143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 -- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faac40ad750 msgr2=0x7faac4143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faac40ad750 0x7faac4143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 -- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faac40614b0 msgr2=0x7faac4156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faac40614b0 0x7faac4156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 -- 172.21.15.32:0/1304042206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab400b1e0 con 0x7faac41405d0 2022-08-01T22:56:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacad8e700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 0x7faac40b0af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7faabc00c000 tx=0x7faabc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faabc011750 con 0x7faac41405d0 2022-08-01T22:56:13.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc0118b0 con 0x7faac41405d0 2022-08-01T22:56:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac4142d10 con 0x7faac41405d0 2022-08-01T22:56:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faabc011c10 con 0x7faac41405d0 2022-08-01T22:56:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.935+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac4143e00 con 0x7faac41405d0 2022-08-01T22:56:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.939+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac40af5f0 con 0x7faac41405d0 2022-08-01T22:56:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.939+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faabc011d70 con 0x7faac41405d0 2022-08-01T22:56:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.939+0000 7faab37fe700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faaa4053950 0x7faaa4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.939+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7faabc06c2d0 con 0x7faac41405d0 2022-08-01T22:56:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.939+0000 7faaca58d700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faaa4053950 0x7faaa4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.939+0000 7faaca58d700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faaa4053950 0x7faaa4055e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7faab401c000 tx=0x7faab4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:13.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:13.947+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faabc043090 con 0x7faac41405d0 2022-08-01T22:56:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.075+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faac40574e0 con 0x7faac41405d0 2022-08-01T22:56:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.075+0000 7faab37fe700 1 -- 172.21.15.32:0/1304042206 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7faabc0466b0 con 0x7faac41405d0 2022-08-01T22:56:14.084 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faaa4053950 msgr2=0x7faaa4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faaa4053950 0x7faaa4055e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7faab401c000 tx=0x7faab4015000).stop 2022-08-01T22:56:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 msgr2=0x7faac40b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 0x7faac40b0af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7faabc00c000 tx=0x7faabc006df0).stop 2022-08-01T22:56:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 shutdown_connections 2022-08-01T22:56:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faaa4053950 0x7faaa4055e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faac41405d0 0x7faac40b0af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faac40ad750 0x7faac4143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 --2- 172.21.15.32:0/1304042206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faac40614b0 0x7faac4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 >> 172.21.15.32:0/1304042206 conn(0x7faac4066f70 msgr2=0x7faac405cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 shutdown_connections 2022-08-01T22:56:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.079+0000 7faacb58f700 1 -- 172.21.15.32:0/1304042206 wait complete. 2022-08-01T22:56:14.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.203+0000 7ff5f6211700 1 Processor -- start 2022-08-01T22:56:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.203+0000 7ff5f6211700 1 -- start start 2022-08-01T22:56:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5f00acd00 0x7ff5f00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f00ac950 con 0x7ff5f00b1250 2022-08-01T22:56:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f00ac1b0 con 0x7ff5f00acd00 2022-08-01T22:56:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01411a0 con 0x7ff5f0140d30 2022-08-01T22:56:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5f00acd00 0x7ff5f00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5f00acd00 0x7ff5f00b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41358/0 (socket says 172.21.15.32:41358) 2022-08-01T22:56:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 -- 172.21.15.32:0/599284454 learned_addr learned my addr 172.21.15.32:0/599284454 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5effff700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 -- 172.21.15.32:0/599284454 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 msgr2=0x7ff5f00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 -- 172.21.15.32:0/599284454 >> v1:172.21.15.55:6789/0 conn(0x7ff5f00b1250 legacy=0x7ff5f013d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 -- 172.21.15.32:0/599284454 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00ac410 con 0x7ff5f00acd00 2022-08-01T22:56:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5effff700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5eeffd700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00acd00 0x7ff5f00b1630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7ff5e00032f0 tx=0x7ff5e0009290).ready entity=mon.0 client_cookie=43c18ff7a2a3e5a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5ee7fc700 1 -- 172.21.15.32:0/599284454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5e000b8a0 con 0x7ff5f00acd00 2022-08-01T22:56:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5ee7fc700 1 -- 172.21.15.32:0/599284454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e000ba00 con 0x7ff5f00acd00 2022-08-01T22:56:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5ee7fc700 1 -- 172.21.15.32:0/599284454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5e000bb70 con 0x7ff5f00acd00 2022-08-01T22:56:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- 172.21.15.32:0/599284454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00acd00 msgr2=0x7ff5f00b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00acd00 0x7ff5f00b1630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7ff5e00032f0 tx=0x7ff5e0009290).stop 2022-08-01T22:56:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- 172.21.15.32:0/599284454 shutdown_connections 2022-08-01T22:56:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00acd00 0x7ff5f00b1630 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 --2- 172.21.15.32:0/599284454 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- 172.21.15.32:0/599284454 >> 172.21.15.32:0/599284454 conn(0x7ff5f005f840 msgr2=0x7ff5f005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- 172.21.15.32:0/599284454 shutdown_connections 2022-08-01T22:56:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.207+0000 7ff5f6211700 1 -- 172.21.15.32:0/599284454 wait complete. 2022-08-01T22:56:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 Processor -- start 2022-08-01T22:56:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 -- start start 2022-08-01T22:56:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f00acd00 0x7ff5f01351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 0x7ff5f00b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0143890 con 0x7ff5f00b1250 2022-08-01T22:56:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00b4c50 con 0x7ff5f00acd00 2022-08-01T22:56:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0143a80 con 0x7ff5f0140d30 2022-08-01T22:56:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 0x7ff5f00b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 0x7ff5f00b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41364/0 (socket says 172.21.15.32:41364) 2022-08-01T22:56:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 -- 172.21.15.32:0/2567400993 learned_addr learned my addr 172.21.15.32:0/2567400993 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5effff700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5ef7fe700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f00acd00 0x7ff5f01351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 -- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 msgr2=0x7ff5f00b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 -- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f00acd00 msgr2=0x7ff5f01351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f00acd00 0x7ff5f01351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 -- 172.21.15.32:0/2567400993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009530 con 0x7ff5f00b1250 2022-08-01T22:56:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5eeffd700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 0x7ff5f00b4fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff5e00032f0 tx=0x7ff5e0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5e000bde0 con 0x7ff5f00b1250 2022-08-01T22:56:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0020430 con 0x7ff5f00b1250 2022-08-01T22:56:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f0143f20 con 0x7ff5f00b1250 2022-08-01T22:56:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5e0020790 con 0x7ff5f00b1250 2022-08-01T22:56:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.211+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f00b59a0 con 0x7ff5f00b1250 2022-08-01T22:56:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.215+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff5e000f070 con 0x7ff5f00b1250 2022-08-01T22:56:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.215+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f0143c40 con 0x7ff5f00b1250 2022-08-01T22:56:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.215+0000 7ff5ecff9700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff5d4053980 0x7ff5d4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.215+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff5e006d290 con 0x7ff5f00b1250 2022-08-01T22:56:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.215+0000 7ff5ef7fe700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff5d4053980 0x7ff5d4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.215+0000 7ff5ef7fe700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff5d4053980 0x7ff5d4055e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff5d800b000 tx=0x7ff5d8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.219+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5e0044050 con 0x7ff5f00b1250 2022-08-01T22:56:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.347+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5f0141710 con 0x7ff5f00b1250 2022-08-01T22:56:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5ecff9700 1 -- 172.21.15.32:0/2567400993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff5e0015020 con 0x7ff5f00b1250 2022-08-01T22:56:14.357 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff5d4053980 msgr2=0x7ff5d4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff5d4053980 0x7ff5d4055e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff5d800b000 tx=0x7ff5d8009040).stop 2022-08-01T22:56:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 msgr2=0x7ff5f00b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 0x7ff5f00b4fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff5e00032f0 tx=0x7ff5e0016000).stop 2022-08-01T22:56:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 shutdown_connections 2022-08-01T22:56:14.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff5d4053980 0x7ff5d4055e40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff5f0140d30 0x7ff5f00b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5f00b1250 0x7ff5f00b4fc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 --2- 172.21.15.32:0/2567400993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f00acd00 0x7ff5f01351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 >> 172.21.15.32:0/2567400993 conn(0x7ff5f005f840 msgr2=0x7ff5f0135e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 shutdown_connections 2022-08-01T22:56:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.351+0000 7ff5f6211700 1 -- 172.21.15.32:0/2567400993 wait complete. 2022-08-01T22:56:14.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.471+0000 7fc55abf9700 1 Processor -- start 2022-08-01T22:56:14.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.471+0000 7fc55abf9700 1 -- start start 2022-08-01T22:56:14.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc5540b3070 0x7fc554065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc55414b040 0x7fc55414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5540b2c20 con 0x7fc5540b7840 2022-08-01T22:56:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc554061830 con 0x7fc55414b040 2022-08-01T22:56:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc554065c10 con 0x7fc5540b3070 2022-08-01T22:56:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc55414b040 0x7fc55414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc55414b040 0x7fc55414b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41374/0 (socket says 172.21.15.32:41374) 2022-08-01T22:56:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 -- 172.21.15.32:0/873660712 learned_addr learned my addr 172.21.15.32:0/873660712 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc5593f6700 1 --2- 172.21.15.32:0/873660712 >> v2:172.21.15.55:3300/0 conn(0x7fc5540b3070 0x7fc554065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 -- 172.21.15.32:0/873660712 >> v2:172.21.15.55:3300/0 conn(0x7fc5540b3070 msgr2=0x7fc554065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 --2- 172.21.15.32:0/873660712 >> v2:172.21.15.55:3300/0 conn(0x7fc5540b3070 0x7fc554065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 -- 172.21.15.32:0/873660712 >> v1:172.21.15.55:6789/0 conn(0x7fc5540b7840 legacy=0x7fc554061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 -- 172.21.15.32:0/873660712 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540b7c20 con 0x7fc55414b040 2022-08-01T22:56:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55a3f8700 1 --2- 172.21.15.32:0/873660712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc55414b040 0x7fc55414b480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc54c002f70 tx=0x7fc54c006eb0).ready entity=mon.0 client_cookie=ec4c5f0de2bb6891 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc558bf5700 1 -- 172.21.15.32:0/873660712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc54c00d5c0 con 0x7fc55414b040 2022-08-01T22:56:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc558bf5700 1 -- 172.21.15.32:0/873660712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c000e90 con 0x7fc55414b040 2022-08-01T22:56:14.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc558bf5700 1 -- 172.21.15.32:0/873660712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc54c00d950 con 0x7fc55414b040 2022-08-01T22:56:14.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- 172.21.15.32:0/873660712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc55414b040 msgr2=0x7fc55414b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 --2- 172.21.15.32:0/873660712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc55414b040 0x7fc55414b480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc54c002f70 tx=0x7fc54c006eb0).stop 2022-08-01T22:56:14.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- 172.21.15.32:0/873660712 shutdown_connections 2022-08-01T22:56:14.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 --2- 172.21.15.32:0/873660712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc55414b040 0x7fc55414b480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 --2- 172.21.15.32:0/873660712 >> v2:172.21.15.55:3300/0 conn(0x7fc5540b3070 0x7fc554065720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- 172.21.15.32:0/873660712 >> 172.21.15.32:0/873660712 conn(0x7fc55405f840 msgr2=0x7fc55405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- 172.21.15.32:0/873660712 shutdown_connections 2022-08-01T22:56:14.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.475+0000 7fc55abf9700 1 -- 172.21.15.32:0/873660712 wait complete. 2022-08-01T22:56:14.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 Processor -- start 2022-08-01T22:56:14.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 -- start start 2022-08-01T22:56:14.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 0x7fc55405c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5540b7840 0x7fc5540afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc55414b040 0x7fc5540b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b0620 con 0x7fc5540b3070 2022-08-01T22:56:14.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540a0750 con 0x7fc55414b040 2022-08-01T22:56:14.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b19b0 con 0x7fc5540b7840 2022-08-01T22:56:14.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 0x7fc55405c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 0x7fc55405c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41376/0 (socket says 172.21.15.32:41376) 2022-08-01T22:56:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 -- 172.21.15.32:0/432562533 learned_addr learned my addr 172.21.15.32:0/432562533 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55a3f8700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc55414b040 0x7fc5540b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc5593f6700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5540b7840 0x7fc5540afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 -- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5540b7840 msgr2=0x7fc5540afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5540b7840 0x7fc5540afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 -- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc55414b040 msgr2=0x7fc5540b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc55414b040 0x7fc5540b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 -- 172.21.15.32:0/432562533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c00b1e0 con 0x7fc5540b3070 2022-08-01T22:56:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55a3f8700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc55414b040 0x7fc5540b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc559bf7700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 0x7fc55405c940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fc554069e10 tx=0x7fc550007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc550007020 con 0x7fc5540b3070 2022-08-01T22:56:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc550007180 con 0x7fc5540b3070 2022-08-01T22:56:14.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5540a0980 con 0x7fc5540b3070 2022-08-01T22:56:14.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc550006560 con 0x7fc5540b3070 2022-08-01T22:56:14.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.479+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5540b0aa0 con 0x7fc5540b3070 2022-08-01T22:56:14.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.483+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc550010070 con 0x7fc5540b3070 2022-08-01T22:56:14.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.483+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5540a0e20 con 0x7fc5540b3070 2022-08-01T22:56:14.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.483+0000 7fc546ffd700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc534053a70 0x7fc534055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.483+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc55006b400 con 0x7fc5540b3070 2022-08-01T22:56:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.483+0000 7fc5593f6700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc534053a70 0x7fc534055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.483+0000 7fc5593f6700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc534053a70 0x7fc534055f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc54800b000 tx=0x7fc5480091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.487+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc5540a0e20 con 0x7fc5540b3070 2022-08-01T22:56:14.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.615+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5540a0e20 con 0x7fc5540b3070 2022-08-01T22:56:14.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.615+0000 7fc546ffd700 1 -- 172.21.15.32:0/432562533 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc5540a0e20 con 0x7fc5540b3070 2022-08-01T22:56:14.623 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:14.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc534053a70 msgr2=0x7fc534055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc534053a70 0x7fc534055f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc54800b000 tx=0x7fc5480091d0).stop 2022-08-01T22:56:14.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 msgr2=0x7fc55405c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 0x7fc55405c940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fc554069e10 tx=0x7fc550007c40).stop 2022-08-01T22:56:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 shutdown_connections 2022-08-01T22:56:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc534053a70 0x7fc534055f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc55414b040 0x7fc5540b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5540b7840 0x7fc5540afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 --2- 172.21.15.32:0/432562533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5540b3070 0x7fc55405c940 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 >> 172.21.15.32:0/432562533 conn(0x7fc55405f840 msgr2=0x7fc5540adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 shutdown_connections 2022-08-01T22:56:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.619+0000 7fc55abf9700 1 -- 172.21.15.32:0/432562533 wait complete. 2022-08-01T22:56:14.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.731+0000 7fda1cb61700 1 Processor -- start 2022-08-01T22:56:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.731+0000 7fda1cb61700 1 -- start start 2022-08-01T22:56:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda18148530 0x7fda180b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda181448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda180b04e0 con 0x7fda180b1f40 2022-08-01T22:56:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda180a9250 con 0x7fda18148530 2022-08-01T22:56:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda180b0780 con 0x7fda180a7250 2022-08-01T22:56:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda181448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda15d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda18148530 0x7fda180b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda181448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:53126/0 (socket says 172.21.15.32:53126) 2022-08-01T22:56:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 -- 172.21.15.32:0/3823521784 learned_addr learned my addr 172.21.15.32:0/3823521784 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 -- 172.21.15.32:0/3823521784 >> v2:172.21.15.55:3300/0 conn(0x7fda18148530 msgr2=0x7fda180b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 --2- 172.21.15.32:0/3823521784 >> v2:172.21.15.55:3300/0 conn(0x7fda18148530 0x7fda180b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 -- 172.21.15.32:0/3823521784 >> v1:172.21.15.55:6789/0 conn(0x7fda180b1f40 legacy=0x7fda180b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 -- 172.21.15.32:0/3823521784 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180b0a20 con 0x7fda180a7250 2022-08-01T22:56:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda15d9b700 1 --2- 172.21.15.32:0/3823521784 >> v2:172.21.15.55:3300/0 conn(0x7fda18148530 0x7fda180b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda16d9d700 1 --2- 172.21.15.32:0/3823521784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda181448b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fda181478c0 tx=0x7fda10009290).ready entity=mon.2 client_cookie=d30e2e3243c09b14 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1559a700 1 -- 172.21.15.32:0/3823521784 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda1000b860 con 0x7fda180a7250 2022-08-01T22:56:14.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1559a700 1 -- 172.21.15.32:0/3823521784 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1000b9c0 con 0x7fda180a7250 2022-08-01T22:56:14.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1559a700 1 -- 172.21.15.32:0/3823521784 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda1000bb30 con 0x7fda180a7250 2022-08-01T22:56:14.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- 172.21.15.32:0/3823521784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 msgr2=0x7fda181448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 --2- 172.21.15.32:0/3823521784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda181448b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fda181478c0 tx=0x7fda10009290).stop 2022-08-01T22:56:14.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- 172.21.15.32:0/3823521784 shutdown_connections 2022-08-01T22:56:14.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 --2- 172.21.15.32:0/3823521784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda181448b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 --2- 172.21.15.32:0/3823521784 >> v2:172.21.15.55:3300/0 conn(0x7fda18148530 0x7fda180b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- 172.21.15.32:0/3823521784 >> 172.21.15.32:0/3823521784 conn(0x7fda1805f840 msgr2=0x7fda1805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- 172.21.15.32:0/3823521784 shutdown_connections 2022-08-01T22:56:14.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.735+0000 7fda1cb61700 1 -- 172.21.15.32:0/3823521784 wait complete. 2022-08-01T22:56:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 Processor -- start 2022-08-01T22:56:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 -- start start 2022-08-01T22:56:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda1814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda180b1f40 0x7fda180adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda18148530 0x7fda180ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1814a300 con 0x7fda18148530 2022-08-01T22:56:14.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1814a5a0 con 0x7fda180b1f40 2022-08-01T22:56:14.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda180a9670 con 0x7fda180a7250 2022-08-01T22:56:14.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda16d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda18148530 0x7fda180ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda16d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda18148530 0x7fda180ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41392/0 (socket says 172.21.15.32:41392) 2022-08-01T22:56:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda180b1f40 0x7fda180adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda16d9d700 1 -- 172.21.15.32:0/2592849631 learned_addr learned my addr 172.21.15.32:0/2592849631 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1659c700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda1814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 -- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 msgr2=0x7fda1814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda1814b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 -- 172.21.15.32:0/2592849631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda18148530 msgr2=0x7fda180ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda18148530 0x7fda180ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 -- 172.21.15.32:0/2592849631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c00b180 con 0x7fda180b1f40 2022-08-01T22:56:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda15d9b700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda180b1f40 0x7fda180adfc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fda0c00c000 tx=0x7fda0c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0c010780 con 0x7fda180b1f40 2022-08-01T22:56:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda10009530 con 0x7fda180b1f40 2022-08-01T22:56:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c0108e0 con 0x7fda180b1f40 2022-08-01T22:56:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0c010c40 con 0x7fda180b1f40 2022-08-01T22:56:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.739+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda181497e0 con 0x7fda180b1f40 2022-08-01T22:56:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.743+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda180574e0 con 0x7fda180b1f40 2022-08-01T22:56:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.743+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fda0c0334d0 con 0x7fda180b1f40 2022-08-01T22:56:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.743+0000 7fda077fe700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9f4053b00 0x7fd9f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.743+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fda0c06c870 con 0x7fda180b1f40 2022-08-01T22:56:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.743+0000 7fda1659c700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9f4053b00 0x7fd9f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.743+0000 7fda1659c700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9f4053b00 0x7fd9f4055fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fda0800d000 tx=0x7fda08007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.751+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda0c043520 con 0x7fda180b1f40 2022-08-01T22:56:14.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.875+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda180447e0 con 0x7fda180b1f40 2022-08-01T22:56:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.879+0000 7fda077fe700 1 -- 172.21.15.32:0/2592849631 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fda0c046b40 con 0x7fda180b1f40 2022-08-01T22:56:14.887 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:14.887 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:55:50.679882+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:55:45.604869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T22:55:48.366315+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9f4053b00 msgr2=0x7fd9f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9f4053b00 0x7fd9f4055fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fda0800d000 tx=0x7fda08007b90).stop 2022-08-01T22:56:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda180b1f40 msgr2=0x7fda180adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda180b1f40 0x7fda180adfc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fda0c00c000 tx=0x7fda0c006df0).stop 2022-08-01T22:56:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 shutdown_connections 2022-08-01T22:56:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9f4053b00 0x7fd9f4055fc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda18148530 0x7fda180ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda180b1f40 0x7fda180adfc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 --2- 172.21.15.32:0/2592849631 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fda180a7250 0x7fda1814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 >> 172.21.15.32:0/2592849631 conn(0x7fda1805f840 msgr2=0x7fda180b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 shutdown_connections 2022-08-01T22:56:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:14.883+0000 7fda1cb61700 1 -- 172.21.15.32:0/2592849631 wait complete. 2022-08-01T22:56:14.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:55:45.604869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:55:48.366315+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:14.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 Processor -- start 2022-08-01T22:56:15.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 -- start start 2022-08-01T22:56:15.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca8c0b31f0 0x7fca8c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fca8c0aadb0 0x7fca8c064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c0b2510 con 0x7fca8c0b7500 2022-08-01T22:56:15.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c1443e0 con 0x7fca8c0b31f0 2022-08-01T22:56:15.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0b3700 con 0x7fca8c0aadb0 2022-08-01T22:56:15.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca89d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca8c0b31f0 0x7fca8c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca8c0b7500 0x7fca8c13e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57172/0 (socket says 172.21.15.32:57172) 2022-08-01T22:56:15.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca89d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca8c0b31f0 0x7fca8c0b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41400/0 (socket says 172.21.15.32:41400) 2022-08-01T22:56:15.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8a59c700 1 -- 172.21.15.32:0/901705319 learned_addr learned my addr 172.21.15.32:0/901705319 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:15.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.007+0000 7fca8ad9d700 1 --2- 172.21.15.32:0/901705319 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0aadb0 0x7fca8c064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca89d9b700 1 -- 172.21.15.32:0/901705319 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0aadb0 msgr2=0x7fca8c064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:15.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca89d9b700 1 --2- 172.21.15.32:0/901705319 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0aadb0 0x7fca8c064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca89d9b700 1 -- 172.21.15.32:0/901705319 >> v1:172.21.15.32:6789/0 conn(0x7fca8c0b7500 legacy=0x7fca8c13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca89d9b700 1 -- 172.21.15.32:0/901705319 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0b2740 con 0x7fca8c0b31f0 2022-08-01T22:56:15.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca89d9b700 1 --2- 172.21.15.32:0/901705319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b31f0 0x7fca8c0b78e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fca74002f70 tx=0x7fca74006df0).ready entity=mon.0 client_cookie=2f81a1ca1143d66b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8959a700 1 -- 172.21.15.32:0/901705319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca7400d570 con 0x7fca8c0b31f0 2022-08-01T22:56:15.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8959a700 1 -- 172.21.15.32:0/901705319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca74000e90 con 0x7fca8c0b31f0 2022-08-01T22:56:15.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8959a700 1 -- 172.21.15.32:0/901705319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca7400d8c0 con 0x7fca8c0b31f0 2022-08-01T22:56:15.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 -- 172.21.15.32:0/901705319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b31f0 msgr2=0x7fca8c0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 --2- 172.21.15.32:0/901705319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b31f0 0x7fca8c0b78e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fca74002f70 tx=0x7fca74006df0).stop 2022-08-01T22:56:15.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 -- 172.21.15.32:0/901705319 shutdown_connections 2022-08-01T22:56:15.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 --2- 172.21.15.32:0/901705319 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0aadb0 0x7fca8c064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 --2- 172.21.15.32:0/901705319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b31f0 0x7fca8c0b78e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 -- 172.21.15.32:0/901705319 >> 172.21.15.32:0/901705319 conn(0x7fca8c05f840 msgr2=0x7fca8c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 -- 172.21.15.32:0/901705319 shutdown_connections 2022-08-01T22:56:15.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 -- 172.21.15.32:0/901705319 wait complete. 2022-08-01T22:56:15.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 Processor -- start 2022-08-01T22:56:15.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.011+0000 7fca8b59e700 1 -- start start 2022-08-01T22:56:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca8c0aadb0 0x7fca8c0ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c0b31f0 0x7fca8c141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 0x7fca8c060950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0acae0 con 0x7fca8c0b7500 2022-08-01T22:56:15.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0ae4b0 con 0x7fca8c0b31f0 2022-08-01T22:56:15.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0acd40 con 0x7fca8c0aadb0 2022-08-01T22:56:15.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 0x7fca8c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 0x7fca8c060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41408/0 (socket says 172.21.15.32:41408) 2022-08-01T22:56:15.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 -- 172.21.15.32:0/3386065408 learned_addr learned my addr 172.21.15.32:0/3386065408 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:15.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca89d9b700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c0b31f0 0x7fca8c141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8a59c700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca8c0aadb0 0x7fca8c0ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 -- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca8c0aadb0 msgr2=0x7fca8c0ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca8c0aadb0 0x7fca8c0ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 -- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c0b31f0 msgr2=0x7fca8c141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c0b31f0 0x7fca8c141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 -- 172.21.15.32:0/3386065408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7400b180 con 0x7fca8c0b7500 2022-08-01T22:56:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8a59c700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca8c0aadb0 0x7fca8c0ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8ad9d700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 0x7fca8c060950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fca84006e00 tx=0x7fca840091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca8400eeb0 con 0x7fca8c0b7500 2022-08-01T22:56:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8400c850 con 0x7fca8c0b7500 2022-08-01T22:56:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c0ae750 con 0x7fca8c0b7500 2022-08-01T22:56:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca8400cbb0 con 0x7fca8c0b7500 2022-08-01T22:56:15.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c0ac450 con 0x7fca8c0b7500 2022-08-01T22:56:15.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.015+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c0acf10 con 0x7fca8c0b7500 2022-08-01T22:56:15.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.019+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fca8400cd10 con 0x7fca8c0b7500 2022-08-01T22:56:15.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.019+0000 7fca837fe700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca70053a70 0x7fca70055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.019+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fca8406c430 con 0x7fca8c0b7500 2022-08-01T22:56:15.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.019+0000 7fca8a59c700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca70053a70 0x7fca70055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.019+0000 7fca8a59c700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca70053a70 0x7fca70055f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fca7c00b000 tx=0x7fca7c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.027+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca840430e0 con 0x7fca8c0b7500 2022-08-01T22:56:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.147+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fca8c0574e0 con 0x7fca8c0b7500 2022-08-01T22:56:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca837fe700 1 -- 172.21.15.32:0/3386065408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fca84046700 con 0x7fca8c0b7500 2022-08-01T22:56:15.157 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca70053a70 msgr2=0x7fca70055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca70053a70 0x7fca70055f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fca7c00b000 tx=0x7fca7c009040).stop 2022-08-01T22:56:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 msgr2=0x7fca8c060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 0x7fca8c060950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fca84006e00 tx=0x7fca840091d0).stop 2022-08-01T22:56:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 shutdown_connections 2022-08-01T22:56:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca70053a70 0x7fca70055f30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca8c0b7500 0x7fca8c060950 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c0b31f0 0x7fca8c141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 --2- 172.21.15.32:0/3386065408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca8c0aadb0 0x7fca8c0ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 >> 172.21.15.32:0/3386065408 conn(0x7fca8c05f840 msgr2=0x7fca8c13dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 shutdown_connections 2022-08-01T22:56:15.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.151+0000 7fca8b59e700 1 -- 172.21.15.32:0/3386065408 wait complete. 2022-08-01T22:56:15.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 Processor -- start 2022-08-01T22:56:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 -- start start 2022-08-01T22:56:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f67980b7840 0x7f6798060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67980b2930 0x7f67980610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6798146720 0x7f6798146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67980671e0 con 0x7f67980b2930 2022-08-01T22:56:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6798065540 con 0x7f67980b7840 2022-08-01T22:56:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679ef2b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67980b7c20 con 0x7f6798146720 2022-08-01T22:56:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679d728700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67980b2930 0x7f67980610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679d728700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67980b2930 0x7f67980610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41414/0 (socket says 172.21.15.32:41414) 2022-08-01T22:56:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.283+0000 7f679d728700 1 -- 172.21.15.32:0/3588860857 learned_addr learned my addr 172.21.15.32:0/3588860857 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679df29700 1 --2- 172.21.15.32:0/3588860857 >> v2:172.21.15.55:3300/0 conn(0x7f67980b7840 0x7f6798060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679e72a700 1 --2- 172.21.15.32:0/3588860857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6798146720 0x7f6798146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679d728700 1 -- 172.21.15.32:0/3588860857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6798146720 msgr2=0x7f6798146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679d728700 1 --2- 172.21.15.32:0/3588860857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6798146720 0x7f6798146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679d728700 1 -- 172.21.15.32:0/3588860857 >> v2:172.21.15.55:3300/0 conn(0x7f67980b7840 msgr2=0x7f6798060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679d728700 1 --2- 172.21.15.32:0/3588860857 >> v2:172.21.15.55:3300/0 conn(0x7f67980b7840 0x7f6798060be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679d728700 1 -- 172.21.15.32:0/3588860857 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798066ce0 con 0x7f67980b2930 2022-08-01T22:56:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679df29700 1 --2- 172.21.15.32:0/3588860857 >> v2:172.21.15.55:3300/0 conn(0x7f67980b7840 0x7f6798060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679d728700 1 --2- 172.21.15.32:0/3588860857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67980b2930 0x7f67980610d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f678c002f70 tx=0x7f678c006eb0).ready entity=mon.0 client_cookie=e40ea00ee8256bc1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679cf27700 1 -- 172.21.15.32:0/3588860857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678c00d570 con 0x7f67980b2930 2022-08-01T22:56:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679cf27700 1 -- 172.21.15.32:0/3588860857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c000e90 con 0x7f67980b2930 2022-08-01T22:56:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679cf27700 1 -- 172.21.15.32:0/3588860857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678c00d900 con 0x7f67980b2930 2022-08-01T22:56:15.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 -- 172.21.15.32:0/3588860857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67980b2930 msgr2=0x7f67980610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 --2- 172.21.15.32:0/3588860857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67980b2930 0x7f67980610d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f678c002f70 tx=0x7f678c006eb0).stop 2022-08-01T22:56:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 -- 172.21.15.32:0/3588860857 shutdown_connections 2022-08-01T22:56:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 --2- 172.21.15.32:0/3588860857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67980b2930 0x7f67980610d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 --2- 172.21.15.32:0/3588860857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6798146720 0x7f6798146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 --2- 172.21.15.32:0/3588860857 >> v2:172.21.15.55:3300/0 conn(0x7f67980b7840 0x7f6798060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 -- 172.21.15.32:0/3588860857 >> 172.21.15.32:0/3588860857 conn(0x7f679805f840 msgr2=0x7f679805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 -- 172.21.15.32:0/3588860857 shutdown_connections 2022-08-01T22:56:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.287+0000 7f679ef2b700 1 -- 172.21.15.32:0/3588860857 wait complete. 2022-08-01T22:56:15.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 Processor -- start 2022-08-01T22:56:15.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 -- start start 2022-08-01T22:56:15.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67980b2930 0x7f6798144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67980b7840 0x7f67980a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 0x7f67980a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6798153700 con 0x7f6798146720 2022-08-01T22:56:15.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67981526f0 con 0x7f67980b2930 2022-08-01T22:56:15.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679ef2b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679813fc90 con 0x7f67980b7840 2022-08-01T22:56:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 0x7f67980a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679df29700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67980b2930 0x7f6798144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 0x7f67980a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41422/0 (socket says 172.21.15.32:41422) 2022-08-01T22:56:15.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 -- 172.21.15.32:0/1359348864 learned_addr learned my addr 172.21.15.32:0/1359348864 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679d728700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67980b7840 0x7f67980a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 -- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67980b7840 msgr2=0x7f67980a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67980b7840 0x7f67980a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 -- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67980b2930 msgr2=0x7f6798144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67980b2930 0x7f6798144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 -- 172.21.15.32:0/1359348864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c00b1e0 con 0x7f6798146720 2022-08-01T22:56:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.291+0000 7f679e72a700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 0x7f67980a4a40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f679000a000 tx=0x7f6790008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67900127e0 con 0x7f6798146720 2022-08-01T22:56:15.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790012940 con 0x7f6798146720 2022-08-01T22:56:15.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67981528e0 con 0x7f6798146720 2022-08-01T22:56:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6790012ca0 con 0x7f6798146720 2022-08-01T22:56:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67980a2ef0 con 0x7f6798146720 2022-08-01T22:56:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67980574e0 con 0x7f6798146720 2022-08-01T22:56:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6790013480 con 0x7f6798146720 2022-08-01T22:56:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f678affd700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6778053950 0x7f6778055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f679006c360 con 0x7f6798146720 2022-08-01T22:56:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f679df29700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6778053950 0x7f6778055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.295+0000 7f679df29700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6778053950 0x7f6778055e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f679400c000 tx=0x7f6794008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.303+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67900430c0 con 0x7f6798146720 2022-08-01T22:56:15.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f67980b4e60 con 0x7f6798146720 2022-08-01T22:56:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f678affd700 1 -- 172.21.15.32:0/1359348864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6790012e00 con 0x7f6798146720 2022-08-01T22:56:15.438 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6778053950 msgr2=0x7f6778055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6778053950 0x7f6778055e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f679400c000 tx=0x7f6794008040).stop 2022-08-01T22:56:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 msgr2=0x7f67980a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 0x7f67980a4a40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f679000a000 tx=0x7f6790008040).stop 2022-08-01T22:56:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 shutdown_connections 2022-08-01T22:56:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6778053950 0x7f6778055e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6798146720 0x7f67980a4a40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67980b7840 0x7f67980a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 --2- 172.21.15.32:0/1359348864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67980b2930 0x7f6798144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.431+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 >> 172.21.15.32:0/1359348864 conn(0x7f679805f840 msgr2=0x7f679805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.435+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 shutdown_connections 2022-08-01T22:56:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.435+0000 7f679ef2b700 1 -- 172.21.15.32:0/1359348864 wait complete. 2022-08-01T22:56:15.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2d32700 1 Processor -- start 2022-08-01T22:56:15.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2d32700 1 -- start start 2022-08-01T22:56:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2d32700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec14b950 0x7fa9ec14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2d32700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0b60e0 con 0x7fa9ec0b6420 2022-08-01T22:56:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2d32700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0b2100 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2d32700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b23a0 con 0x7fa9ec14b950 2022-08-01T22:56:15.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f1d30700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9ec0b6420 0x7fa9ec061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57200/0 (socket says 172.21.15.32:57200) 2022-08-01T22:56:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f1d30700 1 -- 172.21.15.32:0/2452541580 learned_addr learned my addr 172.21.15.32:0/2452541580 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.559+0000 7fa9f2531700 1 --2- 172.21.15.32:0/2452541580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec14b950 0x7fa9ec14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545761566 0 0) 0x7fa9ec0b2100 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d4003260 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818666591 0 0) 0x7fa9e8001110 con 0x7fa9ec0b6420 2022-08-01T22:56:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec0b2100 con 0x7fa9ec0b6420 2022-08-01T22:56:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2531700 1 -- 172.21.15.32:0/2452541580 >> v1:172.21.15.55:6789/0 conn(0x7fa9ec0b41f0 legacy=0x7fa9ec061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2531700 1 -- 172.21.15.32:0/2452541580 >> v1:172.21.15.32:6789/0 conn(0x7fa9ec0b6420 legacy=0x7fa9ec061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2531700 1 -- 172.21.15.32:0/2452541580 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec0b2640 con 0x7fa9ec14b950 2022-08-01T22:56:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2531700 1 --2- 172.21.15.32:0/2452541580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec14b950 0x7fa9ec14dd40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa9e40032f0 tx=0x7fa9e40091d0).ready entity=mon.2 client_cookie=3843a1a65f45aaa4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e400b640 con 0x7fa9ec14b950 2022-08-01T22:56:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e400b7a0 con 0x7fa9ec14b950 2022-08-01T22:56:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f0d2e700 1 -- 172.21.15.32:0/2452541580 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e400b910 con 0x7fa9ec14b950 2022-08-01T22:56:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 -- 172.21.15.32:0/2452541580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec14b950 msgr2=0x7fa9ec14dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/2452541580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec14b950 0x7fa9ec14dd40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa9e40032f0 tx=0x7fa9e40091d0).stop 2022-08-01T22:56:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 -- 172.21.15.32:0/2452541580 shutdown_connections 2022-08-01T22:56:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/2452541580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec14b950 0x7fa9ec14dd40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 -- 172.21.15.32:0/2452541580 >> 172.21.15.32:0/2452541580 conn(0x7fa9ec05c620 msgr2=0x7fa9ec05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 -- 172.21.15.32:0/2452541580 shutdown_connections 2022-08-01T22:56:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.563+0000 7fa9f2d32700 1 -- 172.21.15.32:0/2452541580 wait complete. 2022-08-01T22:56:15.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 Processor -- start 2022-08-01T22:56:15.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 -- start start 2022-08-01T22:56:15.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 0x7fa9ec0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b6420 0x7fa9ec152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec14b950 0x7fa9ec152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec155bf0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec155e90 con 0x7fa9ec14b950 2022-08-01T22:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec156130 con 0x7fa9ec0b6420 2022-08-01T22:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 0x7fa9ec0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2531700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec14b950 0x7fa9ec152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 0x7fa9ec0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41432/0 (socket says 172.21.15.32:41432) 2022-08-01T22:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f152f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b6420 0x7fa9ec152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 -- 172.21.15.32:0/373965703 learned_addr learned my addr 172.21.15.32:0/373965703 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 -- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b6420 msgr2=0x7fa9ec152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b6420 0x7fa9ec152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 -- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec14b950 msgr2=0x7fa9ec152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec14b950 0x7fa9ec152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 -- 172.21.15.32:0/373965703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4009470 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f1d30700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 0x7fa9ec0b49c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fa9e80012a0 tx=0x7fa9e8007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e80070d0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8007230 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec1563d0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e8005610 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.567+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec1501d0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.571+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec0b28e0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.571+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9e801a070 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.571+0000 7fa9deffd700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9cc053b00 0x7fa9cc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.571+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa9e806ae10 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.571+0000 7fa9f152f700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9cc053b00 0x7fa9cc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.571+0000 7fa9f152f700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9cc053b00 0x7fa9cc055fc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa9e000d000 tx=0x7fa9e0007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.583+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9e8041bd0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9ec0b60e0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9deffd700 1 -- 172.21.15.32:0/373965703 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa9ec0b60e0 con 0x7fa9ec0b41f0 2022-08-01T22:56:15.717 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9cc053b00 msgr2=0x7fa9cc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9cc053b00 0x7fa9cc055fc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa9e000d000 tx=0x7fa9e0007a20).stop 2022-08-01T22:56:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 msgr2=0x7fa9ec0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 0x7fa9ec0b49c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fa9e80012a0 tx=0x7fa9e8007900).stop 2022-08-01T22:56:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 shutdown_connections 2022-08-01T22:56:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9cc053b00 0x7fa9cc055fc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec14b950 0x7fa9ec152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b6420 0x7fa9ec152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 --2- 172.21.15.32:0/373965703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b41f0 0x7fa9ec0b49c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 >> 172.21.15.32:0/373965703 conn(0x7fa9ec05c620 msgr2=0x7fa9ec05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 shutdown_connections 2022-08-01T22:56:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.707+0000 7fa9f2d32700 1 -- 172.21.15.32:0/373965703 wait complete. 2022-08-01T22:56:15.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.831+0000 7fc2b940b700 1 Processor -- start 2022-08-01T22:56:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.831+0000 7fc2b940b700 1 -- start start 2022-08-01T22:56:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2b40b0270 0x7fc2b4143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b4067a10 con 0x7fc2b40b2710 2022-08-01T22:56:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b4067cb0 con 0x7fc2b40ac880 2022-08-01T22:56:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b40ad740 con 0x7fc2b40b0270 2022-08-01T22:56:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2b40b2710 0x7fc2b41401e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57214/0 (socket says 172.21.15.32:57214) 2022-08-01T22:56:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b3fff700 1 -- 172.21.15.32:0/1577219144 learned_addr learned my addr 172.21.15.32:0/1577219144 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b37fe700 1 --2- 172.21.15.32:0/1577219144 >> v2:172.21.15.55:3300/0 conn(0x7fc2b40b0270 0x7fc2b4143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b2ffd700 1 -- 172.21.15.32:0/1577219144 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874609024 0 0) 0x7fc2b4067a10 con 0x7fc2b40b2710 2022-08-01T22:56:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b2ffd700 1 -- 172.21.15.32:0/1577219144 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a0003260 con 0x7fc2b40b2710 2022-08-01T22:56:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b37fe700 1 -- 172.21.15.32:0/1577219144 >> v1:172.21.15.55:6789/0 conn(0x7fc2b40ac880 legacy=0x7fc2b41434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b2ffd700 1 -- 172.21.15.32:0/1577219144 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558954790 0 0) 0x7fc2b4067cb0 con 0x7fc2b40ac880 2022-08-01T22:56:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b37fe700 1 -- 172.21.15.32:0/1577219144 >> v1:172.21.15.32:6789/0 conn(0x7fc2b40b2710 legacy=0x7fc2b41401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b37fe700 1 -- 172.21.15.32:0/1577219144 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b4147160 con 0x7fc2b40b0270 2022-08-01T22:56:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b37fe700 1 --2- 172.21.15.32:0/1577219144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b4143000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc2a40032f0 tx=0x7fc2a40091d0).ready entity=mon.1 client_cookie=1d4d737f1a7e0111 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b2ffd700 1 -- 172.21.15.32:0/1577219144 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a400b680 con 0x7fc2b40b0270 2022-08-01T22:56:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b2ffd700 1 -- 172.21.15.32:0/1577219144 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a400b7e0 con 0x7fc2b40b0270 2022-08-01T22:56:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b2ffd700 1 -- 172.21.15.32:0/1577219144 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a400b950 con 0x7fc2b40b0270 2022-08-01T22:56:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- 172.21.15.32:0/1577219144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 msgr2=0x7fc2b4143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1577219144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b4143000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc2a40032f0 tx=0x7fc2a40091d0).stop 2022-08-01T22:56:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- 172.21.15.32:0/1577219144 shutdown_connections 2022-08-01T22:56:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1577219144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b4143000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- 172.21.15.32:0/1577219144 >> 172.21.15.32:0/1577219144 conn(0x7fc2b4060310 msgr2=0x7fc2b40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- 172.21.15.32:0/1577219144 shutdown_connections 2022-08-01T22:56:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.835+0000 7fc2b940b700 1 -- 172.21.15.32:0/1577219144 wait complete. 2022-08-01T22:56:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 Processor -- start 2022-08-01T22:56:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 -- start start 2022-08-01T22:56:15.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2b40ac880 0x7fc2b4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b40aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 0x7fc2b40aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40a0810 con 0x7fc2b40b2710 2022-08-01T22:56:15.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40aac40 con 0x7fc2b40b0270 2022-08-01T22:56:15.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4156470 con 0x7fc2b40ac880 2022-08-01T22:56:15.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 0x7fc2b40aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 0x7fc2b40aa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41450/0 (socket says 172.21.15.32:41450) 2022-08-01T22:56:15.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 -- 172.21.15.32:0/1923758797 learned_addr learned my addr 172.21.15.32:0/1923758797 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:15.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b37fe700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b40aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b3fff700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2b40ac880 0x7fc2b4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 -- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2b40ac880 msgr2=0x7fc2b4155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2b40ac880 0x7fc2b4155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 -- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 msgr2=0x7fc2b40aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b40aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 -- 172.21.15.32:0/1923758797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4009470 con 0x7fc2b40b2710 2022-08-01T22:56:15.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b8c0a700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 0x7fc2b40aa750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fc2b40a0f70 tx=0x7fc2ac0079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac00cdf0 con 0x7fc2b40b2710 2022-08-01T22:56:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b4156710 con 0x7fc2b40b2710 2022-08-01T22:56:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac012400 con 0x7fc2b40b2710 2022-08-01T22:56:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b409fc40 con 0x7fc2b40b2710 2022-08-01T22:56:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.839+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac012760 con 0x7fc2b40b2710 2022-08-01T22:56:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.843+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b4146580 con 0x7fc2b40b2710 2022-08-01T22:56:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.843+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc2ac019030 con 0x7fc2b40b2710 2022-08-01T22:56:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.843+0000 7fc2b17fa700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc294053b30 0x7fc294055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.843+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc2ac06b260 con 0x7fc2b40b2710 2022-08-01T22:56:15.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.843+0000 7fc2b3fff700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc294053b30 0x7fc294055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:15.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.843+0000 7fc2b3fff700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc294053b30 0x7fc294055ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc2a800d000 tx=0x7fc2a8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.851+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2b4146580 con 0x7fc2b40b2710 2022-08-01T22:56:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2b4146580 con 0x7fc2b40b2710 2022-08-01T22:56:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b17fa700 1 -- 172.21.15.32:0/1923758797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc2b4146580 con 0x7fc2b40b2710 2022-08-01T22:56:15.989 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc294053b30 msgr2=0x7fc294055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc294053b30 0x7fc294055ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc2a800d000 tx=0x7fc2a8009040).stop 2022-08-01T22:56:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 msgr2=0x7fc2b40aa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:15.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 0x7fc2b40aa750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fc2b40a0f70 tx=0x7fc2ac0079e0).stop 2022-08-01T22:56:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 shutdown_connections 2022-08-01T22:56:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc294053b30 0x7fc294055ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b40b2710 0x7fc2b40aa750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b40b0270 0x7fc2b40aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:15.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 --2- 172.21.15.32:0/1923758797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2b40ac880 0x7fc2b4155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 >> 172.21.15.32:0/1923758797 conn(0x7fc2b4060310 msgr2=0x7fc2b40b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:16.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 shutdown_connections 2022-08-01T22:56:16.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:15.979+0000 7fc2b940b700 1 -- 172.21.15.32:0/1923758797 wait complete. 2022-08-01T22:56:16.018 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T22:56:16.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-08-01T22:56:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 Processor -- start 2022-08-01T22:56:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 -- start start 2022-08-01T22:56:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fca1c0b1160 0x7fca1c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0b04e0 con 0x7fca1c0b1f40 2022-08-01T22:56:16.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0b0780 con 0x7fca1c148530 2022-08-01T22:56:16.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c0a9250 con 0x7fca1c0b1160 2022-08-01T22:56:16.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca1c0b1f40 0x7fca1c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57228/0 (socket says 172.21.15.32:57228) 2022-08-01T22:56:16.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 -- 172.21.15.32:0/84239065 learned_addr learned my addr 172.21.15.32:0/84239065 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:16.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca21471700 1 --2- 172.21.15.32:0/84239065 >> v2:172.21.15.55:3300/0 conn(0x7fca1c0b1160 0x7fca1c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941012931 0 0) 0x7fca1c0b04e0 con 0x7fca1c0b1f40 2022-08-01T22:56:16.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca08003260 con 0x7fca1c0b1f40 2022-08-01T22:56:16.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1164643510 0 0) 0x7fca08003260 con 0x7fca1c0b1f40 2022-08-01T22:56:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 >> v2:172.21.15.55:3300/0 conn(0x7fca1c0b1160 msgr2=0x7fca1c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 --2- 172.21.15.32:0/84239065 >> v2:172.21.15.55:3300/0 conn(0x7fca1c0b1160 0x7fca1c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 >> v1:172.21.15.55:6789/0 conn(0x7fca1c148530 legacy=0x7fca1c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0b0a20 con 0x7fca1c0b1f40 2022-08-01T22:56:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fca10001660 con 0x7fca1c0b1f40 2022-08-01T22:56:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 >> v1:172.21.15.32:6789/0 conn(0x7fca1c0b1f40 legacy=0x7fca1c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08003750 0x7fca08003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca08004500 0x7fca08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08004e70 0x7fca0800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800bc40 con 0x7fca08003750 2022-08-01T22:56:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800bdb0 con 0x7fca08004e70 2022-08-01T22:56:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800bf70 con 0x7fca08004500 2022-08-01T22:56:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08003750 0x7fca08003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1bfff700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca08004500 0x7fca08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca21471700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08004e70 0x7fca0800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 -- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca08004500 msgr2=0x7fca08004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:16.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca08004500 0x7fca08004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 -- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08004e70 msgr2=0x7fca0800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:16.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08004e70 0x7fca0800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 -- 172.21.15.32:0/84239065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08003260 con 0x7fca08003750 2022-08-01T22:56:16.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca21471700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08004e70 0x7fca0800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:16.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca21471700 1 -- 172.21.15.32:0/84239065 reap_dead start 2022-08-01T22:56:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca20c70700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08003750 0x7fca08003b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fca10001370 tx=0x7fca1c0b0910).ready entity=mon.0 client_cookie=d209abb3a9abe452 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca100070d0 con 0x7fca08003750 2022-08-01T22:56:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10007230 con 0x7fca08003750 2022-08-01T22:56:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca1b7fe700 1 -- 172.21.15.32:0/84239065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca1000c9c0 con 0x7fca08003750 2022-08-01T22:56:16.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 -- 172.21.15.32:0/84239065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08003750 msgr2=0x7fca08003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.123+0000 7fca22ed4700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08003750 0x7fca08003b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fca10001370 tx=0x7fca1c0b0910).stop 2022-08-01T22:56:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- 172.21.15.32:0/84239065 shutdown_connections 2022-08-01T22:56:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 --2- 172.21.15.32:0/84239065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08003750 0x7fca08003b30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- 172.21.15.32:0/84239065 >> 172.21.15.32:0/84239065 conn(0x7fca1c05f840 msgr2=0x7fca1c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- 172.21.15.32:0/84239065 shutdown_connections 2022-08-01T22:56:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- 172.21.15.32:0/84239065 wait complete. 2022-08-01T22:56:16.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 Processor -- start 2022-08-01T22:56:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- start start 2022-08-01T22:56:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08003750 0x7fca1c142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 0x7fca1c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c1472c0 0x7fca1c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c1567b0 con 0x7fca1c159d00 2022-08-01T22:56:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0af750 con 0x7fca08003750 2022-08-01T22:56:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0ad340 con 0x7fca1c1472c0 2022-08-01T22:56:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 0x7fca1c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 0x7fca1c156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41468/0 (socket says 172.21.15.32:41468) 2022-08-01T22:56:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 -- 172.21.15.32:0/3185776597 learned_addr learned my addr 172.21.15.32:0/3185776597 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca20c70700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08003750 0x7fca1c142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca21471700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c1472c0 0x7fca1c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 -- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c1472c0 msgr2=0x7fca1c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c1472c0 0x7fca1c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 -- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08003750 msgr2=0x7fca1c142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08003750 0x7fca1c142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 -- 172.21.15.32:0/3185776597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0ad5e0 con 0x7fca1c159d00 2022-08-01T22:56:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca20c70700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08003750 0x7fca1c142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca1bfff700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 0x7fca1c156e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fca0c00b000 tx=0x7fca0c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca0c0016c0 con 0x7fca1c159d00 2022-08-01T22:56:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c001820 con 0x7fca1c159d00 2022-08-01T22:56:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c1569a0 con 0x7fca1c159d00 2022-08-01T22:56:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca0c001b80 con 0x7fca1c159d00 2022-08-01T22:56:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.127+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c13fe40 con 0x7fca1c159d00 2022-08-01T22:56:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.131+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c156b30 con 0x7fca1c159d00 2022-08-01T22:56:16.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.131+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fca0c001ce0 con 0x7fca1c159d00 2022-08-01T22:56:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.131+0000 7fca19ffb700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc9fc053aa0 0x7fc9fc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.131+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fca0c06c2a0 con 0x7fca1c159d00 2022-08-01T22:56:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.131+0000 7fca20c70700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc9fc053aa0 0x7fc9fc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.131+0000 7fca20c70700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc9fc053aa0 0x7fc9fc055f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fca10016000 tx=0x7fca10013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:16.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.139+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca0c042f50 con 0x7fca1c159d00 2022-08-01T22:56:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.299+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fca1c0ad770 con 0x7fca1c159d00 2022-08-01T22:56:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.323+0000 7fca19ffb700 1 -- 172.21.15.32:0/3185776597 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v43) v1 ==== 142+0+0 (secure 0 0 0) 0x7fca0c046570 con 0x7fca1c159d00 2022-08-01T22:56:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc9fc053aa0 msgr2=0x7fc9fc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc9fc053aa0 0x7fc9fc055f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fca10016000 tx=0x7fca10013000).stop 2022-08-01T22:56:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 msgr2=0x7fca1c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 0x7fca1c156e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fca0c00b000 tx=0x7fca0c009040).stop 2022-08-01T22:56:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 shutdown_connections 2022-08-01T22:56:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc9fc053aa0 0x7fc9fc055f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c1472c0 0x7fca1c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c159d00 0x7fca1c156e40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 --2- 172.21.15.32:0/3185776597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca08003750 0x7fca1c142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 >> 172.21.15.32:0/3185776597 conn(0x7fca1c05f840 msgr2=0x7fca1c1435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 shutdown_connections 2022-08-01T22:56:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.327+0000 7fca22ed4700 1 -- 172.21.15.32:0/3185776597 wait complete. 2022-08-01T22:56:16.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T22:56:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 Processor -- start 2022-08-01T22:56:16.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- start start 2022-08-01T22:56:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32000ab930 0x7f32000abd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3200144890 0x7f3200144d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3200060990 con 0x7f32000ac750 2022-08-01T22:56:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32000ad2d0 con 0x7f32000ab930 2022-08-01T22:56:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3200148120 con 0x7f3200144890 2022-08-01T22:56:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32000ab930 0x7f32000abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32000ab930 0x7f32000abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41474/0 (socket says 172.21.15.32:41474) 2022-08-01T22:56:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 -- 172.21.15.32:0/666975172 learned_addr learned my addr 172.21.15.32:0/666975172 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 -- 172.21.15.32:0/666975172 >> v2:172.21.15.55:3300/0 conn(0x7f3200144890 msgr2=0x7f3200144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 --2- 172.21.15.32:0/666975172 >> v2:172.21.15.55:3300/0 conn(0x7f3200144890 0x7f3200144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T22:56:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 -- 172.21.15.32:0/666975172 >> v1:172.21.15.32:6789/0 conn(0x7f32000ac750 legacy=0x7f32000acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 -- 172.21.15.32:0/666975172 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200060050 con 0x7f32000ab930 2022-08-01T22:56:16.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31ff7fe700 1 --2- 172.21.15.32:0/666975172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ab930 0x7f32000abd10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f32000aa690 tx=0x7f31f8009290).ready entity=mon.0 client_cookie=b8933985b100fd41 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:16.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31fe7fc700 1 -- 172.21.15.32:0/666975172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f800b820 con 0x7f32000ab930 2022-08-01T22:56:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31fe7fc700 1 -- 172.21.15.32:0/666975172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f800b980 con 0x7f32000ab930 2022-08-01T22:56:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f31fe7fc700 1 -- 172.21.15.32:0/666975172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f800baf0 con 0x7f32000ab930 2022-08-01T22:56:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- 172.21.15.32:0/666975172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ab930 msgr2=0x7f32000abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 --2- 172.21.15.32:0/666975172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ab930 0x7f32000abd10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f32000aa690 tx=0x7f31f8009290).stop 2022-08-01T22:56:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- 172.21.15.32:0/666975172 shutdown_connections 2022-08-01T22:56:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 --2- 172.21.15.32:0/666975172 >> v2:172.21.15.55:3300/0 conn(0x7f3200144890 0x7f3200144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T22:56:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 --2- 172.21.15.32:0/666975172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ab930 0x7f32000abd10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- 172.21.15.32:0/666975172 >> 172.21.15.32:0/666975172 conn(0x7f32000b2b00 msgr2=0x7f32000b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- 172.21.15.32:0/666975172 shutdown_connections 2022-08-01T22:56:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.459+0000 7f32060b7700 1 -- 172.21.15.32:0/666975172 wait complete. 2022-08-01T22:56:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 Processor -- start 2022-08-01T22:56:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 -- start start 2022-08-01T22:56:16.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32000ab930 0x7f32000adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 0x7f3200154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3200144890 0x7f3200154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32001574a0 con 0x7f32000ac750 2022-08-01T22:56:16.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200157740 con 0x7f3200144890 2022-08-01T22:56:16.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f32060b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32001579e0 con 0x7f32000ab930 2022-08-01T22:56:16.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f31feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 0x7f3200154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f31feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 0x7f3200154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41482/0 (socket says 172.21.15.32:41482) 2022-08-01T22:56:16.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.463+0000 7f31feffd700 1 -- 172.21.15.32:0/4251034909 learned_addr learned my addr 172.21.15.32:0/4251034909 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:16.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31ff7fe700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32000ab930 0x7f32000adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fffff700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3200144890 0x7f3200154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31feffd700 1 -- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32000ab930 msgr2=0x7f32000adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31feffd700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32000ab930 0x7f32000adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31feffd700 1 -- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3200144890 msgr2=0x7f3200154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31feffd700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3200144890 0x7f3200154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31feffd700 1 -- 172.21.15.32:0/4251034909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009530 con 0x7f32000ac750 2022-08-01T22:56:16.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31ff7fe700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32000ab930 0x7f32000adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:16.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fffff700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3200144890 0x7f3200154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31feffd700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 0x7f3200154170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f31e8002f70 tx=0x7f31e8006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31e8013460 con 0x7f32000ac750 2022-08-01T22:56:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80135c0 con 0x7f32000ac750 2022-08-01T22:56:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3200157c80 con 0x7f32000ac750 2022-08-01T22:56:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31e8013920 con 0x7f32000ac750 2022-08-01T22:56:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3200063c00 con 0x7f32000ac750 2022-08-01T22:56:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3200157e10 con 0x7f32000ac750 2022-08-01T22:56:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f31e801a060 con 0x7f32000ac750 2022-08-01T22:56:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fcff9700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31e4053aa0 0x7f31e4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.467+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f31e806b9c0 con 0x7f32000ac750 2022-08-01T22:56:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.471+0000 7f31ff7fe700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31e4053aa0 0x7f31e4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.471+0000 7f31ff7fe700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31e4053aa0 0x7f31e4055f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f31f8018000 tx=0x7f31f8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.475+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f31e8042780 con 0x7f32000ac750 2022-08-01T22:56:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.603+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3200148340 con 0x7f32000ac750 2022-08-01T22:56:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.611+0000 7f31fcff9700 1 -- 172.21.15.32:0/4251034909 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f31e8018020 con 0x7f32000ac750 2022-08-01T22:56:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31e4053aa0 msgr2=0x7f31e4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31e4053aa0 0x7f31e4055f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f31f8018000 tx=0x7f31f8016000).stop 2022-08-01T22:56:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 msgr2=0x7f3200154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 0x7f3200154170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f31e8002f70 tx=0x7f31e8006e20).stop 2022-08-01T22:56:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 shutdown_connections 2022-08-01T22:56:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f31e4053aa0 0x7f31e4055f60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3200144890 0x7f3200154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000ac750 0x7f3200154170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 --2- 172.21.15.32:0/4251034909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32000ab930 0x7f32000adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 >> 172.21.15.32:0/4251034909 conn(0x7f32000b2b00 msgr2=0x7f32000614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 shutdown_connections 2022-08-01T22:56:16.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:16.615+0000 7f32060b7700 1 -- 172.21.15.32:0/4251034909 wait complete. 2022-08-01T22:56:16.633 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T22:56:16.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:56:16.649 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:56:16.673 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:56:16.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:56:16.718 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T22:56:16.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:16.720 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:16.720 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:16.730 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:56:16.732 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:56:16.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:16.734 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:16.734 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:16.782 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:56:16.786 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:56:16.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:16.789 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:16.790 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:16.797 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T22:56:16.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:56:16.801 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T22:56:16.801 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:56:16.802 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T22:56:16.802 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:16.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:16.834 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:16.834 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:16.881 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T22:56:16.882 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T22:56:16.883 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T22:56:16.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:56:16.884 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:16.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:16.942 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:16.943 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:16.952 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T22:56:16.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T22:56:16.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:16.955 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:16.955 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:17.005 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T22:56:17.007 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T22:56:17.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:17.008 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:17.009 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:17.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:56:17.017 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2022-08-01T22:56:17.018 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2022-08-01T22:56:17.018 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T22:56:17.019 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T22:56:17.019 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:17.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:17.045 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:17.045 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:17.091 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T22:56:17.093 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T22:56:17.094 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T22:56:17.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:56:17.096 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:56:17.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T22:56:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 Processor -- start 2022-08-01T22:56:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- start start 2022-08-01T22:56:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff80b2710 0x7faff814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faff80b01a0 0x7faff814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff80a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff80a7250 con 0x7faff80b2710 2022-08-01T22:56:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff80a7480 con 0x7faff80b01a0 2022-08-01T22:56:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80a7720 con 0x7faff80afd60 2022-08-01T22:56:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff80b2710 0x7faff814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff80b2710 0x7faff814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41490/0 (socket says 172.21.15.32:41490) 2022-08-01T22:56:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 -- 172.21.15.32:0/1715602733 learned_addr learned my addr 172.21.15.32:0/1715602733 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd0c4700 1 --2- 172.21.15.32:0/1715602733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff80a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faff7fff700 1 --2- 172.21.15.32:0/1715602733 >> v2:172.21.15.55:3300/0 conn(0x7faff80b01a0 0x7faff814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 -- 172.21.15.32:0/1715602733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 msgr2=0x7faff80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 --2- 172.21.15.32:0/1715602733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff80a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 -- 172.21.15.32:0/1715602733 >> v2:172.21.15.55:3300/0 conn(0x7faff80b01a0 msgr2=0x7faff814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:17.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 --2- 172.21.15.32:0/1715602733 >> v2:172.21.15.55:3300/0 conn(0x7faff80b01a0 0x7faff814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 -- 172.21.15.32:0/1715602733 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80a79c0 con 0x7faff80b2710 2022-08-01T22:56:17.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffc8c3700 1 --2- 172.21.15.32:0/1715602733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff814afd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fafe0002f70 tx=0x7fafe0006eb0).ready entity=mon.0 client_cookie=32308c7259e7e761 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:17.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faff77fe700 1 -- 172.21.15.32:0/1715602733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafe000d570 con 0x7faff80b2710 2022-08-01T22:56:17.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faff77fe700 1 -- 172.21.15.32:0/1715602733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0000e90 con 0x7faff80b2710 2022-08-01T22:56:17.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faff77fe700 1 -- 172.21.15.32:0/1715602733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafe000d900 con 0x7faff80b2710 2022-08-01T22:56:17.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- 172.21.15.32:0/1715602733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 msgr2=0x7faff814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- 172.21.15.32:0/1715602733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff814afd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fafe0002f70 tx=0x7fafe0006eb0).stop 2022-08-01T22:56:17.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- 172.21.15.32:0/1715602733 shutdown_connections 2022-08-01T22:56:17.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- 172.21.15.32:0/1715602733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff80a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- 172.21.15.32:0/1715602733 >> v2:172.21.15.55:3300/0 conn(0x7faff80b01a0 0x7faff814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 --2- 172.21.15.32:0/1715602733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff814afd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- 172.21.15.32:0/1715602733 >> 172.21.15.32:0/1715602733 conn(0x7faff8060310 msgr2=0x7faff80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:17.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- 172.21.15.32:0/1715602733 shutdown_connections 2022-08-01T22:56:17.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.219+0000 7faffd8c5700 1 -- 172.21.15.32:0/1715602733 wait complete. 2022-08-01T22:56:17.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 Processor -- start 2022-08-01T22:56:17.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 -- start start 2022-08-01T22:56:17.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff805c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80b01a0 0x7faff80aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff80ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80b59e0 con 0x7faff80b2710 2022-08-01T22:56:17.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80b2ff0 con 0x7faff80b01a0 2022-08-01T22:56:17.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80aa610 con 0x7faff80afd60 2022-08-01T22:56:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff80ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffc8c3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff805c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff80ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41498/0 (socket says 172.21.15.32:41498) 2022-08-01T22:56:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 -- 172.21.15.32:0/2921810289 learned_addr learned my addr 172.21.15.32:0/2921810289 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faff7fff700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80b01a0 0x7faff80aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 -- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 msgr2=0x7faff805c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff805c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 -- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80b01a0 msgr2=0x7faff80aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80b01a0 0x7faff80aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 -- 172.21.15.32:0/2921810289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe000b1e0 con 0x7faff80b2710 2022-08-01T22:56:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd0c4700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff80ab1f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7faff80a9880 tx=0x7faff0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff0011780 con 0x7faff80b2710 2022-08-01T22:56:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff00118e0 con 0x7faff80b2710 2022-08-01T22:56:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff80aedd0 con 0x7faff80b2710 2022-08-01T22:56:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff0011c40 con 0x7faff80b2710 2022-08-01T22:56:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.223+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff80b31e0 con 0x7faff80b2710 2022-08-01T22:56:17.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.227+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff80574e0 con 0x7faff80b2710 2022-08-01T22:56:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.227+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faff0012420 con 0x7faff80b2710 2022-08-01T22:56:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.227+0000 7faff5ffb700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafdc053aa0 0x7fafdc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.227+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7faff006be00 con 0x7faff80b2710 2022-08-01T22:56:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.227+0000 7faffc8c3700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafdc053aa0 0x7fafdc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.227+0000 7faffc8c3700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafdc053aa0 0x7fafdc055f60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fafe001c000 tx=0x7fafe0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.231+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faff0042a50 con 0x7faff80b2710 2022-08-01T22:56:17.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.359+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faff80af8b0 con 0x7faff80b2710 2022-08-01T22:56:17.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.359+0000 7faff5ffb700 1 -- 172.21.15.32:0/2921810289 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+34065 (secure 0 0 0) 0x7faff0046070 con 0x7faff80b2710 2022-08-01T22:56:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:17.369 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:56:17.370 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 24, 2022-08-01T22:56:17.370 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T22:56:17.370 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T22:56:17.371 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T22:55:50.679882+0000", 2022-08-01T22:56:17.371 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:56:17.371 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:56:17.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T22:56:17.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T22:56:17.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T22:56:17.372 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T22:56:17.372 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T22:56:17.373 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T22:56:17.373 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T22:56:17.373 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:56:17.373 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T22:56:17.374 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T22:56:17.374 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T22:56:17.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T22:56:17.374 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T22:56:17.375 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 4, 2022-08-01T22:56:17.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T22:56:17.375 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T22:56:17.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T22:56:17.376 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T22:56:17.376 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T22:56:17.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.376 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T22:56:17.377 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T22:56:17.377 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T22:56:17.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:56:17.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:56:17.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:56:17.378 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:56:17.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:56:17.379 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:56:17.379 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:56:17.380 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:56:17.380 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T22:56:17.380 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T22:56:17.381 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T22:56:17.381 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T22:56:17.382 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T22:56:17.382 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:56:17.383 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:56:17.383 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:56:17.383 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:56:17.384 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:56:17.384 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:56:17.385 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:56:17.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.385 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T22:56:17.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:56:17.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:56:17.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:56:17.387 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:56:17.388 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:56:17.388 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:56:17.388 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:56:17.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:56:17.389 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:56:17.390 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:56:17.390 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:56:17.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:56:17.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:56:17.391 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:56:17.392 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:56:17.392 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:56:17.392 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:56:17.393 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:56:17.393 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:56:17.394 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:56:17.394 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:56:17.394 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:56:17.395 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:56:17.395 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:56:17.396 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:56:17.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:56:17.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.397 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:56:17.397 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:56:17.398 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:56:17.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:56:17.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:56:17.399 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:56:17.400 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:56:17.400 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:56:17.401 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:56:17.401 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:56:17.402 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:56:17.402 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:56:17.403 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T22:56:17.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.404 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:56:17.404 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T22:56:17.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.406 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T22:56:17.406 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T22:56:17.407 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T22:56:17.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:56:17.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:56:17.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:56:17.409 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:56:17.409 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:56:17.410 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:56:17.410 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:56:17.411 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:56:17.411 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:56:17.411 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:56:17.412 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:56:17.412 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:56:17.413 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:56:17.413 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:56:17.414 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:56:17.414 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:56:17.415 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:56:17.415 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:56:17.415 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:56:17.416 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:56:17.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T22:56:17.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:56:17.418 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:56:17.418 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:56:17.418 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:56:17.419 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:56:17.419 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:56:17.419 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:56:17.419 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:56:17.419 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:56:17.420 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:56:17.420 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:56:17.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:56:17.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:56:17.421 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:56:17.421 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:56:17.421 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:56:17.421 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:56:17.421 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:56:17.422 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:56:17.422 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:56:17.422 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:56:17.422 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:56:17.423 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:56:17.423 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:56:17.423 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:56:17.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:56:17.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.424 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:56:17.424 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:56:17.424 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:56:17.424 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:56:17.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:56:17.425 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:56:17.425 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:56:17.425 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:56:17.426 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:56:17.426 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:56:17.426 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:56:17.426 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:56:17.426 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:56:17.427 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T22:56:17.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2022-08-01T22:56:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T22:56:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:45.604869+0000", 2022-08-01T22:56:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:56:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:56:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:56:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:56:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:56:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:56:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:56:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:56:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:56:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:56:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:56:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:56:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:56:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:56:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:56:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:56:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:56:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:56:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:56:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:56:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "24", 2022-08-01T22:56:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:56:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:56:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:56:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:56:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:56:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:56:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:56:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:56:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:56:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:56:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:56:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:56:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:56:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:56:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:56:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:56:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:56:17.438 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:56:17.438 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:56:17.438 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:56:17.438 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:56:17.439 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:56:17.439 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:56:17.439 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:56:17.439 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:56:17.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:56:17.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.440 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:56:17.440 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:56:17.440 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:56:17.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:56:17.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:56:17.441 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:56:17.441 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:56:17.442 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:56:17.442 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:56:17.442 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:56:17.443 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:56:17.443 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:56:17.443 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T22:56:17.443 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T22:56:17.444 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T22:56:17.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.444 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:56:17.444 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T22:56:17.445 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T22:56:17.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.447 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 4, 2022-08-01T22:56:17.448 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T22:56:17.448 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:48.366315+0000", 2022-08-01T22:56:17.448 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 5, 2022-08-01T22:56:17.448 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,ec_overwrites", 2022-08-01T22:56:17.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 3, 2022-08-01T22:56:17.449 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 4, 2022-08-01T22:56:17.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 3, 2022-08-01T22:56:17.449 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 1, 2022-08-01T22:56:17.450 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:56:17.450 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:56:17.450 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:56:17.450 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:56:17.450 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:56:17.451 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:56:17.451 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:56:17.451 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:56:17.451 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:56:17.452 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:56:17.452 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:56:17.452 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:56:17.452 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:56:17.452 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:56:17.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.453 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "24", 2022-08-01T22:56:17.453 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:56:17.454 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:56:17.454 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:56:17.454 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:56:17.454 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:56:17.454 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:56:17.455 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:56:17.455 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:56:17.455 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:56:17.455 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:56:17.456 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:56:17.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:56:17.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:56:17.456 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:56:17.456 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:56:17.457 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:56:17.457 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:56:17.457 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:56:17.457 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:56:17.458 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:56:17.458 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:56:17.458 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:56:17.458 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:56:17.458 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "cephfs_data", 2022-08-01T22:56:17.459 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:56:17.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:56:17.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.459 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:56:17.460 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:56:17.460 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:56:17.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:56:17.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:56:17.460 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:56:17.461 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:56:17.461 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:56:17.461 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 8192, 2022-08-01T22:56:17.461 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:56:17.461 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:56:17.462 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:56:17.462 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:56:17.462 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T22:56:17.462 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T22:56:17.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.463 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:56:17.463 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T22:56:17.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.464 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:56:17.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T22:56:17.464 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.465 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.466 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2022-08-01T22:56:17.466 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.466 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.467 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T22:56:17.468 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.469 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T22:56:17.469 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.469 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.470 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.470 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.470 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T22:56:17.471 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T22:56:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T22:56:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.475 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T22:56:17.475 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.476 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.481 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T22:56:17.481 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.482 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T22:56:17.482 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:56:17.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T22:56:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T22:56:17.483 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T22:56:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T22:56:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.484 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:56:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T22:56:17.485 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.486 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2022-08-01T22:56:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.488 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T22:56:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.489 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T22:56:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.491 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T22:56:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T22:56:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.493 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.494 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T22:56:17.495 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.496 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T22:56:17.496 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.497 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.498 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.498 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T22:56:17.498 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.499 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T22:56:17.500 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:56:17.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.500 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.500 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T22:56:17.501 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T22:56:17.501 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T22:56:17.501 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T22:56:17.501 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.501 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.502 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.503 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:56:17.503 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T22:56:17.503 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.504 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.504 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.504 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.505 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.505 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 16, 2022-08-01T22:56:17.505 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.505 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.505 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T22:56:17.507 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.507 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T22:56:17.508 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.508 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.509 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.510 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T22:56:17.510 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T22:56:17.511 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.512 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.512 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T22:56:17.513 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.514 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T22:56:17.515 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.515 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T22:56:17.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.519 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T22:56:17.519 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:56:17.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.520 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T22:56:17.521 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T22:56:17.521 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T22:56:17.522 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T22:56:17.522 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.522 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.523 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.524 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:56:17.524 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T22:56:17.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.525 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.525 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.526 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.527 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.527 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2022-08-01T22:56:17.527 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.528 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.528 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T22:56:17.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T22:56:17.531 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T22:56:17.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T22:56:17.537 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.538 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.539 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T22:56:17.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T22:56:17.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.542 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T22:56:17.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T22:56:17.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:56:17.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.545 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T22:56:17.546 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T22:56:17.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T22:56:17.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T22:56:17.546 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.546 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.547 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.548 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:56:17.548 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T22:56:17.548 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.548 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.548 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.549 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.549 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.549 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.549 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.549 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2022-08-01T22:56:17.550 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.550 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.550 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T22:56:17.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T22:56:17.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T22:56:17.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T22:56:17.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.556 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T22:56:17.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T22:56:17.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T22:56:17.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T22:56:17.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:56:17.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.562 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T22:56:17.562 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T22:56:17.563 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T22:56:17.563 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T22:56:17.563 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.563 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.563 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:56:17.565 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T22:56:17.565 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.565 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.565 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.565 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.566 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.566 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 19, 2022-08-01T22:56:17.566 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.567 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.567 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T22:56:17.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T22:56:17.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.570 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T22:56:17.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T22:56:17.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T22:56:17.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T22:56:17.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.576 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T22:56:17.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T22:56:17.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:56:17.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.579 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T22:56:17.579 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T22:56:17.579 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T22:56:17.579 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T22:56:17.580 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.580 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.580 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.581 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:56:17.581 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T22:56:17.581 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.582 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.582 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.582 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.583 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 22, 2022-08-01T22:56:17.583 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.583 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.583 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T22:56:17.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T22:56:17.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T22:56:17.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T22:56:17.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.589 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T22:56:17.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T22:56:17.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.593 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T22:56:17.593 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.594 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T22:56:17.595 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:56:17.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.596 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T22:56:17.596 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T22:56:17.596 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T22:56:17.597 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T22:56:17.597 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.597 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.597 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.598 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:56:17.598 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T22:56:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:56:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:56:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:56:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:56:17.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:56:17.600 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:56:17.600 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:56:17.600 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 16, 2022-08-01T22:56:17.600 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:56:17.601 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:56:17.601 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:56:17.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T22:56:17.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T22:56:17.604 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.605 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:56:17.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T22:56:17.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T22:56:17.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:56:17.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.609 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T22:56:17.609 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.610 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T22:56:17.610 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.611 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:56:17.611 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:56:17.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:56:17.612 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T22:56:17.612 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:56:17.613 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T22:56:17.613 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:56:17.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.614 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T22:56:17.614 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T22:56:17.615 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T22:56:17.615 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T22:56:17.615 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:56:17.615 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:56:17.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:56:17.616 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:17.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.616 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:56:17.616 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T22:56:17.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.617 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:56:17.617 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.617 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.617 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.618 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.618 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.618 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T22:56:17.619 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:56:17.619 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.620 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.620 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.620 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.620 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.621 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T22:56:17.621 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.621 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:56:17.622 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.622 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.622 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.622 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.623 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.623 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T22:56:17.623 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.624 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:56:17.624 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.624 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.624 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.625 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.625 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T22:56:17.625 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:56:17.626 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.626 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.627 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.627 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.627 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.627 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T22:56:17.627 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.628 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:56:17.628 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.629 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.629 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.629 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.629 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.629 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T22:56:17.630 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:56:17.631 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.631 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.631 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.631 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.632 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.632 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T22:56:17.632 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:17.633 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:56:17.633 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:56:17.633 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:56:17.633 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:56:17.634 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:56:17.634 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:56:17.634 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T22:56:17.634 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:56:17.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:56:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T22:56:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T22:56:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T22:56:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T22:56:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T22:56:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T22:56:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T22:56:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T22:56:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:56:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T22:56:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T22:56:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:56:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T22:56:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:56:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:56:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T22:56:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T22:56:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T22:56:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:56:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:56:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T22:56:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:56:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:56:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T22:56:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T22:56:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T22:56:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T22:56:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T22:56:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T22:56:17.642 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:56:17.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafdc053aa0 msgr2=0x7fafdc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafdc053aa0 0x7fafdc055f60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fafe001c000 tx=0x7fafe0015000).stop 2022-08-01T22:56:17.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 msgr2=0x7faff80ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff80ab1f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7faff80a9880 tx=0x7faff0008040).stop 2022-08-01T22:56:17.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 shutdown_connections 2022-08-01T22:56:17.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafdc053aa0 0x7fafdc055f60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80b2710 0x7faff80ab1f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80b01a0 0x7faff80aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 --2- 172.21.15.32:0/2921810289 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff80afd60 0x7faff805c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 >> 172.21.15.32:0/2921810289 conn(0x7faff8060310 msgr2=0x7faff814a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 shutdown_connections 2022-08-01T22:56:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.363+0000 7faffd8c5700 1 -- 172.21.15.32:0/2921810289 wait complete. 2022-08-01T22:56:17.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8041730700 1 Processor -- start 2022-08-01T22:56:17.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8041730700 1 -- start start 2022-08-01T22:56:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8041730700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 0x7f803c0610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8041730700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c066af0 con 0x7f803c0b6510 2022-08-01T22:56:17.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8041730700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c066d90 con 0x7f803c13d9b0 2022-08-01T22:56:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8041730700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c067030 con 0x7f803c0b0e60 2022-08-01T22:56:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f803c0b6510 0x7f803c13d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57276/0 (socket says 172.21.15.32:57276) 2022-08-01T22:56:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 -- 172.21.15.32:0/4155257724 learned_addr learned my addr 172.21.15.32:0/4155257724 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8040f2f700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 0x7f803c0610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999648474 0 0) 0x7f803c066af0 con 0x7f803c0b6510 2022-08-01T22:56:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8028003260 con 0x7f803c0b6510 2022-08-01T22:56:17.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684953549 0 0) 0x7f803c066d90 con 0x7f803c13d9b0 2022-08-01T22:56:17.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803c066af0 con 0x7f803c13d9b0 2022-08-01T22:56:17.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3259793502 0 0) 0x7f8028003260 con 0x7f803c0b6510 2022-08-01T22:56:17.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 msgr2=0x7f803c0610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:17.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 0x7f803c0610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 >> v1:172.21.15.55:6789/0 conn(0x7f803c13d9b0 legacy=0x7f803c0609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0672d0 con 0x7f803c0b6510 2022-08-01T22:56:17.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f802c001680 con 0x7f803c0b6510 2022-08-01T22:56:17.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 >> v1:172.21.15.32:6789/0 conn(0x7f803c0b6510 legacy=0x7f803c13d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028003910 0x7f8028003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028004500 0x7f8028004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 0x7f8028004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c067030 con 0x7f8028003910 2022-08-01T22:56:17.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8028004320 con 0x7f8028004500 2022-08-01T22:56:17.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802800fd90 con 0x7f803c0b0e60 2022-08-01T22:56:17.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028003910 0x7f8028003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f8040f2f700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 0x7f8028004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803b7fe700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028004500 0x7f8028004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 -- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 msgr2=0x7f8028004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:17.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c0b0e60 0x7f8028004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 -- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028004500 msgr2=0x7f8028004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028004500 0x7f8028004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 -- 172.21.15.32:0/4155257724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028003260 con 0x7f8028003910 2022-08-01T22:56:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803b7fe700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028004500 0x7f8028004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803bfff700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028003910 0x7f8028003d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f802c001450 tx=0x7f803c13ff70).ready entity=mon.0 client_cookie=afd8bc3ebcfd2721 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802c007170 con 0x7f8028003910 2022-08-01T22:56:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.759+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c0072d0 con 0x7f8028003910 2022-08-01T22:56:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803affd700 1 -- 172.21.15.32:0/4155257724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802c00c8f0 con 0x7f8028003910 2022-08-01T22:56:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- 172.21.15.32:0/4155257724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028003910 msgr2=0x7f8028003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 --2- 172.21.15.32:0/4155257724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8028003910 0x7f8028003d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f802c001450 tx=0x7f803c13ff70).stop 2022-08-01T22:56:17.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8040f2f700 1 -- 172.21.15.32:0/4155257724 reap_dead start 2022-08-01T22:56:17.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- 172.21.15.32:0/4155257724 shutdown_connections 2022-08-01T22:56:17.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- 172.21.15.32:0/4155257724 >> 172.21.15.32:0/4155257724 conn(0x7f803c063800 msgr2=0x7f803c063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:17.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- 172.21.15.32:0/4155257724 shutdown_connections 2022-08-01T22:56:17.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- 172.21.15.32:0/4155257724 wait complete. 2022-08-01T22:56:17.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 Processor -- start 2022-08-01T22:56:17.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- start start 2022-08-01T22:56:17.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028003910 0x7f803c142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 0x7f803c13c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c13cc40 0x7f803c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0aa140 con 0x7f803c0ae050 2022-08-01T22:56:17.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0abc80 con 0x7f8028003910 2022-08-01T22:56:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8041730700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0a9ea0 con 0x7f803c13cc40 2022-08-01T22:56:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 0x7f803c13c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 0x7f803c13c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41514/0 (socket says 172.21.15.32:41514) 2022-08-01T22:56:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 -- 172.21.15.32:0/2843266557 learned_addr learned my addr 172.21.15.32:0/2843266557 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f8040f2f700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c13cc40 0x7f803c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803bfff700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028003910 0x7f803c142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 -- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c13cc40 msgr2=0x7f803c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c13cc40 0x7f803c14df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 -- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028003910 msgr2=0x7f803c142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028003910 0x7f803c142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 -- 172.21.15.32:0/2843266557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c13d0c0 con 0x7f803c0ae050 2022-08-01T22:56:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.763+0000 7f803b7fe700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 0x7f803c13c750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f803000b000 tx=0x7f8030009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8030014620 con 0x7f803c0ae050 2022-08-01T22:56:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030014780 con 0x7f803c0ae050 2022-08-01T22:56:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8030014ae0 con 0x7f803c0ae050 2022-08-01T22:56:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c143280 con 0x7f803c0ae050 2022-08-01T22:56:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c14e460 con 0x7f803c0ae050 2022-08-01T22:56:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c0574e0 con 0x7f803c0ae050 2022-08-01T22:56:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8030014c40 con 0x7f803c0ae050 2022-08-01T22:56:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f80397fa700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f801c053a70 0x7f801c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:17.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f803006c2b0 con 0x7f803c0ae050 2022-08-01T22:56:17.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.767+0000 7f803bfff700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f801c053a70 0x7f801c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.771+0000 7f803bfff700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f801c053a70 0x7f801c055f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f802c01b000 tx=0x7f802c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.775+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8030042f00 con 0x7f803c0ae050 2022-08-01T22:56:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.939+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f803c06ac80 con 0x7f803c0ae050 2022-08-01T22:56:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f80397fa700 1 -- 172.21.15.32:0/2843266557 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f8030046520 con 0x7f803c0ae050 2022-08-01T22:56:17.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:17.950 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3750794992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3750794992},{"type":"v1","addr":"172.21.15.32:6835","nonce":3750794992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3123630923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3123630923},{"type":"v1","addr":"172.21.15.55:6835","nonce":3123630923}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:55:50.680148+0000","modified":"2022-08-01T22:55:52.352879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/2632088620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2632088620},{"type":"v1","addr":"172.21.15.32:6833","nonce":2632088620}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T22:56:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f801c053a70 msgr2=0x7f801c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f801c053a70 0x7f801c055f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f802c01b000 tx=0x7f802c015000).stop 2022-08-01T22:56:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 msgr2=0x7f803c13c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 0x7f803c13c750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f803000b000 tx=0x7f8030009040).stop 2022-08-01T22:56:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 shutdown_connections 2022-08-01T22:56:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f801c053a70 0x7f801c055f30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f803c13cc40 0x7f803c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c0ae050 0x7f803c13c750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 --2- 172.21.15.32:0/2843266557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8028003910 0x7f803c142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 >> 172.21.15.32:0/2843266557 conn(0x7f803c063800 msgr2=0x7f803c0607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 shutdown_connections 2022-08-01T22:56:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:17.943+0000 7f8041730700 1 -- 172.21.15.32:0/2843266557 wait complete. 2022-08-01T22:56:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2022-08-01T22:56:17.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T22:56:18.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 Processor -- start 2022-08-01T22:56:18.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 -- start start 2022-08-01T22:56:18.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40619b0 0x7f96c40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96c40b6ee0 0x7f96c4143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96c40b5ed0 0x7f96c40b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c4142f60 con 0x7f96c40b5ed0 2022-08-01T22:56:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c4061d90 con 0x7f96c40b6ee0 2022-08-01T22:56:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.063+0000 7f96cac70700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40ba5a0 con 0x7f96c40619b0 2022-08-01T22:56:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96c40b5ed0 0x7f96c40b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96c40b5ed0 0x7f96c40b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41524/0 (socket says 172.21.15.32:41524) 2022-08-01T22:56:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 -- 172.21.15.32:0/1250590490 learned_addr learned my addr 172.21.15.32:0/1250590490 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96c946d700 1 --2- 172.21.15.32:0/1250590490 >> v2:172.21.15.55:3300/0 conn(0x7f96c40b6ee0 0x7f96c4143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96c9c6e700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40619b0 0x7f96c40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 -- 172.21.15.32:0/1250590490 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40619b0 msgr2=0x7f96c40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40619b0 0x7f96c40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 -- 172.21.15.32:0/1250590490 >> v2:172.21.15.55:3300/0 conn(0x7f96c40b6ee0 msgr2=0x7f96c4143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 --2- 172.21.15.32:0/1250590490 >> v2:172.21.15.55:3300/0 conn(0x7f96c40b6ee0 0x7f96c4143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 -- 172.21.15.32:0/1250590490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4143150 con 0x7f96c40b5ed0 2022-08-01T22:56:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96c9c6e700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40619b0 0x7f96c40b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96ca46f700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40b5ed0 0x7f96c40b5190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f96bc0032f0 tx=0x7f96bc009290).ready entity=mon.0 client_cookie=f3f68ad6ae53b1e2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96c8c6c700 1 -- 172.21.15.32:0/1250590490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96bc00b870 con 0x7f96c40b5ed0 2022-08-01T22:56:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96c8c6c700 1 -- 172.21.15.32:0/1250590490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc00b9d0 con 0x7f96c40b5ed0 2022-08-01T22:56:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96c8c6c700 1 -- 172.21.15.32:0/1250590490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96bc00bb40 con 0x7f96c40b5ed0 2022-08-01T22:56:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 -- 172.21.15.32:0/1250590490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40b5ed0 msgr2=0x7f96c40b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40b5ed0 0x7f96c40b5190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f96bc0032f0 tx=0x7f96bc009290).stop 2022-08-01T22:56:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 -- 172.21.15.32:0/1250590490 shutdown_connections 2022-08-01T22:56:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40b5ed0 0x7f96c40b5190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 --2- 172.21.15.32:0/1250590490 >> v2:172.21.15.55:3300/0 conn(0x7f96c40b6ee0 0x7f96c4143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 --2- 172.21.15.32:0/1250590490 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40619b0 0x7f96c40b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 -- 172.21.15.32:0/1250590490 >> 172.21.15.32:0/1250590490 conn(0x7f96c406af80 msgr2=0x7f96c405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 -- 172.21.15.32:0/1250590490 shutdown_connections 2022-08-01T22:56:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 -- 172.21.15.32:0/1250590490 wait complete. 2022-08-01T22:56:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 Processor -- start 2022-08-01T22:56:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.067+0000 7f96cac70700 1 -- start start 2022-08-01T22:56:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 0x7f96c4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c40b5ed0 0x7f96c40af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40b6ee0 0x7f96c41414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40afa00 con 0x7f96c40619b0 2022-08-01T22:56:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4146d80 con 0x7f96c40b5ed0 2022-08-01T22:56:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c41530a0 con 0x7f96c40b6ee0 2022-08-01T22:56:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 0x7f96c4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96ca46f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40b6ee0 0x7f96c41414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 0x7f96c4155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41526/0 (socket says 172.21.15.32:41526) 2022-08-01T22:56:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c946d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c40b5ed0 0x7f96c40af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 -- 172.21.15.32:0/708769633 learned_addr learned my addr 172.21.15.32:0/708769633 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 -- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40b6ee0 msgr2=0x7f96c41414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40b6ee0 0x7f96c41414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 -- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c40b5ed0 msgr2=0x7f96c40af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c40b5ed0 0x7f96c40af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 -- 172.21.15.32:0/708769633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc009530 con 0x7f96c40619b0 2022-08-01T22:56:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96ca46f700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40b6ee0 0x7f96c41414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c946d700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c40b5ed0 0x7f96c40af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96c9c6e700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 0x7f96c4155a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f96b4006940 tx=0x7f96b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96b4011780 con 0x7f96c40619b0 2022-08-01T22:56:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b40118e0 con 0x7f96c40619b0 2022-08-01T22:56:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c4141990 con 0x7f96c40619b0 2022-08-01T22:56:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96b4011c40 con 0x7f96c40619b0 2022-08-01T22:56:18.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.071+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c4112bd0 con 0x7f96c40619b0 2022-08-01T22:56:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.075+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c4152ee0 con 0x7f96c40619b0 2022-08-01T22:56:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.075+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96b401c460 con 0x7f96c40619b0 2022-08-01T22:56:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.075+0000 7f96b2ffd700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96a4053a10 0x7f96a4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.075+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f96b406c230 con 0x7f96c40619b0 2022-08-01T22:56:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.075+0000 7f96c946d700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96a4053a10 0x7f96a4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.075+0000 7f96c946d700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96a4053a10 0x7f96a4055ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f96c000b000 tx=0x7f96c0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.083+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96b4042e80 con 0x7f96c40619b0 2022-08-01T22:56:18.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.247+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f96c40574e0 con 0x7f96c40619b0 2022-08-01T22:56:18.330 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T22:56:18.323+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T22:56:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.343+0000 7f96b2ffd700 1 -- 172.21.15.32:0/708769633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f96b40464a0 con 0x7f96c40619b0 2022-08-01T22:56:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96a4053a10 msgr2=0x7f96a4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96a4053a10 0x7f96a4055ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f96c000b000 tx=0x7f96c0006df0).stop 2022-08-01T22:56:18.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 msgr2=0x7f96c4155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 0x7f96c4155a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f96b4006940 tx=0x7f96b4008040).stop 2022-08-01T22:56:18.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 shutdown_connections 2022-08-01T22:56:18.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96a4053a10 0x7f96a4055ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96c40b6ee0 0x7f96c41414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c40b5ed0 0x7f96c40af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 --2- 172.21.15.32:0/708769633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c40619b0 0x7f96c4155a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 >> 172.21.15.32:0/708769633 conn(0x7f96c406af80 msgr2=0x7f96c40b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:18.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 shutdown_connections 2022-08-01T22:56:18.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.347+0000 7f96cac70700 1 -- 172.21.15.32:0/708769633 wait complete. 2022-08-01T22:56:18.359 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T22:56:18.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:18.383 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T22:56:18.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.467+0000 7f838a966700 1 Processor -- start 2022-08-01T22:56:18.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.467+0000 7f838a966700 1 -- start start 2022-08-01T22:56:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83840ad750 0x7f8384061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83841405d0 0x7f8384140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83840aee80 con 0x7f83840614b0 2022-08-01T22:56:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840b70b0 con 0x7f83840ad750 2022-08-01T22:56:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840af0b0 con 0x7f83841405d0 2022-08-01T22:56:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83840ad750 0x7f8384061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83840ad750 0x7f8384061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41554/0 (socket says 172.21.15.32:41554) 2022-08-01T22:56:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 -- 172.21.15.32:0/3806344706 learned_addr learned my addr 172.21.15.32:0/3806344706 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a165700 1 --2- 172.21.15.32:0/3806344706 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83841405d0 0x7f8384140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 -- 172.21.15.32:0/3806344706 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83841405d0 msgr2=0x7f8384140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 --2- 172.21.15.32:0/3806344706 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83841405d0 0x7f8384140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 -- 172.21.15.32:0/3806344706 >> v1:172.21.15.55:6789/0 conn(0x7f83840614b0 legacy=0x7f83840b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 -- 172.21.15.32:0/3806344706 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840af350 con 0x7f83840ad750 2022-08-01T22:56:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8389163700 1 --2- 172.21.15.32:0/3806344706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83840ad750 0x7f8384061890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f8374002f70 tx=0x7f8374006eb0).ready entity=mon.0 client_cookie=f2cb5abcbe3abcd1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8388962700 1 -- 172.21.15.32:0/3806344706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837400d5c0 con 0x7f83840ad750 2022-08-01T22:56:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8388962700 1 -- 172.21.15.32:0/3806344706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8374000e90 con 0x7f83840ad750 2022-08-01T22:56:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f8388962700 1 -- 172.21.15.32:0/3806344706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837400d950 con 0x7f83840ad750 2022-08-01T22:56:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- 172.21.15.32:0/3806344706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83840ad750 msgr2=0x7f8384061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 --2- 172.21.15.32:0/3806344706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83840ad750 0x7f8384061890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f8374002f70 tx=0x7f8374006eb0).stop 2022-08-01T22:56:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- 172.21.15.32:0/3806344706 shutdown_connections 2022-08-01T22:56:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 --2- 172.21.15.32:0/3806344706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83840ad750 0x7f8384061890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 --2- 172.21.15.32:0/3806344706 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83841405d0 0x7f8384140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- 172.21.15.32:0/3806344706 >> 172.21.15.32:0/3806344706 conn(0x7f8384066f70 msgr2=0x7f8384067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:18.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- 172.21.15.32:0/3806344706 shutdown_connections 2022-08-01T22:56:18.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.471+0000 7f838a966700 1 -- 172.21.15.32:0/3806344706 wait complete. 2022-08-01T22:56:18.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 Processor -- start 2022-08-01T22:56:18.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 -- start start 2022-08-01T22:56:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83840614b0 0x7f8384153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83840ad750 0x7f83840b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 0x7f838409ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b3970 con 0x7f83841405d0 2022-08-01T22:56:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8384152560 con 0x7f83840ad750 2022-08-01T22:56:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8384152750 con 0x7f83840614b0 2022-08-01T22:56:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 0x7f838409ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 0x7f838409ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41560/0 (socket says 172.21.15.32:41560) 2022-08-01T22:56:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 -- 172.21.15.32:0/3116596597 learned_addr learned my addr 172.21.15.32:0/3116596597 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f8389964700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83840614b0 0x7f8384153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f8389163700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83840ad750 0x7f83840b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 -- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83840614b0 msgr2=0x7f8384153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83840614b0 0x7f8384153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 -- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83840ad750 msgr2=0x7f83840b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83840ad750 0x7f83840b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 -- 172.21.15.32:0/3116596597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837400b1e0 con 0x7f83841405d0 2022-08-01T22:56:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f8389163700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83840ad750 0x7f83840b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a165700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 0x7f838409ea40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f837c00a000 tx=0x7f837c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c0127e0 con 0x7f83841405d0 2022-08-01T22:56:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c012940 con 0x7f83841405d0 2022-08-01T22:56:18.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83841529f0 con 0x7f83841405d0 2022-08-01T22:56:18.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c012ca0 con 0x7f83841405d0 2022-08-01T22:56:18.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.475+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83840b8220 con 0x7f83841405d0 2022-08-01T22:56:18.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.479+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83840af5f0 con 0x7f83841405d0 2022-08-01T22:56:18.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.479+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f837c013480 con 0x7f83841405d0 2022-08-01T22:56:18.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.479+0000 7f8372ffd700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8364053a70 0x7f8364055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.479+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f837c06c500 con 0x7f83841405d0 2022-08-01T22:56:18.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.479+0000 7f8389964700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8364053a70 0x7f8364055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.479+0000 7f8389964700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8364053a70 0x7f8364055f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f838000d000 tx=0x7f8380007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.487+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f837c043160 con 0x7f83841405d0 2022-08-01T22:56:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83840574e0 con 0x7f83841405d0 2022-08-01T22:56:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f8372ffd700 1 -- 172.21.15.32:0/3116596597 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2126 (secure 0 0 0) 0x7f837c012e00 con 0x7f83841405d0 2022-08-01T22:56:18.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:18.657 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3750794992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3750794992},{"type":"v1","addr":"172.21.15.32:6835","nonce":3750794992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3123630923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3123630923},{"type":"v1","addr":"172.21.15.55:6835","nonce":3123630923}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:55:50.680148+0000","modified":"2022-08-01T22:56:18.329369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T22:56:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8364053a70 msgr2=0x7f8364055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8364053a70 0x7f8364055f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f838000d000 tx=0x7f8380007b90).stop 2022-08-01T22:56:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 msgr2=0x7f838409ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 0x7f838409ea40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f837c00a000 tx=0x7f837c008040).stop 2022-08-01T22:56:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 shutdown_connections 2022-08-01T22:56:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8364053a70 0x7f8364055f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83841405d0 0x7f838409ea40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83840ad750 0x7f83840b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 --2- 172.21.15.32:0/3116596597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83840614b0 0x7f8384153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 >> 172.21.15.32:0/3116596597 conn(0x7f8384066f70 msgr2=0x7f838405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:18.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 shutdown_connections 2022-08-01T22:56:18.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.651+0000 7f838a966700 1 -- 172.21.15.32:0/3116596597 wait complete. 2022-08-01T22:56:18.667 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 7 2022-08-01T22:56:18.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T22:56:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21e72a700 1 Processor -- start 2022-08-01T22:56:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21e72a700 1 -- start start 2022-08-01T22:56:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21e72a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa218146ae0 0x7fa2181403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21e72a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa218143cf0 con 0x7fa2180b4820 2022-08-01T22:56:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21e72a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa218143f90 con 0x7fa2180b4310 2022-08-01T22:56:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21e72a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2181476a0 con 0x7fa218146ae0 2022-08-01T22:56:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa217fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa218146ae0 0x7fa2181403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21ccc7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2180b4820 0x7fa2180b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57344/0 (socket says 172.21.15.32:57344) 2022-08-01T22:56:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa21ccc7700 1 -- 172.21.15.32:0/2590152731 learned_addr learned my addr 172.21.15.32:0/2590152731 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa216ffd700 1 -- 172.21.15.32:0/2590152731 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218762282 0 0) 0x7fa218143cf0 con 0x7fa2180b4820 2022-08-01T22:56:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa216ffd700 1 -- 172.21.15.32:0/2590152731 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa204003260 con 0x7fa2180b4820 2022-08-01T22:56:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa216ffd700 1 -- 172.21.15.32:0/2590152731 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668446613 0 0) 0x7fa218143f90 con 0x7fa2180b4310 2022-08-01T22:56:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa217fff700 1 -- 172.21.15.32:0/2590152731 >> v1:172.21.15.55:6789/0 conn(0x7fa2180b4310 legacy=0x7fa2181408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa217fff700 1 -- 172.21.15.32:0/2590152731 >> v1:172.21.15.32:6789/0 conn(0x7fa2180b4820 legacy=0x7fa2180b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa217fff700 1 -- 172.21.15.32:0/2590152731 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180b0e20 con 0x7fa218146ae0 2022-08-01T22:56:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.815+0000 7fa217fff700 1 --2- 172.21.15.32:0/2590152731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2181403c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fa20c002f70 tx=0x7fa20c006df0).ready entity=mon.0 client_cookie=98f59f11e769e1f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa216ffd700 1 -- 172.21.15.32:0/2590152731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa20c00d610 con 0x7fa218146ae0 2022-08-01T22:56:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa216ffd700 1 -- 172.21.15.32:0/2590152731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c000e90 con 0x7fa218146ae0 2022-08-01T22:56:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa216ffd700 1 -- 172.21.15.32:0/2590152731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa20c00d960 con 0x7fa218146ae0 2022-08-01T22:56:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- 172.21.15.32:0/2590152731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 msgr2=0x7fa2181403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 --2- 172.21.15.32:0/2590152731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2181403c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fa20c002f70 tx=0x7fa20c006df0).stop 2022-08-01T22:56:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- 172.21.15.32:0/2590152731 shutdown_connections 2022-08-01T22:56:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 --2- 172.21.15.32:0/2590152731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2181403c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- 172.21.15.32:0/2590152731 >> 172.21.15.32:0/2590152731 conn(0x7fa21805c620 msgr2=0x7fa21805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- 172.21.15.32:0/2590152731 shutdown_connections 2022-08-01T22:56:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- 172.21.15.32:0/2590152731 wait complete. 2022-08-01T22:56:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 Processor -- start 2022-08-01T22:56:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- start start 2022-08-01T22:56:18.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2180b4310 0x7fa218147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2180b4820 0x7fa2180ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2180adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218145d80 con 0x7fa218146ae0 2022-08-01T22:56:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218153180 con 0x7fa2180b4820 2022-08-01T22:56:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21e72a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa218152260 con 0x7fa2180b4310 2022-08-01T22:56:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21ccc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2180adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21ccc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2180adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41576/0 (socket says 172.21.15.32:41576) 2022-08-01T22:56:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa21ccc7700 1 -- 172.21.15.32:0/661356887 learned_addr learned my addr 172.21.15.32:0/661356887 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa2177fe700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2180b4820 0x7fa2180ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.819+0000 7fa217fff700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2180b4310 0x7fa218147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21ccc7700 1 -- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2180b4310 msgr2=0x7fa218147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21ccc7700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2180b4310 0x7fa218147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21ccc7700 1 -- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2180b4820 msgr2=0x7fa2180ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21ccc7700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2180b4820 0x7fa2180ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21ccc7700 1 -- 172.21.15.32:0/661356887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c00b180 con 0x7fa218146ae0 2022-08-01T22:56:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21ccc7700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2180adc70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fa218153310 tx=0x7fa210007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa210005ba0 con 0x7fa218146ae0 2022-08-01T22:56:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210005d00 con 0x7fa218146ae0 2022-08-01T22:56:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa218152500 con 0x7fa218146ae0 2022-08-01T22:56:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2100072c0 con 0x7fa218146ae0 2022-08-01T22:56:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa218152a40 con 0x7fa218146ae0 2022-08-01T22:56:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa218152e80 con 0x7fa218146ae0 2022-08-01T22:56:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa21001a060 con 0x7fa218146ae0 2022-08-01T22:56:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa2157fa700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa1f8053980 0x7fa1f8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.823+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fa21006bad0 con 0x7fa218146ae0 2022-08-01T22:56:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.827+0000 7fa217fff700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa1f8053980 0x7fa1f8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.827+0000 7fa217fff700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa1f8053980 0x7fa1f8055e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa20c018000 tx=0x7fa20c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.831+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa210042830 con 0x7fa218146ae0 2022-08-01T22:56:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:18.991+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa2180574e0 con 0x7fa218146ae0 2022-08-01T22:56:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.347+0000 7fa2157fa700 1 -- 172.21.15.32:0/661356887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa210018020 con 0x7fa218146ae0 2022-08-01T22:56:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa1f8053980 msgr2=0x7fa1f8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa1f8053980 0x7fa1f8055e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa20c018000 tx=0x7fa20c014000).stop 2022-08-01T22:56:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 msgr2=0x7fa2180adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2180adc70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fa218153310 tx=0x7fa210007e00).stop 2022-08-01T22:56:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 shutdown_connections 2022-08-01T22:56:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa1f8053980 0x7fa1f8055e40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa218146ae0 0x7fa2180adc70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2180b4820 0x7fa2180ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 --2- 172.21.15.32:0/661356887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2180b4310 0x7fa218147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 >> 172.21.15.32:0/661356887 conn(0x7fa21805c620 msgr2=0x7fa2180b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:19.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 shutdown_connections 2022-08-01T22:56:19.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.351+0000 7fa21e72a700 1 -- 172.21.15.32:0/661356887 wait complete. 2022-08-01T22:56:19.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T22:56:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.479+0000 7ffa4b59e700 1 Processor -- start 2022-08-01T22:56:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.479+0000 7ffa4b59e700 1 -- start start 2022-08-01T22:56:19.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffa4c0b65c0 0x7ffa4c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0b2100 con 0x7ffa4c0b41f0 2022-08-01T22:56:19.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0b69a0 con 0x7ffa4c147d80 2022-08-01T22:56:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c0b1630 con 0x7ffa4c0b65c0 2022-08-01T22:56:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa49d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa4c0b41f0 0x7ffa4c144630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57358/0 (socket says 172.21.15.32:57358) 2022-08-01T22:56:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa49d9b700 1 -- 172.21.15.32:0/524800248 learned_addr learned my addr 172.21.15.32:0/524800248 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/524800248 >> v2:172.21.15.55:3300/0 conn(0x7ffa4c0b65c0 0x7ffa4c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085781557 0 0) 0x7ffa4c0b2100 con 0x7ffa4c0b41f0 2022-08-01T22:56:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa30003260 con 0x7ffa4c0b41f0 2022-08-01T22:56:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323359906 0 0) 0x7ffa4c0b69a0 con 0x7ffa4c147d80 2022-08-01T22:56:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa4c0b2100 con 0x7ffa4c147d80 2022-08-01T22:56:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2155116569 0 0) 0x7ffa30003260 con 0x7ffa4c0b41f0 2022-08-01T22:56:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 >> v2:172.21.15.55:3300/0 conn(0x7ffa4c0b65c0 msgr2=0x7ffa4c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 --2- 172.21.15.32:0/524800248 >> v2:172.21.15.55:3300/0 conn(0x7ffa4c0b65c0 0x7ffa4c061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 >> v1:172.21.15.55:6789/0 conn(0x7ffa4c147d80 legacy=0x7ffa4c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c1482a0 con 0x7ffa4c0b41f0 2022-08-01T22:56:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7ffa40001660 con 0x7ffa4c0b41f0 2022-08-01T22:56:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 >> v1:172.21.15.32:6789/0 conn(0x7ffa4c0b41f0 legacy=0x7ffa4c144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300038e0 0x7ffa30003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa30004500 0x7ffa30004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa30004e70 0x7ffa3000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0b1630 con 0x7ffa300038e0 2022-08-01T22:56:19.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3000bcf0 con 0x7ffa30004500 2022-08-01T22:56:19.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3000beb0 con 0x7ffa30004e70 2022-08-01T22:56:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/524800248 >> v2:172.21.15.55:3300/0 conn(0x7ffa4c0b65c0 0x7ffa4c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300038e0 0x7ffa30003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa49d9b700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa30004500 0x7ffa30004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4ad9d700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa30004e70 0x7ffa3000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 -- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa30004e70 msgr2=0x7ffa3000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa30004e70 0x7ffa3000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 -- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa30004500 msgr2=0x7ffa30004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa30004500 0x7ffa30004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 -- 172.21.15.32:0/524800248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa30003260 con 0x7ffa300038e0 2022-08-01T22:56:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4ad9d700 1 -- 172.21.15.32:0/524800248 reap_dead start 2022-08-01T22:56:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa49d9b700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa30004500 0x7ffa30004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300038e0 0x7ffa30003ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7ffa3c00c000 tx=0x7ffa3c006df0).ready entity=mon.0 client_cookie=781105366a9828a8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa3c00e800 con 0x7ffa300038e0 2022-08-01T22:56:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c00e960 con 0x7ffa300038e0 2022-08-01T22:56:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.483+0000 7ffa4959a700 1 -- 172.21.15.32:0/524800248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa3c00ecc0 con 0x7ffa300038e0 2022-08-01T22:56:19.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- 172.21.15.32:0/524800248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300038e0 msgr2=0x7ffa30003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300038e0 0x7ffa30003ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7ffa3c00c000 tx=0x7ffa3c006df0).stop 2022-08-01T22:56:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- 172.21.15.32:0/524800248 shutdown_connections 2022-08-01T22:56:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/524800248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300038e0 0x7ffa30003ce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- 172.21.15.32:0/524800248 >> 172.21.15.32:0/524800248 conn(0x7ffa4c05c620 msgr2=0x7ffa4c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- 172.21.15.32:0/524800248 shutdown_connections 2022-08-01T22:56:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- 172.21.15.32:0/524800248 wait complete. 2022-08-01T22:56:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 Processor -- start 2022-08-01T22:56:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- start start 2022-08-01T22:56:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa300038e0 0x7ffa4c14a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 0x7ffa4c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa4c0ad760 0x7ffa4c066070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0b8ea0 con 0x7ffa4c14aee0 2022-08-01T22:56:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c14e9a0 con 0x7ffa4c0ad760 2022-08-01T22:56:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c156950 con 0x7ffa300038e0 2022-08-01T22:56:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 0x7ffa4c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 0x7ffa4c157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41596/0 (socket says 172.21.15.32:41596) 2022-08-01T22:56:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 -- 172.21.15.32:0/2076321835 learned_addr learned my addr 172.21.15.32:0/2076321835 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa300038e0 0x7ffa4c14a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4ad9d700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa4c0ad760 0x7ffa4c066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 -- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa300038e0 msgr2=0x7ffa4c14a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa300038e0 0x7ffa4c14a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 -- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa4c0ad760 msgr2=0x7ffa4c066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:19.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa4c0ad760 0x7ffa4c066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:19.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 -- 172.21.15.32:0/2076321835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c156bf0 con 0x7ffa4c14aee0 2022-08-01T22:56:19.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa300038e0 0x7ffa4c14a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:19.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.487+0000 7ffa49d9b700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 0x7ffa4c157330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7ffa4000b000 tx=0x7ffa4c148430).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:19.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa40010640 con 0x7ffa4c14aee0 2022-08-01T22:56:19.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c156e90 con 0x7ffa4c14aee0 2022-08-01T22:56:19.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa400107a0 con 0x7ffa4c14aee0 2022-08-01T22:56:19.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa40010b00 con 0x7ffa4c14aee0 2022-08-01T22:56:19.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c0b8b20 con 0x7ffa4c14aee0 2022-08-01T22:56:19.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c157020 con 0x7ffa4c14aee0 2022-08-01T22:56:19.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa40010c60 con 0x7ffa4c14aee0 2022-08-01T22:56:19.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa3b7fe700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa28053b00 0x7ffa28055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:19.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6517+0+0 (secure 0 0 0) 0x7ffa4006c490 con 0x7ffa4c14aee0 2022-08-01T22:56:19.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.491+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa28053b00 0x7ffa28055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:19.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.495+0000 7ffa4a59c700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa28053b00 0x7ffa28055fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ffa3c01d000 tx=0x7ffa3c019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:19.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.499+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffa40043190 con 0x7ffa4c14aee0 2022-08-01T22:56:19.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:19.627+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa4c0b2330 con 0x7ffa4c14aee0 2022-08-01T22:56:20.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.359+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/2076321835 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 185+0+0 (secure 0 0 0) 0x7ffa400467b0 con 0x7ffa4c14aee0 2022-08-01T22:56:20.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa28053b00 msgr2=0x7ffa28055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa28053b00 0x7ffa28055fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ffa3c01d000 tx=0x7ffa3c019000).stop 2022-08-01T22:56:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 msgr2=0x7ffa4c157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 0x7ffa4c157330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7ffa4000b000 tx=0x7ffa4c148430).stop 2022-08-01T22:56:20.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 shutdown_connections 2022-08-01T22:56:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa28053b00 0x7ffa28055fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa4c0ad760 0x7ffa4c066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4c14aee0 0x7ffa4c157330 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0x7ffa4000b000 tx=0x7ffa4c148430).stop 2022-08-01T22:56:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 --2- 172.21.15.32:0/2076321835 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa300038e0 0x7ffa4c14a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 >> 172.21.15.32:0/2076321835 conn(0x7ffa4c05c620 msgr2=0x7ffa4c144740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 shutdown_connections 2022-08-01T22:56:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.363+0000 7ffa4b59e700 1 -- 172.21.15.32:0/2076321835 wait complete. 2022-08-01T22:56:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T22:56:20.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T22:56:20.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.495+0000 7f53e8955700 1 Processor -- start 2022-08-01T22:56:20.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.495+0000 7f53e8955700 1 -- start start 2022-08-01T22:56:20.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e8955700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e40af550 0x7f53e40b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:20.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e8955700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53e405c6f0 0x7f53e40a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:20.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e8955700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e40aa630 con 0x7f53e4060330 2022-08-01T22:56:20.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e8955700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40b15e0 con 0x7f53e405c6f0 2022-08-01T22:56:20.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e8955700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40af930 con 0x7f53e40af550 2022-08-01T22:56:20.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53e405c6f0 0x7f53e40a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:20.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53e405c6f0 0x7f53e40a8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41606/0 (socket says 172.21.15.32:41606) 2022-08-01T22:56:20.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 -- 172.21.15.32:0/642131907 learned_addr learned my addr 172.21.15.32:0/642131907 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:20.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e37fe700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e40af550 0x7f53e40b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:20.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 -- 172.21.15.32:0/642131907 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e40af550 msgr2=0x7f53e40b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:20.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e40af550 0x7f53e40b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 -- 172.21.15.32:0/642131907 >> v1:172.21.15.55:6789/0 conn(0x7f53e4060330 legacy=0x7f53e41445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:20.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 -- 172.21.15.32:0/642131907 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e405fd10 con 0x7f53e405c6f0 2022-08-01T22:56:20.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e37fe700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e40af550 0x7f53e40b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:20.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e3fff700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e405c6f0 0x7f53e40a8d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f53dc0032f0 tx=0x7f53dc009290).ready entity=mon.0 client_cookie=18bcafecb4917c89 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:20.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e27fc700 1 -- 172.21.15.32:0/642131907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53dc00b8a0 con 0x7f53e405c6f0 2022-08-01T22:56:20.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e27fc700 1 -- 172.21.15.32:0/642131907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc00ba00 con 0x7f53e405c6f0 2022-08-01T22:56:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.499+0000 7f53e27fc700 1 -- 172.21.15.32:0/642131907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53dc00bb70 con 0x7f53e405c6f0 2022-08-01T22:56:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 -- 172.21.15.32:0/642131907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e405c6f0 msgr2=0x7f53e40a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e405c6f0 0x7f53e40a8d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f53dc0032f0 tx=0x7f53dc009290).stop 2022-08-01T22:56:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 -- 172.21.15.32:0/642131907 shutdown_connections 2022-08-01T22:56:20.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e405c6f0 0x7f53e40a8d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 --2- 172.21.15.32:0/642131907 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e40af550 0x7f53e40b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 -- 172.21.15.32:0/642131907 >> 172.21.15.32:0/642131907 conn(0x7f53e40612d0 msgr2=0x7f53e4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 -- 172.21.15.32:0/642131907 shutdown_connections 2022-08-01T22:56:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 -- 172.21.15.32:0/642131907 wait complete. 2022-08-01T22:56:20.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 Processor -- start 2022-08-01T22:56:20.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.503+0000 7f53e8955700 1 -- start start 2022-08-01T22:56:20.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e405c6f0 0x7f53e4143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e4060330 0x7f53e40652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 0x7f53e4150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4142e40 con 0x7f53e40af550 2022-08-01T22:56:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e41426f0 con 0x7f53e405c6f0 2022-08-01T22:56:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4065950 con 0x7f53e4060330 2022-08-01T22:56:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 0x7f53e4150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 0x7f53e4150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41612/0 (socket says 172.21.15.32:41612) 2022-08-01T22:56:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e405c6f0 0x7f53e4143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 -- 172.21.15.32:0/2461802852 learned_addr learned my addr 172.21.15.32:0/2461802852 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e2ffd700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e4060330 0x7f53e40652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 -- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e4060330 msgr2=0x7f53e40652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:20.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e4060330 0x7f53e40652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 -- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e405c6f0 msgr2=0x7f53e4143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e405c6f0 0x7f53e4143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 -- 172.21.15.32:0/2461802852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc009530 con 0x7f53e40af550 2022-08-01T22:56:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e37fe700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e405c6f0 0x7f53e4143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e3fff700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 0x7f53e4150190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f53dc0032f0 tx=0x7f53dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53dc00b8a0 con 0x7f53e40af550 2022-08-01T22:56:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc00bde0 con 0x7f53e40af550 2022-08-01T22:56:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e405f840 con 0x7f53e40af550 2022-08-01T22:56:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53dc020610 con 0x7f53e40af550 2022-08-01T22:56:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.507+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e4143300 con 0x7f53e40af550 2022-08-01T22:56:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.511+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f53dc00f070 con 0x7f53e40af550 2022-08-01T22:56:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.511+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e4065790 con 0x7f53e40af550 2022-08-01T22:56:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.511+0000 7f53e0ff9700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f53c4053a70 0x7f53c4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.511+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f53dc06cb90 con 0x7f53e40af550 2022-08-01T22:56:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.511+0000 7f53e37fe700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f53c4053a70 0x7f53c4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.511+0000 7f53e37fe700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f53c4053a70 0x7f53c4055f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f53d400c000 tx=0x7f53d4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.519+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f53dc043a20 con 0x7f53e40af550 2022-08-01T22:56:20.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:20.643+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53e405fa70 con 0x7f53e40af550 2022-08-01T22:56:21.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.363+0000 7f53e0ff9700 1 -- 172.21.15.32:0/2461802852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7f53dc015070 con 0x7f53e40af550 2022-08-01T22:56:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f53c4053a70 msgr2=0x7f53c4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f53c4053a70 0x7f53c4055f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f53d400c000 tx=0x7f53d4008040).stop 2022-08-01T22:56:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 msgr2=0x7f53e4150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 0x7f53e4150190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f53dc0032f0 tx=0x7f53dc016000).stop 2022-08-01T22:56:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 shutdown_connections 2022-08-01T22:56:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f53c4053a70 0x7f53c4055f30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e40af550 0x7f53e4150190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53e4060330 0x7f53e40652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 --2- 172.21.15.32:0/2461802852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e405c6f0 0x7f53e4143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 >> 172.21.15.32:0/2461802852 conn(0x7f53e40612d0 msgr2=0x7f53e40a8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 shutdown_connections 2022-08-01T22:56:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.371+0000 7f53e8955700 1 -- 172.21.15.32:0/2461802852 wait complete. 2022-08-01T22:56:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T22:56:21.396 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T22:56:21.396 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T22:56:21.397 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T22:56:21.398 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:56:21.398 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T22:56:21.399 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T22:56:21.400 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T22:56:21.401 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:56:21.402 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T22:56:21.402 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T22:56:21.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T22:56:21.403 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:56:21.409 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T22:56:21.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T22:56:21.413 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T22:56:21.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T22:56:21.418 INFO:tasks.ceph.mds.a:Started 2022-08-01T22:56:21.419 DEBUG:teuthology.parallel:result is None 2022-08-01T22:56:21.422 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T22:56:21.422 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T22:56:21.426 INFO:tasks.ceph.mds.c:Started 2022-08-01T22:56:21.427 DEBUG:teuthology.parallel:result is None 2022-08-01T22:56:21.430 INFO:tasks.ceph.mds.b:Started 2022-08-01T22:56:21.431 DEBUG:teuthology.parallel:result is None 2022-08-01T22:56:21.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T22:56:21.484 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T22:56:21.503 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T22:56:21.508 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T22:56:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 Processor -- start 2022-08-01T22:56:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- start start 2022-08-01T22:56:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58900b1870 0x7f58900abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58900ab8a0 con 0x7f58900b6060 2022-08-01T22:56:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58900b1c50 con 0x7f58900a8cf0 2022-08-01T22:56:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58900aadd0 con 0x7f58900b1870 2022-08-01T22:56:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5895d9f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58900b6060 0x7f58900a9490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57414/0 (socket says 172.21.15.32:57414) 2022-08-01T22:56:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5895d9f700 1 -- 172.21.15.32:0/3339915680 learned_addr learned my addr 172.21.15.32:0/3339915680 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f589559e700 1 --2- 172.21.15.32:0/3339915680 >> v2:172.21.15.55:3300/0 conn(0x7f58900b1870 0x7f58900abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710977824 0 0) 0x7f58900ab8a0 con 0x7f58900b6060 2022-08-01T22:56:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5874003260 con 0x7f58900b6060 2022-08-01T22:56:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423243454 0 0) 0x7f58900b1c50 con 0x7f58900a8cf0 2022-08-01T22:56:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58900ab8a0 con 0x7f58900a8cf0 2022-08-01T22:56:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f589559e700 1 -- 172.21.15.32:0/3339915680 >> v1:172.21.15.55:6789/0 conn(0x7f58900a8cf0 legacy=0x7f58900669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f589559e700 1 -- 172.21.15.32:0/3339915680 >> v1:172.21.15.32:6789/0 conn(0x7f58900b6060 legacy=0x7f58900a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f589559e700 1 -- 172.21.15.32:0/3339915680 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589013f480 con 0x7f58900b1870 2022-08-01T22:56:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f589559e700 1 --2- 172.21.15.32:0/3339915680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900b1870 0x7f58900abf70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f5880002f70 tx=0x7f5880006df0).ready entity=mon.1 client_cookie=eab79bf4cde4f512 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f588000d540 con 0x7f58900b1870 2022-08-01T22:56:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880000e90 con 0x7f58900b1870 2022-08-01T22:56:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5887fff700 1 -- 172.21.15.32:0/3339915680 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f588000d890 con 0x7f58900b1870 2022-08-01T22:56:21.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- 172.21.15.32:0/3339915680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900b1870 msgr2=0x7f58900abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 --2- 172.21.15.32:0/3339915680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900b1870 0x7f58900abf70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f5880002f70 tx=0x7f5880006df0).stop 2022-08-01T22:56:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- 172.21.15.32:0/3339915680 shutdown_connections 2022-08-01T22:56:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 --2- 172.21.15.32:0/3339915680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900b1870 0x7f58900abf70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- 172.21.15.32:0/3339915680 >> 172.21.15.32:0/3339915680 conn(0x7f58900b5570 msgr2=0x7f58900b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- 172.21.15.32:0/3339915680 shutdown_connections 2022-08-01T22:56:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.535+0000 7f5897802700 1 -- 172.21.15.32:0/3339915680 wait complete. 2022-08-01T22:56:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 Processor -- start 2022-08-01T22:56:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 -- start start 2022-08-01T22:56:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900a8cf0 0x7f5890063510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 0x7f5890064910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58900b6060 0x7f5890064e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5890062af0 con 0x7f58900b1870 2022-08-01T22:56:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58900a39e0 con 0x7f58900a8cf0 2022-08-01T22:56:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58900a3c80 con 0x7f58900b6060 2022-08-01T22:56:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 0x7f5890064910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 0x7f5890064910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41646/0 (socket says 172.21.15.32:41646) 2022-08-01T22:56:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 -- 172.21.15.32:0/384367254 learned_addr learned my addr 172.21.15.32:0/384367254 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5895d9f700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58900b6060 0x7f5890064e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f589559e700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900a8cf0 0x7f5890063510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 -- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58900b6060 msgr2=0x7f5890064e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58900b6060 0x7f5890064e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 -- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900a8cf0 msgr2=0x7f5890063510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900a8cf0 0x7f5890063510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 -- 172.21.15.32:0/384367254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588000b180 con 0x7f58900b1870 2022-08-01T22:56:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5895d9f700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58900b6060 0x7f5890064e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f589559e700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900a8cf0 0x7f5890063510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5894d9d700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 0x7f5890064910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f588c001110 tx=0x7f588c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f588c00f470 con 0x7f58900b1870 2022-08-01T22:56:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c00f5d0 con 0x7f58900b1870 2022-08-01T22:56:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58900a3f20 con 0x7f58900b1870 2022-08-01T22:56:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f588c00f930 con 0x7f58900b1870 2022-08-01T22:56:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.539+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5890063a80 con 0x7f58900b1870 2022-08-01T22:56:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.543+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f588c019030 con 0x7f58900b1870 2022-08-01T22:56:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.543+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58900574e0 con 0x7f58900b1870 2022-08-01T22:56:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.543+0000 7f58867fc700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5878053b30 0x7f5878055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.543+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f588c06b3a0 con 0x7f58900b1870 2022-08-01T22:56:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.543+0000 7f589559e700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5878053b30 0x7f5878055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.543+0000 7f589559e700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5878053b30 0x7f5878055ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5880015000 tx=0x7f5880014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.551+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f588c042330 con 0x7f58900b1870 2022-08-01T22:56:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:21.687+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5890069a70 con 0x7f58900b1870 2022-08-01T22:56:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.531+0000 7f58867fc700 1 -- 172.21.15.32:0/384367254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f588c045950 con 0x7f58900b1870 2022-08-01T22:56:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5878053b30 msgr2=0x7f5878055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5878053b30 0x7f5878055ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5880015000 tx=0x7f5880014000).stop 2022-08-01T22:56:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 msgr2=0x7f5890064910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 0x7f5890064910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f588c001110 tx=0x7f588c007ae0).stop 2022-08-01T22:56:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 shutdown_connections 2022-08-01T22:56:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5878053b30 0x7f5878055ff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58900b6060 0x7f5890064e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58900b1870 0x7f5890064910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 --2- 172.21.15.32:0/384367254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58900a8cf0 0x7f5890063510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 >> 172.21.15.32:0/384367254 conn(0x7f58900b5570 msgr2=0x7f58900b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 shutdown_connections 2022-08-01T22:56:22.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.535+0000 7f5897802700 1 -- 172.21.15.32:0/384367254 wait complete. 2022-08-01T22:56:22.551 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T22:56:22.563 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T22:56:22.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T22:56:22.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21db8a2700 1 Processor -- start 2022-08-01T22:56:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21db8a2700 1 -- start start 2022-08-01T22:56:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21db8a2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21d414cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21db8a2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d41453e0 con 0x7f21d40b0160 2022-08-01T22:56:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21db8a2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d4145680 con 0x7f21d40b2390 2022-08-01T22:56:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21db8a2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4141390 con 0x7f21d40ac6a0 2022-08-01T22:56:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21d8e3d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21d40b0160 0x7f21d4141830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57434/0 (socket says 172.21.15.32:57434) 2022-08-01T22:56:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21d8e3d700 1 -- 172.21.15.32:0/4180310474 learned_addr learned my addr 172.21.15.32:0/4180310474 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21d414cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646981768 0 0) 0x7f21d41453e0 con 0x7f21d40b0160 2022-08-01T22:56:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c0003260 con 0x7f21d40b0160 2022-08-01T22:56:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926273500 0 0) 0x7f21c0003260 con 0x7f21d40b0160 2022-08-01T22:56:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 msgr2=0x7f21d414cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21d414cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 >> v1:172.21.15.55:6789/0 conn(0x7f21d40b2390 legacy=0x7f21d40abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40b0670 con 0x7f21d40b0160 2022-08-01T22:56:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f21cc001a00 con 0x7f21d40b0160 2022-08-01T22:56:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 >> v1:172.21.15.32:6789/0 conn(0x7f21d40b0160 legacy=0x7f21d4141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21c0003780 0x7f21c0003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21c0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21c00049f0 0x7f21c0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4141390 con 0x7f21c00049f0 2022-08-01T22:56:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000b780 con 0x7f21c0003780 2022-08-01T22:56:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000b940 con 0x7f21d40ac6a0 2022-08-01T22:56:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21d8e3d700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21c0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21c00049f0 0x7f21c0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.671+0000 7f21d963e700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21c0003780 0x7f21c0003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 -- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 msgr2=0x7f21c0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21c0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 -- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21c0003780 msgr2=0x7f21c0003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21c0003780 0x7f21c0003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 -- 172.21.15.32:0/4180310474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0003260 con 0x7f21c00049f0 2022-08-01T22:56:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d8e3d700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21d40ac6a0 0x7f21c0004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21c00049f0 0x7f21c0004e70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f21c400c000 tx=0x7f21c4006df0).ready entity=mon.0 client_cookie=23db22df80311b2f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c400e830 con 0x7f21c00049f0 2022-08-01T22:56:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400e990 con 0x7f21c00049f0 2022-08-01T22:56:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21cbfff700 1 -- 172.21.15.32:0/4180310474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c400ecf0 con 0x7f21c00049f0 2022-08-01T22:56:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 -- 172.21.15.32:0/4180310474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21c00049f0 msgr2=0x7f21c0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 --2- 172.21.15.32:0/4180310474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21c00049f0 0x7f21c0004e70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f21c400c000 tx=0x7f21c4006df0).stop 2022-08-01T22:56:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21d9e3f700 1 -- 172.21.15.32:0/4180310474 reap_dead start 2022-08-01T22:56:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 -- 172.21.15.32:0/4180310474 shutdown_connections 2022-08-01T22:56:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 -- 172.21.15.32:0/4180310474 >> 172.21.15.32:0/4180310474 conn(0x7f21d405f840 msgr2=0x7f21d405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 -- 172.21.15.32:0/4180310474 shutdown_connections 2022-08-01T22:56:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 -- 172.21.15.32:0/4180310474 wait complete. 2022-08-01T22:56:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 Processor -- start 2022-08-01T22:56:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.675+0000 7f21db8a2700 1 -- start start 2022-08-01T22:56:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21c00049f0 0x7f21d40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21d413e4b0 0x7f21d4138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 0x7f21d4138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d413c9b0 con 0x7f21d40b4b80 2022-08-01T22:56:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d413b990 con 0x7f21d413e4b0 2022-08-01T22:56:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d413be60 con 0x7f21c00049f0 2022-08-01T22:56:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 0x7f21d4138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 0x7f21d4138ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41674/0 (socket says 172.21.15.32:41674) 2022-08-01T22:56:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 -- 172.21.15.32:0/3204812087 learned_addr learned my addr 172.21.15.32:0/3204812087 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d8e3d700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21d413e4b0 0x7f21d4138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d963e700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21c00049f0 0x7f21d40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 -- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21c00049f0 msgr2=0x7f21d40b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21c00049f0 0x7f21d40b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 -- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21d413e4b0 msgr2=0x7f21d4138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21d413e4b0 0x7f21d4138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 -- 172.21.15.32:0/3204812087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d41393e0 con 0x7f21d40b4b80 2022-08-01T22:56:22.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21d9e3f700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 0x7f21d4138ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f21d4144da0 tx=0x7f21c4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:22.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c4012ca0 con 0x7f21d40b4b80 2022-08-01T22:56:22.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4022070 con 0x7f21d40b4b80 2022-08-01T22:56:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d413bbc0 con 0x7f21d40b4b80 2022-08-01T22:56:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c400be60 con 0x7f21d40b4b80 2022-08-01T22:56:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.679+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d40b5470 con 0x7f21d40b4b80 2022-08-01T22:56:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.683+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4154380 con 0x7f21d40b4b80 2022-08-01T22:56:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.683+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f21c402e070 con 0x7f21d40b4b80 2022-08-01T22:56:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.683+0000 7f21ca7fc700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21b4053aa0 0x7f21b4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.683+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f21c4000e40 con 0x7f21d40b4b80 2022-08-01T22:56:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.683+0000 7f21d963e700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21b4053aa0 0x7f21b4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.683+0000 7f21d963e700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21b4053aa0 0x7f21b4055f60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f21d0003000 tx=0x7f21d00079e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.687+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f21d4154380 con 0x7f21d40b4b80 2022-08-01T22:56:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f21d4154380 con 0x7f21d40b4b80 2022-08-01T22:56:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21ca7fc700 1 -- 172.21.15.32:0/3204812087 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+5621 (secure 0 0 0) 0x7f21d4154380 con 0x7f21d40b4b80 2022-08-01T22:56:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:22.866 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:56:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T22:56:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T22:56:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T22:56:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:56:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T22:56:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T22:56:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:56:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T22:56:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T22:56:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:56:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T22:56:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T22:56:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T22:56:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T22:56:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T22:56:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T22:56:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T22:56:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T22:56:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T22:56:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T22:56:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T22:56:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T22:56:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T22:56:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T22:56:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T22:56:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T22:56:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T22:56:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T22:56:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:56:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2022-08-01T22:56:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2022-08-01T22:56:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2022-08-01T22:56:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T22:56:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T22:56:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:56:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2022-08-01T22:56:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2022-08-01T22:56:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2022-08-01T22:56:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T22:56:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T22:56:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:56:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:56:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:56:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2022-08-01T22:56:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjWehis6gKFRAAmMVQsWxqiw5EA8Xhz1OlXQ==", 2022-08-01T22:56:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2022-08-01T22:56:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2022-08-01T22:56:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjWehit8IKFRAANSTq/c3OwS6xtFSIIfgkCQ==", 2022-08-01T22:56:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2022-08-01T22:56:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2022-08-01T22:56:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjWehiWtQKFRAAht8zVHqKismfIrenU0SDGg==", 2022-08-01T22:56:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2022-08-01T22:56:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2022-08-01T22:56:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjWehiBucKFRAAYEzREn/RwkK5aRseC3p+hw==", 2022-08-01T22:56:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2022-08-01T22:56:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2022-08-01T22:56:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjWehi3voKFRAAwDoUDWtQ9zchmFc9VSBVTA==", 2022-08-01T22:56:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2022-08-01T22:56:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2022-08-01T22:56:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjWehiFA0LFRAAeNAVxpUklUZjCU4MsPd78Q==", 2022-08-01T22:56:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2022-08-01T22:56:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T22:56:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T22:56:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:56:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T22:56:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T22:56:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T22:56:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:56:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:56:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T22:56:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:56:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:56:22.962 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:56:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21b4053aa0 msgr2=0x7f21b4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21b4053aa0 0x7f21b4055f60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f21d0003000 tx=0x7f21d00079e0).stop 2022-08-01T22:56:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 msgr2=0x7f21d4138ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 0x7f21d4138ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f21d4144da0 tx=0x7f21c4019000).stop 2022-08-01T22:56:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 shutdown_connections 2022-08-01T22:56:22.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21b4053aa0 0x7f21b4055f60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d40b4b80 0x7f21d4138ef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21d413e4b0 0x7f21d4138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 --2- 172.21.15.32:0/3204812087 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21c00049f0 0x7f21d40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:22.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 >> 172.21.15.32:0/3204812087 conn(0x7f21d405f840 msgr2=0x7f21d4141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 shutdown_connections 2022-08-01T22:56:22.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:22.859+0000 7f21db8a2700 1 -- 172.21.15.32:0/3204812087 wait complete. 2022-08-01T22:56:22.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2022-08-01T22:56:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 Processor -- start 2022-08-01T22:56:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 -- start start 2022-08-01T22:56:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98480b6710 0x7f98480b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f98480b4550 0x7f98480b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9848144240 con 0x7f9848061ec0 2022-08-01T22:56:23.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98480b6fe0 con 0x7f98480b6710 2022-08-01T22:56:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984fc79700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9848143d00 con 0x7f98480b4550 2022-08-01T22:56:23.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984e216700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9848061ec0 0x7f9848146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57456/0 (socket says 172.21.15.32:57456) 2022-08-01T22:56:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984e216700 1 -- 172.21.15.32:0/2531551141 learned_addr learned my addr 172.21.15.32:0/2531551141 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984d214700 1 --2- 172.21.15.32:0/2531551141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f98480b4550 0x7f98480b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984da15700 1 --2- 172.21.15.32:0/2531551141 >> v2:172.21.15.55:3300/0 conn(0x7f98480b6710 0x7f98480b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984ca13700 1 -- 172.21.15.32:0/2531551141 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892745140 0 0) 0x7f9848144240 con 0x7f9848061ec0 2022-08-01T22:56:23.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984ca13700 1 -- 172.21.15.32:0/2531551141 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9830003260 con 0x7f9848061ec0 2022-08-01T22:56:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984d214700 1 -- 172.21.15.32:0/2531551141 >> v2:172.21.15.55:3300/0 conn(0x7f98480b6710 msgr2=0x7f98480b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984d214700 1 --2- 172.21.15.32:0/2531551141 >> v2:172.21.15.55:3300/0 conn(0x7f98480b6710 0x7f98480b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984d214700 1 -- 172.21.15.32:0/2531551141 >> v1:172.21.15.32:6789/0 conn(0x7f9848061ec0 legacy=0x7f9848146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.071+0000 7f984d214700 1 -- 172.21.15.32:0/2531551141 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848143fa0 con 0x7f98480b4550 2022-08-01T22:56:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984d214700 1 --2- 172.21.15.32:0/2531551141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f98480b4550 0x7f98480b6af0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9844002f70 tx=0x7f9844006df0).ready entity=mon.2 client_cookie=30e01aa7ce6c61ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984ca13700 1 -- 172.21.15.32:0/2531551141 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984400d4a0 con 0x7f98480b4550 2022-08-01T22:56:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984ca13700 1 -- 172.21.15.32:0/2531551141 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844000e90 con 0x7f98480b4550 2022-08-01T22:56:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984ca13700 1 -- 172.21.15.32:0/2531551141 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984400d7f0 con 0x7f98480b4550 2022-08-01T22:56:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 -- 172.21.15.32:0/2531551141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f98480b4550 msgr2=0x7f98480b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 --2- 172.21.15.32:0/2531551141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f98480b4550 0x7f98480b6af0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9844002f70 tx=0x7f9844006df0).stop 2022-08-01T22:56:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 -- 172.21.15.32:0/2531551141 shutdown_connections 2022-08-01T22:56:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 --2- 172.21.15.32:0/2531551141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f98480b4550 0x7f98480b6af0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 --2- 172.21.15.32:0/2531551141 >> v2:172.21.15.55:3300/0 conn(0x7f98480b6710 0x7f98480b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 -- 172.21.15.32:0/2531551141 >> 172.21.15.32:0/2531551141 conn(0x7f984805c620 msgr2=0x7f984805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 -- 172.21.15.32:0/2531551141 shutdown_connections 2022-08-01T22:56:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 -- 172.21.15.32:0/2531551141 wait complete. 2022-08-01T22:56:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 Processor -- start 2022-08-01T22:56:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 -- start start 2022-08-01T22:56:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9848061ec0 0x7f98480adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98480b4550 0x7f98481253f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.075+0000 7f984fc79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 0x7f98481258e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984fc79700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98480aedc0 con 0x7f98480b6710 2022-08-01T22:56:23.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984fc79700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9848152f40 con 0x7f98480b4550 2022-08-01T22:56:23.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984fc79700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98480af760 con 0x7f9848061ec0 2022-08-01T22:56:23.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 0x7f98481258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984da15700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9848061ec0 0x7f98480adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 0x7f98481258e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41688/0 (socket says 172.21.15.32:41688) 2022-08-01T22:56:23.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984da15700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9848061ec0 0x7f98480adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:53422/0 (socket says 172.21.15.32:53422) 2022-08-01T22:56:23.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 -- 172.21.15.32:0/2227581251 learned_addr learned my addr 172.21.15.32:0/2227581251 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:23.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984d214700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98480b4550 0x7f98481253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 -- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9848061ec0 msgr2=0x7f98480adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9848061ec0 0x7f98480adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 -- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98480b4550 msgr2=0x7f98481253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98480b4550 0x7f98481253f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 -- 172.21.15.32:0/2227581251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984400b180 con 0x7f98480b6710 2022-08-01T22:56:23.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984e216700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 0x7f98481258e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f9840001490 tx=0x7f9840007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9840005ba0 con 0x7f98480b6710 2022-08-01T22:56:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9840005d00 con 0x7f98480b6710 2022-08-01T22:56:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98480afa00 con 0x7f98480b6710 2022-08-01T22:56:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98400072c0 con 0x7f98480b6710 2022-08-01T22:56:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9848152c00 con 0x7f98480b6710 2022-08-01T22:56:23.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9840010070 con 0x7f98480b6710 2022-08-01T22:56:23.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.079+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9848061780 con 0x7f98480b6710 2022-08-01T22:56:23.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.083+0000 7f9836ffd700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9828053a70 0x7f9828055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.083+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f984006b540 con 0x7f98480b6710 2022-08-01T22:56:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.083+0000 7f984da15700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9828053a70 0x7f9828055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.083+0000 7f984da15700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9828053a70 0x7f9828055f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f983800b000 tx=0x7f9838006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.087+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9840042640 con 0x7f98480b6710 2022-08-01T22:56:23.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:23.253 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:23.256 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.251+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f98480574e0 con 0x7f98480b6710 2022-08-01T22:56:23.260 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:23.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f9836ffd700 1 -- 172.21.15.32:0/2227581251 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9840018070 con 0x7f98480b6710 2022-08-01T22:56:23.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9828053a70 msgr2=0x7f9828055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9828053a70 0x7f9828055f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f983800b000 tx=0x7f9838006df0).stop 2022-08-01T22:56:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 msgr2=0x7f98481258e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 0x7f98481258e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f9840001490 tx=0x7f9840007e00).stop 2022-08-01T22:56:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 shutdown_connections 2022-08-01T22:56:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9828053a70 0x7f9828055f30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98480b6710 0x7f98481258e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98480b4550 0x7f98481253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 --2- 172.21.15.32:0/2227581251 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9848061ec0 0x7f98480adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 >> 172.21.15.32:0/2227581251 conn(0x7f984805c620 msgr2=0x7f9848148a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 shutdown_connections 2022-08-01T22:56:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.259+0000 7f984fc79700 1 -- 172.21.15.32:0/2227581251 wait complete. 2022-08-01T22:56:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2022-08-01T22:56:23.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2022-08-01T22:56:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.447+0000 7fc09aec1700 1 Processor -- start 2022-08-01T22:56:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.447+0000 7fc09aec1700 1 -- start start 2022-08-01T22:56:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0940b01a0 0x7fc0940a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc0940afd60 0x7fc0940a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940a7250 con 0x7fc0940b2710 2022-08-01T22:56:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0940a7480 con 0x7fc0940b01a0 2022-08-01T22:56:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0940a7720 con 0x7fc0940afd60 2022-08-01T22:56:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc0996be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0940b01a0 0x7fc0940a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc099ebf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc0940b2710 0x7fc09414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57466/0 (socket says 172.21.15.32:57466) 2022-08-01T22:56:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc099ebf700 1 -- 172.21.15.32:0/1581564164 learned_addr learned my addr 172.21.15.32:0/1581564164 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09a6c0700 1 --2- 172.21.15.32:0/1581564164 >> v2:172.21.15.55:3300/0 conn(0x7fc0940afd60 0x7fc0940a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc0996be700 1 -- 172.21.15.32:0/1581564164 >> v2:172.21.15.55:3300/0 conn(0x7fc0940afd60 msgr2=0x7fc0940a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc0996be700 1 --2- 172.21.15.32:0/1581564164 >> v2:172.21.15.55:3300/0 conn(0x7fc0940afd60 0x7fc0940a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc0996be700 1 -- 172.21.15.32:0/1581564164 >> v1:172.21.15.32:6789/0 conn(0x7fc0940b2710 legacy=0x7fc09414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc0996be700 1 -- 172.21.15.32:0/1581564164 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940a79c0 con 0x7fc0940b01a0 2022-08-01T22:56:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc098ebd700 1 -- 172.21.15.32:0/1581564164 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123944619 0 0) 0x7fc0940a7250 con 0x7fc0940b2710 2022-08-01T22:56:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc0996be700 1 --2- 172.21.15.32:0/1581564164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a8540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fc088002f70 tx=0x7fc088006df0).ready entity=mon.0 client_cookie=2289b0f8715cb669 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc098ebd700 1 -- 172.21.15.32:0/1581564164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc08800d570 con 0x7fc0940b01a0 2022-08-01T22:56:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc098ebd700 1 -- 172.21.15.32:0/1581564164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc088000e90 con 0x7fc0940b01a0 2022-08-01T22:56:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc098ebd700 1 -- 172.21.15.32:0/1581564164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc08800d8c0 con 0x7fc0940b01a0 2022-08-01T22:56:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- 172.21.15.32:0/1581564164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 msgr2=0x7fc0940a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 --2- 172.21.15.32:0/1581564164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a8540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fc088002f70 tx=0x7fc088006df0).stop 2022-08-01T22:56:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- 172.21.15.32:0/1581564164 shutdown_connections 2022-08-01T22:56:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 --2- 172.21.15.32:0/1581564164 >> v2:172.21.15.55:3300/0 conn(0x7fc0940afd60 0x7fc0940a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 --2- 172.21.15.32:0/1581564164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a8540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- 172.21.15.32:0/1581564164 >> 172.21.15.32:0/1581564164 conn(0x7fc094060310 msgr2=0x7fc0940b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- 172.21.15.32:0/1581564164 shutdown_connections 2022-08-01T22:56:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 -- 172.21.15.32:0/1581564164 wait complete. 2022-08-01T22:56:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.451+0000 7fc09aec1700 1 Processor -- start 2022-08-01T22:56:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 -- start start 2022-08-01T22:56:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0940afd60 0x7fc0940b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0940b2710 0x7fc0940a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940aac20 con 0x7fc0940b01a0 2022-08-01T22:56:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940aaec0 con 0x7fc0940afd60 2022-08-01T22:56:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940a9450 con 0x7fc0940b2710 2022-08-01T22:56:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41700/0 (socket says 172.21.15.32:41700) 2022-08-01T22:56:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 -- 172.21.15.32:0/2529105319 learned_addr learned my addr 172.21.15.32:0/2529105319 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc099ebf700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0940afd60 0x7fc0940b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09a6c0700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0940b2710 0x7fc0940a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 -- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0940b2710 msgr2=0x7fc0940a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0940b2710 0x7fc0940a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 -- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0940afd60 msgr2=0x7fc0940b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0940afd60 0x7fc0940b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 -- 172.21.15.32:0/2529105319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08800b180 con 0x7fc0940b01a0 2022-08-01T22:56:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc0996be700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a98f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc088002f70 tx=0x7fc088014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc088016070 con 0x7fc0940b01a0 2022-08-01T22:56:23.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880139c0 con 0x7fc0940b01a0 2022-08-01T22:56:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0940b5380 con 0x7fc0940b01a0 2022-08-01T22:56:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc088013d20 con 0x7fc0940b01a0 2022-08-01T22:56:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.455+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09414b890 con 0x7fc0940b01a0 2022-08-01T22:56:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.459+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0940a9610 con 0x7fc0940b01a0 2022-08-01T22:56:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.459+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc088025030 con 0x7fc0940b01a0 2022-08-01T22:56:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.459+0000 7fc092ffd700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc074053aa0 0x7fc074055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.459+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc08806cdc0 con 0x7fc0940b01a0 2022-08-01T22:56:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.459+0000 7fc099ebf700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc074053aa0 0x7fc074055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.459+0000 7fc099ebf700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc074053aa0 0x7fc074055f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc08c00d000 tx=0x7fc08c001360).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.467+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc088043db0 con 0x7fc0940b01a0 2022-08-01T22:56:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.635+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc0940574e0 con 0x7fc0940b01a0 2022-08-01T22:56:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.643+0000 7fc092ffd700 1 -- 172.21.15.32:0/2529105319 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc088023070 con 0x7fc0940b01a0 2022-08-01T22:56:23.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc074053aa0 msgr2=0x7fc074055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc074053aa0 0x7fc074055f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc08c00d000 tx=0x7fc08c001360).stop 2022-08-01T22:56:23.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 msgr2=0x7fc0940a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a98f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc088002f70 tx=0x7fc088014000).stop 2022-08-01T22:56:23.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 shutdown_connections 2022-08-01T22:56:23.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc074053aa0 0x7fc074055f60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0940b2710 0x7fc0940a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0940b01a0 0x7fc0940a98f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 --2- 172.21.15.32:0/2529105319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0940afd60 0x7fc0940b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 >> 172.21.15.32:0/2529105319 conn(0x7fc094060310 msgr2=0x7fc0940ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:23.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 shutdown_connections 2022-08-01T22:56:23.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.647+0000 7fc09aec1700 1 -- 172.21.15.32:0/2529105319 wait complete. 2022-08-01T22:56:23.663 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2022-08-01T22:56:23.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2022-08-01T22:56:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 Processor -- start 2022-08-01T22:56:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 -- start start 2022-08-01T22:56:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6cd40b2710 0x7f6cd414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cd40a8c50 0x7f6cd414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd40a7250 con 0x7f6cd40b01a0 2022-08-01T22:56:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40a7480 con 0x7f6cd40b2710 2022-08-01T22:56:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd996d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40a7720 con 0x7f6cd40a8c50 2022-08-01T22:56:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd896b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6cd40b2710 0x7f6cd414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6cd40b01a0 0x7f6cd40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57480/0 (socket says 172.21.15.32:57480) 2022-08-01T22:56:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd3fff700 1 -- 172.21.15.32:0/755164026 learned_addr learned my addr 172.21.15.32:0/755164026 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd916c700 1 --2- 172.21.15.32:0/755164026 >> v2:172.21.15.55:3300/0 conn(0x7f6cd40a8c50 0x7f6cd414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd37fe700 1 -- 172.21.15.32:0/755164026 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276127181 0 0) 0x7f6cd40a7250 con 0x7f6cd40b01a0 2022-08-01T22:56:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd37fe700 1 -- 172.21.15.32:0/755164026 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc0003260 con 0x7f6cd40b01a0 2022-08-01T22:56:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd896b700 1 -- 172.21.15.32:0/755164026 >> v2:172.21.15.55:3300/0 conn(0x7f6cd40a8c50 msgr2=0x7f6cd414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd896b700 1 --2- 172.21.15.32:0/755164026 >> v2:172.21.15.55:3300/0 conn(0x7f6cd40a8c50 0x7f6cd414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd896b700 1 -- 172.21.15.32:0/755164026 >> v1:172.21.15.32:6789/0 conn(0x7f6cd40b01a0 legacy=0x7f6cd40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.763+0000 7f6cd896b700 1 -- 172.21.15.32:0/755164026 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40a79c0 con 0x7f6cd40b2710 2022-08-01T22:56:23.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd896b700 1 --2- 172.21.15.32:0/755164026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd414afd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f6cc8002f70 tx=0x7f6cc8006df0).ready entity=mon.0 client_cookie=50d0eac64488ca60 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd37fe700 1 -- 172.21.15.32:0/755164026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cc800d4f0 con 0x7f6cd40b2710 2022-08-01T22:56:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd37fe700 1 -- 172.21.15.32:0/755164026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8000e90 con 0x7f6cd40b2710 2022-08-01T22:56:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd37fe700 1 -- 172.21.15.32:0/755164026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cc800d840 con 0x7f6cd40b2710 2022-08-01T22:56:23.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- 172.21.15.32:0/755164026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 msgr2=0x7f6cd414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 --2- 172.21.15.32:0/755164026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd414afd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f6cc8002f70 tx=0x7f6cc8006df0).stop 2022-08-01T22:56:23.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- 172.21.15.32:0/755164026 shutdown_connections 2022-08-01T22:56:23.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 --2- 172.21.15.32:0/755164026 >> v2:172.21.15.55:3300/0 conn(0x7f6cd40a8c50 0x7f6cd414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 --2- 172.21.15.32:0/755164026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd414afd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- 172.21.15.32:0/755164026 >> 172.21.15.32:0/755164026 conn(0x7f6cd4060310 msgr2=0x7f6cd40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- 172.21.15.32:0/755164026 shutdown_connections 2022-08-01T22:56:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- 172.21.15.32:0/755164026 wait complete. 2022-08-01T22:56:23.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 Processor -- start 2022-08-01T22:56:23.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- start start 2022-08-01T22:56:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd40a8c50 0x7f6cd40b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6cd40b01a0 0x7f6cd40ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd4156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40b31a0 con 0x7f6cd40b2710 2022-08-01T22:56:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41571d0 con 0x7f6cd40a8c50 2022-08-01T22:56:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd996d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4157470 con 0x7f6cd40b01a0 2022-08-01T22:56:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd4156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd4156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41716/0 (socket says 172.21.15.32:41716) 2022-08-01T22:56:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 -- 172.21.15.32:0/1903940096 learned_addr learned my addr 172.21.15.32:0/1903940096 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd3fff700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6cd40b01a0 0x7f6cd40ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd896b700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd40a8c50 0x7f6cd40b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 -- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6cd40b01a0 msgr2=0x7f6cd40ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6cd40b01a0 0x7f6cd40ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 -- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd40a8c50 msgr2=0x7f6cd40b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd40a8c50 0x7f6cd40b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 -- 172.21.15.32:0/1903940096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc800b180 con 0x7f6cd40b2710 2022-08-01T22:56:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd896b700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd40a8c50 0x7f6cd40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd916c700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd4156a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f6ccc00c000 tx=0x7f6ccc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.767+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ccc00c670 con 0x7f6cd40b2710 2022-08-01T22:56:23.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00c7d0 con 0x7f6cd40b2710 2022-08-01T22:56:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd4156ef0 con 0x7f6cd40b2710 2022-08-01T22:56:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ccc00cb30 con 0x7f6cd40b2710 2022-08-01T22:56:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd40b4900 con 0x7f6cd40b2710 2022-08-01T22:56:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd40b4bb0 con 0x7f6cd40b2710 2022-08-01T22:56:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6ccc022070 con 0x7f6cd40b2710 2022-08-01T22:56:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd1ffb700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cb4053a70 0x7f6cb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6ccc06c390 con 0x7f6cd40b2710 2022-08-01T22:56:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd896b700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cb4053a70 0x7f6cb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.771+0000 7f6cd896b700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cb4053a70 0x7f6cb4055f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6cc8015000 tx=0x7f6cc8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.779+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ccc043490 con 0x7f6cd40b2710 2022-08-01T22:56:23.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.943+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f6cd40574e0 con 0x7f6cd40b2710 2022-08-01T22:56:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.947+0000 7f6cd1ffb700 1 -- 172.21.15.32:0/1903940096 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6ccc016020 con 0x7f6cd40b2710 2022-08-01T22:56:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cb4053a70 msgr2=0x7f6cb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cb4053a70 0x7f6cb4055f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6cc8015000 tx=0x7f6cc8014000).stop 2022-08-01T22:56:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 msgr2=0x7f6cd4156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd4156a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f6ccc00c000 tx=0x7f6ccc0091d0).stop 2022-08-01T22:56:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 shutdown_connections 2022-08-01T22:56:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cb4053a70 0x7f6cb4055f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cd40b2710 0x7f6cd4156a00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6cd40b01a0 0x7f6cd40ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 --2- 172.21.15.32:0/1903940096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd40a8c50 0x7f6cd40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 >> 172.21.15.32:0/1903940096 conn(0x7f6cd4060310 msgr2=0x7f6cd4144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 shutdown_connections 2022-08-01T22:56:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:23.951+0000 7f6cd996d700 1 -- 172.21.15.32:0/1903940096 wait complete. 2022-08-01T22:56:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2022-08-01T22:56:23.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2022-08-01T22:56:24.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.067+0000 7f06873de700 1 Processor -- start 2022-08-01T22:56:24.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.067+0000 7f06873de700 1 -- start start 2022-08-01T22:56:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0680060290 0x7f06800b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 0x7f068014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800aad40 con 0x7f0680067490 2022-08-01T22:56:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0680060670 con 0x7f0680060290 2022-08-01T22:56:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0680067030 con 0x7f06800b4dd0 2022-08-01T22:56:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 0x7f068014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 0x7f068014b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:53462/0 (socket says 172.21.15.32:53462) 2022-08-01T22:56:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 -- 172.21.15.32:0/537765077 learned_addr learned my addr 172.21.15.32:0/537765077 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:24.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068517a700 1 --2- 172.21.15.32:0/537765077 >> v2:172.21.15.55:3300/0 conn(0x7f0680060290 0x7f06800b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 -- 172.21.15.32:0/537765077 >> v2:172.21.15.55:3300/0 conn(0x7f0680060290 msgr2=0x7f06800b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 --2- 172.21.15.32:0/537765077 >> v2:172.21.15.55:3300/0 conn(0x7f0680060290 0x7f06800b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 -- 172.21.15.32:0/537765077 >> v1:172.21.15.55:6789/0 conn(0x7f0680067490 legacy=0x7f0680143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 -- 172.21.15.32:0/537765077 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06800b3540 con 0x7f06800b4dd0 2022-08-01T22:56:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f068597b700 1 --2- 172.21.15.32:0/537765077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 0x7f068014b9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f06780032f0 tx=0x7f0678009290).ready entity=mon.2 client_cookie=f2f4676bcb95f491 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f0673fff700 1 -- 172.21.15.32:0/537765077 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f067800b8a0 con 0x7f06800b4dd0 2022-08-01T22:56:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f0673fff700 1 -- 172.21.15.32:0/537765077 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067800ba00 con 0x7f06800b4dd0 2022-08-01T22:56:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f0673fff700 1 -- 172.21.15.32:0/537765077 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f067800bb70 con 0x7f06800b4dd0 2022-08-01T22:56:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- 172.21.15.32:0/537765077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 msgr2=0x7f068014b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 --2- 172.21.15.32:0/537765077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 0x7f068014b9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f06780032f0 tx=0x7f0678009290).stop 2022-08-01T22:56:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- 172.21.15.32:0/537765077 shutdown_connections 2022-08-01T22:56:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 --2- 172.21.15.32:0/537765077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06800b4dd0 0x7f068014b9e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 --2- 172.21.15.32:0/537765077 >> v2:172.21.15.55:3300/0 conn(0x7f0680060290 0x7f06800b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- 172.21.15.32:0/537765077 >> 172.21.15.32:0/537765077 conn(0x7f068005d0c0 msgr2=0x7f068005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- 172.21.15.32:0/537765077 shutdown_connections 2022-08-01T22:56:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.071+0000 7f06873de700 1 -- 172.21.15.32:0/537765077 wait complete. 2022-08-01T22:56:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 Processor -- start 2022-08-01T22:56:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 -- start start 2022-08-01T22:56:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0680060290 0x7f06801506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0680067490 0x7f06800b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 0x7f06800ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06800a0d80 con 0x7f06800b4dd0 2022-08-01T22:56:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06800a1020 con 0x7f0680060290 2022-08-01T22:56:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f06873de700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06800a12c0 con 0x7f0680067490 2022-08-01T22:56:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f068597b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 0x7f06800ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f068597b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 0x7f06800ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41730/0 (socket says 172.21.15.32:41730) 2022-08-01T22:56:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f068597b700 1 -- 172.21.15.32:0/1060417208 learned_addr learned my addr 172.21.15.32:0/1060417208 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f068517a700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0680060290 0x7f06801506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.075+0000 7f0684979700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0680067490 0x7f06800b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f068597b700 1 -- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0680067490 msgr2=0x7f06800b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f068597b700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0680067490 0x7f06800b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f068597b700 1 -- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0680060290 msgr2=0x7f06801506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f068597b700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0680060290 0x7f06801506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f068597b700 1 -- 172.21.15.32:0/1060417208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009530 con 0x7f06800b4dd0 2022-08-01T22:56:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f068597b700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 0x7f06800ae6c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f06780032f0 tx=0x7f0678016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f067800b8a0 con 0x7f06800b4dd0 2022-08-01T22:56:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678021440 con 0x7f06800b4dd0 2022-08-01T22:56:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06800ad480 con 0x7f06800b4dd0 2022-08-01T22:56:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06780217a0 con 0x7f06800b4dd0 2022-08-01T22:56:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06800aebb0 con 0x7f06800b4dd0 2022-08-01T22:56:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06800ad610 con 0x7f06800b4dd0 2022-08-01T22:56:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.079+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0678027020 con 0x7f06800b4dd0 2022-08-01T22:56:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.083+0000 7f06727fc700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0668053a10 0x7f0668055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.083+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f067806d1f0 con 0x7f06800b4dd0 2022-08-01T22:56:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.083+0000 7f068517a700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0668053a10 0x7f0668055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.083+0000 7f068517a700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0668053a10 0x7f0668055ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f067400b000 tx=0x7f0674006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.087+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06780442f0 con 0x7f06800b4dd0 2022-08-01T22:56:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.255+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f06800574e0 con 0x7f06800b4dd0 2022-08-01T22:56:24.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06727fc700 1 -- 172.21.15.32:0/1060417208 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0678015020 con 0x7f06800b4dd0 2022-08-01T22:56:24.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0668053a10 msgr2=0x7f0668055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0668053a10 0x7f0668055ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f067400b000 tx=0x7f0674006df0).stop 2022-08-01T22:56:24.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 msgr2=0x7f06800ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 0x7f06800ae6c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f06780032f0 tx=0x7f0678016000).stop 2022-08-01T22:56:24.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 shutdown_connections 2022-08-01T22:56:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0668053a10 0x7f0668055ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06800b4dd0 0x7f06800ae6c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0680067490 0x7f06800b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 --2- 172.21.15.32:0/1060417208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0680060290 0x7f06801506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 >> 172.21.15.32:0/1060417208 conn(0x7f068005d0c0 msgr2=0x7f06800b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 shutdown_connections 2022-08-01T22:56:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.263+0000 7f06873de700 1 -- 172.21.15.32:0/1060417208 wait complete. 2022-08-01T22:56:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2022-08-01T22:56:24.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2022-08-01T22:56:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 Processor -- start 2022-08-01T22:56:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 -- start start 2022-08-01T22:56:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b2710 0x7f8a940abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a940b13f0 0x7f8a940ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a940b1920 0x7f8a940afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a940adf10 con 0x7f8a940b1920 2022-08-01T22:56:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a940ae1b0 con 0x7f8a940b13f0 2022-08-01T22:56:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9ad5f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940a9520 con 0x7f8a940b2710 2022-08-01T22:56:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9a55e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a940b1920 0x7f8a940afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9955c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a940b13f0 0x7f8a940ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9955c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a940b13f0 0x7f8a940ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:51520/0 (socket says 172.21.15.32:51520) 2022-08-01T22:56:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9955c700 1 -- 172.21.15.32:0/3302519615 learned_addr learned my addr 172.21.15.32:0/3302519615 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a9a55e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a940b1920 0x7f8a940afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41738/0 (socket says 172.21.15.32:41738) 2022-08-01T22:56:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.403+0000 7f8a99d5d700 1 --2- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b2710 0x7f8a940abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9955c700 1 -- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b2710 msgr2=0x7f8a940abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9955c700 1 --2- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b2710 0x7f8a940abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9955c700 1 -- 172.21.15.32:0/3302519615 >> v2:172.21.15.32:3300/0 conn(0x7f8a940b1920 msgr2=0x7f8a940afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9955c700 1 --2- 172.21.15.32:0/3302519615 >> v2:172.21.15.32:3300/0 conn(0x7f8a940b1920 0x7f8a940afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9955c700 1 -- 172.21.15.32:0/3302519615 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a940a99c0 con 0x7f8a940b13f0 2022-08-01T22:56:24.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9955c700 1 --2- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a940ac190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f8a94067bc0 tx=0x7f8a90006eb0).ready entity=mon.1 client_cookie=157fc29697819ab5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a98d5b700 1 -- 172.21.15.32:0/3302519615 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a9000e540 con 0x7f8a940b13f0 2022-08-01T22:56:24.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a98d5b700 1 -- 172.21.15.32:0/3302519615 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90000e90 con 0x7f8a940b13f0 2022-08-01T22:56:24.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a98d5b700 1 -- 172.21.15.32:0/3302519615 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a9000e8d0 con 0x7f8a940b13f0 2022-08-01T22:56:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 msgr2=0x7f8a940ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a940ac190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f8a94067bc0 tx=0x7f8a90006eb0).stop 2022-08-01T22:56:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/3302519615 shutdown_connections 2022-08-01T22:56:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/3302519615 >> v2:172.21.15.32:3300/0 conn(0x7f8a940b1920 0x7f8a940afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a940ac190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/3302519615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b2710 0x7f8a940abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/3302519615 >> 172.21.15.32:0/3302519615 conn(0x7f8a94060310 msgr2=0x7f8a940b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/3302519615 shutdown_connections 2022-08-01T22:56:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/3302519615 wait complete. 2022-08-01T22:56:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 Processor -- start 2022-08-01T22:56:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.407+0000 7f8a9ad5f700 1 -- start start 2022-08-01T22:56:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a94066470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b1920 0x7f8a94144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 0x7f8a941411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94140d80 con 0x7f8a940b2710 2022-08-01T22:56:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9413f9a0 con 0x7f8a940b13f0 2022-08-01T22:56:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9413f6f0 con 0x7f8a940b1920 2022-08-01T22:56:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 0x7f8a941411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 0x7f8a941411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41742/0 (socket says 172.21.15.32:41742) 2022-08-01T22:56:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 -- 172.21.15.32:0/1508928340 learned_addr learned my addr 172.21.15.32:0/1508928340 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9955c700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b1920 0x7f8a94144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a99d5d700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a94066470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 -- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b1920 msgr2=0x7f8a94144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b1920 0x7f8a94144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 -- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 msgr2=0x7f8a94066470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a94066470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 -- 172.21.15.32:0/1508928340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9000b1e0 con 0x7f8a940b2710 2022-08-01T22:56:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9a55e700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 0x7f8a941411f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f8a8c00c000 tx=0x7f8a8c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8c00c610 con 0x7f8a940b2710 2022-08-01T22:56:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c00c770 con 0x7f8a940b2710 2022-08-01T22:56:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a94142e10 con 0x7f8a940b2710 2022-08-01T22:56:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8c00cad0 con 0x7f8a940b2710 2022-08-01T22:56:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a94066aa0 con 0x7f8a940b2710 2022-08-01T22:56:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.411+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a940574e0 con 0x7f8a940b2710 2022-08-01T22:56:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.415+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a8c022070 con 0x7f8a940b2710 2022-08-01T22:56:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.415+0000 7f8a86ffd700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8a74053950 0x7f8a74055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.415+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8a8c06c2d0 con 0x7f8a940b2710 2022-08-01T22:56:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.415+0000 7f8a99d5d700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8a74053950 0x7f8a74055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.415+0000 7f8a99d5d700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8a74053950 0x7f8a74055e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f8a8800b000 tx=0x7f8a88012040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.419+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a8c0433d0 con 0x7f8a940b2710 2022-08-01T22:56:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.583+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8a94069a70 con 0x7f8a940b2710 2022-08-01T22:56:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.591+0000 7f8a86ffd700 1 -- 172.21.15.32:0/1508928340 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a8c016020 con 0x7f8a940b2710 2022-08-01T22:56:24.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8a74053950 msgr2=0x7f8a74055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8a74053950 0x7f8a74055e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f8a8800b000 tx=0x7f8a88012040).stop 2022-08-01T22:56:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 msgr2=0x7f8a941411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 0x7f8a941411f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f8a8c00c000 tx=0x7f8a8c0091d0).stop 2022-08-01T22:56:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 shutdown_connections 2022-08-01T22:56:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8a74053950 0x7f8a74055e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a940b2710 0x7f8a941411f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8a940b1920 0x7f8a94144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 --2- 172.21.15.32:0/1508928340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a940b13f0 0x7f8a94066470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 >> 172.21.15.32:0/1508928340 conn(0x7f8a94060310 msgr2=0x7f8a940a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 shutdown_connections 2022-08-01T22:56:24.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.595+0000 7f8a9ad5f700 1 -- 172.21.15.32:0/1508928340 wait complete. 2022-08-01T22:56:24.605 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2022-08-01T22:56:24.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2022-08-01T22:56:24.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 Processor -- start 2022-08-01T22:56:24.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 -- start start 2022-08-01T22:56:24.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51b00b0570 0x7f51b00af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51b013fa40 0x7f51b013fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b00af610 con 0x7f51b00ad670 2022-08-01T22:56:24.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b00b0c40 con 0x7f51b013fa40 2022-08-01T22:56:24.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b72a3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b00b0ee0 con 0x7f51b00b0570 2022-08-01T22:56:24.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b62a1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51b00ad670 0x7f51b00b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57522/0 (socket says 172.21.15.32:57522) 2022-08-01T22:56:24.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b62a1700 1 -- 172.21.15.32:0/920163040 learned_addr learned my addr 172.21.15.32:0/920163040 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:24.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b6aa2700 1 --2- 172.21.15.32:0/920163040 >> v2:172.21.15.32:3300/0 conn(0x7f51b013fa40 0x7f51b013fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b5aa0700 1 --2- 172.21.15.32:0/920163040 >> v2:172.21.15.55:3300/0 conn(0x7f51b00b0570 0x7f51b00af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b529f700 1 -- 172.21.15.32:0/920163040 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155599278 0 0) 0x7f51ac001490 con 0x7f51b00ad670 2022-08-01T22:56:24.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b529f700 1 -- 172.21.15.32:0/920163040 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5198003260 con 0x7f51b00ad670 2022-08-01T22:56:24.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b6aa2700 1 -- 172.21.15.32:0/920163040 >> v2:172.21.15.55:3300/0 conn(0x7f51b00b0570 msgr2=0x7f51b00af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:24.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b6aa2700 1 --2- 172.21.15.32:0/920163040 >> v2:172.21.15.55:3300/0 conn(0x7f51b00b0570 0x7f51b00af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b6aa2700 1 -- 172.21.15.32:0/920163040 >> v1:172.21.15.32:6789/0 conn(0x7f51b00ad670 legacy=0x7f51b00b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b6aa2700 1 -- 172.21.15.32:0/920163040 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00ae9e0 con 0x7f51b013fa40 2022-08-01T22:56:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b6aa2700 1 --2- 172.21.15.32:0/920163040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b013fa40 0x7f51b013fea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f51a8002f70 tx=0x7f51a8006df0).ready entity=mon.0 client_cookie=ba05d9da442e94aa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b529f700 1 -- 172.21.15.32:0/920163040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a800d4f0 con 0x7f51b013fa40 2022-08-01T22:56:24.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b529f700 1 -- 172.21.15.32:0/920163040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a8000e90 con 0x7f51b013fa40 2022-08-01T22:56:24.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.719+0000 7f51b529f700 1 -- 172.21.15.32:0/920163040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a800d840 con 0x7f51b013fa40 2022-08-01T22:56:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 -- 172.21.15.32:0/920163040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b013fa40 msgr2=0x7f51b013fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 --2- 172.21.15.32:0/920163040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b013fa40 0x7f51b013fea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f51a8002f70 tx=0x7f51a8006df0).stop 2022-08-01T22:56:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 -- 172.21.15.32:0/920163040 shutdown_connections 2022-08-01T22:56:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 --2- 172.21.15.32:0/920163040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b013fa40 0x7f51b013fea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 --2- 172.21.15.32:0/920163040 >> v2:172.21.15.55:3300/0 conn(0x7f51b00b0570 0x7f51b00af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 -- 172.21.15.32:0/920163040 >> 172.21.15.32:0/920163040 conn(0x7f51b0061360 msgr2=0x7f51b0062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 -- 172.21.15.32:0/920163040 shutdown_connections 2022-08-01T22:56:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 -- 172.21.15.32:0/920163040 wait complete. 2022-08-01T22:56:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 Processor -- start 2022-08-01T22:56:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.723+0000 7f51b72a3700 1 -- start start 2022-08-01T22:56:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 0x7f51b013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51b00b0570 0x7f51b00abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b013fa40 0x7f51b0153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0064060 con 0x7f51b00ad670 2022-08-01T22:56:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0063dc0 con 0x7f51b013fa40 2022-08-01T22:56:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0067770 con 0x7f51b00b0570 2022-08-01T22:56:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 0x7f51b013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 0x7f51b013eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41754/0 (socket says 172.21.15.32:41754) 2022-08-01T22:56:24.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 -- 172.21.15.32:0/1048599213 learned_addr learned my addr 172.21.15.32:0/1048599213 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:24.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b5aa0700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51b00b0570 0x7f51b00abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 -- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51b00b0570 msgr2=0x7f51b00abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b6aa2700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b013fa40 0x7f51b0153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51b00b0570 0x7f51b00abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 -- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b013fa40 msgr2=0x7f51b0153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b013fa40 0x7f51b0153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 -- 172.21.15.32:0/1048599213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a800b180 con 0x7f51b00ad670 2022-08-01T22:56:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b5aa0700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51b00b0570 0x7f51b00abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b6aa2700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b013fa40 0x7f51b0153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b62a1700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 0x7f51b013eea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f51ac001620 tx=0x7f51ac007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ac005ba0 con 0x7f51b00ad670 2022-08-01T22:56:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac005d00 con 0x7f51b00ad670 2022-08-01T22:56:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0067a10 con 0x7f51b00ad670 2022-08-01T22:56:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ac007300 con 0x7f51b00ad670 2022-08-01T22:56:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b00ac280 con 0x7f51b00ad670 2022-08-01T22:56:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.727+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f51ac010030 con 0x7f51b00ad670 2022-08-01T22:56:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.731+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b00aec80 con 0x7f51b00ad670 2022-08-01T22:56:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.731+0000 7f519f7fe700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5190053a70 0x7f5190055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.731+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f51ac06bd10 con 0x7f51b00ad670 2022-08-01T22:56:24.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.731+0000 7f51b5aa0700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5190053a70 0x7f5190055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.731+0000 7f51b5aa0700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5190053a70 0x7f5190055f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f51a000b000 tx=0x7f51a0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.735+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51ac071020 con 0x7f51b00ad670 2022-08-01T22:56:24.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.903+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f51b00af610 con 0x7f51b00ad670 2022-08-01T22:56:24.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.911+0000 7f519f7fe700 1 -- 172.21.15.32:0/1048599213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f51b00af610 con 0x7f51b00ad670 2022-08-01T22:56:24.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5190053a70 msgr2=0x7f5190055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5190053a70 0x7f5190055f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f51a000b000 tx=0x7f51a0008040).stop 2022-08-01T22:56:24.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 msgr2=0x7f51b013eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:24.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 0x7f51b013eea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f51ac001620 tx=0x7f51ac007c80).stop 2022-08-01T22:56:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 shutdown_connections 2022-08-01T22:56:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5190053a70 0x7f5190055f30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b013fa40 0x7f51b0153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51b00b0570 0x7f51b00abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 --2- 172.21.15.32:0/1048599213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00ad670 0x7f51b013eea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 >> 172.21.15.32:0/1048599213 conn(0x7f51b0061360 msgr2=0x7f51b00b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 shutdown_connections 2022-08-01T22:56:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:24.915+0000 7f51b72a3700 1 -- 172.21.15.32:0/1048599213 wait complete. 2022-08-01T22:56:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:updated 2022-08-01T22:56:24.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe69d722700 1 Processor -- start 2022-08-01T22:56:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe69d722700 1 -- start start 2022-08-01T22:56:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe69d722700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980ad750 0x7fe698061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe69d722700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6980aee80 con 0x7fe6980614b0 2022-08-01T22:56:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe69d722700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6980b70b0 con 0x7fe6981405d0 2022-08-01T22:56:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe69d722700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980af0b0 con 0x7fe6980ad750 2022-08-01T22:56:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe697fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6980614b0 0x7fe6980b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57536/0 (socket says 172.21.15.32:57536) 2022-08-01T22:56:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe697fff700 1 -- 172.21.15.32:0/3417638461 learned_addr learned my addr 172.21.15.32:0/3417638461 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe6977fe700 1 --2- 172.21.15.32:0/3417638461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980ad750 0x7fe698061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198593117 0 0) 0x7fe6980aee80 con 0x7fe6980614b0 2022-08-01T22:56:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe684003260 con 0x7fe6980614b0 2022-08-01T22:56:25.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144093464 0 0) 0x7fe6980b70b0 con 0x7fe6981405d0 2022-08-01T22:56:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6980aee80 con 0x7fe6981405d0 2022-08-01T22:56:25.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe6977fe700 1 -- 172.21.15.32:0/3417638461 >> v1:172.21.15.55:6789/0 conn(0x7fe6981405d0 legacy=0x7fe6981466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe6977fe700 1 -- 172.21.15.32:0/3417638461 >> v1:172.21.15.32:6789/0 conn(0x7fe6980614b0 legacy=0x7fe6980b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe6977fe700 1 -- 172.21.15.32:0/3417638461 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980af350 con 0x7fe6980ad750 2022-08-01T22:56:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe6977fe700 1 --2- 172.21.15.32:0/3417638461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980ad750 0x7fe698061890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe6880032f0 tx=0x7fe6880091d0).ready entity=mon.2 client_cookie=2c16d796eeed472c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe68800b640 con 0x7fe6980ad750 2022-08-01T22:56:25.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68800b7a0 con 0x7fe6980ad750 2022-08-01T22:56:25.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.039+0000 7fe696ffd700 1 -- 172.21.15.32:0/3417638461 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe68800b910 con 0x7fe6980ad750 2022-08-01T22:56:25.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- 172.21.15.32:0/3417638461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980ad750 msgr2=0x7fe698061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 --2- 172.21.15.32:0/3417638461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980ad750 0x7fe698061890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe6880032f0 tx=0x7fe6880091d0).stop 2022-08-01T22:56:25.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- 172.21.15.32:0/3417638461 shutdown_connections 2022-08-01T22:56:25.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 --2- 172.21.15.32:0/3417638461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980ad750 0x7fe698061890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- 172.21.15.32:0/3417638461 >> 172.21.15.32:0/3417638461 conn(0x7fe698066f70 msgr2=0x7fe698067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- 172.21.15.32:0/3417638461 shutdown_connections 2022-08-01T22:56:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- 172.21.15.32:0/3417638461 wait complete. 2022-08-01T22:56:25.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 Processor -- start 2022-08-01T22:56:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- start start 2022-08-01T22:56:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980614b0 0x7fe6980afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6980ad750 0x7fe6980b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 0x7fe6980b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69809f300 con 0x7fe6981405d0 2022-08-01T22:56:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe698145db0 con 0x7fe6980ad750 2022-08-01T22:56:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69d722700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6981533c0 con 0x7fe6980614b0 2022-08-01T22:56:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 0x7fe6980b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 0x7fe6980b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41772/0 (socket says 172.21.15.32:41772) 2022-08-01T22:56:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 -- 172.21.15.32:0/2535759965 learned_addr learned my addr 172.21.15.32:0/2535759965 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe697fff700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980614b0 0x7fe6980afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe6977fe700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6980ad750 0x7fe6980b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 -- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980614b0 msgr2=0x7fe6980afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980614b0 0x7fe6980afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 -- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6980ad750 msgr2=0x7fe6980b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6980ad750 0x7fe6980b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 -- 172.21.15.32:0/2535759965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009470 con 0x7fe6981405d0 2022-08-01T22:56:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.043+0000 7fe69cf21700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 0x7fe6980b1450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fe690001110 tx=0x7fe690007a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69000cd60 con 0x7fe6981405d0 2022-08-01T22:56:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690012400 con 0x7fe6981405d0 2022-08-01T22:56:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69809ebc0 con 0x7fe6981405d0 2022-08-01T22:56:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe690012760 con 0x7fe6981405d0 2022-08-01T22:56:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698067740 con 0x7fe6981405d0 2022-08-01T22:56:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe698153580 con 0x7fe6981405d0 2022-08-01T22:56:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe690019030 con 0x7fe6981405d0 2022-08-01T22:56:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe6957fa700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe678053a70 0x7fe678055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe69006af70 con 0x7fe6981405d0 2022-08-01T22:56:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe697fff700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe678053a70 0x7fe678055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.047+0000 7fe697fff700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe678053a70 0x7fe678055f30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe68c00d000 tx=0x7fe68c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.055+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe690042070 con 0x7fe6981405d0 2022-08-01T22:56:25.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.183+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6980af5f0 con 0x7fe6981405d0 2022-08-01T22:56:25.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.183+0000 7fe6957fa700 1 -- 172.21.15.32:0/2535759965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13414 (secure 0 0 0) 0x7fe690045690 con 0x7fe6981405d0 2022-08-01T22:56:25.190 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:25.190 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":30,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:56:22.523467+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.183+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe678053a70 msgr2=0x7fe678055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.183+0000 7fe69d722700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe678053a70 0x7fe678055f30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe68c00d000 tx=0x7fe68c009040).stop 2022-08-01T22:56:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.183+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 msgr2=0x7fe6980b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.183+0000 7fe69d722700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 0x7fe6980b1450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fe690001110 tx=0x7fe690007a10).stop 2022-08-01T22:56:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 shutdown_connections 2022-08-01T22:56:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe678053a70 0x7fe678055f30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6981405d0 0x7fe6980b1450 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6980ad750 0x7fe6980b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 --2- 172.21.15.32:0/2535759965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6980614b0 0x7fe6980afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 >> 172.21.15.32:0/2535759965 conn(0x7fe698066f70 msgr2=0x7fe6980b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 shutdown_connections 2022-08-01T22:56:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.187+0000 7fe69d722700 1 -- 172.21.15.32:0/2535759965 wait complete. 2022-08-01T22:56:25.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T22:56:25.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:25.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.307+0000 7fa97399c700 1 Processor -- start 2022-08-01T22:56:25.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.307+0000 7fa97399c700 1 -- start start 2022-08-01T22:56:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa97399c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa96c0b0740 0x7fa96c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa97399c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c143ab0 con 0x7fa96c0b38a0 2022-08-01T22:56:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa97399c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c143d50 con 0x7fa96c0a9da0 2022-08-01T22:56:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa97399c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c143ff0 con 0x7fa96c0b0740 2022-08-01T22:56:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa970f37700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa96c0b38a0 0x7fa96c13d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57554/0 (socket says 172.21.15.32:57554) 2022-08-01T22:56:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa970f37700 1 -- 172.21.15.32:0/964085883 learned_addr learned my addr 172.21.15.32:0/964085883 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971738700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa96c0b0740 0x7fa96c0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908935363 0 0) 0x7fa96c143ab0 con 0x7fa96c0b38a0 2022-08-01T22:56:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa954003260 con 0x7fa96c0b38a0 2022-08-01T22:56:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709749134 0 0) 0x7fa96c143d50 con 0x7fa96c0a9da0 2022-08-01T22:56:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa96c143ab0 con 0x7fa96c0a9da0 2022-08-01T22:56:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103792021 0 0) 0x7fa954003260 con 0x7fa96c0b38a0 2022-08-01T22:56:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa96c0b0740 msgr2=0x7fa96c0b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa96c0b0740 0x7fa96c0b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 >> v1:172.21.15.55:6789/0 conn(0x7fa96c0a9da0 legacy=0x7fa96c0b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c144290 con 0x7fa96c0b38a0 2022-08-01T22:56:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fa968001660 con 0x7fa96c0b38a0 2022-08-01T22:56:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971738700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa96c0b0740 0x7fa96c0b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 >> v1:172.21.15.32:6789/0 conn(0x7fa96c0b38a0 legacy=0x7fa96c13d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954003910 0x7fa954003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa954004500 0x7fa954004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa954004e70 0x7fa95400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c143ff0 con 0x7fa954004e70 2022-08-01T22:56:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400bcf0 con 0x7fa954004500 2022-08-01T22:56:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400beb0 con 0x7fa954003910 2022-08-01T22:56:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa954004e70 0x7fa95400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971738700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954003910 0x7fa954003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa970f37700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa954004500 0x7fa954004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 -- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954003910 msgr2=0x7fa954003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954003910 0x7fa954003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 -- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa954004500 msgr2=0x7fa954004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa954004500 0x7fa954004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 -- 172.21.15.32:0/964085883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954003260 con 0x7fa954004e70 2022-08-01T22:56:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971738700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954003910 0x7fa954003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa971f39700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa954004e70 0x7fa95400b750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fa964001490 tx=0x7fa964007d80).ready entity=mon.0 client_cookie=995e52ab999410ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa964001780 con 0x7fa954004e70 2022-08-01T22:56:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964006890 con 0x7fa954004e70 2022-08-01T22:56:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.311+0000 7fa95bfff700 1 -- 172.21.15.32:0/964085883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa964006bf0 con 0x7fa954004e70 2022-08-01T22:56:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- 172.21.15.32:0/964085883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa954004e70 msgr2=0x7fa95400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 --2- 172.21.15.32:0/964085883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa954004e70 0x7fa95400b750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fa964001490 tx=0x7fa964007d80).stop 2022-08-01T22:56:25.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa971f39700 1 -- 172.21.15.32:0/964085883 reap_dead start 2022-08-01T22:56:25.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- 172.21.15.32:0/964085883 shutdown_connections 2022-08-01T22:56:25.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- 172.21.15.32:0/964085883 >> 172.21.15.32:0/964085883 conn(0x7fa96c060950 msgr2=0x7fa96c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- 172.21.15.32:0/964085883 shutdown_connections 2022-08-01T22:56:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- 172.21.15.32:0/964085883 wait complete. 2022-08-01T22:56:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 Processor -- start 2022-08-01T22:56:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- start start 2022-08-01T22:56:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954004e70 0x7fa96c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 0x7fa96c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa96c1396a0 0x7fa96c139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c141050 con 0x7fa96c0ab750 2022-08-01T22:56:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c1409e0 con 0x7fa96c1396a0 2022-08-01T22:56:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa97399c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c13fe20 con 0x7fa954004e70 2022-08-01T22:56:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa970f37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 0x7fa96c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa971f39700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa96c1396a0 0x7fa96c139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa970f37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 0x7fa96c0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41792/0 (socket says 172.21.15.32:41792) 2022-08-01T22:56:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.315+0000 7fa970f37700 1 -- 172.21.15.32:0/3633209232 learned_addr learned my addr 172.21.15.32:0/3633209232 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa971738700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954004e70 0x7fa96c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa970f37700 1 -- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954004e70 msgr2=0x7fa96c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa970f37700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954004e70 0x7fa96c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa970f37700 1 -- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa96c1396a0 msgr2=0x7fa96c139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa970f37700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa96c1396a0 0x7fa96c139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa970f37700 1 -- 172.21.15.32:0/3633209232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0ad820 con 0x7fa96c0ab750 2022-08-01T22:56:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa970f37700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 0x7fa96c0add80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fa96800b000 tx=0x7fa96c1411b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa968015630 con 0x7fa96c0ab750 2022-08-01T22:56:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968010040 con 0x7fa96c0ab750 2022-08-01T22:56:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c1402c0 con 0x7fa96c0ab750 2022-08-01T22:56:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa968015980 con 0x7fa96c0ab750 2022-08-01T22:56:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c140c40 con 0x7fa96c0ab750 2022-08-01T22:56:25.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c13ffe0 con 0x7fa96c0ab750 2022-08-01T22:56:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa968015ae0 con 0x7fa96c0ab750 2022-08-01T22:56:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa95a7fc700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa94c053b00 0x7fa94c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.319+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa96806b740 con 0x7fa96c0ab750 2022-08-01T22:56:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.323+0000 7fa971738700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa94c053b00 0x7fa94c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.323+0000 7fa971738700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa94c053b00 0x7fa94c055fc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fa95c00b000 tx=0x7fa95c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.323+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa968042840 con 0x7fa96c0ab750 2022-08-01T22:56:25.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.455+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa96c13ddf0 con 0x7fa96c0ab750 2022-08-01T22:56:25.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa95a7fc700 1 -- 172.21.15.32:0/3633209232 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v30) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa968045e60 con 0x7fa96c0ab750 2022-08-01T22:56:25.464 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:25.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa94c053b00 msgr2=0x7fa94c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa94c053b00 0x7fa94c055fc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fa95c00b000 tx=0x7fa95c008040).stop 2022-08-01T22:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 msgr2=0x7fa96c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 0x7fa96c0add80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fa96800b000 tx=0x7fa96c1411b0).stop 2022-08-01T22:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 shutdown_connections 2022-08-01T22:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa94c053b00 0x7fa94c055fc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa96c1396a0 0x7fa96c139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa96c0ab750 0x7fa96c0add80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 --2- 172.21.15.32:0/3633209232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa954004e70 0x7fa96c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 >> 172.21.15.32:0/3633209232 conn(0x7fa96c060950 msgr2=0x7fa96c142880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 shutdown_connections 2022-08-01T22:56:25.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.459+0000 7fa97399c700 1 -- 172.21.15.32:0/3633209232 wait complete. 2022-08-01T22:56:25.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 Processor -- start 2022-08-01T22:56:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 -- start start 2022-08-01T22:56:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b2710 0x7fb10414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1040b01a0 0x7fb10414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1040a7250 con 0x7fb1040afd60 2022-08-01T22:56:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040a7480 con 0x7fb1040b01a0 2022-08-01T22:56:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb103fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040a7720 con 0x7fb1040b2710 2022-08-01T22:56:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb1027fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1040b01a0 0x7fb10414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb1037fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb1040afd60 0x7fb1040a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57574/0 (socket says 172.21.15.32:57574) 2022-08-01T22:56:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb1037fe700 1 -- 172.21.15.32:0/1228431392 learned_addr learned my addr 172.21.15.32:0/1228431392 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.587+0000 7fb102ffd700 1 --2- 172.21.15.32:0/1228431392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b2710 0x7fb10414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb1027fc700 1 -- 172.21.15.32:0/1228431392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b2710 msgr2=0x7fb10414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb1027fc700 1 --2- 172.21.15.32:0/1228431392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b2710 0x7fb10414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb1027fc700 1 -- 172.21.15.32:0/1228431392 >> v1:172.21.15.32:6789/0 conn(0x7fb1040afd60 legacy=0x7fb1040a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb1027fc700 1 -- 172.21.15.32:0/1228431392 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040a79c0 con 0x7fb1040b01a0 2022-08-01T22:56:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb1027fc700 1 --2- 172.21.15.32:0/1228431392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b01a0 0x7fb10414b4c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fb0f4002f70 tx=0x7fb0f4006df0).ready entity=mon.0 client_cookie=7f198712cd818f1c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb101ffb700 1 -- 172.21.15.32:0/1228431392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0f400d570 con 0x7fb1040b01a0 2022-08-01T22:56:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb101ffb700 1 -- 172.21.15.32:0/1228431392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4000e90 con 0x7fb1040b01a0 2022-08-01T22:56:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb101ffb700 1 -- 172.21.15.32:0/1228431392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0f400d8c0 con 0x7fb1040b01a0 2022-08-01T22:56:25.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- 172.21.15.32:0/1228431392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b01a0 msgr2=0x7fb10414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 --2- 172.21.15.32:0/1228431392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b01a0 0x7fb10414b4c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fb0f4002f70 tx=0x7fb0f4006df0).stop 2022-08-01T22:56:25.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- 172.21.15.32:0/1228431392 shutdown_connections 2022-08-01T22:56:25.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 --2- 172.21.15.32:0/1228431392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b01a0 0x7fb10414b4c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 --2- 172.21.15.32:0/1228431392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b2710 0x7fb10414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- 172.21.15.32:0/1228431392 >> 172.21.15.32:0/1228431392 conn(0x7fb104060310 msgr2=0x7fb1040b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- 172.21.15.32:0/1228431392 shutdown_connections 2022-08-01T22:56:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- 172.21.15.32:0/1228431392 wait complete. 2022-08-01T22:56:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 Processor -- start 2022-08-01T22:56:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- start start 2022-08-01T22:56:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1040afd60 0x7fb104148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b01a0 0x7fb1040b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 0x7fb10415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040b40c0 con 0x7fb1040b2710 2022-08-01T22:56:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040b42f0 con 0x7fb1040afd60 2022-08-01T22:56:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.591+0000 7fb103fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040b4590 con 0x7fb1040b01a0 2022-08-01T22:56:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 0x7fb10415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 0x7fb10415a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41806/0 (socket says 172.21.15.32:41806) 2022-08-01T22:56:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 -- 172.21.15.32:0/934143836 learned_addr learned my addr 172.21.15.32:0/934143836 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb102ffd700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1040afd60 0x7fb104148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1027fc700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b01a0 0x7fb1040b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 -- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b01a0 msgr2=0x7fb1040b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b01a0 0x7fb1040b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 -- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1040afd60 msgr2=0x7fb104148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1040afd60 0x7fb104148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 -- 172.21.15.32:0/934143836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f400b180 con 0x7fb1040b2710 2022-08-01T22:56:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1027fc700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b01a0 0x7fb1040b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb102ffd700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1040afd60 0x7fb104148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb1037fe700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 0x7fb10415a010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fb104061c10 tx=0x7fb0fc007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc005ba0 con 0x7fb1040b2710 2022-08-01T22:56:25.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc005d00 con 0x7fb1040b2710 2022-08-01T22:56:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc0072c0 con 0x7fb1040b2710 2022-08-01T22:56:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb104159b40 con 0x7fb1040b2710 2022-08-01T22:56:25.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1040b5fd0 con 0x7fb1040b2710 2022-08-01T22:56:25.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1040b5680 con 0x7fb1040b2710 2022-08-01T22:56:25.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.595+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb0fc010070 con 0x7fb1040b2710 2022-08-01T22:56:25.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.599+0000 7fb0f3fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0e0053b30 0x7fb0e0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.599+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb0fc06b5a0 con 0x7fb1040b2710 2022-08-01T22:56:25.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.599+0000 7fb102ffd700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0e0053b30 0x7fb0e0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.599+0000 7fb102ffd700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0e0053b30 0x7fb0e0055ff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fb0f800b000 tx=0x7fb0f8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.603+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1040b5680 con 0x7fb1040b2710 2022-08-01T22:56:25.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.727+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb1040b5680 con 0x7fb1040b2710 2022-08-01T22:56:25.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.727+0000 7fb0f3fff700 1 -- 172.21.15.32:0/934143836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb1040b5680 con 0x7fb1040b2710 2022-08-01T22:56:25.735 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:25.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0e0053b30 msgr2=0x7fb0e0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0e0053b30 0x7fb0e0055ff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fb0f800b000 tx=0x7fb0f8008040).stop 2022-08-01T22:56:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 msgr2=0x7fb10415a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 0x7fb10415a010 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fb104061c10 tx=0x7fb0fc007f00).stop 2022-08-01T22:56:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 shutdown_connections 2022-08-01T22:56:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0e0053b30 0x7fb0e0055ff0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1040b2710 0x7fb10415a010 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1040b01a0 0x7fb1040b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 --2- 172.21.15.32:0/934143836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1040afd60 0x7fb104148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 >> 172.21.15.32:0/934143836 conn(0x7fb104060310 msgr2=0x7fb10414a210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 shutdown_connections 2022-08-01T22:56:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.731+0000 7fb103fff700 1 -- 172.21.15.32:0/934143836 wait complete. 2022-08-01T22:56:25.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T22:56:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 Processor -- start 2022-08-01T22:56:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 -- start start 2022-08-01T22:56:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8780b5630 0x7ff8780af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780b5a10 con 0x7ff878067730 2022-08-01T22:56:25.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff878145470 con 0x7ff8780b5630 2022-08-01T22:56:25.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.843+0000 7ff87ec46700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780ae180 con 0x7ff8780b07a0 2022-08-01T22:56:25.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87e445700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff878067730 0x7ff8780acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57588/0 (socket says 172.21.15.32:57588) 2022-08-01T22:56:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87e445700 1 -- 172.21.15.32:0/334690662 learned_addr learned my addr 172.21.15.32:0/334690662 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:25.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87dc44700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8780b5630 0x7ff8780af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87d443700 1 --2- 172.21.15.32:0/334690662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87cc42700 1 -- 172.21.15.32:0/334690662 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309510741 0 0) 0x7ff8780b5a10 con 0x7ff878067730 2022-08-01T22:56:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87cc42700 1 -- 172.21.15.32:0/334690662 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff864003260 con 0x7ff878067730 2022-08-01T22:56:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87d443700 1 -- 172.21.15.32:0/334690662 >> v2:172.21.15.32:3300/0 conn(0x7ff8780b5630 msgr2=0x7ff8780af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87d443700 1 --2- 172.21.15.32:0/334690662 >> v2:172.21.15.32:3300/0 conn(0x7ff8780b5630 0x7ff8780af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87d443700 1 -- 172.21.15.32:0/334690662 >> v1:172.21.15.32:6789/0 conn(0x7ff878067730 legacy=0x7ff8780acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87cc42700 1 -- 172.21.15.32:0/334690662 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3733185487 0 0) 0x7ff864003260 con 0x7ff878067730 2022-08-01T22:56:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87d443700 1 -- 172.21.15.32:0/334690662 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780ac990 con 0x7ff8780b07a0 2022-08-01T22:56:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87d443700 1 --2- 172.21.15.32:0/334690662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878147730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7ff870001800 tx=0x7ff8680091d0).ready entity=mon.2 client_cookie=a1efe48b114de47a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87cc42700 1 -- 172.21.15.32:0/334690662 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff870001660 con 0x7ff8780b07a0 2022-08-01T22:56:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87cc42700 1 -- 172.21.15.32:0/334690662 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800f510 con 0x7ff8780b07a0 2022-08-01T22:56:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87cc42700 1 -- 172.21.15.32:0/334690662 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86800f680 con 0x7ff8780b07a0 2022-08-01T22:56:25.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 -- 172.21.15.32:0/334690662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 msgr2=0x7ff878147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 --2- 172.21.15.32:0/334690662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878147730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7ff870001800 tx=0x7ff8680091d0).stop 2022-08-01T22:56:25.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 -- 172.21.15.32:0/334690662 shutdown_connections 2022-08-01T22:56:25.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 --2- 172.21.15.32:0/334690662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878147730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 --2- 172.21.15.32:0/334690662 >> v2:172.21.15.32:3300/0 conn(0x7ff8780b5630 0x7ff8780af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 -- 172.21.15.32:0/334690662 >> 172.21.15.32:0/334690662 conn(0x7ff8780626f0 msgr2=0x7ff878062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 -- 172.21.15.32:0/334690662 shutdown_connections 2022-08-01T22:56:25.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 -- 172.21.15.32:0/334690662 wait complete. 2022-08-01T22:56:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 Processor -- start 2022-08-01T22:56:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.847+0000 7ff87ec46700 1 -- start start 2022-08-01T22:56:25.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff878067730 0x7ff878144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8780b5630 0x7ff878063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780633d0 con 0x7ff878067730 2022-08-01T22:56:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780a7330 con 0x7ff8780b5630 2022-08-01T22:56:25.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780641e0 con 0x7ff8780b07a0 2022-08-01T22:56:25.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87dc44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff878067730 0x7ff878144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87dc44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff878067730 0x7ff878144b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41816/0 (socket says 172.21.15.32:41816) 2022-08-01T22:56:25.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87dc44700 1 -- 172.21.15.32:0/2018904672 learned_addr learned my addr 172.21.15.32:0/2018904672 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:25.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87e445700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8780b5630 0x7ff878063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 -- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8780b5630 msgr2=0x7ff878063cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8780b5630 0x7ff878063cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 -- 172.21.15.32:0/2018904672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff878067730 msgr2=0x7ff878144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff878067730 0x7ff878144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 -- 172.21.15.32:0/2018904672 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff868009470 con 0x7ff8780b07a0 2022-08-01T22:56:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87d443700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878063800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff870001800 tx=0x7ff868016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86800f680 con 0x7ff8780b07a0 2022-08-01T22:56:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8680032d0 con 0x7ff8780b07a0 2022-08-01T22:56:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff878064c80 con 0x7ff8780b07a0 2022-08-01T22:56:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff868021630 con 0x7ff8780b07a0 2022-08-01T22:56:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.851+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8780651c0 con 0x7ff8780b07a0 2022-08-01T22:56:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.855+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff878063540 con 0x7ff8780b07a0 2022-08-01T22:56:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.855+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff868027020 con 0x7ff8780b07a0 2022-08-01T22:56:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.855+0000 7ff876ffd700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff858053aa0 0x7ff858055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.855+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff86806cdf0 con 0x7ff8780b07a0 2022-08-01T22:56:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.855+0000 7ff87dc44700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff858053aa0 0x7ff858055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.855+0000 7ff87dc44700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff858053aa0 0x7ff858055f60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff86c00b000 tx=0x7ff86c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.863+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff878063540 con 0x7ff8780b07a0 2022-08-01T22:56:25.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.991+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff878063540 con 0x7ff8780b07a0 2022-08-01T22:56:25.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.991+0000 7ff876ffd700 1 -- 172.21.15.32:0/2018904672 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff878063540 con 0x7ff8780b07a0 2022-08-01T22:56:25.999 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:25.999 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T22:56:26.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:26.000 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T22:56:26.000 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T22:56:26.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:56:26.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:56:26.001 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T22:56:26.001 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T22:56:26.001 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:56:26.001 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T22:56:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff858053aa0 msgr2=0x7ff858055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:26.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff858053aa0 0x7ff858055f60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff86c00b000 tx=0x7ff86c006df0).stop 2022-08-01T22:56:26.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 msgr2=0x7ff878063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:26.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878063800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff870001800 tx=0x7ff868016000).stop 2022-08-01T22:56:26.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 shutdown_connections 2022-08-01T22:56:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff858053aa0 0x7ff858055f60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8780b5630 0x7ff878063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff8780b07a0 0x7ff878063800 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 --2- 172.21.15.32:0/2018904672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff878067730 0x7ff878144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 >> 172.21.15.32:0/2018904672 conn(0x7ff8780626f0 msgr2=0x7ff878149970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 shutdown_connections 2022-08-01T22:56:26.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:25.995+0000 7ff87ec46700 1 -- 172.21.15.32:0/2018904672 wait complete. 2022-08-01T22:56:26.015 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T22:56:26.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T22:56:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 Processor -- start 2022-08-01T22:56:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 -- start start 2022-08-01T22:56:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9680b7840 0x7fa968060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa9680b2930 0x7fa9680610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9680671e0 con 0x7fa968146720 2022-08-01T22:56:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa968065540 con 0x7fa9680b7840 2022-08-01T22:56:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96e724700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9680b7c20 con 0x7fa9680b2930 2022-08-01T22:56:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9680b7840 0x7fa968060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9680b7840 0x7fa968060be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41824/0 (socket says 172.21.15.32:41824) 2022-08-01T22:56:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 -- 172.21.15.32:0/615112130 learned_addr learned my addr 172.21.15.32:0/615112130 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96cf21700 1 --2- 172.21.15.32:0/615112130 >> v2:172.21.15.55:3300/0 conn(0x7fa9680b2930 0x7fa9680610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 -- 172.21.15.32:0/615112130 >> v2:172.21.15.55:3300/0 conn(0x7fa9680b2930 msgr2=0x7fa9680610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 --2- 172.21.15.32:0/615112130 >> v2:172.21.15.55:3300/0 conn(0x7fa9680b2930 0x7fa9680610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 -- 172.21.15.32:0/615112130 >> v1:172.21.15.55:6789/0 conn(0x7fa968146720 legacy=0x7fa968146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 -- 172.21.15.32:0/615112130 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968066ce0 con 0x7fa9680b7840 2022-08-01T22:56:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.123+0000 7fa96d722700 1 --2- 172.21.15.32:0/615112130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b7840 0x7fa968060be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa958002f70 tx=0x7fa958006e50).ready entity=mon.0 client_cookie=b6eaa5a27298d14a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa957fff700 1 -- 172.21.15.32:0/615112130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95800d610 con 0x7fa9680b7840 2022-08-01T22:56:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa957fff700 1 -- 172.21.15.32:0/615112130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa958000e90 con 0x7fa9680b7840 2022-08-01T22:56:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa957fff700 1 -- 172.21.15.32:0/615112130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95800d9a0 con 0x7fa9680b7840 2022-08-01T22:56:26.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- 172.21.15.32:0/615112130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b7840 msgr2=0x7fa968060be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:26.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 --2- 172.21.15.32:0/615112130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b7840 0x7fa968060be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa958002f70 tx=0x7fa958006e50).stop 2022-08-01T22:56:26.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- 172.21.15.32:0/615112130 shutdown_connections 2022-08-01T22:56:26.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 --2- 172.21.15.32:0/615112130 >> v2:172.21.15.55:3300/0 conn(0x7fa9680b2930 0x7fa9680610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 --2- 172.21.15.32:0/615112130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b7840 0x7fa968060be0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- 172.21.15.32:0/615112130 >> 172.21.15.32:0/615112130 conn(0x7fa96805f840 msgr2=0x7fa96805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:26.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- 172.21.15.32:0/615112130 shutdown_connections 2022-08-01T22:56:26.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- 172.21.15.32:0/615112130 wait complete. 2022-08-01T22:56:26.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 Processor -- start 2022-08-01T22:56:26.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- start start 2022-08-01T22:56:26.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 0x7fa9680ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:26.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9680b7840 0x7fa968152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:26.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa968146720 0x7fa968153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:26.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa968152580 con 0x7fa9680b2930 2022-08-01T22:56:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa968147e40 con 0x7fa9680b7840 2022-08-01T22:56:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681480e0 con 0x7fa968146720 2022-08-01T22:56:26.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 0x7fa9680ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:26.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 0x7fa9680ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41830/0 (socket says 172.21.15.32:41830) 2022-08-01T22:56:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 -- 172.21.15.32:0/2152233189 learned_addr learned my addr 172.21.15.32:0/2152233189 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96cf21700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9680b7840 0x7fa968152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96df23700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa968146720 0x7fa968153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:26.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 -- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa968146720 msgr2=0x7fa968153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa968146720 0x7fa968153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 -- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9680b7840 msgr2=0x7fa968152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9680b7840 0x7fa968152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 -- 172.21.15.32:0/2152233189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95800b1e0 con 0x7fa9680b2930 2022-08-01T22:56:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96d722700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 0x7fa9680ae6f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fa958002f70 tx=0x7fa958014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95800d290 con 0x7fa9680b2930 2022-08-01T22:56:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95801e400 con 0x7fa9680b2930 2022-08-01T22:56:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa968152770 con 0x7fa9680b2930 2022-08-01T22:56:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.127+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95801e760 con 0x7fa9680b2930 2022-08-01T22:56:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.131+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa968147b30 con 0x7fa9680b2930 2022-08-01T22:56:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.131+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9680574e0 con 0x7fa9680b2930 2022-08-01T22:56:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.131+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa958025030 con 0x7fa9680b2930 2022-08-01T22:56:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.131+0000 7fa9567fc700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa948053b00 0x7fa948055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.131+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa95806cb40 con 0x7fa9680b2930 2022-08-01T22:56:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.131+0000 7fa96cf21700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa948053b00 0x7fa948055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.135+0000 7fa96cf21700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa948053b00 0x7fa948055fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa96400b000 tx=0x7fa964008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.135+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa958043c40 con 0x7fa9680b2930 2022-08-01T22:56:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:26.263+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa968060330 con 0x7fa9680b2930 2022-08-01T22:56:27.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.091+0000 7fa9567fc700 1 -- 172.21.15.32:0/2152233189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa958023020 con 0x7fa9680b2930 2022-08-01T22:56:27.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa948053b00 msgr2=0x7fa948055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa948053b00 0x7fa948055fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa96400b000 tx=0x7fa964008040).stop 2022-08-01T22:56:27.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 msgr2=0x7fa9680ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 0x7fa9680ae6f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fa958002f70 tx=0x7fa958014000).stop 2022-08-01T22:56:27.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 shutdown_connections 2022-08-01T22:56:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa948053b00 0x7fa948055fc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa968146720 0x7fa968153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9680b7840 0x7fa968152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 --2- 172.21.15.32:0/2152233189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9680b2930 0x7fa9680ae6f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 >> 172.21.15.32:0/2152233189 conn(0x7fa96805f840 msgr2=0x7fa96805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:27.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 shutdown_connections 2022-08-01T22:56:27.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.095+0000 7fa96e724700 1 -- 172.21.15.32:0/2152233189 wait complete. 2022-08-01T22:56:27.110 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T22:56:27.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T22:56:27.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.227+0000 7fdfa6439700 1 Processor -- start 2022-08-01T22:56:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.227+0000 7fdfa6439700 1 -- start start 2022-08-01T22:56:27.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa6439700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00ad000 0x7fdfa01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa6439700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa0143ab0 con 0x7fdfa00b08a0 2022-08-01T22:56:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa6439700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00b0c80 con 0x7fdfa00b3260 2022-08-01T22:56:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa6439700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00ae810 con 0x7fdfa00ad000 2022-08-01T22:56:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5437700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdfa00b08a0 0x7fdfa0142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57612/0 (socket says 172.21.15.32:57612) 2022-08-01T22:56:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5437700 1 -- 172.21.15.32:0/1161652679 learned_addr learned my addr 172.21.15.32:0/1161652679 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5c38700 1 --2- 172.21.15.32:0/1161652679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00ad000 0x7fdfa01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:27.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1161652679 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064149534 0 0) 0x7fdfa0143ab0 con 0x7fdfa00b08a0 2022-08-01T22:56:27.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1161652679 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf88003260 con 0x7fdfa00b08a0 2022-08-01T22:56:27.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1161652679 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168693454 0 0) 0x7fdfa00b0c80 con 0x7fdfa00b3260 2022-08-01T22:56:27.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5c38700 1 -- 172.21.15.32:0/1161652679 >> v1:172.21.15.55:6789/0 conn(0x7fdfa00b3260 legacy=0x7fdfa013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5c38700 1 -- 172.21.15.32:0/1161652679 >> v1:172.21.15.32:6789/0 conn(0x7fdfa00b08a0 legacy=0x7fdfa0142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5c38700 1 -- 172.21.15.32:0/1161652679 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00aeab0 con 0x7fdfa00ad000 2022-08-01T22:56:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdfa5c38700 1 --2- 172.21.15.32:0/1161652679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00ad000 0x7fdfa01435c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fdf980032f0 tx=0x7fdf980091d0).ready entity=mon.2 client_cookie=4c227b04a6ec317e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1161652679 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9800b6c0 con 0x7fdfa00ad000 2022-08-01T22:56:27.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1161652679 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800b820 con 0x7fdfa00ad000 2022-08-01T22:56:27.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.231+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1161652679 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9800b990 con 0x7fdfa00ad000 2022-08-01T22:56:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 -- 172.21.15.32:0/1161652679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00ad000 msgr2=0x7fdfa01435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1161652679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00ad000 0x7fdfa01435c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fdf980032f0 tx=0x7fdf980091d0).stop 2022-08-01T22:56:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 -- 172.21.15.32:0/1161652679 shutdown_connections 2022-08-01T22:56:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1161652679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00ad000 0x7fdfa01435c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 -- 172.21.15.32:0/1161652679 >> 172.21.15.32:0/1161652679 conn(0x7fdfa00612d0 msgr2=0x7fdfa0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 -- 172.21.15.32:0/1161652679 shutdown_connections 2022-08-01T22:56:27.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 -- 172.21.15.32:0/1161652679 wait complete. 2022-08-01T22:56:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 Processor -- start 2022-08-01T22:56:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.235+0000 7fdfa6439700 1 -- start start 2022-08-01T22:56:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 0x7fdfa00649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfa00b08a0 0x7fdfa0066300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00b3260 0x7fdfa00667f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00674c0 con 0x7fdfa00ad000 2022-08-01T22:56:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0066ce0 con 0x7fdfa00b08a0 2022-08-01T22:56:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0138a40 con 0x7fdfa00b3260 2022-08-01T22:56:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 0x7fdfa00649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 0x7fdfa00649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41844/0 (socket says 172.21.15.32:41844) 2022-08-01T22:56:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 -- 172.21.15.32:0/1036933880 learned_addr learned my addr 172.21.15.32:0/1036933880 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa4c36700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfa00b08a0 0x7fdfa0066300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5c38700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00b3260 0x7fdfa00667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 -- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00b3260 msgr2=0x7fdfa00667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00b3260 0x7fdfa00667f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 -- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfa00b08a0 msgr2=0x7fdfa0066300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfa00b08a0 0x7fdfa0066300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 -- 172.21.15.32:0/1036933880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98009470 con 0x7fdfa00ad000 2022-08-01T22:56:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa5437700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 0x7fdfa00649f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fdf9c001110 tx=0x7fdf9c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9c00ce80 con 0x7fdfa00ad000 2022-08-01T22:56:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0070d0 con 0x7fdfa00ad000 2022-08-01T22:56:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa00654d0 con 0x7fdfa00ad000 2022-08-01T22:56:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9c007430 con 0x7fdfa00ad000 2022-08-01T22:56:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.239+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa0065e60 con 0x7fdfa00ad000 2022-08-01T22:56:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.243+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa0066020 con 0x7fdfa00ad000 2022-08-01T22:56:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.243+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf9c010070 con 0x7fdfa00ad000 2022-08-01T22:56:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.243+0000 7fdf8e7fc700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf80053a10 0x7fdf80055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.243+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fdf9c06b3a0 con 0x7fdfa00ad000 2022-08-01T22:56:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.243+0000 7fdfa4c36700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf80053a10 0x7fdf80055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.243+0000 7fdfa4c36700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf80053a10 0x7fdf80055ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fdf9000d000 tx=0x7fdf90007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.251+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf9c042330 con 0x7fdfa00ad000 2022-08-01T22:56:27.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:27.379+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fdfa00574e0 con 0x7fdfa00ad000 2022-08-01T22:56:28.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.095+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1036933880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v33) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdf9c045950 con 0x7fdfa00ad000 2022-08-01T22:56:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf80053a10 msgr2=0x7fdf80055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf80053a10 0x7fdf80055ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fdf9000d000 tx=0x7fdf90007a00).stop 2022-08-01T22:56:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 msgr2=0x7fdfa00649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 0x7fdfa00649f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fdf9c001110 tx=0x7fdf9c007aa0).stop 2022-08-01T22:56:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 shutdown_connections 2022-08-01T22:56:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf80053a10 0x7fdf80055ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdfa00b3260 0x7fdfa00667f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfa00b08a0 0x7fdfa0066300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 --2- 172.21.15.32:0/1036933880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfa00ad000 0x7fdfa00649f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 >> 172.21.15.32:0/1036933880 conn(0x7fdfa00612d0 msgr2=0x7fdfa0142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 shutdown_connections 2022-08-01T22:56:28.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.099+0000 7fdfa6439700 1 -- 172.21.15.32:0/1036933880 wait complete. 2022-08-01T22:56:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T22:56:28.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T22:56:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.231+0000 7fd9daa91700 1 Processor -- start 2022-08-01T22:56:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.231+0000 7fd9daa91700 1 -- start start 2022-08-01T22:56:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9d40ada20 0x7fd9d40b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4143d00 0x7fd9d40a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d405c620 con 0x7fd9d40b2730 2022-08-01T22:56:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40ae440 con 0x7fd9d40ada20 2022-08-01T22:56:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d405c850 con 0x7fd9d4143d00 2022-08-01T22:56:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9d40ada20 0x7fd9d40b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d9a8f700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd9d40b2730 0x7fd9d413fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:50300/0 (socket says 172.21.15.32:50300) 2022-08-01T22:56:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9d40ada20 0x7fd9d40b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41854/0 (socket says 172.21.15.32:41854) 2022-08-01T22:56:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d9a8f700 1 -- 172.21.15.32:0/299009296 learned_addr learned my addr 172.21.15.32:0/299009296 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9da290700 1 --2- 172.21.15.32:0/299009296 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4143d00 0x7fd9d40a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 -- 172.21.15.32:0/299009296 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4143d00 msgr2=0x7fd9d40a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 --2- 172.21.15.32:0/299009296 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4143d00 0x7fd9d40a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 -- 172.21.15.32:0/299009296 >> v1:172.21.15.55:6789/0 conn(0x7fd9d40b2730 legacy=0x7fd9d413fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 -- 172.21.15.32:0/299009296 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d41435c0 con 0x7fd9d40ada20 2022-08-01T22:56:28.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d928e700 1 --2- 172.21.15.32:0/299009296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40ada20 0x7fd9d40b1fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fd9c4002f70 tx=0x7fd9c4006df0).ready entity=mon.0 client_cookie=d308071e9bfa1345 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d8a8d700 1 -- 172.21.15.32:0/299009296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c400d570 con 0x7fd9d40ada20 2022-08-01T22:56:28.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d8a8d700 1 -- 172.21.15.32:0/299009296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4000e90 con 0x7fd9d40ada20 2022-08-01T22:56:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9d8a8d700 1 -- 172.21.15.32:0/299009296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c400d8c0 con 0x7fd9d40ada20 2022-08-01T22:56:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- 172.21.15.32:0/299009296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40ada20 msgr2=0x7fd9d40b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 --2- 172.21.15.32:0/299009296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40ada20 0x7fd9d40b1fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fd9c4002f70 tx=0x7fd9c4006df0).stop 2022-08-01T22:56:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- 172.21.15.32:0/299009296 shutdown_connections 2022-08-01T22:56:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 --2- 172.21.15.32:0/299009296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40ada20 0x7fd9d40b1fe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 --2- 172.21.15.32:0/299009296 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4143d00 0x7fd9d40a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- 172.21.15.32:0/299009296 >> 172.21.15.32:0/299009296 conn(0x7fd9d4061360 msgr2=0x7fd9d4062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- 172.21.15.32:0/299009296 shutdown_connections 2022-08-01T22:56:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.235+0000 7fd9daa91700 1 -- 172.21.15.32:0/299009296 wait complete. 2022-08-01T22:56:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 Processor -- start 2022-08-01T22:56:28.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 -- start start 2022-08-01T22:56:28.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d40ada20 0x7fd9d40a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 0x7fd9d4155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d4143d00 0x7fd9d4155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4141b70 con 0x7fd9d40b2730 2022-08-01T22:56:28.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d413ef20 con 0x7fd9d4143d00 2022-08-01T22:56:28.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d41564c0 con 0x7fd9d40ada20 2022-08-01T22:56:28.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 0x7fd9d4155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 0x7fd9d4155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41862/0 (socket says 172.21.15.32:41862) 2022-08-01T22:56:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9da290700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d4143d00 0x7fd9d4155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 -- 172.21.15.32:0/1905541358 learned_addr learned my addr 172.21.15.32:0/1905541358 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d9a8f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d40ada20 0x7fd9d40a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 -- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d40ada20 msgr2=0x7fd9d40a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d40ada20 0x7fd9d40a9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 -- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d4143d00 msgr2=0x7fd9d4155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d4143d00 0x7fd9d4155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 -- 172.21.15.32:0/1905541358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c400b180 con 0x7fd9d40b2730 2022-08-01T22:56:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9d928e700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 0x7fd9d4155ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fd9c4002f70 tx=0x7fd9c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c400dea0 con 0x7fd9d40b2730 2022-08-01T22:56:28.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4013de0 con 0x7fd9d40b2730 2022-08-01T22:56:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d413ec80 con 0x7fd9d40b2730 2022-08-01T22:56:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c401e5f0 con 0x7fd9d40b2730 2022-08-01T22:56:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.239+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d413f6a0 con 0x7fd9d40b2730 2022-08-01T22:56:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.243+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d4141f90 con 0x7fd9d40b2730 2022-08-01T22:56:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.243+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd9c4025030 con 0x7fd9d40b2730 2022-08-01T22:56:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.243+0000 7fd9c2ffd700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9b4053b00 0x7fd9b4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.243+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fd9c406ccb0 con 0x7fd9d40b2730 2022-08-01T22:56:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.243+0000 7fd9d9a8f700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9b4053b00 0x7fd9b4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.243+0000 7fd9d9a8f700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9b4053b00 0x7fd9b4055fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fd9d000d000 tx=0x7fd9d0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.247+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9c4043af0 con 0x7fd9d40b2730 2022-08-01T22:56:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.411+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd9d413f3b0 con 0x7fd9d40b2730 2022-08-01T22:56:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.423+0000 7fd9c2ffd700 1 -- 172.21.15.32:0/1905541358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 187+0+0 (secure 0 0 0) 0x7fd9c4023070 con 0x7fd9d40b2730 2022-08-01T22:56:28.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9b4053b00 msgr2=0x7fd9b4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9b4053b00 0x7fd9b4055fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fd9d000d000 tx=0x7fd9d0009040).stop 2022-08-01T22:56:28.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 msgr2=0x7fd9d4155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 0x7fd9d4155ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fd9c4002f70 tx=0x7fd9c4014000).stop 2022-08-01T22:56:28.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 shutdown_connections 2022-08-01T22:56:28.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9b4053b00 0x7fd9b4055fc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d4143d00 0x7fd9d4155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d40b2730 0x7fd9d4155ae0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 --2- 172.21.15.32:0/1905541358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d40ada20 0x7fd9d40a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 >> 172.21.15.32:0/1905541358 conn(0x7fd9d4061360 msgr2=0x7fd9d4140480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 shutdown_connections 2022-08-01T22:56:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.427+0000 7fd9daa91700 1 -- 172.21.15.32:0/1905541358 wait complete. 2022-08-01T22:56:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 5 and data pool 6 2022-08-01T22:56:28.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57ad2f4700 1 Processor -- start 2022-08-01T22:56:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57ad2f4700 1 -- start start 2022-08-01T22:56:28.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57ad2f4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f57a80609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57ad2f4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a8066af0 con 0x7f57a80b0e60 2022-08-01T22:56:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57ad2f4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a8066d90 con 0x7f57a80b6510 2022-08-01T22:56:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57ad2f4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8067030 con 0x7f57a813d9b0 2022-08-01T22:56:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57acaf3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57a80b0e60 0x7f57a8060eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57654/0 (socket says 172.21.15.32:57654) 2022-08-01T22:56:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57acaf3700 1 -- 172.21.15.32:0/776304118 learned_addr learned my addr 172.21.15.32:0/776304118 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57a77fe700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f57a80609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133974877 0 0) 0x7f57a8066af0 con 0x7f57a80b0e60 2022-08-01T22:56:28.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.535+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5794003260 con 0x7f57a80b0e60 2022-08-01T22:56:28.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461804454 0 0) 0x7f57a8066d90 con 0x7f57a80b6510 2022-08-01T22:56:28.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a8066af0 con 0x7f57a80b6510 2022-08-01T22:56:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988447652 0 0) 0x7f5794003260 con 0x7f57a80b0e60 2022-08-01T22:56:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 msgr2=0x7f57a80609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f57a80609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 >> v1:172.21.15.55:6789/0 conn(0x7f57a80b6510 legacy=0x7f57a813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80672d0 con 0x7f57a80b0e60 2022-08-01T22:56:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f57a00019e0 con 0x7f57a80b0e60 2022-08-01T22:56:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 >> v1:172.21.15.32:6789/0 conn(0x7f57a80b0e60 legacy=0x7f57a8060eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5794003910 0x7f5794003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f5794004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57940049f0 0x7f5794004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8067030 con 0x7f5794003910 2022-08-01T22:56:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579400b780 con 0x7f57940049f0 2022-08-01T22:56:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579400b940 con 0x7f57a813d9b0 2022-08-01T22:56:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5794003910 0x7f5794003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a77fe700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f5794004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57acaf3700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57940049f0 0x7f5794004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 -- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 msgr2=0x7f5794004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f5794004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 -- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57940049f0 msgr2=0x7f5794004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57940049f0 0x7f5794004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 -- 172.21.15.32:0/776304118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794003260 con 0x7f5794003910 2022-08-01T22:56:28.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a77fe700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a813d9b0 0x7f5794004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:28.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57acaf3700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57940049f0 0x7f5794004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:28.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a7fff700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5794003910 0x7f5794003d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f5798001110 tx=0x7f5798007a00).ready entity=mon.0 client_cookie=c67d0c34dd67c0c3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5798007d10 con 0x7f5794003910 2022-08-01T22:56:28.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798007030 con 0x7f5794003910 2022-08-01T22:56:28.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57a6ffd700 1 -- 172.21.15.32:0/776304118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5798007390 con 0x7f5794003910 2022-08-01T22:56:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57ad2f4700 1 -- 172.21.15.32:0/776304118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5794003910 msgr2=0x7f5794003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/776304118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5794003910 0x7f5794003d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f5798001110 tx=0x7f5798007a00).stop 2022-08-01T22:56:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57acaf3700 1 -- 172.21.15.32:0/776304118 reap_dead start 2022-08-01T22:56:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57ad2f4700 1 -- 172.21.15.32:0/776304118 shutdown_connections 2022-08-01T22:56:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57ad2f4700 1 -- 172.21.15.32:0/776304118 >> 172.21.15.32:0/776304118 conn(0x7f57a8063800 msgr2=0x7f57a8063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57ad2f4700 1 -- 172.21.15.32:0/776304118 shutdown_connections 2022-08-01T22:56:28.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.539+0000 7f57ad2f4700 1 -- 172.21.15.32:0/776304118 wait complete. 2022-08-01T22:56:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 Processor -- start 2022-08-01T22:56:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- start start 2022-08-01T22:56:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a8142250 0x7f57a8142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 0x7f57a8137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a8137b00 0x7f57a80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80ad000 con 0x7f57a80aed20 2022-08-01T22:56:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80ae4f0 con 0x7f57a8137b00 2022-08-01T22:56:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80ae790 con 0x7f57a8142250 2022-08-01T22:56:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 0x7f57a8137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 0x7f57a8137610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41894/0 (socket says 172.21.15.32:41894) 2022-08-01T22:56:28.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 -- 172.21.15.32:0/1865392905 learned_addr learned my addr 172.21.15.32:0/1865392905 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:28.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a7fff700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a8142250 0x7f57a8142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57acaf3700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a8137b00 0x7f57a80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 -- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a8142250 msgr2=0x7f57a8142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a8142250 0x7f57a8142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 -- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a8137b00 msgr2=0x7f57a80af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a8137b00 0x7f57a80af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 -- 172.21.15.32:0/1865392905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a813fcc0 con 0x7f57a80aed20 2022-08-01T22:56:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a77fe700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 0x7f57a8137610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f579c00b000 tx=0x7f579c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f579c014680 con 0x7f57a80aed20 2022-08-01T22:56:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c0147e0 con 0x7f57a80aed20 2022-08-01T22:56:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8137f80 con 0x7f57a80aed20 2022-08-01T22:56:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f579c014b40 con 0x7f57a80aed20 2022-08-01T22:56:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a80afa30 con 0x7f57a80aed20 2022-08-01T22:56:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.543+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a80b0080 con 0x7f57a80aed20 2022-08-01T22:56:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.547+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f579c014ca0 con 0x7f57a80aed20 2022-08-01T22:56:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.547+0000 7f57a57fa700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5788053950 0x7f5788055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.547+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f579c06c3e0 con 0x7f57a80aed20 2022-08-01T22:56:28.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.547+0000 7f57a7fff700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5788053950 0x7f5788055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.547+0000 7f57a7fff700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5788053950 0x7f5788055e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f579801b000 tx=0x7f5798016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.555+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f579c043140 con 0x7f57a80aed20 2022-08-01T22:56:28.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.679+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a80ae350 con 0x7f57a80aed20 2022-08-01T22:56:28.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.679+0000 7f57a57fa700 1 -- 172.21.15.32:0/1865392905 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f57a80ae350 con 0x7f57a80aed20 2022-08-01T22:56:28.687 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:28.687 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:56:28.419279+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:56:26.319594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T22:56:27.441691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:28.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5788053950 msgr2=0x7f5788055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5788053950 0x7f5788055e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f579801b000 tx=0x7f5798016000).stop 2022-08-01T22:56:28.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 msgr2=0x7f57a8137610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 0x7f57a8137610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f579c00b000 tx=0x7f579c008040).stop 2022-08-01T22:56:28.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 shutdown_connections 2022-08-01T22:56:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5788053950 0x7f5788055e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a8137b00 0x7f57a80af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a80aed20 0x7f57a8137610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 --2- 172.21.15.32:0/1865392905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f57a8142250 0x7f57a8142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 >> 172.21.15.32:0/1865392905 conn(0x7f57a8063800 msgr2=0x7f57a8060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 shutdown_connections 2022-08-01T22:56:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.683+0000 7f57ad2f4700 1 -- 172.21.15.32:0/1865392905 wait complete. 2022-08-01T22:56:28.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3aa6c6700 1 Processor -- start 2022-08-01T22:56:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3aa6c6700 1 -- start start 2022-08-01T22:56:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3aa6c6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3a40b01a0 0x7ff3a40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3aa6c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a40a7250 con 0x7ff3a40b2710 2022-08-01T22:56:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3aa6c6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a40a7480 con 0x7ff3a40afd60 2022-08-01T22:56:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3aa6c6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40a7720 con 0x7ff3a40b01a0 2022-08-01T22:56:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3a40b2710 0x7ff3a414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57672/0 (socket says 172.21.15.32:57672) 2022-08-01T22:56:28.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 -- 172.21.15.32:0/30846968 learned_addr learned my addr 172.21.15.32:0/30846968 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:28.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a8ec3700 1 --2- 172.21.15.32:0/30846968 >> v2:172.21.15.55:3300/0 conn(0x7ff3a40b01a0 0x7ff3a40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093082792 0 0) 0x7ff3a40a7250 con 0x7ff3a40b2710 2022-08-01T22:56:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff390003260 con 0x7ff3a40b2710 2022-08-01T22:56:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654354285 0 0) 0x7ff390003260 con 0x7ff3a40b2710 2022-08-01T22:56:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 >> v2:172.21.15.55:3300/0 conn(0x7ff3a40b01a0 msgr2=0x7ff3a40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 --2- 172.21.15.32:0/30846968 >> v2:172.21.15.55:3300/0 conn(0x7ff3a40b01a0 0x7ff3a40a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 >> v1:172.21.15.55:6789/0 conn(0x7ff3a40afd60 legacy=0x7ff3a4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40a79c0 con 0x7ff3a40b2710 2022-08-01T22:56:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7ff3940019e0 con 0x7ff3a40b2710 2022-08-01T22:56:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 >> v1:172.21.15.32:6789/0 conn(0x7ff3a40b2710 legacy=0x7ff3a414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3900037a0 0x7ff390003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff390004500 0x7ff390004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff390004e70 0x7ff39000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40a7720 con 0x7ff3900037a0 2022-08-01T22:56:28.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39000bda0 con 0x7ff390004500 2022-08-01T22:56:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39000bf60 con 0x7ff390004e70 2022-08-01T22:56:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3900037a0 0x7ff390003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a8ec3700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff390004500 0x7ff390004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a9ec5700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff390004e70 0x7ff39000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 -- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff390004e70 msgr2=0x7ff39000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff390004e70 0x7ff39000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 -- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff390004500 msgr2=0x7ff390004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff390004500 0x7ff390004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 -- 172.21.15.32:0/30846968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390003260 con 0x7ff3900037a0 2022-08-01T22:56:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a9ec5700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff390004e70 0x7ff39000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a8ec3700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff390004500 0x7ff390004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a9ec5700 1 -- 172.21.15.32:0/30846968 reap_dead start 2022-08-01T22:56:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3900037a0 0x7ff390003bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7ff3940016f0 tx=0x7ff3a40a7610).ready entity=mon.0 client_cookie=c20d71bdf91a7276 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff39400cba0 con 0x7ff3900037a0 2022-08-01T22:56:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39400cd00 con 0x7ff3900037a0 2022-08-01T22:56:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.807+0000 7ff3a3fff700 1 -- 172.21.15.32:0/30846968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff394005b80 con 0x7ff3900037a0 2022-08-01T22:56:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/30846968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3900037a0 msgr2=0x7ff390003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3900037a0 0x7ff390003bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7ff3940016f0 tx=0x7ff3a40a7610).stop 2022-08-01T22:56:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/30846968 shutdown_connections 2022-08-01T22:56:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/30846968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3900037a0 0x7ff390003bc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/30846968 >> 172.21.15.32:0/30846968 conn(0x7ff3a4060310 msgr2=0x7ff3a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/30846968 shutdown_connections 2022-08-01T22:56:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/30846968 wait complete. 2022-08-01T22:56:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 Processor -- start 2022-08-01T22:56:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- start start 2022-08-01T22:56:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3900037a0 0x7ff3a4064650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 0x7ff3a4065b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a4066060 0x7ff3a411de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a4066510 con 0x7ff3a4064e30 2022-08-01T22:56:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a4156ac0 con 0x7ff3a4066060 2022-08-01T22:56:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.811+0000 7ff3aa6c6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a4156d60 con 0x7ff3900037a0 2022-08-01T22:56:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 0x7ff3a4065b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 0x7ff3a4065b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41914/0 (socket says 172.21.15.32:41914) 2022-08-01T22:56:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 -- 172.21.15.32:0/3518279710 learned_addr learned my addr 172.21.15.32:0/3518279710 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3900037a0 0x7ff3a4064650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a9ec5700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a4066060 0x7ff3a411de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 -- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3900037a0 msgr2=0x7ff3a4064650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3900037a0 0x7ff3a4064650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 -- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a4066060 msgr2=0x7ff3a411de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a4066060 0x7ff3a411de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 -- 172.21.15.32:0/3518279710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4157000 con 0x7ff3a4064e30 2022-08-01T22:56:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a8ec3700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 0x7ff3a4065b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7ff39800b000 tx=0x7ff398008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3980135e0 con 0x7ff3a4064e30 2022-08-01T22:56:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398013740 con 0x7ff3a4064e30 2022-08-01T22:56:28.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a41572a0 con 0x7ff3a4064e30 2022-08-01T22:56:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff398013aa0 con 0x7ff3a4064e30 2022-08-01T22:56:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a4157740 con 0x7ff3a4064e30 2022-08-01T22:56:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.815+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a4065430 con 0x7ff3a4064e30 2022-08-01T22:56:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.819+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff398013c00 con 0x7ff3a4064e30 2022-08-01T22:56:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.819+0000 7ff3a27fc700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff384053b00 0x7ff384055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.819+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff39806c740 con 0x7ff3a4064e30 2022-08-01T22:56:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.819+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff384053b00 0x7ff384055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.819+0000 7ff3a96c4700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff384053b00 0x7ff384055fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff394019000 tx=0x7ff394013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:28.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.823+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3980433f0 con 0x7ff3a4064e30 2022-08-01T22:56:28.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.951+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3a4157430 con 0x7ff3a4064e30 2022-08-01T22:56:28.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.951+0000 7ff3a27fc700 1 -- 172.21.15.32:0/3518279710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16284 (secure 0 0 0) 0x7ff398046a10 con 0x7ff3a4064e30 2022-08-01T22:56:28.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:28.961 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:56:28.419279+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:56:26.319594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T22:56:27.441691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:28.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff384053b00 msgr2=0x7ff384055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff384053b00 0x7ff384055fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff394019000 tx=0x7ff394013000).stop 2022-08-01T22:56:28.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 msgr2=0x7ff3a4065b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:28.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 0x7ff3a4065b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7ff39800b000 tx=0x7ff398008040).stop 2022-08-01T22:56:28.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 shutdown_connections 2022-08-01T22:56:28.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff384053b00 0x7ff384055fc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a4066060 0x7ff3a411de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a4064e30 0x7ff3a4065b70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 --2- 172.21.15.32:0/3518279710 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3900037a0 0x7ff3a4064650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:28.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 >> 172.21.15.32:0/3518279710 conn(0x7ff3a4060310 msgr2=0x7ff3a405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:28.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 shutdown_connections 2022-08-01T22:56:28.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:28.955+0000 7ff3aa6c6700 1 -- 172.21.15.32:0/3518279710 wait complete. 2022-08-01T22:56:28.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T22:56:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 Processor -- start 2022-08-01T22:56:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 -- start start 2022-08-01T22:56:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20740b2710 0x7f207414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740a8c50 0x7f207414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740a7250 con 0x7f20740b01a0 2022-08-01T22:56:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20740a7480 con 0x7f20740b2710 2022-08-01T22:56:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207b370700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740a7720 con 0x7f20740a8c50 2022-08-01T22:56:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207a36e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20740b2710 0x7f207414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f2079b6d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f20740b01a0 0x7f20740a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57692/0 (socket says 172.21.15.32:57692) 2022-08-01T22:56:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f2079b6d700 1 -- 172.21.15.32:0/4204186238 learned_addr learned my addr 172.21.15.32:0/4204186238 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207ab6f700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740a8c50 0x7f207414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207936c700 1 -- 172.21.15.32:0/4204186238 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902270322 0 0) 0x7f20740a7250 con 0x7f20740b01a0 2022-08-01T22:56:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207936c700 1 -- 172.21.15.32:0/4204186238 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205c003260 con 0x7f20740b01a0 2022-08-01T22:56:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207a36e700 1 -- 172.21.15.32:0/4204186238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740a8c50 msgr2=0x7f207414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207a36e700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740a8c50 0x7f207414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207a36e700 1 -- 172.21.15.32:0/4204186238 >> v1:172.21.15.32:6789/0 conn(0x7f20740b01a0 legacy=0x7f20740a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207a36e700 1 -- 172.21.15.32:0/4204186238 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740a79c0 con 0x7f20740b2710 2022-08-01T22:56:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207ab6f700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740a8c50 0x7f207414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.083+0000 7f207a36e700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f207414afd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f2064002f70 tx=0x7f2064006df0).ready entity=mon.0 client_cookie=189d080778808391 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207936c700 1 -- 172.21.15.32:0/4204186238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206400d4a0 con 0x7f20740b2710 2022-08-01T22:56:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207936c700 1 -- 172.21.15.32:0/4204186238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064000e90 con 0x7f20740b2710 2022-08-01T22:56:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207936c700 1 -- 172.21.15.32:0/4204186238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206400d7f0 con 0x7f20740b2710 2022-08-01T22:56:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- 172.21.15.32:0/4204186238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 msgr2=0x7f207414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f207414afd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f2064002f70 tx=0x7f2064006df0).stop 2022-08-01T22:56:29.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- 172.21.15.32:0/4204186238 shutdown_connections 2022-08-01T22:56:29.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740a8c50 0x7f207414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 --2- 172.21.15.32:0/4204186238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f207414afd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- 172.21.15.32:0/4204186238 >> 172.21.15.32:0/4204186238 conn(0x7f2074060310 msgr2=0x7f20740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- 172.21.15.32:0/4204186238 shutdown_connections 2022-08-01T22:56:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- 172.21.15.32:0/4204186238 wait complete. 2022-08-01T22:56:29.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 Processor -- start 2022-08-01T22:56:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- start start 2022-08-01T22:56:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20740a8c50 0x7f2074142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740b01a0 0x7f20740ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f20740abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740b5180 con 0x7f20740b2710 2022-08-01T22:56:29.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074144060 con 0x7f20740a8c50 2022-08-01T22:56:29.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.087+0000 7f207b370700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740b4c40 con 0x7f20740b01a0 2022-08-01T22:56:29.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f20740abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f20740abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41928/0 (socket says 172.21.15.32:41928) 2022-08-01T22:56:29.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 -- 172.21.15.32:0/1283840487 learned_addr learned my addr 172.21.15.32:0/1283840487 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:29.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f2079b6d700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740b01a0 0x7f20740ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207a36e700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20740a8c50 0x7f2074142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 -- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740b01a0 msgr2=0x7f20740ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740b01a0 0x7f20740ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 -- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20740a8c50 msgr2=0x7f2074142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20740a8c50 0x7f2074142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 -- 172.21.15.32:0/1283840487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c00b180 con 0x7f20740b2710 2022-08-01T22:56:29.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207ab6f700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f20740abe60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f206c00c000 tx=0x7f206c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206c010780 con 0x7f20740b2710 2022-08-01T22:56:29.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c0108e0 con 0x7f20740b2710 2022-08-01T22:56:29.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206400b180 con 0x7f20740b2710 2022-08-01T22:56:29.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206c010c40 con 0x7f20740b2710 2022-08-01T22:56:29.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20740ab4d0 con 0x7f20740b2710 2022-08-01T22:56:29.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20740b49e0 con 0x7f20740b2710 2022-08-01T22:56:29.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.091+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f206c01c460 con 0x7f20740b2710 2022-08-01T22:56:29.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.095+0000 7f20637fe700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053b00 0x7f2054055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.095+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f206c06c610 con 0x7f20740b2710 2022-08-01T22:56:29.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.095+0000 7f207a36e700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053b00 0x7f2054055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.095+0000 7f207a36e700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053b00 0x7f2054055fc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f206401a000 tx=0x7f2064014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.099+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f206c0431e0 con 0x7f20740b2710 2022-08-01T22:56:29.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.263+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f20740574e0 con 0x7f20740b2710 2022-08-01T22:56:29.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.467+0000 7f20637fe700 1 -- 172.21.15.32:0/1283840487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f206c046800 con 0x7f20740b2710 2022-08-01T22:56:29.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053b00 msgr2=0x7f2054055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053b00 0x7f2054055fc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f206401a000 tx=0x7f2064014000).stop 2022-08-01T22:56:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 msgr2=0x7f20740abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f20740abe60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f206c00c000 tx=0x7f206c006df0).stop 2022-08-01T22:56:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 shutdown_connections 2022-08-01T22:56:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053b00 0x7f2054055fc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20740b2710 0x7f20740abe60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20740b01a0 0x7f20740ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 --2- 172.21.15.32:0/1283840487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20740a8c50 0x7f2074142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 >> 172.21.15.32:0/1283840487 conn(0x7f2074060310 msgr2=0x7f2074148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 shutdown_connections 2022-08-01T22:56:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.471+0000 7f207b370700 1 -- 172.21.15.32:0/1283840487 wait complete. 2022-08-01T22:56:29.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 Processor -- start 2022-08-01T22:56:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- start start 2022-08-01T22:56:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a7414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a740a8c50 0x7f3a7414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a740a7250 con 0x7f3a740b01a0 2022-08-01T22:56:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a740a7480 con 0x7f3a740a8c50 2022-08-01T22:56:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740a7720 con 0x7f3a740b2710 2022-08-01T22:56:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a79d70700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a740b01a0 0x7f3a740a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57708/0 (socket says 172.21.15.32:57708) 2022-08-01T22:56:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a79d70700 1 -- 172.21.15.32:0/2349583625 learned_addr learned my addr 172.21.15.32:0/2349583625 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7ad72700 1 --2- 172.21.15.32:0/2349583625 >> v2:172.21.15.55:3300/0 conn(0x7f3a740a8c50 0x7f3a7414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7a571700 1 --2- 172.21.15.32:0/2349583625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a7414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7956f700 1 -- 172.21.15.32:0/2349583625 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152247108 0 0) 0x7f3a740a7250 con 0x7f3a740b01a0 2022-08-01T22:56:29.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7956f700 1 -- 172.21.15.32:0/2349583625 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a5c003260 con 0x7f3a740b01a0 2022-08-01T22:56:29.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7a571700 1 -- 172.21.15.32:0/2349583625 >> v2:172.21.15.55:3300/0 conn(0x7f3a740a8c50 msgr2=0x7f3a7414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7a571700 1 --2- 172.21.15.32:0/2349583625 >> v2:172.21.15.55:3300/0 conn(0x7f3a740a8c50 0x7f3a7414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7a571700 1 -- 172.21.15.32:0/2349583625 >> v1:172.21.15.32:6789/0 conn(0x7f3a740b01a0 legacy=0x7f3a740a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7a571700 1 -- 172.21.15.32:0/2349583625 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740a79c0 con 0x7f3a740b2710 2022-08-01T22:56:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7ad72700 1 --2- 172.21.15.32:0/2349583625 >> v2:172.21.15.55:3300/0 conn(0x7f3a740a8c50 0x7f3a7414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7a571700 1 --2- 172.21.15.32:0/2349583625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a7414afd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3a70002f70 tx=0x7f3a70006df0).ready entity=mon.2 client_cookie=901f6a81bc976d4f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7956f700 1 -- 172.21.15.32:0/2349583625 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a7000d4a0 con 0x7f3a740b2710 2022-08-01T22:56:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7956f700 1 -- 172.21.15.32:0/2349583625 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70000e90 con 0x7f3a740b2710 2022-08-01T22:56:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7956f700 1 -- 172.21.15.32:0/2349583625 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a7000d7f0 con 0x7f3a740b2710 2022-08-01T22:56:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- 172.21.15.32:0/2349583625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 msgr2=0x7f3a7414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 --2- 172.21.15.32:0/2349583625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a7414afd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3a70002f70 tx=0x7f3a70006df0).stop 2022-08-01T22:56:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- 172.21.15.32:0/2349583625 shutdown_connections 2022-08-01T22:56:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 --2- 172.21.15.32:0/2349583625 >> v2:172.21.15.55:3300/0 conn(0x7f3a740a8c50 0x7f3a7414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 --2- 172.21.15.32:0/2349583625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a7414afd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- 172.21.15.32:0/2349583625 >> 172.21.15.32:0/2349583625 conn(0x7f3a74060310 msgr2=0x7f3a740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- 172.21.15.32:0/2349583625 shutdown_connections 2022-08-01T22:56:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.599+0000 7f3a7b573700 1 -- 172.21.15.32:0/2349583625 wait complete. 2022-08-01T22:56:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 Processor -- start 2022-08-01T22:56:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 -- start start 2022-08-01T22:56:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 0x7f3a740aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a740b01a0 0x7f3a74141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a74141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7414d410 con 0x7f3a740a8c50 2022-08-01T22:56:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7414ceb0 con 0x7f3a740b01a0 2022-08-01T22:56:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740b47b0 con 0x7f3a740b2710 2022-08-01T22:56:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 0x7f3a740aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 0x7f3a740aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41938/0 (socket says 172.21.15.32:41938) 2022-08-01T22:56:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 -- 172.21.15.32:0/1461664531 learned_addr learned my addr 172.21.15.32:0/1461664531 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7ad72700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a74141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a79d70700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a740b01a0 0x7f3a74141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 -- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 msgr2=0x7f3a74141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a74141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 -- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a740b01a0 msgr2=0x7f3a74141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a740b01a0 0x7f3a74141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 -- 172.21.15.32:0/1461664531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009470 con 0x7f3a740a8c50 2022-08-01T22:56:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7a571700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 0x7f3a740aef50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f3a70002f70 tx=0x7f3a70014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a70016070 con 0x7f3a740a8c50 2022-08-01T22:56:29.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a700138d0 con 0x7f3a740a8c50 2022-08-01T22:56:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7000b180 con 0x7f3a740a8c50 2022-08-01T22:56:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.603+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a70013c30 con 0x7f3a740a8c50 2022-08-01T22:56:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.607+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7414d170 con 0x7f3a740a8c50 2022-08-01T22:56:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.607+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a741420d0 con 0x7f3a740a8c50 2022-08-01T22:56:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.607+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a7000f030 con 0x7f3a740a8c50 2022-08-01T22:56:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.607+0000 7f3a637fe700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a54053a70 0x7f3a54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.607+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3a7006ced0 con 0x7f3a740a8c50 2022-08-01T22:56:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.607+0000 7f3a79d70700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a54053a70 0x7f3a54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.611+0000 7f3a79d70700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a54053a70 0x7f3a54055f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3a6400d000 tx=0x7f3a64009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.615+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a741420d0 con 0x7f3a740a8c50 2022-08-01T22:56:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a741420d0 con 0x7f3a740a8c50 2022-08-01T22:56:29.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a637fe700 1 -- 172.21.15.32:0/1461664531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f3a741420d0 con 0x7f3a740a8c50 2022-08-01T22:56:29.781 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:29.782 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:56:29.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a54053a70 msgr2=0x7f3a54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a54053a70 0x7f3a54055f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3a6400d000 tx=0x7f3a64009040).stop 2022-08-01T22:56:29.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 msgr2=0x7f3a740aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 0x7f3a740aef50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f3a70002f70 tx=0x7f3a70014000).stop 2022-08-01T22:56:29.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 shutdown_connections 2022-08-01T22:56:29.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a54053a70 0x7f3a54055f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a740b2710 0x7f3a74141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a740b01a0 0x7f3a74141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 --2- 172.21.15.32:0/1461664531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a740a8c50 0x7f3a740aef50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 >> 172.21.15.32:0/1461664531 conn(0x7f3a74060310 msgr2=0x7f3a74146030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:29.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 shutdown_connections 2022-08-01T22:56:29.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.775+0000 7f3a7b573700 1 -- 172.21.15.32:0/1461664531 wait complete. 2022-08-01T22:56:29.786 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:56:29.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 Processor -- start 2022-08-01T22:56:29.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 -- start start 2022-08-01T22:56:29.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70680614b0 0x7f7068061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70681405d0 0x7f7068140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680aee80 con 0x7f70680ad750 2022-08-01T22:56:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70680b70b0 con 0x7f70680614b0 2022-08-01T22:56:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706d5d0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70680af0b0 con 0x7f70681405d0 2022-08-01T22:56:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7067fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70680614b0 0x7f7068061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f70677fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f70680ad750 0x7f70680b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57720/0 (socket says 172.21.15.32:57720) 2022-08-01T22:56:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f70677fe700 1 -- 172.21.15.32:0/2955012295 learned_addr learned my addr 172.21.15.32:0/2955012295 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706cdcf700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70681405d0 0x7f7068140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7067fff700 1 -- 172.21.15.32:0/2955012295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70681405d0 msgr2=0x7f7068140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7067fff700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70681405d0 0x7f7068140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7067fff700 1 -- 172.21.15.32:0/2955012295 >> v1:172.21.15.32:6789/0 conn(0x7f70680ad750 legacy=0x7f70680b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7067fff700 1 -- 172.21.15.32:0/2955012295 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680af350 con 0x7f70680614b0 2022-08-01T22:56:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f706cdcf700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70681405d0 0x7f7068140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:29.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7067fff700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680614b0 0x7f7068061890 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f70680aed30 tx=0x7f7058006df0).ready entity=mon.0 client_cookie=a1bf027aff7e5ba8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7066ffd700 1 -- 172.21.15.32:0/2955012295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f705800d570 con 0x7f70680614b0 2022-08-01T22:56:29.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7066ffd700 1 -- 172.21.15.32:0/2955012295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7058000e90 con 0x7f70680614b0 2022-08-01T22:56:29.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.895+0000 7f7066ffd700 1 -- 172.21.15.32:0/2955012295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f705800d8c0 con 0x7f70680614b0 2022-08-01T22:56:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/2955012295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680614b0 msgr2=0x7f7068061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680614b0 0x7f7068061890 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f70680aed30 tx=0x7f7058006df0).stop 2022-08-01T22:56:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/2955012295 shutdown_connections 2022-08-01T22:56:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70681405d0 0x7f7068140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 --2- 172.21.15.32:0/2955012295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680614b0 0x7f7068061890 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/2955012295 >> 172.21.15.32:0/2955012295 conn(0x7f7068066f70 msgr2=0x7f7068067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/2955012295 shutdown_connections 2022-08-01T22:56:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/2955012295 wait complete. 2022-08-01T22:56:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 Processor -- start 2022-08-01T22:56:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- start start 2022-08-01T22:56:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680614b0 0x7f70680aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 0x7f70680accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70681405d0 0x7f7068152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068141c90 con 0x7f70680ad750 2022-08-01T22:56:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70681412a0 con 0x7f70681405d0 2022-08-01T22:56:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7068142c30 con 0x7f70680614b0 2022-08-01T22:56:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 0x7f70680accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 0x7f70680accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41950/0 (socket says 172.21.15.32:41950) 2022-08-01T22:56:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 -- 172.21.15.32:0/1546462343 learned_addr learned my addr 172.21.15.32:0/1546462343 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f7067fff700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680614b0 0x7f70680aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 -- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680614b0 msgr2=0x7f70680aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706cdcf700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70681405d0 0x7f7068152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680614b0 0x7f70680aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 -- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70681405d0 msgr2=0x7f7068152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70681405d0 0x7f7068152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 -- 172.21.15.32:0/1546462343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705800b180 con 0x7f70680ad750 2022-08-01T22:56:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706cdcf700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70681405d0 0x7f7068152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f7067fff700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680614b0 0x7f70680aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:56:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70677fe700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 0x7f70680accf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f70680af970 tx=0x7f7050007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f705000ce30 con 0x7f70680ad750 2022-08-01T22:56:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7068141000 con 0x7f70680ad750 2022-08-01T22:56:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70500072e0 con 0x7f70680ad750 2022-08-01T22:56:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7050005800 con 0x7f70680ad750 2022-08-01T22:56:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.899+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70680ad1e0 con 0x7f70680ad750 2022-08-01T22:56:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.903+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7068142a80 con 0x7f70680ad750 2022-08-01T22:56:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.903+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7050010070 con 0x7f70680ad750 2022-08-01T22:56:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.903+0000 7f70657fa700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7040053950 0x7f7040055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.903+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f705006b6b0 con 0x7f70680ad750 2022-08-01T22:56:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.903+0000 7f7067fff700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7040053950 0x7f7040055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.903+0000 7f7067fff700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7040053950 0x7f7040055e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f7058017000 tx=0x7f7058015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:29.907+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7050042360 con 0x7f70680ad750 2022-08-01T22:56:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.035+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70680af5f0 con 0x7f70680ad750 2022-08-01T22:56:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.035+0000 7f70657fa700 1 -- 172.21.15.32:0/1546462343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f7050045980 con 0x7f70680ad750 2022-08-01T22:56:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:30.043 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:56:29.422501+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:56:26.319594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T22:56:27.441691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7040053950 msgr2=0x7f7040055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7040053950 0x7f7040055e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f7058017000 tx=0x7f7058015000).stop 2022-08-01T22:56:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 msgr2=0x7f70680accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 0x7f70680accf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f70680af970 tx=0x7f7050007ba0).stop 2022-08-01T22:56:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 shutdown_connections 2022-08-01T22:56:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7040053950 0x7f7040055e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70681405d0 0x7f7068152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680ad750 0x7f70680accf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 --2- 172.21.15.32:0/1546462343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680614b0 0x7f70680aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 >> 172.21.15.32:0/1546462343 conn(0x7f7068066f70 msgr2=0x7f70680b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 shutdown_connections 2022-08-01T22:56:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.039+0000 7f706d5d0700 1 -- 172.21.15.32:0/1546462343 wait complete. 2022-08-01T22:56:30.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T22:56:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 Processor -- start 2022-08-01T22:56:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 -- start start 2022-08-01T22:56:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8dc80a9320 0x7f8dc81466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8dc80b0560 0x7f8dc8140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc80b12d0 con 0x7f8dc80b0d10 2022-08-01T22:56:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc80a9700 con 0x7f8dc80a9320 2022-08-01T22:56:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc7fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc80b1500 con 0x7f8dc80b0560 2022-08-01T22:56:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8dc80a9320 0x7f8dc81466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8dc80a9320 0x7f8dc81466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41958/0 (socket says 172.21.15.32:41958) 2022-08-01T22:56:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.155+0000 7f8dc6ffd700 1 -- 172.21.15.32:0/340418345 learned_addr learned my addr 172.21.15.32:0/340418345 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc77fe700 1 --2- 172.21.15.32:0/340418345 >> v2:172.21.15.55:3300/0 conn(0x7f8dc80b0560 0x7f8dc8140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc6ffd700 1 -- 172.21.15.32:0/340418345 >> v2:172.21.15.55:3300/0 conn(0x7f8dc80b0560 msgr2=0x7f8dc8140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc6ffd700 1 --2- 172.21.15.32:0/340418345 >> v2:172.21.15.55:3300/0 conn(0x7f8dc80b0560 0x7f8dc8140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc6ffd700 1 -- 172.21.15.32:0/340418345 >> v1:172.21.15.55:6789/0 conn(0x7f8dc80b0d10 legacy=0x7f8dc8146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc6ffd700 1 -- 172.21.15.32:0/340418345 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80b17a0 con 0x7f8dc80a9320 2022-08-01T22:56:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc77fe700 1 --2- 172.21.15.32:0/340418345 >> v2:172.21.15.55:3300/0 conn(0x7f8dc80b0560 0x7f8dc8140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc6ffd700 1 --2- 172.21.15.32:0/340418345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80a9320 0x7f8dc81466e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8dc80b6bd0 tx=0x7f8db8006eb0).ready entity=mon.0 client_cookie=2ab4be14d584602d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc5ffb700 1 -- 172.21.15.32:0/340418345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8db800d5c0 con 0x7f8dc80a9320 2022-08-01T22:56:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc5ffb700 1 -- 172.21.15.32:0/340418345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8000e90 con 0x7f8dc80a9320 2022-08-01T22:56:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc5ffb700 1 -- 172.21.15.32:0/340418345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8db800d910 con 0x7f8dc80a9320 2022-08-01T22:56:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 -- 172.21.15.32:0/340418345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80a9320 msgr2=0x7f8dc81466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/340418345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80a9320 0x7f8dc81466e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8dc80b6bd0 tx=0x7f8db8006eb0).stop 2022-08-01T22:56:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 -- 172.21.15.32:0/340418345 shutdown_connections 2022-08-01T22:56:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/340418345 >> v2:172.21.15.55:3300/0 conn(0x7f8dc80b0560 0x7f8dc8140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/340418345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80a9320 0x7f8dc81466e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 -- 172.21.15.32:0/340418345 >> 172.21.15.32:0/340418345 conn(0x7f8dc80612d0 msgr2=0x7f8dc8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 -- 172.21.15.32:0/340418345 shutdown_connections 2022-08-01T22:56:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.159+0000 7f8dc7fff700 1 -- 172.21.15.32:0/340418345 wait complete. 2022-08-01T22:56:30.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 Processor -- start 2022-08-01T22:56:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 -- start start 2022-08-01T22:56:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8dc80a9320 0x7f8dc813f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 0x7f8dc80aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8dc80b0d10 0x7f8dc813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80aa9c0 con 0x7f8dc80b0560 2022-08-01T22:56:30.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80b39d0 con 0x7f8dc80a9320 2022-08-01T22:56:30.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc7fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80a99d0 con 0x7f8dc80b0d10 2022-08-01T22:56:30.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 0x7f8dc80aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 0x7f8dc80aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41966/0 (socket says 172.21.15.32:41966) 2022-08-01T22:56:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 -- 172.21.15.32:0/3171400360 learned_addr learned my addr 172.21.15.32:0/3171400360 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc6ffd700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8dc80a9320 0x7f8dc813f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc77fe700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8dc80b0d10 0x7f8dc813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 -- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8dc80b0d10 msgr2=0x7f8dc813add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8dc80b0d10 0x7f8dc813add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 -- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8dc80a9320 msgr2=0x7f8dc813f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8dc80a9320 0x7f8dc813f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.163+0000 7f8dc67fc700 1 -- 172.21.15.32:0/3171400360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db800b1e0 con 0x7f8dc80b0560 2022-08-01T22:56:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8dc67fc700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 0x7f8dc80aae30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f8dbc000ed0 tx=0x7f8dbc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dbc012400 con 0x7f8dc80b0560 2022-08-01T22:56:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc012560 con 0x7f8dc80b0560 2022-08-01T22:56:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc80b5a20 con 0x7f8dc80b0560 2022-08-01T22:56:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dbc0128c0 con 0x7f8dc80b0560 2022-08-01T22:56:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc80ab320 con 0x7f8dc80b0560 2022-08-01T22:56:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc80aabc0 con 0x7f8dc80b0560 2022-08-01T22:56:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8dbc019030 con 0x7f8dc80b0560 2022-08-01T22:56:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8db7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8da4053a70 0x7f8da4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8dbc06b430 con 0x7f8dc80b0560 2022-08-01T22:56:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8dc6ffd700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8da4053a70 0x7f8da4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.167+0000 7f8dc6ffd700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8da4053a70 0x7f8da4055f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f8db801c000 tx=0x7f8db8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.175+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8dbc0421f0 con 0x7f8dc80b0560 2022-08-01T22:56:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.339+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8dc80aa6c0 con 0x7f8dc80b0560 2022-08-01T22:56:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.347+0000 7f8db7fff700 1 -- 172.21.15.32:0/3171400360 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8dbc045810 con 0x7f8dc80b0560 2022-08-01T22:56:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8da4053a70 msgr2=0x7f8da4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8da4053a70 0x7f8da4055f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f8db801c000 tx=0x7f8db8015000).stop 2022-08-01T22:56:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 msgr2=0x7f8dc80aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 0x7f8dc80aae30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f8dbc000ed0 tx=0x7f8dbc007c50).stop 2022-08-01T22:56:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 shutdown_connections 2022-08-01T22:56:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8da4053a70 0x7f8da4055f30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8dc80b0d10 0x7f8dc813add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dc80b0560 0x7f8dc80aae30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 --2- 172.21.15.32:0/3171400360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8dc80a9320 0x7f8dc813f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 >> 172.21.15.32:0/3171400360 conn(0x7f8dc80612d0 msgr2=0x7f8dc8143560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 shutdown_connections 2022-08-01T22:56:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.351+0000 7f8dc7fff700 1 -- 172.21.15.32:0/3171400360 wait complete. 2022-08-01T22:56:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T22:56:30.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T22:56:30.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 Processor -- start 2022-08-01T22:56:30.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 -- start start 2022-08-01T22:56:30.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb07c0b3190 0x7fb07c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb07c0666b0 0x7fb07c066b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c062190 con 0x7fb07c0b3190 2022-08-01T22:56:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c0654c0 con 0x7fb07c0666b0 2022-08-01T22:56:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.475+0000 7fb080ad1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0b3570 con 0x7fb07c0b2730 2022-08-01T22:56:30.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb07c0b3190 0x7fb07c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb07c0b3190 0x7fb07c060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41972/0 (socket says 172.21.15.32:41972) 2022-08-01T22:56:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1743984202 learned_addr learned my addr 172.21.15.32:0/1743984202 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07affd700 1 --2- 172.21.15.32:0/1743984202 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07bfff700 1 --2- 172.21.15.32:0/1743984202 >> v2:172.21.15.55:3300/0 conn(0x7fb07c0666b0 0x7fb07c066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1743984202 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 msgr2=0x7fb07c061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- 172.21.15.32:0/1743984202 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c061190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1743984202 >> v2:172.21.15.55:3300/0 conn(0x7fb07c0666b0 msgr2=0x7fb07c066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:30.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- 172.21.15.32:0/1743984202 >> v2:172.21.15.55:3300/0 conn(0x7fb07c0666b0 0x7fb07c066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1743984202 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c05fbf0 con 0x7fb07c0b3190 2022-08-01T22:56:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07bfff700 1 --2- 172.21.15.32:0/1743984202 >> v2:172.21.15.55:3300/0 conn(0x7fb07c0666b0 0x7fb07c066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- 172.21.15.32:0/1743984202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0b3190 0x7fb07c060ca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fb064002f70 tx=0x7fb064006eb0).ready entity=mon.0 client_cookie=c703c7d7f65ea911 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07a7fc700 1 -- 172.21.15.32:0/1743984202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06400d570 con 0x7fb07c0b3190 2022-08-01T22:56:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07a7fc700 1 -- 172.21.15.32:0/1743984202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb064000e90 con 0x7fb07c0b3190 2022-08-01T22:56:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07a7fc700 1 -- 172.21.15.32:0/1743984202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06400d900 con 0x7fb07c0b3190 2022-08-01T22:56:30.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- 172.21.15.32:0/1743984202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0b3190 msgr2=0x7fb07c060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1743984202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0b3190 0x7fb07c060ca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fb064002f70 tx=0x7fb064006eb0).stop 2022-08-01T22:56:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- 172.21.15.32:0/1743984202 shutdown_connections 2022-08-01T22:56:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1743984202 >> v2:172.21.15.55:3300/0 conn(0x7fb07c0666b0 0x7fb07c066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1743984202 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1743984202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0b3190 0x7fb07c060ca0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- 172.21.15.32:0/1743984202 >> 172.21.15.32:0/1743984202 conn(0x7fb07c063870 msgr2=0x7fb07c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- 172.21.15.32:0/1743984202 shutdown_connections 2022-08-01T22:56:30.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- 172.21.15.32:0/1743984202 wait complete. 2022-08-01T22:56:30.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 Processor -- start 2022-08-01T22:56:30.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- start start 2022-08-01T22:56:30.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 0x7fb07c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c0a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb07c0b3190 0x7fb07c0a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c152de0 con 0x7fb07c0666b0 2022-08-01T22:56:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0acf30 con 0x7fb07c0b3190 2022-08-01T22:56:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb080ad1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0ad1d0 con 0x7fb07c0b2730 2022-08-01T22:56:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 0x7fb07c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 0x7fb07c0a38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41978/0 (socket says 172.21.15.32:41978) 2022-08-01T22:56:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1167370757 learned_addr learned my addr 172.21.15.32:0/1167370757 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07bfff700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb07c0b3190 0x7fb07c0a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07affd700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c0a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 msgr2=0x7fb07c0a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.479+0000 7fb07b7fe700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c0a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb07c0b3190 msgr2=0x7fb07c0a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb07b7fe700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb07c0b3190 0x7fb07c0a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb07b7fe700 1 -- 172.21.15.32:0/1167370757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06400b1e0 con 0x7fb07c0666b0 2022-08-01T22:56:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb07b7fe700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 0x7fb07c0a38d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fb064002f70 tx=0x7fb064015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb064006be0 con 0x7fb07c0666b0 2022-08-01T22:56:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0640068b0 con 0x7fb07c0666b0 2022-08-01T22:56:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c152b00 con 0x7fb07c0666b0 2022-08-01T22:56:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06401e460 con 0x7fb07c0666b0 2022-08-01T22:56:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c0a3dc0 con 0x7fb07c0666b0 2022-08-01T22:56:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c152fe0 con 0x7fb07c0666b0 2022-08-01T22:56:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb064025030 con 0x7fb07c0666b0 2022-08-01T22:56:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb078ff9700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb060053b30 0x7fb060055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb06406d040 con 0x7fb07c0666b0 2022-08-01T22:56:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb07affd700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb060053b30 0x7fb060055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.483+0000 7fb07affd700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb060053b30 0x7fb060055ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fb06c00b000 tx=0x7fb06c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.491+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb064043e00 con 0x7fb07c0666b0 2022-08-01T22:56:30.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.659+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb07c0ad390 con 0x7fb07c0666b0 2022-08-01T22:56:30.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.667+0000 7fb078ff9700 1 -- 172.21.15.32:0/1167370757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb064023070 con 0x7fb07c0666b0 2022-08-01T22:56:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb060053b30 msgr2=0x7fb060055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb060053b30 0x7fb060055ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fb06c00b000 tx=0x7fb06c008040).stop 2022-08-01T22:56:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 msgr2=0x7fb07c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 0x7fb07c0a38d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fb064002f70 tx=0x7fb064015000).stop 2022-08-01T22:56:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 shutdown_connections 2022-08-01T22:56:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb060053b30 0x7fb060055ff0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb07c0b3190 0x7fb07c0a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb07c0b2730 0x7fb07c0a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 --2- 172.21.15.32:0/1167370757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb07c0666b0 0x7fb07c0a38d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 >> 172.21.15.32:0/1167370757 conn(0x7fb07c063870 msgr2=0x7fb07c0b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 shutdown_connections 2022-08-01T22:56:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.671+0000 7fb080ad1700 1 -- 172.21.15.32:0/1167370757 wait complete. 2022-08-01T22:56:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T22:56:30.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 Processor -- start 2022-08-01T22:56:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 -- start start 2022-08-01T22:56:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20700b01a0 0x7f20700a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20700afd60 0x7f20700a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20700a7250 con 0x7f20700b2710 2022-08-01T22:56:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700a7480 con 0x7f20700afd60 2022-08-01T22:56:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700a7720 con 0x7f20700b01a0 2022-08-01T22:56:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20700afd60 0x7f20700a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20700afd60 0x7f20700a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41990/0 (socket says 172.21.15.32:41990) 2022-08-01T22:56:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 -- 172.21.15.32:0/1136455158 learned_addr learned my addr 172.21.15.32:0/1136455158 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206dd9b700 1 --2- 172.21.15.32:0/1136455158 >> v2:172.21.15.55:3300/0 conn(0x7f20700b01a0 0x7f20700a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 -- 172.21.15.32:0/1136455158 >> v2:172.21.15.55:3300/0 conn(0x7f20700b01a0 msgr2=0x7f20700a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 --2- 172.21.15.32:0/1136455158 >> v2:172.21.15.55:3300/0 conn(0x7f20700b01a0 0x7f20700a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 -- 172.21.15.32:0/1136455158 >> v1:172.21.15.55:6789/0 conn(0x7f20700b2710 legacy=0x7f207014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 -- 172.21.15.32:0/1136455158 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700a79c0 con 0x7f20700afd60 2022-08-01T22:56:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206ed9d700 1 --2- 172.21.15.32:0/1136455158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a8a30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f2068002f70 tx=0x7f2068006eb0).ready entity=mon.0 client_cookie=952433de2303fcb2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206d59a700 1 -- 172.21.15.32:0/1136455158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206800d5c0 con 0x7f20700afd60 2022-08-01T22:56:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206d59a700 1 -- 172.21.15.32:0/1136455158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068000e90 con 0x7f20700afd60 2022-08-01T22:56:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206d59a700 1 -- 172.21.15.32:0/1136455158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206800d950 con 0x7f20700afd60 2022-08-01T22:56:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 -- 172.21.15.32:0/1136455158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 msgr2=0x7f20700a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.791+0000 7f206f59e700 1 --2- 172.21.15.32:0/1136455158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a8a30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f2068002f70 tx=0x7f2068006eb0).stop 2022-08-01T22:56:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- 172.21.15.32:0/1136455158 shutdown_connections 2022-08-01T22:56:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 --2- 172.21.15.32:0/1136455158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a8a30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 --2- 172.21.15.32:0/1136455158 >> v2:172.21.15.55:3300/0 conn(0x7f20700b01a0 0x7f20700a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- 172.21.15.32:0/1136455158 >> 172.21.15.32:0/1136455158 conn(0x7f2070060310 msgr2=0x7f20700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- 172.21.15.32:0/1136455158 shutdown_connections 2022-08-01T22:56:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- 172.21.15.32:0/1136455158 wait complete. 2022-08-01T22:56:30.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 Processor -- start 2022-08-01T22:56:30.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- start start 2022-08-01T22:56:30.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20700b01a0 0x7f2070146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20700b2710 0x7f2070146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20700aad70 con 0x7f20700afd60 2022-08-01T22:56:30.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20700ab010 con 0x7f20700b2710 2022-08-01T22:56:30.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20700ab2b0 con 0x7f20700b01a0 2022-08-01T22:56:30.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206dd9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20700b01a0 0x7f2070146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41994/0 (socket says 172.21.15.32:41994) 2022-08-01T22:56:30.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 -- 172.21.15.32:0/2716017436 learned_addr learned my addr 172.21.15.32:0/2716017436 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:30.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206ed9d700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20700b2710 0x7f2070146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 -- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20700b01a0 msgr2=0x7f2070146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20700b01a0 0x7f2070146630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 -- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20700b2710 msgr2=0x7f2070146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20700b2710 0x7f2070146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 -- 172.21.15.32:0/2716017436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206800b1e0 con 0x7f20700afd60 2022-08-01T22:56:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206e59c700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a9850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f207006b950 tx=0x7f2060007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2060007e70 con 0x7f20700afd60 2022-08-01T22:56:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060007030 con 0x7f20700afd60 2022-08-01T22:56:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20700ab550 con 0x7f20700afd60 2022-08-01T22:56:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2060007390 con 0x7f20700afd60 2022-08-01T22:56:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.795+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2070147010 con 0x7f20700afd60 2022-08-01T22:56:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.799+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20700ab6e0 con 0x7f20700afd60 2022-08-01T22:56:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.799+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2060010030 con 0x7f20700afd60 2022-08-01T22:56:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.799+0000 7f20677fe700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053a70 0x7f2054055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.799+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f206006bab0 con 0x7f20700afd60 2022-08-01T22:56:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.799+0000 7f206dd9b700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053a70 0x7f2054055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.803+0000 7f206dd9b700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053a70 0x7f2054055f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f205800b000 tx=0x7f20580091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:30.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.807+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2060042870 con 0x7f20700afd60 2022-08-01T22:56:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.967+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20700574e0 con 0x7f20700afd60 2022-08-01T22:56:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f20677fe700 1 -- 172.21.15.32:0/2716017436 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f2060045e90 con 0x7f20700afd60 2022-08-01T22:56:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:30.977 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:56:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053a70 msgr2=0x7f2054055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053a70 0x7f2054055f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f205800b000 tx=0x7f20580091d0).stop 2022-08-01T22:56:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 msgr2=0x7f20700a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a9850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f207006b950 tx=0x7f2060007c00).stop 2022-08-01T22:56:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 shutdown_connections 2022-08-01T22:56:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2054053a70 0x7f2054055f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20700b2710 0x7f2070146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20700b01a0 0x7f2070146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 --2- 172.21.15.32:0/2716017436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20700afd60 0x7f20700a9850 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:30.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 >> 172.21.15.32:0/2716017436 conn(0x7f2070060310 msgr2=0x7f207014d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 shutdown_connections 2022-08-01T22:56:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:30.971+0000 7f206f59e700 1 -- 172.21.15.32:0/2716017436 wait complete. 2022-08-01T22:56:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:56:31.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:56:28.419646+0000', 'modified': '2022-08-01T22:56:29.452004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4863': {'gid': 4863, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6833/3560980402', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6832', 'nonce': 3560980402}, {'type': 'v1', 'addr': '172.21.15.32:6833', 'nonce': 3560980402}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:56:31.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:56:31.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T22:56:31.408 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":4863,"want_state":"up:active","state":"up:active","rank_uptime":2.97478233,"mdsmap_epoch":12,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":9.9132150630000009} 2022-08-01T22:56:31.409 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 4863, "mdsmap_epoch": 12, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 2.97478233, "state": "up:active", "uptime": 9.913215063, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:56:31.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:56:31.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.507+0000 7f352525e700 1 Processor -- start 2022-08-01T22:56:31.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.507+0000 7f352525e700 1 -- start start 2022-08-01T22:56:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3520066dc0 0x7f352014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200654c0 con 0x7f35200b3190 2022-08-01T22:56:31.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200b3570 con 0x7f35200b2730 2022-08-01T22:56:31.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3520062190 con 0x7f3520066dc0 2022-08-01T22:56:31.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f3524a5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3520066dc0 0x7f352014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f351ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f35200b3190 0x7f3520060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57774/0 (socket says 172.21.15.32:57774) 2022-08-01T22:56:31.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f3524a5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3520066dc0 0x7f352014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42004/0 (socket says 172.21.15.32:42004) 2022-08-01T22:56:31.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f351ffff700 1 -- 172.21.15.32:0/3611568720 learned_addr learned my addr 172.21.15.32:0/3611568720 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:31.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f3524a5d700 1 -- 172.21.15.32:0/3611568720 >> v1:172.21.15.55:6789/0 conn(0x7f35200b2730 legacy=0x7f35200666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:31.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f3524a5d700 1 -- 172.21.15.32:0/3611568720 >> v1:172.21.15.32:6789/0 conn(0x7f35200b3190 legacy=0x7f3520060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f3524a5d700 1 -- 172.21.15.32:0/3611568720 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352005fbf0 con 0x7f3520066dc0 2022-08-01T22:56:31.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f3524a5d700 1 --2- 172.21.15.32:0/3611568720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3520066dc0 0x7f352014eac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f3520068010 tx=0x7f35180091d0).ready entity=mon.0 client_cookie=c13384223584ee7e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:31.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f351f7fe700 1 -- 172.21.15.32:0/3611568720 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3518009ea0 con 0x7f3520066dc0 2022-08-01T22:56:31.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f351f7fe700 1 -- 172.21.15.32:0/3611568720 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351800b860 con 0x7f3520066dc0 2022-08-01T22:56:31.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f351f7fe700 1 -- 172.21.15.32:0/3611568720 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f351800b9d0 con 0x7f3520066dc0 2022-08-01T22:56:31.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- 172.21.15.32:0/3611568720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3520066dc0 msgr2=0x7f352014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 --2- 172.21.15.32:0/3611568720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3520066dc0 0x7f352014eac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f3520068010 tx=0x7f35180091d0).stop 2022-08-01T22:56:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- 172.21.15.32:0/3611568720 shutdown_connections 2022-08-01T22:56:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 --2- 172.21.15.32:0/3611568720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3520066dc0 0x7f352014eac0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- 172.21.15.32:0/3611568720 >> 172.21.15.32:0/3611568720 conn(0x7f3520063870 msgr2=0x7f3520063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- 172.21.15.32:0/3611568720 shutdown_connections 2022-08-01T22:56:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.511+0000 7f352525e700 1 -- 172.21.15.32:0/3611568720 wait complete. 2022-08-01T22:56:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 Processor -- start 2022-08-01T22:56:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 -- start start 2022-08-01T22:56:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3520066dc0 0x7f3520144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35200b2730 0x7f35200a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 0x7f35200a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200a5ea0 con 0x7f35200b3190 2022-08-01T22:56:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201455f0 con 0x7f35200b2730 2022-08-01T22:56:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f352525e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200a6a40 con 0x7f3520066dc0 2022-08-01T22:56:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 0x7f35200a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 0x7f35200a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42008/0 (socket says 172.21.15.32:42008) 2022-08-01T22:56:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 -- 172.21.15.32:0/82222386 learned_addr learned my addr 172.21.15.32:0/82222386 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3517fff700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35200b2730 0x7f35200a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f351ffff700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3520066dc0 0x7f3520144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 -- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3520066dc0 msgr2=0x7f3520144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3520066dc0 0x7f3520144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 -- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35200b2730 msgr2=0x7f35200a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35200b2730 0x7f35200a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 -- 172.21.15.32:0/82222386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009470 con 0x7f35200b3190 2022-08-01T22:56:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3517fff700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35200b2730 0x7f35200a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.515+0000 7f3524a5d700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 0x7f35200a4b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f3520068010 tx=0x7f3518016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3518009ea0 con 0x7f35200b3190 2022-08-01T22:56:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35180032f0 con 0x7f35200b3190 2022-08-01T22:56:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35200a6ce0 con 0x7f35200b3190 2022-08-01T22:56:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f351801f5e0 con 0x7f35200b3190 2022-08-01T22:56:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3520147110 con 0x7f35200b3190 2022-08-01T22:56:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35200a7180 con 0x7f35200b3190 2022-08-01T22:56:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3518026030 con 0x7f35200b3190 2022-08-01T22:56:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351dffb700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3504053a10 0x7f3504055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f351806cdf0 con 0x7f35200b3190 2022-08-01T22:56:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.519+0000 7f351ffff700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3504053a10 0x7f3504055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.523+0000 7f351ffff700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3504053a10 0x7f3504055ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f351000b000 tx=0x7f3510009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.527+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3518043b10 con 0x7f35200b3190 2022-08-01T22:56:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.655+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f352005fe90 con 0x7f35200b3190 2022-08-01T22:56:31.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.655+0000 7f351dffb700 1 -- 172.21.15.32:0/82222386 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f3518024090 con 0x7f35200b3190 2022-08-01T22:56:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:31.663 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:56:29.422501+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:56:26.319594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T22:56:27.441691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:56:31.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3504053a10 msgr2=0x7f3504055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3504053a10 0x7f3504055ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f351000b000 tx=0x7f3510009040).stop 2022-08-01T22:56:31.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 msgr2=0x7f35200a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 0x7f35200a4b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f3520068010 tx=0x7f3518016000).stop 2022-08-01T22:56:31.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 shutdown_connections 2022-08-01T22:56:31.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3504053a10 0x7f3504055ed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35200b3190 0x7f35200a4b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35200b2730 0x7f35200a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 --2- 172.21.15.32:0/82222386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3520066dc0 0x7f3520144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 >> 172.21.15.32:0/82222386 conn(0x7f3520063870 msgr2=0x7f3520063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:31.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 shutdown_connections 2022-08-01T22:56:31.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.659+0000 7f352525e700 1 -- 172.21.15.32:0/82222386 wait complete. 2022-08-01T22:56:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:56:26.319594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:56:27.441691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:56:31.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:56:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 Processor -- start 2022-08-01T22:56:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 -- start start 2022-08-01T22:56:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b2710 0x7f77f014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77f00b01a0 0x7f77f014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f00a7250 con 0x7f77f00afd60 2022-08-01T22:56:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f00a7480 con 0x7f77f00b01a0 2022-08-01T22:56:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f6bbd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00a7720 con 0x7f77f00b2710 2022-08-01T22:56:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f53ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77f00b01a0 0x7f77f014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f5bbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b2710 0x7f77f014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f63bc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f77f00afd60 0x7f77f00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:50464/0 (socket says 172.21.15.32:50464) 2022-08-01T22:56:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f63bc700 1 -- 172.21.15.32:0/1441691035 learned_addr learned my addr 172.21.15.32:0/1441691035 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f4bb9700 1 -- 172.21.15.32:0/1441691035 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901558607 0 0) 0x7f77f00a7250 con 0x7f77f00afd60 2022-08-01T22:56:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f4bb9700 1 -- 172.21.15.32:0/1441691035 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77d4003260 con 0x7f77f00afd60 2022-08-01T22:56:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f5bbb700 1 -- 172.21.15.32:0/1441691035 >> v2:172.21.15.32:3300/0 conn(0x7f77f00b01a0 msgr2=0x7f77f014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f5bbb700 1 --2- 172.21.15.32:0/1441691035 >> v2:172.21.15.32:3300/0 conn(0x7f77f00b01a0 0x7f77f014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f5bbb700 1 -- 172.21.15.32:0/1441691035 >> v1:172.21.15.55:6789/0 conn(0x7f77f00afd60 legacy=0x7f77f00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f5bbb700 1 -- 172.21.15.32:0/1441691035 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00a79c0 con 0x7f77f00b2710 2022-08-01T22:56:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f5bbb700 1 --2- 172.21.15.32:0/1441691035 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b2710 0x7f77f014afd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f77ec002f70 tx=0x7f77ec006df0).ready entity=mon.2 client_cookie=2df8d1d11d4b4bde server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f4bb9700 1 -- 172.21.15.32:0/1441691035 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77ec00d4a0 con 0x7f77f00b2710 2022-08-01T22:56:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f4bb9700 1 -- 172.21.15.32:0/1441691035 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec000e90 con 0x7f77f00b2710 2022-08-01T22:56:31.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.787+0000 7f77f4bb9700 1 -- 172.21.15.32:0/1441691035 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77ec00d7f0 con 0x7f77f00b2710 2022-08-01T22:56:31.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1441691035 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b2710 msgr2=0x7f77f014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1441691035 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b2710 0x7f77f014afd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f77ec002f70 tx=0x7f77ec006df0).stop 2022-08-01T22:56:31.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1441691035 shutdown_connections 2022-08-01T22:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1441691035 >> v2:172.21.15.32:3300/0 conn(0x7f77f00b01a0 0x7f77f014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1441691035 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b2710 0x7f77f014afd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1441691035 >> 172.21.15.32:0/1441691035 conn(0x7f77f0060310 msgr2=0x7f77f00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1441691035 shutdown_connections 2022-08-01T22:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1441691035 wait complete. 2022-08-01T22:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 Processor -- start 2022-08-01T22:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- start start 2022-08-01T22:56:31.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77f00afd60 0x7f77f00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b01a0 0x7f77f0159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 0x7f77f015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f014d580 con 0x7f77f00b2710 2022-08-01T22:56:31.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00b4770 con 0x7f77f00afd60 2022-08-01T22:56:31.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f6bbd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00ab980 con 0x7f77f00b01a0 2022-08-01T22:56:31.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f63bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 0x7f77f015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f63bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 0x7f77f015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42020/0 (socket says 172.21.15.32:42020) 2022-08-01T22:56:31.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f63bc700 1 -- 172.21.15.32:0/1492946484 learned_addr learned my addr 172.21.15.32:0/1492946484 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f5bbb700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77f00afd60 0x7f77f00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f53ba700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b01a0 0x7f77f0159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.791+0000 7f77f63bc700 1 -- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b01a0 msgr2=0x7f77f0159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f63bc700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b01a0 0x7f77f0159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f63bc700 1 -- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77f00afd60 msgr2=0x7f77f00b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f63bc700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77f00afd60 0x7f77f00b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f63bc700 1 -- 172.21.15.32:0/1492946484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec00b180 con 0x7f77f00b2710 2022-08-01T22:56:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f63bc700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 0x7f77f015a300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f77e8001490 tx=0x7f77e8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:31.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e8005ba0 con 0x7f77f00b2710 2022-08-01T22:56:31.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8005d00 con 0x7f77f00b2710 2022-08-01T22:56:31.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e80072c0 con 0x7f77f00b2710 2022-08-01T22:56:31.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f00abc20 con 0x7f77f00b2710 2022-08-01T22:56:31.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f00b4460 con 0x7f77f00b2710 2022-08-01T22:56:31.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f00b3ac0 con 0x7f77f00b2710 2022-08-01T22:56:31.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f77e8010070 con 0x7f77f00b2710 2022-08-01T22:56:31.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77e6ffd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f77d8053a70 0x7f77d8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:31.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f77e806bc50 con 0x7f77f00b2710 2022-08-01T22:56:31.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.795+0000 7f77f5bbb700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f77d8053a70 0x7f77d8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:31.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.799+0000 7f77f5bbb700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f77d8053a70 0x7f77d8055f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f77ec018000 tx=0x7f77ec014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:31.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.803+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77e8042a20 con 0x7f77f00b2710 2022-08-01T22:56:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f77f00574e0 con 0x7f77f00b2710 2022-08-01T22:56:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77e6ffd700 1 -- 172.21.15.32:0/1492946484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v35) v1 ==== 110+0+10 (secure 0 0 0) 0x7f77e8046040 con 0x7f77f00b2710 2022-08-01T22:56:31.941 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:56:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f77d8053a70 msgr2=0x7f77d8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f77d8053a70 0x7f77d8055f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f77ec018000 tx=0x7f77ec014000).stop 2022-08-01T22:56:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 msgr2=0x7f77f015a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 0x7f77f015a300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f77e8001490 tx=0x7f77e8007e00).stop 2022-08-01T22:56:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 shutdown_connections 2022-08-01T22:56:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f77d8053a70 0x7f77d8055f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77f00b2710 0x7f77f015a300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77f00b01a0 0x7f77f0159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 --2- 172.21.15.32:0/1492946484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77f00afd60 0x7f77f00b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 >> 172.21.15.32:0/1492946484 conn(0x7f77f0060310 msgr2=0x7f77f0145f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 shutdown_connections 2022-08-01T22:56:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:31.935+0000 7f77f6bbd700 1 -- 172.21.15.32:0/1492946484 wait complete. 2022-08-01T22:56:31.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:56:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.055+0000 7fe4d359e700 1 Processor -- start 2022-08-01T22:56:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.055+0000 7fe4d359e700 1 -- start start 2022-08-01T22:56:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4d40b65c0 0x7fe4d4061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4d40b41f0 0x7fe4d4061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d40b2100 con 0x7fe4d4147d80 2022-08-01T22:56:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d40b1630 con 0x7fe4d40b65c0 2022-08-01T22:56:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d4061fe0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4d40b65c0 0x7fe4d4061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4d40b65c0 0x7fe4d4061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42028/0 (socket says 172.21.15.32:42028) 2022-08-01T22:56:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 -- 172.21.15.32:0/4009558388 learned_addr learned my addr 172.21.15.32:0/4009558388 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/4009558388 >> v2:172.21.15.55:3300/0 conn(0x7fe4d40b41f0 0x7fe4d4061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 -- 172.21.15.32:0/4009558388 >> v2:172.21.15.55:3300/0 conn(0x7fe4d40b41f0 msgr2=0x7fe4d4061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 --2- 172.21.15.32:0/4009558388 >> v2:172.21.15.55:3300/0 conn(0x7fe4d40b41f0 0x7fe4d4061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 -- 172.21.15.32:0/4009558388 >> v1:172.21.15.55:6789/0 conn(0x7fe4d4147d80 legacy=0x7fe4d4144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 -- 172.21.15.32:0/4009558388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d40b69a0 con 0x7fe4d40b65c0 2022-08-01T22:56:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d259c700 1 --2- 172.21.15.32:0/4009558388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b65c0 0x7fe4d4061600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fe4d4062120 tx=0x7fe4bc006eb0).ready entity=mon.0 client_cookie=ffcad141b9b91b98 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d159a700 1 -- 172.21.15.32:0/4009558388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4bc00d610 con 0x7fe4d40b65c0 2022-08-01T22:56:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d159a700 1 -- 172.21.15.32:0/4009558388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc000e90 con 0x7fe4d40b65c0 2022-08-01T22:56:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d159a700 1 -- 172.21.15.32:0/4009558388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4bc00d9a0 con 0x7fe4d40b65c0 2022-08-01T22:56:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- 172.21.15.32:0/4009558388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b65c0 msgr2=0x7fe4d4061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 --2- 172.21.15.32:0/4009558388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b65c0 0x7fe4d4061600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fe4d4062120 tx=0x7fe4bc006eb0).stop 2022-08-01T22:56:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- 172.21.15.32:0/4009558388 shutdown_connections 2022-08-01T22:56:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 --2- 172.21.15.32:0/4009558388 >> v2:172.21.15.55:3300/0 conn(0x7fe4d40b41f0 0x7fe4d4061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 --2- 172.21.15.32:0/4009558388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b65c0 0x7fe4d4061600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- 172.21.15.32:0/4009558388 >> 172.21.15.32:0/4009558388 conn(0x7fe4d405c620 msgr2=0x7fe4d405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- 172.21.15.32:0/4009558388 shutdown_connections 2022-08-01T22:56:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.059+0000 7fe4d359e700 1 -- 172.21.15.32:0/4009558388 wait complete. 2022-08-01T22:56:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 Processor -- start 2022-08-01T22:56:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 -- start start 2022-08-01T22:56:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 0x7fe4d414ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe4d40b65c0 0x7fe4d4064610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4147d80 0x7fe4d40aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4064b00 con 0x7fe4d40b41f0 2022-08-01T22:56:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40af5d0 con 0x7fe4d4147d80 2022-08-01T22:56:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40b0520 con 0x7fe4d40b65c0 2022-08-01T22:56:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 0x7fe4d414ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 0x7fe4d414ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42034/0 (socket says 172.21.15.32:42034) 2022-08-01T22:56:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 -- 172.21.15.32:0/2915685978 learned_addr learned my addr 172.21.15.32:0/2915685978 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d2d9d700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4147d80 0x7fe4d40aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe4d40b65c0 0x7fe4d4064610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 -- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe4d40b65c0 msgr2=0x7fe4d4064610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe4d40b65c0 0x7fe4d4064610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 -- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4147d80 msgr2=0x7fe4d40aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4147d80 0x7fe4d40aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 -- 172.21.15.32:0/2915685978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc00b1e0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d2d9d700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4147d80 0x7fe4d40aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d259c700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 0x7fe4d414ac40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fe4d4062120 tx=0x7fe4bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4bc021070 con 0x7fe4d40b41f0 2022-08-01T22:56:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc011860 con 0x7fe4d40b41f0 2022-08-01T22:56:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d40b07c0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4bc011bc0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.063+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d414b270 con 0x7fe4d40b41f0 2022-08-01T22:56:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.067+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d40af7d0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.067+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe4bc00f070 con 0x7fe4d40b41f0 2022-08-01T22:56:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.067+0000 7fe4cb7fe700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe4b8053b30 0x7fe4b8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.067+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe4bc06d2c0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.067+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe4b8053b30 0x7fe4b8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.067+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe4b8053b30 0x7fe4b8055ff0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fe4c400b000 tx=0x7fe4c4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.075+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe4bc044020 con 0x7fe4d40b41f0 2022-08-01T22:56:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.203+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe4d41482a0 con 0x7fe4d40b41f0 2022-08-01T22:56:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.203+0000 7fe4cb7fe700 1 -- 172.21.15.32:0/2915685978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe4bc047640 con 0x7fe4d40b41f0 2022-08-01T22:56:32.210 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:32.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe4b8053b30 msgr2=0x7fe4b8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe4b8053b30 0x7fe4b8055ff0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fe4c400b000 tx=0x7fe4c4008040).stop 2022-08-01T22:56:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 msgr2=0x7fe4d414ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 0x7fe4d414ac40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fe4d4062120 tx=0x7fe4bc015000).stop 2022-08-01T22:56:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 shutdown_connections 2022-08-01T22:56:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe4b8053b30 0x7fe4b8055ff0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4147d80 0x7fe4d40aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe4d40b65c0 0x7fe4d4064610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 --2- 172.21.15.32:0/2915685978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d40b41f0 0x7fe4d414ac40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 >> 172.21.15.32:0/2915685978 conn(0x7fe4d405c620 msgr2=0x7fe4d4143cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 shutdown_connections 2022-08-01T22:56:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.207+0000 7fe4d359e700 1 -- 172.21.15.32:0/2915685978 wait complete. 2022-08-01T22:56:32.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:56:32.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989f7e7700 1 Processor -- start 2022-08-01T22:56:32.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989f7e7700 1 -- start start 2022-08-01T22:56:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989f7e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98980b4310 0x7f9898140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989f7e7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9898143cf0 con 0x7f98980b4820 2022-08-01T22:56:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989f7e7700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9898143f90 con 0x7f9898146ae0 2022-08-01T22:56:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989f7e7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98981476a0 con 0x7f98980b4310 2022-08-01T22:56:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98980b4310 0x7f9898140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98980b4310 0x7f9898140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42044/0 (socket says 172.21.15.32:42044) 2022-08-01T22:56:32.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 -- 172.21.15.32:0/3606316281 learned_addr learned my addr 172.21.15.32:0/3606316281 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:32.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989dd84700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98980b4820 0x7f98980b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57820/0 (socket says 172.21.15.32:57820) 2022-08-01T22:56:32.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 -- 172.21.15.32:0/3606316281 >> v1:172.21.15.55:6789/0 conn(0x7f9898146ae0 legacy=0x7f98981403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f988ffff700 1 -- 172.21.15.32:0/3606316281 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106892209 0 0) 0x7f9898143cf0 con 0x7f98980b4820 2022-08-01T22:56:32.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 -- 172.21.15.32:0/3606316281 >> v1:172.21.15.32:6789/0 conn(0x7f98980b4820 legacy=0x7f98980b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 -- 172.21.15.32:0/3606316281 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980b0e20 con 0x7f98980b4310 2022-08-01T22:56:32.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f989cd82700 1 --2- 172.21.15.32:0/3606316281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f9898140ad0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f9888002f70 tx=0x7f9888006eb0).ready entity=mon.0 client_cookie=bea7a9ade84de70d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f988ffff700 1 -- 172.21.15.32:0/3606316281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f988800d660 con 0x7f98980b4310 2022-08-01T22:56:32.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f988ffff700 1 -- 172.21.15.32:0/3606316281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9888000e90 con 0x7f98980b4310 2022-08-01T22:56:32.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.339+0000 7f988ffff700 1 -- 172.21.15.32:0/3606316281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f988800d9b0 con 0x7f98980b4310 2022-08-01T22:56:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- 172.21.15.32:0/3606316281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 msgr2=0x7f9898140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 --2- 172.21.15.32:0/3606316281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f9898140ad0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f9888002f70 tx=0x7f9888006eb0).stop 2022-08-01T22:56:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- 172.21.15.32:0/3606316281 shutdown_connections 2022-08-01T22:56:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 --2- 172.21.15.32:0/3606316281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f9898140ad0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- 172.21.15.32:0/3606316281 >> 172.21.15.32:0/3606316281 conn(0x7f989805c620 msgr2=0x7f989805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- 172.21.15.32:0/3606316281 shutdown_connections 2022-08-01T22:56:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- 172.21.15.32:0/3606316281 wait complete. 2022-08-01T22:56:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 Processor -- start 2022-08-01T22:56:32.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- start start 2022-08-01T22:56:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f98981524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98980b4820 0x7f98981529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9898146ae0 0x7f9898155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981534e0 con 0x7f98980b4310 2022-08-01T22:56:32.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9898153240 con 0x7f98980b4820 2022-08-01T22:56:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989f7e7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980b7930 con 0x7f9898146ae0 2022-08-01T22:56:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989d583700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f98981524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989d583700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f98981524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42048/0 (socket says 172.21.15.32:42048) 2022-08-01T22:56:32.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989d583700 1 -- 172.21.15.32:0/1854834185 learned_addr learned my addr 172.21.15.32:0/1854834185 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989cd82700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98980b4820 0x7f98981529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.343+0000 7f989dd84700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9898146ae0 0x7f9898155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989d583700 1 -- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9898146ae0 msgr2=0x7f9898155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989d583700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9898146ae0 0x7f9898155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989d583700 1 -- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98980b4820 msgr2=0x7f98981529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989d583700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98980b4820 0x7f98981529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989d583700 1 -- 172.21.15.32:0/1854834185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988800b1e0 con 0x7f98980b4310 2022-08-01T22:56:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989cd82700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98980b4820 0x7f98981529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:32.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989dd84700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9898146ae0 0x7f9898155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989d583700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f98981524f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f9898060e20 tx=0x7f9890007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f989000ce30 con 0x7f98980b4310 2022-08-01T22:56:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989000f470 con 0x7f98980b4310 2022-08-01T22:56:32.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98980b7bd0 con 0x7f98980b4310 2022-08-01T22:56:32.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f989000f7d0 con 0x7f98980b4310 2022-08-01T22:56:32.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9898152ed0 con 0x7f98980b4310 2022-08-01T22:56:32.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98980b8770 con 0x7f98980b4310 2022-08-01T22:56:32.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9890019030 con 0x7f98980b4310 2022-08-01T22:56:32.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f988e7fc700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9880053a70 0x7f9880055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.347+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f989006b440 con 0x7f98980b4310 2022-08-01T22:56:32.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.351+0000 7f989cd82700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9880053a70 0x7f9880055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.351+0000 7f989cd82700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9880053a70 0x7f9880055f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f988801a000 tx=0x7f9888015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.355+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98980b8770 con 0x7f98980b4310 2022-08-01T22:56:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.475+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98980b8770 con 0x7f98980b4310 2022-08-01T22:56:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f988e7fc700 1 -- 172.21.15.32:0/1854834185 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+10 (secure 0 0 0) 0x7f98980b8770 con 0x7f98980b4310 2022-08-01T22:56:32.483 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9880053a70 msgr2=0x7f9880055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9880053a70 0x7f9880055f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f988801a000 tx=0x7f9888015000).stop 2022-08-01T22:56:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 msgr2=0x7f98981524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f98981524f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f9898060e20 tx=0x7f9890007c50).stop 2022-08-01T22:56:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 shutdown_connections 2022-08-01T22:56:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9880053a70 0x7f9880055f30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9898146ae0 0x7f9898155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98980b4820 0x7f98981529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 --2- 172.21.15.32:0/1854834185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98980b4310 0x7f98981524f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 >> 172.21.15.32:0/1854834185 conn(0x7f989805c620 msgr2=0x7f9898062050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 shutdown_connections 2022-08-01T22:56:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.479+0000 7f989f7e7700 1 -- 172.21.15.32:0/1854834185 wait complete. 2022-08-01T22:56:32.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:56:32.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.595+0000 7f6292ef5700 1 Processor -- start 2022-08-01T22:56:32.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.595+0000 7f6292ef5700 1 -- start start 2022-08-01T22:56:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f628c0b2730 0x7f628c0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0654c0 con 0x7f628c066ba0 2022-08-01T22:56:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0b3570 con 0x7f628c0b3190 2022-08-01T22:56:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c062190 con 0x7f628c0b2730 2022-08-01T22:56:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62926f4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f628c066ba0 0x7f628c14dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57834/0 (socket says 172.21.15.32:57834) 2022-08-01T22:56:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62926f4700 1 -- 172.21.15.32:0/1722301827 learned_addr learned my addr 172.21.15.32:0/1722301827 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62916f2700 1 --2- 172.21.15.32:0/1722301827 >> v2:172.21.15.55:3300/0 conn(0x7f628c0b2730 0x7f628c0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6290ef1700 1 -- 172.21.15.32:0/1722301827 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462646772 0 0) 0x7f628c0654c0 con 0x7f628c066ba0 2022-08-01T22:56:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6290ef1700 1 -- 172.21.15.32:0/1722301827 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6274003260 con 0x7f628c066ba0 2022-08-01T22:56:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62916f2700 1 -- 172.21.15.32:0/1722301827 >> v1:172.21.15.55:6789/0 conn(0x7f628c0b3190 legacy=0x7f628c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62916f2700 1 -- 172.21.15.32:0/1722301827 >> v1:172.21.15.32:6789/0 conn(0x7f628c066ba0 legacy=0x7f628c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62916f2700 1 -- 172.21.15.32:0/1722301827 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c05fbf0 con 0x7f628c0b2730 2022-08-01T22:56:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f62916f2700 1 --2- 172.21.15.32:0/1722301827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c0b2730 0x7f628c0666b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6288002f70 tx=0x7f6288006df0).ready entity=mon.1 client_cookie=c4bdb1dc70dc9989 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6290ef1700 1 -- 172.21.15.32:0/1722301827 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628800d610 con 0x7f628c0b2730 2022-08-01T22:56:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6290ef1700 1 -- 172.21.15.32:0/1722301827 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288000e90 con 0x7f628c0b2730 2022-08-01T22:56:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6290ef1700 1 -- 172.21.15.32:0/1722301827 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628800d960 con 0x7f628c0b2730 2022-08-01T22:56:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- 172.21.15.32:0/1722301827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c0b2730 msgr2=0x7f628c0666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 --2- 172.21.15.32:0/1722301827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c0b2730 0x7f628c0666b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6288002f70 tx=0x7f6288006df0).stop 2022-08-01T22:56:32.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- 172.21.15.32:0/1722301827 shutdown_connections 2022-08-01T22:56:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 --2- 172.21.15.32:0/1722301827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c0b2730 0x7f628c0666b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- 172.21.15.32:0/1722301827 >> 172.21.15.32:0/1722301827 conn(0x7f628c063870 msgr2=0x7f628c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- 172.21.15.32:0/1722301827 shutdown_connections 2022-08-01T22:56:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.599+0000 7f6292ef5700 1 -- 172.21.15.32:0/1722301827 wait complete. 2022-08-01T22:56:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 Processor -- start 2022-08-01T22:56:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 -- start start 2022-08-01T22:56:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c066ba0 0x7f628c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 0x7f628c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f628c0b3190 0x7f628c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0ad710 con 0x7f628c0b2730 2022-08-01T22:56:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0ab7b0 con 0x7f628c066ba0 2022-08-01T22:56:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6292ef5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0ac250 con 0x7f628c0b3190 2022-08-01T22:56:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 0x7f628c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 0x7f628c0b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42064/0 (socket says 172.21.15.32:42064) 2022-08-01T22:56:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 -- 172.21.15.32:0/2853908859 learned_addr learned my addr 172.21.15.32:0/2853908859 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f6291ef3700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c066ba0 0x7f628c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62926f4700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f628c0b3190 0x7f628c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 -- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f628c0b3190 msgr2=0x7f628c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f628c0b3190 0x7f628c0b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 -- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c066ba0 msgr2=0x7f628c0b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c066ba0 0x7f628c0b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 -- 172.21.15.32:0/2853908859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628800b180 con 0x7f628c0b2730 2022-08-01T22:56:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62926f4700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f628c0b3190 0x7f628c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.603+0000 7f62916f2700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 0x7f628c0b0080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6288002f70 tx=0x7f6288014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628800d290 con 0x7f628c0b2730 2022-08-01T22:56:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288013e90 con 0x7f628c0b2730 2022-08-01T22:56:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0b0a60 con 0x7f628c0b2730 2022-08-01T22:56:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628801e5f0 con 0x7f628c0b2730 2022-08-01T22:56:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0ac4b0 con 0x7f628c0b2730 2022-08-01T22:56:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c05fe90 con 0x7f628c0b2730 2022-08-01T22:56:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6288025030 con 0x7f628c0b2730 2022-08-01T22:56:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f627effd700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f626c053a70 0x7f626c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f628806ce10 con 0x7f628c0b2730 2022-08-01T22:56:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.607+0000 7f6291ef3700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f626c053a70 0x7f626c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.611+0000 7f6291ef3700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f626c053a70 0x7f626c055f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f628000d000 tx=0x7f6280007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.619+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6288043bd0 con 0x7f628c0b2730 2022-08-01T22:56:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.743+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f628c0574e0 con 0x7f628c0b2730 2022-08-01T22:56:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.743+0000 7f627effd700 1 -- 172.21.15.32:0/2853908859 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6288023070 con 0x7f628c0b2730 2022-08-01T22:56:32.752 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:56:32.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f626c053a70 msgr2=0x7f626c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f626c053a70 0x7f626c055f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f628000d000 tx=0x7f6280007b90).stop 2022-08-01T22:56:32.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 msgr2=0x7f628c0b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 0x7f628c0b0080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6288002f70 tx=0x7f6288014000).stop 2022-08-01T22:56:32.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 shutdown_connections 2022-08-01T22:56:32.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f626c053a70 0x7f626c055f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f628c0b3190 0x7f628c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f628c0b2730 0x7f628c0b0080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 --2- 172.21.15.32:0/2853908859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f628c066ba0 0x7f628c0b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 >> 172.21.15.32:0/2853908859 conn(0x7f628c063870 msgr2=0x7f628c145040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 shutdown_connections 2022-08-01T22:56:32.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.747+0000 7f6292ef5700 1 -- 172.21.15.32:0/2853908859 wait complete. 2022-08-01T22:56:32.763 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:56:32.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.859+0000 7f6d8f801700 1 Processor -- start 2022-08-01T22:56:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.859+0000 7f6d8f801700 1 -- start start 2022-08-01T22:56:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8f801700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d88148530 0x7f6d880b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8f801700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6d880a7250 0x7f6d881448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8f801700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d880b04e0 con 0x7f6d880b1f40 2022-08-01T22:56:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8f801700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880a9250 con 0x7f6d88148530 2022-08-01T22:56:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8f801700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880b0780 con 0x7f6d880a7250 2022-08-01T22:56:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8d59d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d880b1f40 0x7f6d880b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57844/0 (socket says 172.21.15.32:57844) 2022-08-01T22:56:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8d59d700 1 -- 172.21.15.32:0/4237828609 learned_addr learned my addr 172.21.15.32:0/4237828609 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8cd9c700 1 --2- 172.21.15.32:0/4237828609 >> v2:172.21.15.32:3300/0 conn(0x7f6d88148530 0x7f6d880b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 --2- 172.21.15.32:0/4237828609 >> v2:172.21.15.55:3300/0 conn(0x7f6d880a7250 0x7f6d881448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707761298 0 0) 0x7f6d880b04e0 con 0x7f6d880b1f40 2022-08-01T22:56:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74003260 con 0x7f6d880b1f40 2022-08-01T22:56:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 632959643 0 0) 0x7f6d74003260 con 0x7f6d880b1f40 2022-08-01T22:56:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 >> v2:172.21.15.55:3300/0 conn(0x7f6d880a7250 msgr2=0x7f6d881448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 --2- 172.21.15.32:0/4237828609 >> v2:172.21.15.55:3300/0 conn(0x7f6d880a7250 0x7f6d881448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 >> v2:172.21.15.32:3300/0 conn(0x7f6d88148530 msgr2=0x7f6d880b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 --2- 172.21.15.32:0/4237828609 >> v2:172.21.15.32:3300/0 conn(0x7f6d88148530 0x7f6d880b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880b0a20 con 0x7f6d880b1f40 2022-08-01T22:56:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f6d78001660 con 0x7f6d880b1f40 2022-08-01T22:56:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 >> v1:172.21.15.32:6789/0 conn(0x7f6d880b1f40 legacy=0x7f6d880b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74003750 0x7f6d74003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d74004500 0x7f6d74004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d74004e70 0x7f6d7400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880b0780 con 0x7f6d74004e70 2022-08-01T22:56:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400bc40 con 0x7f6d74004500 2022-08-01T22:56:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400bd80 con 0x7f6d74003750 2022-08-01T22:56:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d74004e70 0x7f6d7400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8d59d700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74003750 0x7f6d74003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8cd9c700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d74004500 0x7f6d74004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 -- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74003750 msgr2=0x7f6d74003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74003750 0x7f6d74003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 -- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d74004500 msgr2=0x7f6d74004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d74004500 0x7f6d74004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 -- 172.21.15.32:0/4237828609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74003260 con 0x7f6d74004e70 2022-08-01T22:56:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8cd9c700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d74004500 0x7f6d74004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 -- 172.21.15.32:0/4237828609 reap_dead start 2022-08-01T22:56:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d8dd9e700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d74004e70 0x7f6d7400b750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f6d8000c000 tx=0x7f6d800091d0).ready entity=mon.0 client_cookie=2c0ff7d9e872e383 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d8000e780 con 0x7f6d74004e70 2022-08-01T22:56:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000e8e0 con 0x7f6d74004e70 2022-08-01T22:56:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.863+0000 7f6d87fff700 1 -- 172.21.15.32:0/4237828609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d8000ec40 con 0x7f6d74004e70 2022-08-01T22:56:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 -- 172.21.15.32:0/4237828609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d74004e70 msgr2=0x7f6d7400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d74004e70 0x7f6d7400b750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f6d8000c000 tx=0x7f6d800091d0).stop 2022-08-01T22:56:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 -- 172.21.15.32:0/4237828609 shutdown_connections 2022-08-01T22:56:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 --2- 172.21.15.32:0/4237828609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d74004e70 0x7f6d7400b750 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 -- 172.21.15.32:0/4237828609 >> 172.21.15.32:0/4237828609 conn(0x7f6d8805f840 msgr2=0x7f6d8805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 -- 172.21.15.32:0/4237828609 shutdown_connections 2022-08-01T22:56:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 -- 172.21.15.32:0/4237828609 wait complete. 2022-08-01T22:56:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 Processor -- start 2022-08-01T22:56:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.867+0000 7f6d8f801700 1 -- start start 2022-08-01T22:56:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74004e70 0x7f6d88144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 0x7f6d88156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d880ac260 0x7f6d88157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8814a470 con 0x7f6d88062950 2022-08-01T22:56:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88156a10 con 0x7f6d880ac260 2022-08-01T22:56:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88157850 con 0x7f6d74004e70 2022-08-01T22:56:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 0x7f6d88156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 0x7f6d88156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42084/0 (socket says 172.21.15.32:42084) 2022-08-01T22:56:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 -- 172.21.15.32:0/2671380585 learned_addr learned my addr 172.21.15.32:0/2671380585 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8d59d700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74004e70 0x7f6d88144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8dd9e700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d880ac260 0x7f6d88157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 -- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74004e70 msgr2=0x7f6d88144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74004e70 0x7f6d88144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 -- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d880ac260 msgr2=0x7f6d88157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d880ac260 0x7f6d88157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 -- 172.21.15.32:0/2671380585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8814a6a0 con 0x7f6d88062950 2022-08-01T22:56:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8cd9c700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 0x7f6d88156e70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f6d7c00b000 tx=0x7f6d7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7c012610 con 0x7f6d88062950 2022-08-01T22:56:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c012770 con 0x7f6d88062950 2022-08-01T22:56:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d880ab910 con 0x7f6d88062950 2022-08-01T22:56:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7c012ad0 con 0x7f6d88062950 2022-08-01T22:56:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88148c90 con 0x7f6d88062950 2022-08-01T22:56:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d88156b80 con 0x7f6d88062950 2022-08-01T22:56:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6d7c012c30 con 0x7f6d88062950 2022-08-01T22:56:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.871+0000 7f6d867fc700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d68053a10 0x7f6d68055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.875+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6d7c06c7a0 con 0x7f6d88062950 2022-08-01T22:56:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.875+0000 7f6d8d59d700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d68053a10 0x7f6d68055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.875+0000 7f6d8d59d700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d68053a10 0x7f6d68055ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6d78005000 tx=0x7f6d880b0bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:32.875+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d7c043500 con 0x7f6d88062950 2022-08-01T22:56:33.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.039+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d880b0cc0 con 0x7f6d88062950 2022-08-01T22:56:33.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.039+0000 7f6d867fc700 1 -- 172.21.15.32:0/2671380585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f6d7c046b20 con 0x7f6d88062950 2022-08-01T22:56:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:33.046 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:56:33.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.039+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d68053a10 msgr2=0x7f6d68055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:33.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.039+0000 7f6d8f801700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d68053a10 0x7f6d68055ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6d78005000 tx=0x7f6d880b0bb0).stop 2022-08-01T22:56:33.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.039+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 msgr2=0x7f6d88156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.039+0000 7f6d8f801700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 0x7f6d88156e70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f6d7c00b000 tx=0x7f6d7c008040).stop 2022-08-01T22:56:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 shutdown_connections 2022-08-01T22:56:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d68053a10 0x7f6d68055ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d880ac260 0x7f6d88157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d88062950 0x7f6d88156e70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 --2- 172.21.15.32:0/2671380585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d74004e70 0x7f6d88144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 >> 172.21.15.32:0/2671380585 conn(0x7f6d8805f840 msgr2=0x7f6d880b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 shutdown_connections 2022-08-01T22:56:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:33.043+0000 7f6d8f801700 1 -- 172.21.15.32:0/2671380585 wait complete. 2022-08-01T22:56:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:56:33.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:56:28.419646+0000', 'modified': '2022-08-01T22:56:29.452004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4863': {'gid': 4863, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6833/3560980402', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6832', 'nonce': 3560980402}, {'type': 'v1', 'addr': '172.21.15.32:6833', 'nonce': 3560980402}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:56:33.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:56:33.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T22:56:33.488 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":4863,"want_state":"up:active","state":"up:active","rank_uptime":5.0546255760000003,"mdsmap_epoch":12,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":11.993069154000001} 2022-08-01T22:56:33.489 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 4863, "mdsmap_epoch": 12, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 5.054625576, "state": "up:active", "uptime": 11.993069154, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:56:33.490 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:56:33.490 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T22:56:33.491 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T22:56:33.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:33.493 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:33.503 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:56:33.563 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:33.563 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:33.578 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:33.579 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:33.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:33.581 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:56:33.589 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:56:33.590 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:33.636 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:56:33.655 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:33.656 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:33.666 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:33.667 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:33.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:33.668 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:56:33.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T22:56:33.709+0000 7fe2df1d2080 -1 init, newargv = 0x55fce0419840 newargc=9 2022-08-01T22:56:33.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22430]: starting ceph client 2022-08-01T22:56:33.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22430]: starting fuse 2022-08-01T22:56:34.775 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:56:34.795 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:56:34.796 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:56:34.853 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:56:34.854 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:56:34.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:34.855 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:56:34.861 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T22:56:34.862 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:56:34.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:34.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T22:56:34.864 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T22:56:34.865 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T22:56:34.865 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:56:34.866 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T22:56:34.866 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:34.867 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T22:56:34.867 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T22:56:34.867 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T22:56:34.868 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:34.868 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T22:56:34.869 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T22:56:34.869 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T22:56:34.870 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T22:56:34.871 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:34.871 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T22:56:34.872 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:56:34.873 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:56:34.873 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T22:56:34.874 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:56:34.875 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:34.876 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T22:56:34.876 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:56:34.949 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22430.asok 2022-08-01T22:56:34.954 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22430.asok 2022-08-01T22:56:34.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:34.955 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22430.asok status 2022-08-01T22:56:35.044 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T22:56:35.044 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T22:56:35.044 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:56:35.044 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:56:35.045 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T22:56:35.045 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T22:56:35.045 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T22:56:35.045 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22430", 2022-08-01T22:56:35.046 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T22:56:35.046 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:35.046 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T22:56:35.046 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T22:56:35.047 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 5085, 2022-08-01T22:56:35.047 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T22:56:35.047 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T22:56:35.047 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T22:56:35.047 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 5085 2022-08-01T22:56:35.048 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:35.048 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:56:35.048 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:56:35.048 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:56:35.049 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3582106783 2022-08-01T22:56:35.049 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T22:56:35.049 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:35.049 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:56:35.050 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:56:35.050 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:56:35.050 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3582106783 2022-08-01T22:56:35.050 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:56:35.050 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.5085 172.21.15.130:0/3582106783", 2022-08-01T22:56:35.051 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/3582106783", 2022-08-01T22:56:35.051 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T22:56:35.051 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 12, 2022-08-01T22:56:35.051 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 35, 2022-08-01T22:56:35.052 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:56:35.052 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T22:56:35.052 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T22:56:35.052 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T22:56:35.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:35.054 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:35.055 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:35.062 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:56:35.063 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:56:35.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:35.064 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:35.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2022-08-01T22:56:35.229 INFO:teuthology.orchestra.run.smithi032.stdout:{"mds_min_caps_per_client":"100"} 2022-08-01T22:56:35.230 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2022-08-01T22:56:35.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:56:35.231 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:56:35.231 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:56:35.242 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:56:35.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:56:35.244 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T22:56:35.245 DEBUG:teuthology.orchestra.run.smithi130:> import sys 2022-08-01T22:56:35.245 DEBUG:teuthology.orchestra.run.smithi130:> import time 2022-08-01T22:56:35.246 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:56:35.247 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:35.248 DEBUG:teuthology.orchestra.run.smithi130:> n = 1000 2022-08-01T22:56:35.249 DEBUG:teuthology.orchestra.run.smithi130:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T22:56:35.249 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:35.250 DEBUG:teuthology.orchestra.run.smithi130:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T22:56:35.251 DEBUG:teuthology.orchestra.run.smithi130:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T22:56:35.252 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:56:35.252 DEBUG:teuthology.orchestra.run.smithi130:> for i in range(0, n): 2022-08-01T22:56:35.253 DEBUG:teuthology.orchestra.run.smithi130:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T22:56:35.254 DEBUG:teuthology.orchestra.run.smithi130:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T22:56:35.255 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content'"'"') 2022-08-01T22:56:35.255 DEBUG:teuthology.orchestra.run.smithi130:> if True: 2022-08-01T22:56:35.256 DEBUG:teuthology.orchestra.run.smithi130:> f.flush() 2022-08-01T22:56:35.257 DEBUG:teuthology.orchestra.run.smithi130:> os.fsync(f.fileno()) 2022-08-01T22:56:35.258 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:56:46.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.479+0000 7f8d1efc5700 1 Processor -- start 2022-08-01T22:56:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.479+0000 7f8d1efc5700 1 -- start start 2022-08-01T22:56:46.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.479+0000 7f8d1efc5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d18148530 0x7f8d180b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d180a7250 0x7f8d181448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d180b04e0 con 0x7f8d180b1f40 2022-08-01T22:56:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d180a9250 con 0x7f8d18148530 2022-08-01T22:56:46.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180b0780 con 0x7f8d180a7250 2022-08-01T22:56:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d18148530 0x7f8d180b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d18148530 0x7f8d180b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42094/0 (socket says 172.21.15.32:42094) 2022-08-01T22:56:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 -- 172.21.15.32:0/4159053562 learned_addr learned my addr 172.21.15.32:0/4159053562 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1d562700 1 --2- 172.21.15.32:0/4159053562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d180a7250 0x7f8d181448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 -- 172.21.15.32:0/4159053562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d180a7250 msgr2=0x7f8d181448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 --2- 172.21.15.32:0/4159053562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d180a7250 0x7f8d181448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 -- 172.21.15.32:0/4159053562 >> v1:172.21.15.55:6789/0 conn(0x7f8d180b1f40 legacy=0x7f8d180b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 -- 172.21.15.32:0/4159053562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180b0a20 con 0x7f8d18148530 2022-08-01T22:56:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0ffff700 1 --2- 172.21.15.32:0/4159053562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d18148530 0x7f8d180b1160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f8d181478c0 tx=0x7f8d08006eb0).ready entity=mon.0 client_cookie=4d7a72b10e17103c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0f7fe700 1 -- 172.21.15.32:0/4159053562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d0800d5c0 con 0x7f8d18148530 2022-08-01T22:56:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0f7fe700 1 -- 172.21.15.32:0/4159053562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08000e90 con 0x7f8d18148530 2022-08-01T22:56:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d0f7fe700 1 -- 172.21.15.32:0/4159053562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d0800d950 con 0x7f8d18148530 2022-08-01T22:56:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- 172.21.15.32:0/4159053562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d18148530 msgr2=0x7f8d180b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/4159053562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d18148530 0x7f8d180b1160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f8d181478c0 tx=0x7f8d08006eb0).stop 2022-08-01T22:56:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- 172.21.15.32:0/4159053562 shutdown_connections 2022-08-01T22:56:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/4159053562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d18148530 0x7f8d180b1160 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/4159053562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d180a7250 0x7f8d181448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- 172.21.15.32:0/4159053562 >> 172.21.15.32:0/4159053562 conn(0x7f8d1805f840 msgr2=0x7f8d1805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- 172.21.15.32:0/4159053562 shutdown_connections 2022-08-01T22:56:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.483+0000 7f8d1efc5700 1 -- 172.21.15.32:0/4159053562 wait complete. 2022-08-01T22:56:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 Processor -- start 2022-08-01T22:56:46.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 -- start start 2022-08-01T22:56:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d180a7250 0x7f8d180aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 0x7f8d18156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d18148530 0x7f8d180ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180ab750 con 0x7f8d180b1f40 2022-08-01T22:56:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180ab9f0 con 0x7f8d180a7250 2022-08-01T22:56:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180abfe0 con 0x7f8d18148530 2022-08-01T22:56:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 0x7f8d18156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1cd61700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d180a7250 0x7f8d180aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 0x7f8d18156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42100/0 (socket says 172.21.15.32:42100) 2022-08-01T22:56:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 -- 172.21.15.32:0/177758420 learned_addr learned my addr 172.21.15.32:0/177758420 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1d562700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d18148530 0x7f8d180ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 -- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d18148530 msgr2=0x7f8d180ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d18148530 0x7f8d180ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 -- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d180a7250 msgr2=0x7f8d180aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d180a7250 0x7f8d180aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 -- 172.21.15.32:0/177758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0800b1e0 con 0x7f8d180b1f40 2022-08-01T22:56:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0ffff700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 0x7f8d18156c20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f8d181478c0 tx=0x7f8d08015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d08011610 con 0x7f8d180b1f40 2022-08-01T22:56:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08011770 con 0x7f8d180b1f40 2022-08-01T22:56:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d180af730 con 0x7f8d180b1f40 2022-08-01T22:56:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d08011ad0 con 0x7f8d180b1f40 2022-08-01T22:56:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.487+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d180ac970 con 0x7f8d180b1f40 2022-08-01T22:56:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.491+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d0800f070 con 0x7f8d180b1f40 2022-08-01T22:56:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.491+0000 7f8d0dffb700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf8053aa0 0x7f8cf8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.491+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8d0806d470 con 0x7f8d180b1f40 2022-08-01T22:56:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.491+0000 7f8d1cd61700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf8053aa0 0x7f8cf8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.491+0000 7f8d1cd61700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf8053aa0 0x7f8cf8055f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8d1400d000 tx=0x7f8d14007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:46.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.499+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d180ac1a0 con 0x7f8d180b1f40 2022-08-01T22:56:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.503+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d080442e0 con 0x7f8d180b1f40 2022-08-01T22:56:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d180574e0 con 0x7f8d180b1f40 2022-08-01T22:56:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d0dffb700 1 -- 172.21.15.32:0/177758420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f8d08047900 con 0x7f8d180b1f40 2022-08-01T22:56:46.677 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:46.677 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:56:46.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf8053aa0 msgr2=0x7f8cf8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf8053aa0 0x7f8cf8055f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8d1400d000 tx=0x7f8d14007b90).stop 2022-08-01T22:56:46.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 msgr2=0x7f8d18156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 0x7f8d18156c20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f8d181478c0 tx=0x7f8d08015000).stop 2022-08-01T22:56:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 shutdown_connections 2022-08-01T22:56:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf8053aa0 0x7f8cf8055f60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d18148530 0x7f8d180ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d180b1f40 0x7f8d18156c20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 --2- 172.21.15.32:0/177758420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d180a7250 0x7f8d180aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 >> 172.21.15.32:0/177758420 conn(0x7f8d1805f840 msgr2=0x7f8d18144de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 shutdown_connections 2022-08-01T22:56:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.671+0000 7f8d1efc5700 1 -- 172.21.15.32:0/177758420 wait complete. 2022-08-01T22:56:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:56:46.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-08-01T22:56:46.799 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T22:56:46.800 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T22:56:46.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:46.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 Processor -- start 2022-08-01T22:56:46.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 -- start start 2022-08-01T22:56:46.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f86c40b6a90 0x7f86c40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40aa760 0x7f86c413c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c40b6e70 con 0x7f86c4060aa0 2022-08-01T22:56:46.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c4061370 con 0x7f86c40b6a90 2022-08-01T22:56:46.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86ca227700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4061610 con 0x7f86c40aa760 2022-08-01T22:56:46.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f86c40b6a90 0x7f86c40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c2ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f86c4060aa0 0x7f86c40aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:50554/0 (socket says 172.21.15.32:50554) 2022-08-01T22:56:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c2ffd700 1 -- 172.21.15.32:0/3530230771 learned_addr learned my addr 172.21.15.32:0/3530230771 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40aa760 0x7f86c413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c27fc700 1 -- 172.21.15.32:0/3530230771 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269148705 0 0) 0x7f86c40b6e70 con 0x7f86c4060aa0 2022-08-01T22:56:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c27fc700 1 -- 172.21.15.32:0/3530230771 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b0003260 con 0x7f86c4060aa0 2022-08-01T22:56:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c3fff700 1 -- 172.21.15.32:0/3530230771 >> v2:172.21.15.55:3300/0 conn(0x7f86c40b6a90 msgr2=0x7f86c40acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:46.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c3fff700 1 --2- 172.21.15.32:0/3530230771 >> v2:172.21.15.55:3300/0 conn(0x7f86c40b6a90 0x7f86c40acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c3fff700 1 -- 172.21.15.32:0/3530230771 >> v1:172.21.15.55:6789/0 conn(0x7f86c4060aa0 legacy=0x7f86c40aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c3fff700 1 -- 172.21.15.32:0/3530230771 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40618b0 con 0x7f86c40aa760 2022-08-01T22:56:46.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.891+0000 7f86c3fff700 1 --2- 172.21.15.32:0/3530230771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40aa760 0x7f86c413c600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f86bc0032f0 tx=0x7f86bc0091d0).ready entity=mon.2 client_cookie=1ae50ff2fd62a16 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86c27fc700 1 -- 172.21.15.32:0/3530230771 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc00b600 con 0x7f86c40aa760 2022-08-01T22:56:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86c27fc700 1 -- 172.21.15.32:0/3530230771 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00b760 con 0x7f86c40aa760 2022-08-01T22:56:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86c27fc700 1 -- 172.21.15.32:0/3530230771 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc00b8d0 con 0x7f86c40aa760 2022-08-01T22:56:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 -- 172.21.15.32:0/3530230771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40aa760 msgr2=0x7f86c413c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 --2- 172.21.15.32:0/3530230771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40aa760 0x7f86c413c600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f86bc0032f0 tx=0x7f86bc0091d0).stop 2022-08-01T22:56:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 -- 172.21.15.32:0/3530230771 shutdown_connections 2022-08-01T22:56:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 --2- 172.21.15.32:0/3530230771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40aa760 0x7f86c413c600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 --2- 172.21.15.32:0/3530230771 >> v2:172.21.15.55:3300/0 conn(0x7f86c40b6a90 0x7f86c40acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 -- 172.21.15.32:0/3530230771 >> 172.21.15.32:0/3530230771 conn(0x7f86c40b3610 msgr2=0x7f86c40b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 -- 172.21.15.32:0/3530230771 shutdown_connections 2022-08-01T22:56:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 -- 172.21.15.32:0/3530230771 wait complete. 2022-08-01T22:56:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.895+0000 7f86ca227700 1 Processor -- start 2022-08-01T22:56:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 -- start start 2022-08-01T22:56:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 0x7f86c40ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c40aa760 0x7f86c40b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40b6a90 0x7f86c4064b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b03d0 con 0x7f86c4060aa0 2022-08-01T22:56:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40631c0 con 0x7f86c40aa760 2022-08-01T22:56:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40aeb70 con 0x7f86c40b6a90 2022-08-01T22:56:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 0x7f86c40ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 0x7f86c40ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42112/0 (socket says 172.21.15.32:42112) 2022-08-01T22:56:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 -- 172.21.15.32:0/1739882093 learned_addr learned my addr 172.21.15.32:0/1739882093 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c2ffd700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c40aa760 0x7f86c40b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c3fff700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40b6a90 0x7f86c4064b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 -- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40b6a90 msgr2=0x7f86c4064b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40b6a90 0x7f86c4064b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 -- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c40aa760 msgr2=0x7f86c40b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c40aa760 0x7f86c40b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 -- 172.21.15.32:0/1739882093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009470 con 0x7f86c4060aa0 2022-08-01T22:56:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c37fe700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 0x7f86c40ae450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f86b800c000 tx=0x7f86b8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86b8010780 con 0x7f86c4060aa0 2022-08-01T22:56:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80108e0 con 0x7f86c4060aa0 2022-08-01T22:56:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c4063420 con 0x7f86c4060aa0 2022-08-01T22:56:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86b8010c40 con 0x7f86c4060aa0 2022-08-01T22:56:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.899+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c40aedd0 con 0x7f86c4060aa0 2022-08-01T22:56:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.903+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c40ae280 con 0x7f86c4060aa0 2022-08-01T22:56:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.903+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f86b8026430 con 0x7f86c4060aa0 2022-08-01T22:56:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.903+0000 7f86c0ff9700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86a4053980 0x7f86a4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.903+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f86b806cf40 con 0x7f86c4060aa0 2022-08-01T22:56:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.903+0000 7f86c2ffd700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86a4053980 0x7f86a4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.903+0000 7f86c2ffd700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86a4053980 0x7f86a4055e40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f86b400b000 tx=0x7f86b4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:46.911+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86b8043ca0 con 0x7f86c4060aa0 2022-08-01T22:56:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.071+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86c40aef60 con 0x7f86c4060aa0 2022-08-01T22:56:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.071+0000 7f86c0ff9700 1 -- 172.21.15.32:0/1739882093 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f86b80472c0 con 0x7f86c4060aa0 2022-08-01T22:56:47.078 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:47.079 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:56:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86a4053980 msgr2=0x7f86a4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86a4053980 0x7f86a4055e40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f86b400b000 tx=0x7f86b4009040).stop 2022-08-01T22:56:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 msgr2=0x7f86c40ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 0x7f86c40ae450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f86b800c000 tx=0x7f86b8006df0).stop 2022-08-01T22:56:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 shutdown_connections 2022-08-01T22:56:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f86a4053980 0x7f86a4055e40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86c40b6a90 0x7f86c4064b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c40aa760 0x7f86c40b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 --2- 172.21.15.32:0/1739882093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86c4060aa0 0x7f86c40ae450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 >> 172.21.15.32:0/1739882093 conn(0x7f86c40b3610 msgr2=0x7f86c413b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 shutdown_connections 2022-08-01T22:56:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.075+0000 7f86ca227700 1 -- 172.21.15.32:0/1739882093 wait complete. 2022-08-01T22:56:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:56:47.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T22:56:47.190 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T22:56:47.191 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T22:56:47.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:56:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.291+0000 7f88e90cc700 1 Processor -- start 2022-08-01T22:56:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.291+0000 7f88e90cc700 1 -- start start 2022-08-01T22:56:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.291+0000 7f88e90cc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88e40b6420 0x7f88e4061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e40b60e0 con 0x7f88e40b41f0 2022-08-01T22:56:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e40b2100 con 0x7f88e4142ff0 2022-08-01T22:56:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e40b23a0 con 0x7f88e40b6420 2022-08-01T22:56:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88e40b41f0 0x7f88e4061af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57894/0 (socket says 172.21.15.32:57894) 2022-08-01T22:56:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 -- 172.21.15.32:0/4219957958 learned_addr learned my addr 172.21.15.32:0/4219957958 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:56:47.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e3fff700 1 --2- 172.21.15.32:0/4219957958 >> v2:172.21.15.55:3300/0 conn(0x7f88e40b6420 0x7f88e4061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720670421 0 0) 0x7f88cc001490 con 0x7f88e40b41f0 2022-08-01T22:56:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c8003260 con 0x7f88e40b41f0 2022-08-01T22:56:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753546231 0 0) 0x7f88e40b2100 con 0x7f88e4142ff0 2022-08-01T22:56:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88cc001490 con 0x7f88e4142ff0 2022-08-01T22:56:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400737030 0 0) 0x7f88c8003260 con 0x7f88e40b41f0 2022-08-01T22:56:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 >> v2:172.21.15.55:3300/0 conn(0x7f88e40b6420 msgr2=0x7f88e4061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 --2- 172.21.15.32:0/4219957958 >> v2:172.21.15.55:3300/0 conn(0x7f88e40b6420 0x7f88e4061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 >> v1:172.21.15.55:6789/0 conn(0x7f88e4142ff0 legacy=0x7f88e4143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e40b2640 con 0x7f88e40b41f0 2022-08-01T22:56:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f88cc001b90 con 0x7f88e40b41f0 2022-08-01T22:56:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 >> v1:172.21.15.32:6789/0 conn(0x7f88e40b41f0 legacy=0x7f88e4061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c80038e0 0x7f88c8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c8004500 0x7f88c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88c8004e70 0x7f88c800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40b23a0 con 0x7f88c80038e0 2022-08-01T22:56:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c800bcf0 con 0x7f88c8004500 2022-08-01T22:56:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c800beb0 con 0x7f88c8004e70 2022-08-01T22:56:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e3fff700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c80038e0 0x7f88c8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c8004500 0x7f88c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e88cb700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88c8004e70 0x7f88c800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 -- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88c8004e70 msgr2=0x7f88c800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88c8004e70 0x7f88c800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 -- 172.21.15.32:0/4219957958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c80038e0 msgr2=0x7f88c8003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c80038e0 0x7f88c8003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 -- 172.21.15.32:0/4219957958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8003260 con 0x7f88c8004500 2022-08-01T22:56:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e88cb700 1 -- 172.21.15.32:0/4219957958 reap_dead start 2022-08-01T22:56:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e37fe700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c8004500 0x7f88c8004980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f88c80054d0 tx=0x7f88e40ad630).ready entity=mon.1 client_cookie=6ac411c6b1c76ca3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88cc00cd60 con 0x7f88c8004500 2022-08-01T22:56:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc004960 con 0x7f88c8004500 2022-08-01T22:56:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e2ffd700 1 -- 172.21.15.32:0/4219957958 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88cc004cc0 con 0x7f88c8004500 2022-08-01T22:56:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c8004500 msgr2=0x7f88c8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c8004500 0x7f88c8004980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f88c80054d0 tx=0x7f88e40ad630).stop 2022-08-01T22:56:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- 172.21.15.32:0/4219957958 shutdown_connections 2022-08-01T22:56:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 --2- 172.21.15.32:0/4219957958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c8004500 0x7f88c8004980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- 172.21.15.32:0/4219957958 >> 172.21.15.32:0/4219957958 conn(0x7f88e405c620 msgr2=0x7f88e405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- 172.21.15.32:0/4219957958 shutdown_connections 2022-08-01T22:56:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.295+0000 7f88e90cc700 1 -- 172.21.15.32:0/4219957958 wait complete. 2022-08-01T22:56:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 Processor -- start 2022-08-01T22:56:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 -- start start 2022-08-01T22:56:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 0x7f88e40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88e40b0780 0x7f88e40b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88e40b9cb0 0x7f88e40ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40ae8c0 con 0x7f88c8004500 2022-08-01T22:56:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40ba160 con 0x7f88e40b9cb0 2022-08-01T22:56:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e90cc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4156750 con 0x7f88e40b0780 2022-08-01T22:56:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 0x7f88e40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 0x7f88e40b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42132/0 (socket says 172.21.15.32:42132) 2022-08-01T22:56:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 -- 172.21.15.32:0/3609613184 learned_addr learned my addr 172.21.15.32:0/3609613184 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e37fe700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88e40b0780 0x7f88e40b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e88cb700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88e40b9cb0 0x7f88e40ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 -- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88e40b0780 msgr2=0x7f88e40b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88e40b0780 0x7f88e40b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 -- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88e40b9cb0 msgr2=0x7f88e40ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88e40b9cb0 0x7f88e40ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 -- 172.21.15.32:0/3609613184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e41569f0 con 0x7f88c8004500 2022-08-01T22:56:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e88cb700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88e40b9cb0 0x7f88e40ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:56:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.299+0000 7f88e3fff700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 0x7f88e40b0290 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f88dc00b000 tx=0x7f88dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88dc013680 con 0x7f88c8004500 2022-08-01T22:56:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc000e90 con 0x7f88c8004500 2022-08-01T22:56:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e40aeaf0 con 0x7f88c8004500 2022-08-01T22:56:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88dc0139d0 con 0x7f88c8004500 2022-08-01T22:56:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e4145e20 con 0x7f88c8004500 2022-08-01T22:56:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e40b28e0 con 0x7f88c8004500 2022-08-01T22:56:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f88dc013b30 con 0x7f88c8004500 2022-08-01T22:56:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e17fa700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f88bc053b30 0x7f88bc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.303+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f88dc06c490 con 0x7f88c8004500 2022-08-01T22:56:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.307+0000 7f88e37fe700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f88bc053b30 0x7f88bc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.307+0000 7f88e37fe700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f88bc053b30 0x7f88bc055ff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f88cc017000 tx=0x7f88cc013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.311+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f88dc043250 con 0x7f88c8004500 2022-08-01T22:56:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.475+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88e40574e0 con 0x7f88c8004500 2022-08-01T22:56:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.475+0000 7f88e17fa700 1 -- 172.21.15.32:0/3609613184 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f88dc018020 con 0x7f88c8004500 2022-08-01T22:56:47.482 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:56:47.482 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:56:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f88bc053b30 msgr2=0x7f88bc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f88bc053b30 0x7f88bc055ff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f88cc017000 tx=0x7f88cc013000).stop 2022-08-01T22:56:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 msgr2=0x7f88e40b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 0x7f88e40b0290 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f88dc00b000 tx=0x7f88dc006df0).stop 2022-08-01T22:56:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 shutdown_connections 2022-08-01T22:56:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f88bc053b30 0x7f88bc055ff0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88e40b9cb0 0x7f88e40ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88e40b0780 0x7f88e40b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 --2- 172.21.15.32:0/3609613184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88c8004500 0x7f88e40b0290 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 >> 172.21.15.32:0/3609613184 conn(0x7f88e405c620 msgr2=0x7f88e4140e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 shutdown_connections 2022-08-01T22:56:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.479+0000 7f88e90cc700 1 -- 172.21.15.32:0/3609613184 wait complete. 2022-08-01T22:56:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:56:47.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 2022-08-01T22:56:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 Processor -- start 2022-08-01T22:56:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 -- start start 2022-08-01T22:56:47.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7ff00b2710 0x7f7ff014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ff00b01a0 0x7f7ff014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00afd60 0x7f7ff00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff00a7250 con 0x7f7ff00b01a0 2022-08-01T22:56:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff00a7480 con 0x7f7ff00b2710 2022-08-01T22:56:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff5389700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff00a7720 con 0x7f7ff00afd60 2022-08-01T22:56:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ff00b01a0 0x7f7ff014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ff00b01a0 0x7f7ff014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42142/0 (socket says 172.21.15.32:42142) 2022-08-01T22:56:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 -- 172.21.15.32:0/713039911 learned_addr learned my addr 172.21.15.32:0/713039911 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7feffff700 1 --2- 172.21.15.32:0/713039911 >> v2:172.21.15.55:3300/0 conn(0x7f7ff00b2710 0x7f7ff014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00afd60 0x7f7ff00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 -- 172.21.15.32:0/713039911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00afd60 msgr2=0x7f7ff00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00afd60 0x7f7ff00a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 -- 172.21.15.32:0/713039911 >> v2:172.21.15.55:3300/0 conn(0x7f7ff00b2710 msgr2=0x7f7ff014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/713039911 >> v2:172.21.15.55:3300/0 conn(0x7f7ff00b2710 0x7f7ff014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 -- 172.21.15.32:0/713039911 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff00a79c0 con 0x7f7ff00b01a0 2022-08-01T22:56:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00afd60 0x7f7ff00a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00b01a0 0x7f7ff014b4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f7fe00032f0 tx=0x7f7fe0009290).ready entity=mon.0 client_cookie=cf406913de6700dc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7feeffd700 1 -- 172.21.15.32:0/713039911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe000b870 con 0x7f7ff00b01a0 2022-08-01T22:56:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7feeffd700 1 -- 172.21.15.32:0/713039911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe000b9d0 con 0x7f7ff00b01a0 2022-08-01T22:56:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.607+0000 7f7feeffd700 1 -- 172.21.15.32:0/713039911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe000bb40 con 0x7f7ff00b01a0 2022-08-01T22:56:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 -- 172.21.15.32:0/713039911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00b01a0 msgr2=0x7f7ff014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00b01a0 0x7f7ff014b4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f7fe00032f0 tx=0x7f7fe0009290).stop 2022-08-01T22:56:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 -- 172.21.15.32:0/713039911 shutdown_connections 2022-08-01T22:56:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00b01a0 0x7f7ff014b4c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 --2- 172.21.15.32:0/713039911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00afd60 0x7f7ff00a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 --2- 172.21.15.32:0/713039911 >> v2:172.21.15.55:3300/0 conn(0x7f7ff00b2710 0x7f7ff014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 -- 172.21.15.32:0/713039911 >> 172.21.15.32:0/713039911 conn(0x7f7ff0060310 msgr2=0x7f7ff00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 -- 172.21.15.32:0/713039911 shutdown_connections 2022-08-01T22:56:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 -- 172.21.15.32:0/713039911 wait complete. 2022-08-01T22:56:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 Processor -- start 2022-08-01T22:56:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.611+0000 7f7ff5389700 1 -- start start 2022-08-01T22:56:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 0x7f7ff00aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00b2710 0x7f7ff014aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ff00abc00 0x7f7ff0157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff014b530 con 0x7f7ff00afd60 2022-08-01T22:56:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0156810 con 0x7f7ff00abc00 2022-08-01T22:56:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff00aa750 con 0x7f7ff00b2710 2022-08-01T22:56:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 0x7f7ff00aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 0x7f7ff00aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42146/0 (socket says 172.21.15.32:42146) 2022-08-01T22:56:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 -- 172.21.15.32:0/4152089831 learned_addr learned my addr 172.21.15.32:0/4152089831 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00b2710 0x7f7ff014aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ff00abc00 0x7f7ff0157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 -- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00b2710 msgr2=0x7f7ff014aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00b2710 0x7f7ff014aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 -- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ff00abc00 msgr2=0x7f7ff0157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ff00abc00 0x7f7ff0157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 -- 172.21.15.32:0/4152089831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0009530 con 0x7f7ff00afd60 2022-08-01T22:56:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ff00abc00 0x7f7ff0157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00b2710 0x7f7ff014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7feffff700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 0x7f7ff00aa260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f7fd800c000 tx=0x7f7fd8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7fed7fa700 1 -- 172.21.15.32:0/4152089831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fd8010780 con 0x7f7ff00afd60 2022-08-01T22:56:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7fed7fa700 1 -- 172.21.15.32:0/4152089831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd80108e0 con 0x7f7ff00afd60 2022-08-01T22:56:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff00ac110 con 0x7f7ff00afd60 2022-08-01T22:56:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7fed7fa700 1 -- 172.21.15.32:0/4152089831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fd8010c40 con 0x7f7ff00afd60 2022-08-01T22:56:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff0156ca0 con 0x7f7ff00afd60 2022-08-01T22:56:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.615+0000 7f7fed7fa700 1 -- 172.21.15.32:0/4152089831 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7fd801c460 con 0x7f7ff00afd60 2022-08-01T22:56:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f7fed7fa700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7fc8053950 0x7f7fc8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f7fed7fa700 1 -- 172.21.15.32:0/4152089831 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7fd806c4a0 con 0x7f7ff00afd60 2022-08-01T22:56:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7fc8053950 0x7f7fc8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7fc8053950 0x7f7fc8055e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f7fe0018000 tx=0x7f7fe0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 Processor -- start 2022-08-01T22:56:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 -- start start 2022-08-01T22:56:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1335e00 0x1334e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13361e0 0x1335380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1368920 0x1368dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1293320 con 0x13361e0 2022-08-01T22:56:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x12a6c50 con 0x1335e00 2022-08-01T22:56:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x12871f0 con 0x1368920 2022-08-01T22:56:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.619+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13361e0 0x1335380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1335e00 0x1334e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13361e0 0x1335380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42156/0 (socket says 172.21.15.32:42156) 2022-08-01T22:56:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7fef7fe700 1 -- 172.21.15.32:0/1574649560 learned_addr learned my addr 172.21.15.32:0/1574649560 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1368920 0x1368dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 -- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1368920 msgr2=0x1368dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1368920 0x1368dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 -- 172.21.15.32:0/1574649560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13361e0 msgr2=0x1335380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13361e0 0x1335380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 -- 172.21.15.32:0/1574649560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12deec0 con 0x1335e00 2022-08-01T22:56:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1368920 0x1368dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7feffff700 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1335e00 0x1334e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x1297d80 tx=0x7f7fd8014000).ready entity=mon.1 client_cookie=a4f6afd2eb4a40bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7ff5389700 1 -- 172.21.15.32:0/1574649560 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fd8045260 con 0x1335e00 2022-08-01T22:56:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7ff5389700 1 -- 172.21.15.32:0/1574649560 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800fd50 con 0x1335e00 2022-08-01T22:56:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f7ff5389700 1 -- 172.21.15.32:0/1574649560 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fd806c0d0 con 0x1335e00 2022-08-01T22:56:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 -- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1335e00 msgr2=0x1334e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1335e00 0x1334e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x1297d80 tx=0x7f7fd8014000).stop 2022-08-01T22:56:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 -- 172.21.15.32:0/1574649560 shutdown_connections 2022-08-01T22:56:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1368920 0x1368dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13361e0 0x1335380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 --2- 172.21.15.32:0/1574649560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1335e00 0x1334e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 -- 172.21.15.32:0/1574649560 >> 172.21.15.32:0/1574649560 conn(0x1333180 msgr2=0x1333e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 -- 172.21.15.32:0/1574649560 shutdown_connections 2022-08-01T22:56:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 -- 172.21.15.32:0/1574649560 wait complete. 2022-08-01T22:56:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.623+0000 7f8000842740 1 -- start start 2022-08-01T22:56:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 Processor -- start 2022-08-01T22:56:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x13361e0 0x1367410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1368920 0x1367900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 0x1339c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x1367df0 con 0x136f740 2022-08-01T22:56:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x136c010 con 0x1368920 2022-08-01T22:56:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x137b830 con 0x13361e0 2022-08-01T22:56:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 10 client.0 ll_register_callbacks cb 0x128c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T22:56:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 0x1339c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 0x1339c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42164/0 (socket says 172.21.15.32:42164) 2022-08-01T22:56:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7feffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x13361e0 0x1367410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 -- 172.21.15.32:0/3948857466 learned_addr learned my addr 172.21.15.32:0/3948857466 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1368920 0x1367900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 -- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x13361e0 msgr2=0x1367410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x13361e0 0x1367410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 -- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1368920 msgr2=0x1367900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1368920 0x1367900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 -- 172.21.15.32:0/3948857466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff0156ca0 con 0x136f740 2022-08-01T22:56:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 0x1339c50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7fe800c000 tx=0x7f7fe8008040).ready entity=mon.0 client_cookie=f95fa68407debb3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7fdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T22:56:47.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7fdeffd700 1 -- 172.21.15.32:0/3948857466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe8011d70 con 0x136f740 2022-08-01T22:56:47.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7fdeffd700 1 -- 172.21.15.32:0/3948857466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7fe8015480 con 0x136f740 2022-08-01T22:56:47.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x129cc70 con 0x136f740 2022-08-01T22:56:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f7fdeffd700 1 -- 172.21.15.32:0/3948857466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe8011ed0 con 0x136f740 2022-08-01T22:56:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 10 client.5112 fetch_fsmap learned FSMap version 12 2022-08-01T22:56:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.627+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x1279950 con 0x136f740 2022-08-01T22:56:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7fdeffd700 1 -- 172.21.15.32:0/3948857466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x129cc70 con 0x136f740 2022-08-01T22:56:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7fdeffd700 1 -- 172.21.15.32:0/3948857466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 12) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f7fe8013040 con 0x136f740 2022-08-01T22:56:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 10 client.5112 fetch_fsmap finished waiting for FSMap version 12 2022-08-01T22:56:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 10 client.5112 resolve_mds: resolved ID 'a' to GID 4863 2022-08-01T22:56:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 20 client.5112 populate_metadata read hostname 'smithi032' 2022-08-01T22:56:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x133b0f0 0x137e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 4 client.5112 mds_command: new command op to 4863 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T22:56:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 --> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1294fe0 con 0x133b0f0 2022-08-01T22:56:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7feffff700 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x133b0f0 0x137e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7fdeffd700 10 client.5112 ms_handle_connect on v2:172.21.15.32:6832/3560980402 2022-08-01T22:56:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7feffff700 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x133b0f0 0x137e200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6b9bd48dd8d1d1ac server_cookie=26e06c6894597ae2 in_seq=0 out_seq=0 2022-08-01T22:56:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7fdeffd700 1 -- 172.21.15.32:0/3948857466 <== mds.0 v2:172.21.15.32:6832/3560980402 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1294fe0 con 0x133b0f0 2022-08-01T22:56:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7fdeffd700 10 client.5112 handle_command_reply: tid=0 2022-08-01T22:56:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 client.5112 shutdown 2022-08-01T22:56:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 msgr2=0x1339c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 0x1339c50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7fe800c000 tx=0x7f7fe8008040).stop 2022-08-01T22:56:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 shutdown_connections 2022-08-01T22:56:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x133b0f0 0x137e200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x136f740 0x1339c50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x1368920 0x1367900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 --2- 172.21.15.32:0/3948857466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x13361e0 0x1367410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f7fdeffd700 0 client.5112 ms_handle_reset on v2:172.21.15.32:6832/3560980402 2022-08-01T22:56:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 >> 172.21.15.32:0/3948857466 conn(0x1335e00 msgr2=0x13368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 shutdown_connections 2022-08-01T22:56:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 1 -- 172.21.15.32:0/3948857466 wait complete. 2022-08-01T22:56:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.631+0000 7f8000842740 20 client.5112 trim_cache size 0 max 16384 2022-08-01T22:56:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 Processor -- start 2022-08-01T22:56:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 -- start start 2022-08-01T22:56:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x13671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x1334630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x133c0d0 con 0x1368920 2022-08-01T22:56:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x133b880 con 0x13361e0 2022-08-01T22:56:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x129cc70 con 0x136f740 2022-08-01T22:56:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42172/0 (socket says 172.21.15.32:42172) 2022-08-01T22:56:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 -- 172.21.15.32:0/4030335423 learned_addr learned my addr 172.21.15.32:0/4030335423 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x1334630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7feffff700 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x13671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 -- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 msgr2=0x1334630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x1334630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 -- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 msgr2=0x13671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x13671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 -- 172.21.15.32:0/4030335423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1296550 con 0x1368920 2022-08-01T22:56:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x1334630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:56:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13676d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x133aef0 tx=0x7f7fe001e000).ready entity=mon.0 client_cookie=527b70b7025d951a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fe57fa700 1 -- 172.21.15.32:0/4030335423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe00209a0 con 0x1368920 2022-08-01T22:56:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fe57fa700 1 -- 172.21.15.32:0/4030335423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe0020b00 con 0x1368920 2022-08-01T22:56:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.647+0000 7f7fe57fa700 1 -- 172.21.15.32:0/4030335423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe0020e60 con 0x1368920 2022-08-01T22:56:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- 172.21.15.32:0/4030335423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 msgr2=0x13676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:56:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13676d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x133aef0 tx=0x7f7fe001e000).stop 2022-08-01T22:56:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- 172.21.15.32:0/4030335423 shutdown_connections 2022-08-01T22:56:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x1334630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13676d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- 172.21.15.32:0/4030335423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x13671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- 172.21.15.32:0/4030335423 >> 172.21.15.32:0/4030335423 conn(0x1335e00 msgr2=0x1369c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:56:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- 172.21.15.32:0/4030335423 shutdown_connections 2022-08-01T22:56:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- 172.21.15.32:0/4030335423 wait complete. 2022-08-01T22:56:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- start start 2022-08-01T22:56:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 Processor -- start 2022-08-01T22:56:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x136e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x13d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x136e6a0 con 0x1368920 2022-08-01T22:56:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x1373680 con 0x13361e0 2022-08-01T22:56:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x136c230 con 0x136f740 2022-08-01T22:56:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f8000842740 10 client.0 ll_register_callbacks cb 0x137b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T22:56:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42176/0 (socket says 172.21.15.32:42176) 2022-08-01T22:56:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f7fef7fe700 1 -- 172.21.15.32:0/1728999123 learned_addr learned my addr 172.21.15.32:0/1728999123 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:56:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f7feffff700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x136e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.651+0000 7f7ff4b88700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x13d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fef7fe700 1 -- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 msgr2=0x13d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x13d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fef7fe700 1 -- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 msgr2=0x136e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:56:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x136e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:56:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fef7fe700 1 -- 172.21.15.32:0/1728999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff00a79c0 con 0x1368920 2022-08-01T22:56:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fef7fe700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13ce570 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x13731e0 tx=0x7f7fe002d000).ready entity=mon.0 client_cookie=a0ebdf8db796b9a8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:56:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T22:56:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 1 -- 172.21.15.32:0/1728999123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fe003b070 con 0x1368920 2022-08-01T22:56:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 1 -- 172.21.15.32:0/1728999123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7fe0038dc0 con 0x1368920 2022-08-01T22:56:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fd800fd50 con 0x1368920 2022-08-01T22:56:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 1 -- 172.21.15.32:0/1728999123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe00209a0 con 0x1368920 2022-08-01T22:56:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 10 client.5118 fetch_fsmap learned FSMap version 12 2022-08-01T22:56:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x129ee50 con 0x1368920 2022-08-01T22:56:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 1 -- 172.21.15.32:0/1728999123 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800fd50 con 0x1368920 2022-08-01T22:56:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 1 -- 172.21.15.32:0/1728999123 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 12) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f7fe002eb50 con 0x1368920 2022-08-01T22:56:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 10 client.5118 fetch_fsmap finished waiting for FSMap version 12 2022-08-01T22:56:47.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 10 client.5118 resolve_mds: resolved ID 'a' to GID 4863 2022-08-01T22:56:47.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 20 client.5118 populate_metadata read hostname 'smithi032' 2022-08-01T22:56:47.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x13d2030 0x13d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:56:47.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 4 client.5118 mds_command: new command op to 4863 tid=0[{"prefix": "cache drop"}] 2022-08-01T22:56:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 --> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x118a590 con 0x13d2030 2022-08-01T22:56:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7feffff700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x13d2030 0x13d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:56:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7fdffff700 10 client.5118 ms_handle_connect on v2:172.21.15.32:6832/3560980402 2022-08-01T22:56:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:56:47.655+0000 7f7feffff700 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x13d2030 0x13d2410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=65b3f79bfdbc17a9 server_cookie=e9dd620fe3304aa6 in_seq=0 out_seq=0 2022-08-01T22:56:53.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:53.331 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:53.334 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:56:53.337 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:23.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:23.389 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:23.394 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:23.397 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7fdffff700 1 -- 172.21.15.32:0/1728999123 <== mds.0 v2:172.21.15.32:6832/3560980402 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x118a590 con 0x13d2030 2022-08-01T22:57:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7fdffff700 10 client.5118 handle_command_reply: tid=0 2022-08-01T22:57:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 client.5118 shutdown 2022-08-01T22:57:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 msgr2=0x13ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13ce570 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x13731e0 tx=0x7f7fe002d000).stop 2022-08-01T22:57:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 shutdown_connections 2022-08-01T22:57:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:6832/3560980402,v1:172.21.15.32:6833/3560980402] conn(0x13d2030 0x13d2410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x136f740 0x13d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1368920 0x13ce570 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 --2- 172.21.15.32:0/1728999123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x13361e0 0x136e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 >> 172.21.15.32:0/1728999123 conn(0x1335e00 msgr2=0x1333570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7fdffff700 0 client.5118 ms_handle_reset on v2:172.21.15.32:6832/3560980402 2022-08-01T22:57:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 shutdown_connections 2022-08-01T22:57:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 1 -- 172.21.15.32:0/1728999123 wait complete. 2022-08-01T22:57:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f8000842740 20 client.5118 trim_cache size 0 max 16384 2022-08-01T22:57:32.012 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:57:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2022-08-01T22:57:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2022-08-01T22:57:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(0) Success", 2022-08-01T22:57:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 920 2022-08-01T22:57:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2022-08-01T22:57:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2022-08-01T22:57:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2022-08-01T22:57:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2022-08-01T22:57:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 900 2022-08-01T22:57:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2022-08-01T22:57:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2022-08-01T22:57:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 2537, 2022-08-01T22:57:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 432016 2022-08-01T22:57:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 44.346182263999999 2022-08-01T22:57:32.017 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:57:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7fc8053950 msgr2=0x7f7fc8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7fc8053950 0x7f7fc8055e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f7fe0018000 tx=0x7f7fe0016000).stop 2022-08-01T22:57:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 msgr2=0x7f7ff00aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 0x7f7ff00aa260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f7fd800c000 tx=0x7f7fd8006df0).stop 2022-08-01T22:57:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 shutdown_connections 2022-08-01T22:57:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7fc8053950 0x7f7fc8055e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ff00abc00 0x7f7ff0157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7ff00b2710 0x7f7ff014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 --2- 172.21.15.32:0/4152089831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ff00afd60 0x7f7ff00aa260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 >> 172.21.15.32:0/4152089831 conn(0x7f7ff0060310 msgr2=0x7f7ff00ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 shutdown_connections 2022-08-01T22:57:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.003+0000 7f7ff5389700 1 -- 172.21.15.32:0/4152089831 wait complete. 2022-08-01T22:57:32.023 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T22:57:32.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:32.041 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:32.064 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:32.088 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:32.110 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T22:57:32.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:32.111 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:32.111 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:32.121 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:57:32.123 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:57:32.124 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:57:32.125 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T22:57:32.125 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:32.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22430]: fuse finished with error 0 and tester_r 0 2022-08-01T22:57:32.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:32.144 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:32.145 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:32.234 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T22:57:32.236 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T22:57:32.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:57:32.237 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:32.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:32.245 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:32.245 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:57:32.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:57:32.257 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T22:57:32.257 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T22:57:32.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:57:32.259 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:57:32.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:57:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T22:57:32.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T22:57:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02c927700 1 Processor -- start 2022-08-01T22:57:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02c927700 1 -- start start 2022-08-01T22:57:32.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02c927700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0280b0d10 0x7ff028146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02c927700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0280b12d0 con 0x7ff0280b0560 2022-08-01T22:57:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02c927700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0280a9700 con 0x7ff0280a9320 2022-08-01T22:57:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02c927700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0280b1500 con 0x7ff0280b0d10 2022-08-01T22:57:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0280b0d10 0x7ff028146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02759e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0280b0560 0x7ff028140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57962/0 (socket says 172.21.15.32:57962) 2022-08-01T22:57:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.375+0000 7ff02759e700 1 -- 172.21.15.32:0/989270455 learned_addr learned my addr 172.21.15.32:0/989270455 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 -- 172.21.15.32:0/989270455 >> v1:172.21.15.55:6789/0 conn(0x7ff0280a9320 legacy=0x7ff0281466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 -- 172.21.15.32:0/989270455 >> v1:172.21.15.32:6789/0 conn(0x7ff0280b0560 legacy=0x7ff028140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 -- 172.21.15.32:0/989270455 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280b17a0 con 0x7ff0280b0d10 2022-08-01T22:57:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 --2- 172.21.15.32:0/989270455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0d10 0x7ff028146df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018006df0).ready entity=mon.0 client_cookie=ffd87651b9220246 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:32.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff025d9b700 1 -- 172.21.15.32:0/989270455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01800d610 con 0x7ff0280b0d10 2022-08-01T22:57:32.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff025d9b700 1 -- 172.21.15.32:0/989270455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018000e90 con 0x7ff0280b0d10 2022-08-01T22:57:32.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff025d9b700 1 -- 172.21.15.32:0/989270455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01800d9a0 con 0x7ff0280b0d10 2022-08-01T22:57:32.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- 172.21.15.32:0/989270455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0d10 msgr2=0x7ff028146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 --2- 172.21.15.32:0/989270455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0d10 0x7ff028146df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018006df0).stop 2022-08-01T22:57:32.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- 172.21.15.32:0/989270455 shutdown_connections 2022-08-01T22:57:32.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 --2- 172.21.15.32:0/989270455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0d10 0x7ff028146df0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- 172.21.15.32:0/989270455 >> 172.21.15.32:0/989270455 conn(0x7ff0280612d0 msgr2=0x7ff028062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:32.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- 172.21.15.32:0/989270455 shutdown_connections 2022-08-01T22:57:32.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- 172.21.15.32:0/989270455 wait complete. 2022-08-01T22:57:32.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 Processor -- start 2022-08-01T22:57:32.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- start start 2022-08-01T22:57:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0280a9320 0x7ff0280b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 0x7ff02813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0280b0d10 0x7ff0280b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280b59b0 con 0x7ff0280b0560 2022-08-01T22:57:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0281463c0 con 0x7ff0280b0d10 2022-08-01T22:57:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02c927700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280b5be0 con 0x7ff0280a9320 2022-08-01T22:57:32.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 0x7ff02813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 0x7ff02813f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42192/0 (socket says 172.21.15.32:42192) 2022-08-01T22:57:32.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.379+0000 7ff02659c700 1 -- 172.21.15.32:0/2860982728 learned_addr learned my addr 172.21.15.32:0/2860982728 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:32.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02759e700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0280b0d10 0x7ff0280b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff026d9d700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0280a9320 0x7ff0280b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02659c700 1 -- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0280a9320 msgr2=0x7ff0280b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02659c700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0280a9320 0x7ff0280b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02659c700 1 -- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0280b0d10 msgr2=0x7ff0280b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02659c700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0280b0d10 0x7ff0280b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02659c700 1 -- 172.21.15.32:0/2860982728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01800b180 con 0x7ff0280b0560 2022-08-01T22:57:32.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02659c700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 0x7ff02813f6b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:32.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff018016070 con 0x7ff0280b0560 2022-08-01T22:57:32.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01800d990 con 0x7ff0280b0560 2022-08-01T22:57:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0280b5e80 con 0x7ff0280b0560 2022-08-01T22:57:32.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff018013de0 con 0x7ff0280b0560 2022-08-01T22:57:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0280ab1e0 con 0x7ff0280b0560 2022-08-01T22:57:32.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0280574e0 con 0x7ff0280b0560 2022-08-01T22:57:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff018025030 con 0x7ff0280b0560 2022-08-01T22:57:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.383+0000 7ff01ffff700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff00c0539e0 0x7ff00c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.387+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff01806d090 con 0x7ff0280b0560 2022-08-01T22:57:32.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.387+0000 7ff026d9d700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff00c0539e0 0x7ff00c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.387+0000 7ff026d9d700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff00c0539e0 0x7ff00c055ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7ff01000d000 tx=0x7ff010007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:32.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.391+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff018043e50 con 0x7ff0280b0560 2022-08-01T22:57:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.523+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff028147590 con 0x7ff0280b0560 2022-08-01T22:57:32.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff01ffff700 1 -- 172.21.15.32:0/2860982728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v35) v1 ==== 81+0+34037 (secure 0 0 0) 0x7ff018023090 con 0x7ff0280b0560 2022-08-01T22:57:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:32.532 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:57:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 35, 2022-08-01T22:57:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T22:57:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T22:57:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T22:56:29.422501+0000", 2022-08-01T22:57:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:57:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:57:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T22:57:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T22:57:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T22:57:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T22:57:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T22:57:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T22:57:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T22:57:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T22:57:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T22:57:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T22:57:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T22:57:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T22:57:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 6, 2022-08-01T22:57:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T22:57:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T22:57:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T22:57:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T22:57:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T22:57:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T22:57:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T22:57:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T22:57:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:57:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:57:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:57:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:57:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:57:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:57:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:57:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:57:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T22:57:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T22:57:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T22:57:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T22:57:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T22:57:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:57:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:57:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:57:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:57:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:57:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:57:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:57:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T22:57:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:57:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:57:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:57:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:57:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:57:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:57:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:57:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:57:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:57:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:57:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:57:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:57:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:57:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:57:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:57:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:57:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:57:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:57:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:57:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:57:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:57:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:57:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:57:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:57:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:57:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:57:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:57:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:57:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:57:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:57:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:57:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:57:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:57:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:57:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:57:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:57:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:57:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:57:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T22:57:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:57:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T22:57:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T22:57:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T22:57:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T22:57:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:57:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:57:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:57:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:57:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:57:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:57:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:57:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:57:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:57:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:57:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:57:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:57:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:57:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:57:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:57:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:57:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:57:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:57:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:57:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:57:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T22:57:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:57:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:57:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:57:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:57:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:57:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:57:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:57:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:57:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:57:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:57:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:57:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:57:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:57:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:57:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:57:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:57:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:57:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:57:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:57:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:57:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:57:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:57:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:57:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:57:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:57:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:57:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:57:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:57:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:57:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:57:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:57:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:57:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:57:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:57:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:57:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:57:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:57:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:57:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:57:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T22:57:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 5, 2022-08-01T22:57:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T22:57:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:56:26.319594+0000", 2022-08-01T22:57:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:57:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:57:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:57:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:57:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:57:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:57:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:57:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:57:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:57:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:57:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:57:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:57:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:57:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:57:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:57:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:57:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:57:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:57:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:57:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:57:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "34", 2022-08-01T22:57:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:57:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:57:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:57:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:57:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:57:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:57:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:57:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:57:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:57:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:57:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:57:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:57:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:57:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:57:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:57:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:57:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:57:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:57:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:57:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:57:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:57:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:57:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:57:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:57:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:57:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:57:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:57:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:57:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:57:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:57:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:57:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:57:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:57:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:57:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:57:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:57:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:57:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:57:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T22:57:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T22:57:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T22:57:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:57:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T22:57:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T22:57:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 6, 2022-08-01T22:57:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T22:57:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:56:27.441691+0000", 2022-08-01T22:57:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:57:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:57:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:57:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:57:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:57:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:57:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:57:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:57:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:57:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:57:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:57:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:57:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:57:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:57:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:57:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:57:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:57:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:57:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:57:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:57:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "35", 2022-08-01T22:57:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:57:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:57:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:57:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:57:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:57:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:57:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:57:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:57:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:57:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:57:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:57:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:57:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:57:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:57:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:57:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:57:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:57:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:57:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:57:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:57:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:57:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:57:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:57:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:57:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:57:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:57:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:57:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:57:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:57:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:57:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:57:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:57:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:57:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:57:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:57:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:57:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:57:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:57:32.612 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:57:32.612 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T22:57:32.612 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T22:57:32.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T22:57:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:57:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T22:57:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T22:57:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T22:57:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T22:57:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T22:57:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T22:57:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T22:57:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T22:57:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T22:57:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T22:57:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T22:57:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T22:57:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T22:57:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:57:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T22:57:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T22:57:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T22:57:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T22:57:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T22:57:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T22:57:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T22:57:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T22:57:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T22:57:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T22:57:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T22:57:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T22:57:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T22:57:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:57:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T22:57:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2022-08-01T22:57:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T22:57:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T22:57:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T22:57:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T22:57:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T22:57:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T22:57:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T22:57:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T22:57:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T22:57:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T22:57:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T22:57:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T22:57:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:57:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T22:57:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T22:57:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T22:57:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T22:57:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T22:57:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T22:57:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T22:57:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T22:57:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T22:57:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T22:57:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T22:57:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T22:57:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T22:57:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:57:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T22:57:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T22:57:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T22:57:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T22:57:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T22:57:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T22:57:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T22:57:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T22:57:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T22:57:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T22:57:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T22:57:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T22:57:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T22:57:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:57:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T22:57:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T22:57:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T22:57:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T22:57:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T22:57:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T22:57:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T22:57:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T22:57:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T22:57:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T22:57:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T22:57:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T22:57:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T22:57:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:57:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T22:57:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2022-08-01T22:57:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T22:57:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T22:57:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T22:57:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T22:57:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T22:57:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T22:57:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T22:57:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T22:57:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T22:57:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T22:57:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T22:57:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T22:57:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:57:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T22:57:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T22:57:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T22:57:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T22:57:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T22:57:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T22:57:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T22:57:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T22:57:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T22:57:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T22:57:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T22:57:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T22:57:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T22:57:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T22:57:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:57:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T22:57:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:57:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T22:57:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:57:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T22:57:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:57:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T22:57:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:57:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T22:57:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:57:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T22:57:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:57:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T22:57:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:57:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T22:57:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T22:57:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T22:57:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T22:57:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T22:57:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T22:57:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T22:57:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T22:57:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T22:57:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:57:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T22:57:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T22:57:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:57:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T22:57:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:57:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:57:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T22:57:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T22:57:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T22:57:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:57:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:57:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T22:57:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:57:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:57:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T22:57:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T22:57:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T22:57:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T22:57:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T22:57:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T22:57:32.823 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:57:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff00c0539e0 msgr2=0x7ff00c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff00c0539e0 0x7ff00c055ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7ff01000d000 tx=0x7ff010007b70).stop 2022-08-01T22:57:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 msgr2=0x7ff02813f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 0x7ff02813f6b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018014000).stop 2022-08-01T22:57:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 shutdown_connections 2022-08-01T22:57:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff00c0539e0 0x7ff00c055ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0280b0d10 0x7ff0280b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0280b0560 0x7ff02813f6b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 --2- 172.21.15.32:0/2860982728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0280a9320 0x7ff0280b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 >> 172.21.15.32:0/2860982728 conn(0x7ff0280612d0 msgr2=0x7ff028141d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 shutdown_connections 2022-08-01T22:57:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.527+0000 7ff02c927700 1 -- 172.21.15.32:0/2860982728 wait complete. 2022-08-01T22:57:32.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1359e700 1 Processor -- start 2022-08-01T22:57:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1359e700 1 -- start start 2022-08-01T22:57:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b01a0 0x7f8d140a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d140a7250 con 0x7f8d140afd60 2022-08-01T22:57:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d140a7480 con 0x7f8d140b2710 2022-08-01T22:57:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140a7720 con 0x7f8d140b01a0 2022-08-01T22:57:32.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d12d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8d140afd60 0x7f8d14151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57976/0 (socket says 172.21.15.32:57976) 2022-08-01T22:57:32.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d12d9d700 1 -- 172.21.15.32:0/3709478480 learned_addr learned my addr 172.21.15.32:0/3709478480 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:32.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d11d9b700 1 --2- 172.21.15.32:0/3709478480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b01a0 0x7f8d140a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1159a700 1 -- 172.21.15.32:0/3709478480 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615444569 0 0) 0x7f8d140a7250 con 0x7f8d140afd60 2022-08-01T22:57:32.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1159a700 1 -- 172.21.15.32:0/3709478480 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf8003260 con 0x7f8d140afd60 2022-08-01T22:57:32.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d11d9b700 1 -- 172.21.15.32:0/3709478480 >> v1:172.21.15.55:6789/0 conn(0x7f8d140b2710 legacy=0x7f8d1414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d11d9b700 1 -- 172.21.15.32:0/3709478480 >> v1:172.21.15.32:6789/0 conn(0x7f8d140afd60 legacy=0x7f8d14151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d11d9b700 1 -- 172.21.15.32:0/3709478480 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140a79c0 con 0x7f8d140b01a0 2022-08-01T22:57:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d11d9b700 1 --2- 172.21.15.32:0/3709478480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b01a0 0x7f8d140a8540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8d040032f0 tx=0x7f8d040091d0).ready entity=mon.2 client_cookie=2dc3bbe363ba6c81 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1159a700 1 -- 172.21.15.32:0/3709478480 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d0400b6c0 con 0x7f8d140b01a0 2022-08-01T22:57:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1159a700 1 -- 172.21.15.32:0/3709478480 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0400b820 con 0x7f8d140b01a0 2022-08-01T22:57:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.939+0000 7f8d1159a700 1 -- 172.21.15.32:0/3709478480 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d0400b990 con 0x7f8d140b01a0 2022-08-01T22:57:32.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 -- 172.21.15.32:0/3709478480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b01a0 msgr2=0x7f8d140a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 --2- 172.21.15.32:0/3709478480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b01a0 0x7f8d140a8540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8d040032f0 tx=0x7f8d040091d0).stop 2022-08-01T22:57:32.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 -- 172.21.15.32:0/3709478480 shutdown_connections 2022-08-01T22:57:32.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 --2- 172.21.15.32:0/3709478480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b01a0 0x7f8d140a8540 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 -- 172.21.15.32:0/3709478480 >> 172.21.15.32:0/3709478480 conn(0x7f8d14060310 msgr2=0x7f8d140b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:32.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 -- 172.21.15.32:0/3709478480 shutdown_connections 2022-08-01T22:57:32.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 -- 172.21.15.32:0/3709478480 wait complete. 2022-08-01T22:57:32.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 Processor -- start 2022-08-01T22:57:32.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.943+0000 7f8d1359e700 1 -- start start 2022-08-01T22:57:32.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 0x7f8d141457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140b01a0 0x7f8d141415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b2710 0x7f8d14141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140af870 con 0x7f8d140afd60 2022-08-01T22:57:32.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1415ad00 con 0x7f8d140b01a0 2022-08-01T22:57:32.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140b3390 con 0x7f8d140b2710 2022-08-01T22:57:32.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 0x7f8d141457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 0x7f8d141457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42206/0 (socket says 172.21.15.32:42206) 2022-08-01T22:57:32.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 -- 172.21.15.32:0/1550187458 learned_addr learned my addr 172.21.15.32:0/1550187458 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d12d9d700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b2710 0x7f8d14141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d11d9b700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140b01a0 0x7f8d141415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 -- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b2710 msgr2=0x7f8d14141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b2710 0x7f8d14141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 -- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140b01a0 msgr2=0x7f8d141415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140b01a0 0x7f8d141415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 -- 172.21.15.32:0/1550187458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04009470 con 0x7f8d140afd60 2022-08-01T22:57:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1259c700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 0x7f8d141457b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f8d14062040 tx=0x7f8d08007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d08007e70 con 0x7f8d140afd60 2022-08-01T22:57:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08007030 con 0x7f8d140afd60 2022-08-01T22:57:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d08007390 con 0x7f8d140afd60 2022-08-01T22:57:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d140b3630 con 0x7f8d140afd60 2022-08-01T22:57:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d140af2b0 con 0x7f8d140afd60 2022-08-01T22:57:32.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1415aec0 con 0x7f8d140afd60 2022-08-01T22:57:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.947+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d08010030 con 0x7f8d140afd60 2022-08-01T22:57:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.951+0000 7f8d037fe700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf0053a10 0x7f8cf0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.951+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8d0806bc20 con 0x7f8d140afd60 2022-08-01T22:57:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.951+0000 7f8d11d9b700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf0053a10 0x7f8cf0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.951+0000 7f8d11d9b700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf0053a10 0x7f8cf0055ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f8d0401b000 tx=0x7f8d04016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:32.959+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d080428d0 con 0x7f8d140afd60 2022-08-01T22:57:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.119+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d140574e0 con 0x7f8d140afd60 2022-08-01T22:57:33.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.119+0000 7f8d037fe700 1 -- 172.21.15.32:0/1550187458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f8d08045ef0 con 0x7f8d140afd60 2022-08-01T22:57:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:33.126 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:56:29.452004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6833/3560980402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3560980402},{"type":"v1","addr":"172.21.15.32:6833","nonce":3560980402}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:57:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf0053a10 msgr2=0x7f8cf0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf0053a10 0x7f8cf0055ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f8d0401b000 tx=0x7f8d04016000).stop 2022-08-01T22:57:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 msgr2=0x7f8d141457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 0x7f8d141457b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f8d14062040 tx=0x7f8d08007c00).stop 2022-08-01T22:57:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 shutdown_connections 2022-08-01T22:57:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cf0053a10 0x7f8cf0055ed0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d140b2710 0x7f8d14141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140b01a0 0x7f8d141415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 --2- 172.21.15.32:0/1550187458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d140afd60 0x7f8d141457b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 >> 172.21.15.32:0/1550187458 conn(0x7f8d14060310 msgr2=0x7f8d1414cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 shutdown_connections 2022-08-01T22:57:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.123+0000 7f8d1359e700 1 -- 172.21.15.32:0/1550187458 wait complete. 2022-08-01T22:57:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2022-08-01T22:57:33.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T22:57:33.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 Processor -- start 2022-08-01T22:57:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 -- start start 2022-08-01T22:57:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3498060e90 0x7f34980ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f34980aff10 0x7f34981435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3498067370 con 0x7f3498066f90 2022-08-01T22:57:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34980b0320 con 0x7f3498060e90 2022-08-01T22:57:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349e984700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980a95e0 con 0x7f34980aff10 2022-08-01T22:57:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3498060e90 0x7f34980ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:33.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3498060e90 0x7f34980ab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42212/0 (socket says 172.21.15.32:42212) 2022-08-01T22:57:33.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 -- 172.21.15.32:0/880325174 learned_addr learned my addr 172.21.15.32:0/880325174 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f349cf21700 1 --2- 172.21.15.32:0/880325174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f34980aff10 0x7f34981435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 -- 172.21.15.32:0/880325174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f34980aff10 msgr2=0x7f34981435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 --2- 172.21.15.32:0/880325174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f34980aff10 0x7f34981435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 -- 172.21.15.32:0/880325174 >> v1:172.21.15.55:6789/0 conn(0x7f3498066f90 legacy=0x7f34980aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:33.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 -- 172.21.15.32:0/880325174 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34980b0540 con 0x7f3498060e90 2022-08-01T22:57:33.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f34977fe700 1 --2- 172.21.15.32:0/880325174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f34980ab0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f34880032f0 tx=0x7f3488009290).ready entity=mon.0 client_cookie=47a01b592247c2cb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:33.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f3496ffd700 1 -- 172.21.15.32:0/880325174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348800b8a0 con 0x7f3498060e90 2022-08-01T22:57:33.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.243+0000 7f3496ffd700 1 -- 172.21.15.32:0/880325174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348800ba00 con 0x7f3498060e90 2022-08-01T22:57:33.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3496ffd700 1 -- 172.21.15.32:0/880325174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348800bb70 con 0x7f3498060e90 2022-08-01T22:57:33.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- 172.21.15.32:0/880325174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 msgr2=0x7f34980ab0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:33.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 --2- 172.21.15.32:0/880325174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f34980ab0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f34880032f0 tx=0x7f3488009290).stop 2022-08-01T22:57:33.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- 172.21.15.32:0/880325174 shutdown_connections 2022-08-01T22:57:33.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 --2- 172.21.15.32:0/880325174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f34980ab0a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 --2- 172.21.15.32:0/880325174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f34980aff10 0x7f34981435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- 172.21.15.32:0/880325174 >> 172.21.15.32:0/880325174 conn(0x7f34980b5570 msgr2=0x7f34980b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:33.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- 172.21.15.32:0/880325174 shutdown_connections 2022-08-01T22:57:33.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- 172.21.15.32:0/880325174 wait complete. 2022-08-01T22:57:33.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 Processor -- start 2022-08-01T22:57:33.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- start start 2022-08-01T22:57:33.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f3498153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3498066f90 0x7f3498149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34980aff10 0x7f34980ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498152460 con 0x7f3498060e90 2022-08-01T22:57:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980b4f40 con 0x7f34980aff10 2022-08-01T22:57:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349e984700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3498152650 con 0x7f3498066f90 2022-08-01T22:57:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f3498153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f349cf21700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34980aff10 0x7f34980ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f34977fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3498066f90 0x7f3498149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f3498153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42220/0 (socket says 172.21.15.32:42220) 2022-08-01T22:57:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 -- 172.21.15.32:0/484167470 learned_addr learned my addr 172.21.15.32:0/484167470 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 -- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3498066f90 msgr2=0x7f3498149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3498066f90 0x7f3498149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 -- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34980aff10 msgr2=0x7f34980ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:33.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.247+0000 7f3497fff700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34980aff10 0x7f34980ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:33.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f3497fff700 1 -- 172.21.15.32:0/484167470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009530 con 0x7f3498060e90 2022-08-01T22:57:33.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f349cf21700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34980aff10 0x7f34980ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f3497fff700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f3498153030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f3498062540 tx=0x7f348c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c007030 con 0x7f3498060e90 2022-08-01T22:57:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c007190 con 0x7f3498060e90 2022-08-01T22:57:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34981528f0 con 0x7f3498060e90 2022-08-01T22:57:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c006510 con 0x7f3498060e90 2022-08-01T22:57:33.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498060470 con 0x7f3498060e90 2022-08-01T22:57:33.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34980574e0 con 0x7f3498060e90 2022-08-01T22:57:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f348c006670 con 0x7f3498060e90 2022-08-01T22:57:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.251+0000 7f34957fa700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3478053a70 0x7f3478055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.255+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f348c06b9f0 con 0x7f3498060e90 2022-08-01T22:57:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.255+0000 7f34977fe700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3478053a70 0x7f3478055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.255+0000 7f34977fe700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3478053a70 0x7f3478055f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f348801b000 tx=0x7f3488016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:33.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.259+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f348c06de60 con 0x7f3498060e90 2022-08-01T22:57:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:33.423+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3498152d90 con 0x7f3498060e90 2022-08-01T22:57:34.325 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T22:57:34.319+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T22:57:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.335+0000 7f34957fa700 1 -- 172.21.15.32:0/484167470 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v13) v1 ==== 161+0+0 (secure 0 0 0) 0x7f348c046180 con 0x7f3498060e90 2022-08-01T22:57:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3478053a70 msgr2=0x7f3478055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3478053a70 0x7f3478055f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f348801b000 tx=0x7f3488016000).stop 2022-08-01T22:57:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 msgr2=0x7f3498153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f3498153030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f3498062540 tx=0x7f348c007be0).stop 2022-08-01T22:57:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 shutdown_connections 2022-08-01T22:57:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3478053a70 0x7f3478055f30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34980aff10 0x7f34980ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3498066f90 0x7f3498149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 --2- 172.21.15.32:0/484167470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3498060e90 0x7f3498153030 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 >> 172.21.15.32:0/484167470 conn(0x7f34980b5570 msgr2=0x7f34980b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 shutdown_connections 2022-08-01T22:57:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.339+0000 7f349e984700 1 -- 172.21.15.32:0/484167470 wait complete. 2022-08-01T22:57:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T22:57:34.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:34.379 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T22:57:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 Processor -- start 2022-08-01T22:57:34.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- start start 2022-08-01T22:57:34.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c0af450 0x7f610c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f610c0ad4b0 0x7f610c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f610c0aff30 con 0x7f610c1437d0 2022-08-01T22:57:34.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610c05c7a0 con 0x7f610c0ad4b0 2022-08-01T22:57:34.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610c140990 con 0x7f610c0af450 2022-08-01T22:57:34.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f610c0ad4b0 0x7f610c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610affd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c0af450 0x7f610c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f610c0ad4b0 0x7f610c1432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42242/0 (socket says 172.21.15.32:42242) 2022-08-01T22:57:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610affd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c0af450 0x7f610c0af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:53976/0 (socket says 172.21.15.32:53976) 2022-08-01T22:57:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 -- 172.21.15.32:0/1063097670 learned_addr learned my addr 172.21.15.32:0/1063097670 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:34.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 -- 172.21.15.32:0/1063097670 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c0af450 msgr2=0x7f610c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 --2- 172.21.15.32:0/1063097670 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c0af450 0x7f610c0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 -- 172.21.15.32:0/1063097670 >> v1:172.21.15.55:6789/0 conn(0x7f610c1437d0 legacy=0x7f610c0a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 -- 172.21.15.32:0/1063097670 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c0a9550 con 0x7f610c0ad4b0 2022-08-01T22:57:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610a7fc700 1 --2- 172.21.15.32:0/1063097670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0ad4b0 0x7f610c1432e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f610c13fdc0 tx=0x7f60f4006f40).ready entity=mon.0 client_cookie=8cf282d9d711bd23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f6109ffb700 1 -- 172.21.15.32:0/1063097670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60f400e9b0 con 0x7f610c0ad4b0 2022-08-01T22:57:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f6109ffb700 1 -- 172.21.15.32:0/1063097670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f4000e90 con 0x7f610c0ad4b0 2022-08-01T22:57:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f6109ffb700 1 -- 172.21.15.32:0/1063097670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60f400ed00 con 0x7f610c0ad4b0 2022-08-01T22:57:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- 172.21.15.32:0/1063097670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0ad4b0 msgr2=0x7f610c1432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 --2- 172.21.15.32:0/1063097670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0ad4b0 0x7f610c1432e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f610c13fdc0 tx=0x7f60f4006f40).stop 2022-08-01T22:57:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- 172.21.15.32:0/1063097670 shutdown_connections 2022-08-01T22:57:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 --2- 172.21.15.32:0/1063097670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0ad4b0 0x7f610c1432e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 --2- 172.21.15.32:0/1063097670 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c0af450 0x7f610c0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- 172.21.15.32:0/1063097670 >> 172.21.15.32:0/1063097670 conn(0x7f610c0612d0 msgr2=0x7f610c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- 172.21.15.32:0/1063097670 shutdown_connections 2022-08-01T22:57:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.451+0000 7f610bfff700 1 -- 172.21.15.32:0/1063097670 wait complete. 2022-08-01T22:57:34.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 Processor -- start 2022-08-01T22:57:34.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 -- start start 2022-08-01T22:57:34.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f610c0ad4b0 0x7f610c13a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 0x7f610c0abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c1437d0 0x7f610c0ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c14f070 con 0x7f610c0af450 2022-08-01T22:57:34.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c14edd0 con 0x7f610c0ad4b0 2022-08-01T22:57:34.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610c142930 con 0x7f610c1437d0 2022-08-01T22:57:34.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 0x7f610c0abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f610c0ad4b0 0x7f610c13a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c1437d0 0x7f610c0ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 0x7f610c0abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42248/0 (socket says 172.21.15.32:42248) 2022-08-01T22:57:34.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 -- 172.21.15.32:0/4147880078 learned_addr learned my addr 172.21.15.32:0/4147880078 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:34.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 -- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c1437d0 msgr2=0x7f610c0ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c1437d0 0x7f610c0ac400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 -- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f610c0ad4b0 msgr2=0x7f610c13a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f610c0ad4b0 0x7f610c13a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 -- 172.21.15.32:0/4147880078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f400c2e0 con 0x7f610c0af450 2022-08-01T22:57:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610affd700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f610c0ad4b0 0x7f610c13a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610a7fc700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 0x7f610c0abf10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f610c13fdc0 tx=0x7f60f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:34.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60f400f860 con 0x7f610c0af450 2022-08-01T22:57:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f400f9c0 con 0x7f610c0af450 2022-08-01T22:57:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60f400fd20 con 0x7f610c0af450 2022-08-01T22:57:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.455+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610c142bd0 con 0x7f610c0af450 2022-08-01T22:57:34.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610c0ac8f0 con 0x7f610c0af450 2022-08-01T22:57:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610c142d60 con 0x7f610c0af450 2022-08-01T22:57:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f60f402a070 con 0x7f610c0af450 2022-08-01T22:57:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f6103fff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f60f0053980 0x7f60f0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f60f406d380 con 0x7f610c0af450 2022-08-01T22:57:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f610affd700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f60f0053980 0x7f60f0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.459+0000 7f610affd700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f60f0053980 0x7f60f0055e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f610400f000 tx=0x7f610400f040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.467+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f60f4043fe0 con 0x7f610c0af450 2022-08-01T22:57:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f610c0574e0 con 0x7f610c0af450 2022-08-01T22:57:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f6103fff700 1 -- 172.21.15.32:0/4147880078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2126 (secure 0 0 0) 0x7f60f4047600 con 0x7f610c0af450 2022-08-01T22:57:34.637 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:34.637 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:56:28.419646+0000","modified":"2022-08-01T22:57:34.325153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T22:57:34.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f60f0053980 msgr2=0x7f60f0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f60f0053980 0x7f60f0055e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f610400f000 tx=0x7f610400f040).stop 2022-08-01T22:57:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 msgr2=0x7f610c0abf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 0x7f610c0abf10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f610c13fdc0 tx=0x7f60f4015000).stop 2022-08-01T22:57:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 shutdown_connections 2022-08-01T22:57:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f60f0053980 0x7f60f0055e40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f610c1437d0 0x7f610c0ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f610c0af450 0x7f610c0abf10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 --2- 172.21.15.32:0/4147880078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f610c0ad4b0 0x7f610c13a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 >> 172.21.15.32:0/4147880078 conn(0x7f610c0612d0 msgr2=0x7f610c137bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 shutdown_connections 2022-08-01T22:57:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.627+0000 7f610bfff700 1 -- 172.21.15.32:0/4147880078 wait complete. 2022-08-01T22:57:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2022-08-01T22:57:34.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T22:57:34.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee78a12700 1 Processor -- start 2022-08-01T22:57:34.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee78a12700 1 -- start start 2022-08-01T22:57:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee78a12700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee740b4310 0x7fee74140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee78a12700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee74143cf0 con 0x7fee740b4820 2022-08-01T22:57:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee78a12700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee74143f90 con 0x7fee74146ae0 2022-08-01T22:57:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee78a12700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee741476a0 con 0x7fee740b4310 2022-08-01T22:57:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee71d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee740b4310 0x7fee74140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee72d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee740b4820 0x7fee740b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58030/0 (socket says 172.21.15.32:58030) 2022-08-01T22:57:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.747+0000 7fee72d9d700 1 -- 172.21.15.32:0/1599201614 learned_addr learned my addr 172.21.15.32:0/1599201614 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187602943 0 0) 0x7fee74143cf0 con 0x7fee740b4820 2022-08-01T22:57:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee54003260 con 0x7fee740b4820 2022-08-01T22:57:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660662452 0 0) 0x7fee74143f90 con 0x7fee74146ae0 2022-08-01T22:57:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee74143cf0 con 0x7fee74146ae0 2022-08-01T22:57:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee71d9b700 1 -- 172.21.15.32:0/1599201614 >> v1:172.21.15.55:6789/0 conn(0x7fee74146ae0 legacy=0x7fee741403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee71d9b700 1 -- 172.21.15.32:0/1599201614 >> v1:172.21.15.32:6789/0 conn(0x7fee740b4820 legacy=0x7fee740b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee71d9b700 1 -- 172.21.15.32:0/1599201614 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee740b0e20 con 0x7fee740b4310 2022-08-01T22:57:34.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee71d9b700 1 --2- 172.21.15.32:0/1599201614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee740b4310 0x7fee74140ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fee64002f70 tx=0x7fee64006df0).ready entity=mon.0 client_cookie=fa927acce332396b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee6400d540 con 0x7fee740b4310 2022-08-01T22:57:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64000e90 con 0x7fee740b4310 2022-08-01T22:57:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee7159a700 1 -- 172.21.15.32:0/1599201614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee6400d890 con 0x7fee740b4310 2022-08-01T22:57:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- 172.21.15.32:0/1599201614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee740b4310 msgr2=0x7fee74140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 --2- 172.21.15.32:0/1599201614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee740b4310 0x7fee74140ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fee64002f70 tx=0x7fee64006df0).stop 2022-08-01T22:57:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- 172.21.15.32:0/1599201614 shutdown_connections 2022-08-01T22:57:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 --2- 172.21.15.32:0/1599201614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee740b4310 0x7fee74140ad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- 172.21.15.32:0/1599201614 >> 172.21.15.32:0/1599201614 conn(0x7fee7405c620 msgr2=0x7fee7405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- 172.21.15.32:0/1599201614 shutdown_connections 2022-08-01T22:57:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- 172.21.15.32:0/1599201614 wait complete. 2022-08-01T22:57:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 Processor -- start 2022-08-01T22:57:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- start start 2022-08-01T22:57:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740b4310 0x7fee740b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee740b4820 0x7fee740b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 0x7fee740ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee741446b0 con 0x7fee74146ae0 2022-08-01T22:57:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.751+0000 7fee78a12700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740ba690 con 0x7fee740b4820 2022-08-01T22:57:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee78a12700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee74140600 con 0x7fee740b4310 2022-08-01T22:57:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 0x7fee740ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 0x7fee740ba1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42262/0 (socket says 172.21.15.32:42262) 2022-08-01T22:57:34.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 -- 172.21.15.32:0/2625970170 learned_addr learned my addr 172.21.15.32:0/2625970170 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:34.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee71d9b700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee740b4820 0x7fee740b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee7259c700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740b4310 0x7fee740b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 -- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740b4310 msgr2=0x7fee740b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740b4310 0x7fee740b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 -- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee740b4820 msgr2=0x7fee740b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:34.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee740b4820 0x7fee740b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:34.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 -- 172.21.15.32:0/2625970170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6400b180 con 0x7fee74146ae0 2022-08-01T22:57:34.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee71d9b700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee740b4820 0x7fee740b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:34.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee7259c700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740b4310 0x7fee740b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:34.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee72d9d700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 0x7fee740ba1a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fee6c001490 tx=0x7fee6c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:34.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee6c005ba0 con 0x7fee74146ae0 2022-08-01T22:57:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c005d00 con 0x7fee74146ae0 2022-08-01T22:57:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee74144920 con 0x7fee74146ae0 2022-08-01T22:57:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee6c0072c0 con 0x7fee74146ae0 2022-08-01T22:57:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee74146340 con 0x7fee74146ae0 2022-08-01T22:57:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee6c01a060 con 0x7fee74146ae0 2022-08-01T22:57:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee74145ba0 con 0x7fee74146ae0 2022-08-01T22:57:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee6b7fe700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee58053b00 0x7fee58055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fee6c06bc10 con 0x7fee74146ae0 2022-08-01T22:57:34.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.755+0000 7fee7259c700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee58053b00 0x7fee58055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:34.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.759+0000 7fee7259c700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee58053b00 0x7fee58055fc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fee5c00b000 tx=0x7fee5c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.763+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee6c0463b0 con 0x7fee74146ae0 2022-08-01T22:57:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:34.927+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fee740574e0 con 0x7fee74146ae0 2022-08-01T22:57:35.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.339+0000 7fee6b7fe700 1 -- 172.21.15.32:0/2625970170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee6c018020 con 0x7fee74146ae0 2022-08-01T22:57:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee58053b00 msgr2=0x7fee58055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee58053b00 0x7fee58055fc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fee5c00b000 tx=0x7fee5c009040).stop 2022-08-01T22:57:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 msgr2=0x7fee740ba1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:35.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 0x7fee740ba1a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fee6c001490 tx=0x7fee6c007e00).stop 2022-08-01T22:57:35.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 shutdown_connections 2022-08-01T22:57:35.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee58053b00 0x7fee58055fc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee74146ae0 0x7fee740ba1a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee740b4820 0x7fee740b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 --2- 172.21.15.32:0/2625970170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740b4310 0x7fee740b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 >> 172.21.15.32:0/2625970170 conn(0x7fee7405c620 msgr2=0x7fee740b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:35.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 shutdown_connections 2022-08-01T22:57:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.343+0000 7fee78a12700 1 -- 172.21.15.32:0/2625970170 wait complete. 2022-08-01T22:57:35.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T22:57:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.471+0000 7f92ecd64700 1 Processor -- start 2022-08-01T22:57:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.471+0000 7f92ecd64700 1 -- start start 2022-08-01T22:57:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92ecd64700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f92e80b3190 0x7f92e8060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92ecd64700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e8061190 0x7f92e8066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92ecd64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e80654c0 con 0x7f92e80b2730 2022-08-01T22:57:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92ecd64700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e8062190 con 0x7f92e80b3190 2022-08-01T22:57:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92ecd64700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80615d0 con 0x7f92e8061190 2022-08-01T22:57:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92dffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92e80b2730 0x7f92e80666b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58042/0 (socket says 172.21.15.32:58042) 2022-08-01T22:57:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92dffff700 1 -- 172.21.15.32:0/394868232 learned_addr learned my addr 172.21.15.32:0/394868232 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e7fff700 1 --2- 172.21.15.32:0/394868232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e8061190 0x7f92e8066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e77fe700 1 --2- 172.21.15.32:0/394868232 >> v2:172.21.15.55:3300/0 conn(0x7f92e80b3190 0x7f92e8060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e6ffd700 1 -- 172.21.15.32:0/394868232 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337892858 0 0) 0x7f92e80654c0 con 0x7f92e80b2730 2022-08-01T22:57:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e6ffd700 1 -- 172.21.15.32:0/394868232 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c8003260 con 0x7f92e80b2730 2022-08-01T22:57:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e7fff700 1 -- 172.21.15.32:0/394868232 >> v2:172.21.15.55:3300/0 conn(0x7f92e80b3190 msgr2=0x7f92e8060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e7fff700 1 --2- 172.21.15.32:0/394868232 >> v2:172.21.15.55:3300/0 conn(0x7f92e80b3190 0x7f92e8060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e7fff700 1 -- 172.21.15.32:0/394868232 >> v1:172.21.15.32:6789/0 conn(0x7f92e80b2730 legacy=0x7f92e80666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e7fff700 1 -- 172.21.15.32:0/394868232 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80b3570 con 0x7f92e8061190 2022-08-01T22:57:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e7fff700 1 --2- 172.21.15.32:0/394868232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e8061190 0x7f92e8066dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f92e00032f0 tx=0x7f92e00091d0).ready entity=mon.2 client_cookie=1c65b3fac07fd656 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e6ffd700 1 -- 172.21.15.32:0/394868232 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e000b600 con 0x7f92e8061190 2022-08-01T22:57:35.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e6ffd700 1 -- 172.21.15.32:0/394868232 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e000b760 con 0x7f92e8061190 2022-08-01T22:57:35.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.475+0000 7f92e6ffd700 1 -- 172.21.15.32:0/394868232 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e000b8d0 con 0x7f92e8061190 2022-08-01T22:57:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 -- 172.21.15.32:0/394868232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e8061190 msgr2=0x7f92e8066dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 --2- 172.21.15.32:0/394868232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e8061190 0x7f92e8066dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f92e00032f0 tx=0x7f92e00091d0).stop 2022-08-01T22:57:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 -- 172.21.15.32:0/394868232 shutdown_connections 2022-08-01T22:57:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 --2- 172.21.15.32:0/394868232 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e8061190 0x7f92e8066dc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 --2- 172.21.15.32:0/394868232 >> v2:172.21.15.55:3300/0 conn(0x7f92e80b3190 0x7f92e8060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 -- 172.21.15.32:0/394868232 >> 172.21.15.32:0/394868232 conn(0x7f92e8063870 msgr2=0x7f92e8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 -- 172.21.15.32:0/394868232 shutdown_connections 2022-08-01T22:57:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 -- 172.21.15.32:0/394868232 wait complete. 2022-08-01T22:57:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 Processor -- start 2022-08-01T22:57:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.479+0000 7f92ecd64700 1 -- start start 2022-08-01T22:57:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92e8061190 0x7f92e80b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:35.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e80b2730 0x7f92e80a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:35.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 0x7f92e80a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:35.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80ac5d0 con 0x7f92e80b3190 2022-08-01T22:57:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8156230 con 0x7f92e8061190 2022-08-01T22:57:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80ac800 con 0x7f92e80b2730 2022-08-01T22:57:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 0x7f92e80a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 0x7f92e80a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42278/0 (socket says 172.21.15.32:42278) 2022-08-01T22:57:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 -- 172.21.15.32:0/1808340025 learned_addr learned my addr 172.21.15.32:0/1808340025 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92dffff700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e80b2730 0x7f92e80a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e77fe700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92e8061190 0x7f92e80b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 -- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e80b2730 msgr2=0x7f92e80a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e80b2730 0x7f92e80a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 -- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92e8061190 msgr2=0x7f92e80b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92e8061190 0x7f92e80b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 -- 172.21.15.32:0/1808340025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009470 con 0x7f92e80b3190 2022-08-01T22:57:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e7fff700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 0x7f92e80a9b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f92e00032f0 tx=0x7f92e0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e000b600 con 0x7f92e80b3190 2022-08-01T22:57:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e000b8d0 con 0x7f92e80b3190 2022-08-01T22:57:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8156490 con 0x7f92e80b3190 2022-08-01T22:57:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e001f5e0 con 0x7f92e80b3190 2022-08-01T22:57:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.483+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e80aec20 con 0x7f92e80b3190 2022-08-01T22:57:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.487+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92e0026030 con 0x7f92e80b3190 2022-08-01T22:57:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.487+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e8156620 con 0x7f92e80b3190 2022-08-01T22:57:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.487+0000 7f92e57fa700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f92cc0539e0 0x7f92cc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.487+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f92e006cb20 con 0x7f92e80b3190 2022-08-01T22:57:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.487+0000 7f92e77fe700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f92cc0539e0 0x7f92cc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.487+0000 7f92e77fe700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f92cc0539e0 0x7f92cc055ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f92d800b000 tx=0x7f92d8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.491+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92e0043870 con 0x7f92e80b3190 2022-08-01T22:57:35.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:35.619+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92e805fbf0 con 0x7f92e80b3190 2022-08-01T22:57:36.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.339+0000 7f92e57fa700 1 -- 172.21.15.32:0/1808340025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 185+0+0 (secure 0 0 0) 0x7f92e0015020 con 0x7f92e80b3190 2022-08-01T22:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.343+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f92cc0539e0 msgr2=0x7f92cc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.343+0000 7f92ecd64700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f92cc0539e0 0x7f92cc055ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f92d800b000 tx=0x7f92d8006df0).stop 2022-08-01T22:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 msgr2=0x7f92e80a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 0x7f92e80a9b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f92e00032f0 tx=0x7f92e0016000).stop 2022-08-01T22:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 shutdown_connections 2022-08-01T22:57:36.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f92cc0539e0 0x7f92cc055ea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e80b3190 0x7f92e80a9b40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f92e80b2730 0x7f92e80a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 --2- 172.21.15.32:0/1808340025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92e8061190 0x7f92e80b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 >> 172.21.15.32:0/1808340025 conn(0x7f92e8063870 msgr2=0x7f92e8066510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 shutdown_connections 2022-08-01T22:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.347+0000 7f92ecd64700 1 -- 172.21.15.32:0/1808340025 wait complete. 2022-08-01T22:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T22:57:36.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T22:57:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.479+0000 7f395d5d9700 1 Processor -- start 2022-08-01T22:57:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.479+0000 7f395d5d9700 1 -- start start 2022-08-01T22:57:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f39580b20a0 0x7f39580b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39580b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39580aa520 con 0x7f39580b0250 2022-08-01T22:57:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39580ab3f0 con 0x7f39580b20a0 2022-08-01T22:57:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b0630 con 0x7f39580a9d00 2022-08-01T22:57:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f39580b20a0 0x7f39580b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f39577fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39580b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f39580b20a0 0x7f39580b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52070/0 (socket says 172.21.15.32:52070) 2022-08-01T22:57:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f39577fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39580b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:54022/0 (socket says 172.21.15.32:54022) 2022-08-01T22:57:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 -- 172.21.15.32:0/2142429630 learned_addr learned my addr 172.21.15.32:0/2142429630 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 -- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 msgr2=0x7f39580b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 --2- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39580b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 -- 172.21.15.32:0/2142429630 >> v1:172.21.15.55:6789/0 conn(0x7f39580b0250 legacy=0x7f39580aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:36.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 -- 172.21.15.32:0/2142429630 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580aa280 con 0x7f39580b20a0 2022-08-01T22:57:36.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f39577fe700 1 --2- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39580b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f394ffff700 1 --2- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b20a0 0x7f39580b1ab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3940002f70 tx=0x7f3940006eb0).ready entity=mon.1 client_cookie=4ee5918b47d7cc5a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f39567fc700 1 -- 172.21.15.32:0/2142429630 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f394000d570 con 0x7f39580b20a0 2022-08-01T22:57:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f39567fc700 1 -- 172.21.15.32:0/2142429630 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3940000e90 con 0x7f39580b20a0 2022-08-01T22:57:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f39567fc700 1 -- 172.21.15.32:0/2142429630 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f394000d900 con 0x7f39580b20a0 2022-08-01T22:57:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 -- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b20a0 msgr2=0x7f39580b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.483+0000 7f395d5d9700 1 --2- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b20a0 0x7f39580b1ab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3940002f70 tx=0x7f3940006eb0).stop 2022-08-01T22:57:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 -- 172.21.15.32:0/2142429630 shutdown_connections 2022-08-01T22:57:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 --2- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39580b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 --2- 172.21.15.32:0/2142429630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b20a0 0x7f39580b1ab0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 -- 172.21.15.32:0/2142429630 >> 172.21.15.32:0/2142429630 conn(0x7f3958060950 msgr2=0x7f3958060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 -- 172.21.15.32:0/2142429630 shutdown_connections 2022-08-01T22:57:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 -- 172.21.15.32:0/2142429630 wait complete. 2022-08-01T22:57:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 Processor -- start 2022-08-01T22:57:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.487+0000 7f395d5d9700 1 -- start start 2022-08-01T22:57:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39581434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b0250 0x7f39580656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 0x7f3958065b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395813e8d0 con 0x7f39580b20a0 2022-08-01T22:57:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395813ed70 con 0x7f39580b0250 2022-08-01T22:57:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3958063fb0 con 0x7f39580a9d00 2022-08-01T22:57:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 0x7f3958065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 0x7f3958065b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42290/0 (socket says 172.21.15.32:42290) 2022-08-01T22:57:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f3956ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39581434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f394ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b0250 0x7f39580656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 -- 172.21.15.32:0/1121536212 learned_addr learned my addr 172.21.15.32:0/1121536212 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 -- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 msgr2=0x7f39581434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39581434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 -- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b0250 msgr2=0x7f39580656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b0250 0x7f39580656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 -- 172.21.15.32:0/1121536212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3950009470 con 0x7f39580b20a0 2022-08-01T22:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f39577fe700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 0x7f3958065b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f39580652e0 tx=0x7f39500091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f395000c850 con 0x7f39580b20a0 2022-08-01T22:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395000c9b0 con 0x7f39580b20a0 2022-08-01T22:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f395000cd10 con 0x7f39580b20a0 2022-08-01T22:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394000b1e0 con 0x7f39580b20a0 2022-08-01T22:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39580641e0 con 0x7f39580b20a0 2022-08-01T22:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.491+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3958065390 con 0x7f39580b20a0 2022-08-01T22:57:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.495+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3950022070 con 0x7f39580b20a0 2022-08-01T22:57:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.495+0000 7f3954ff9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f393c053a70 0x7f393c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.495+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f395006c680 con 0x7f39580b20a0 2022-08-01T22:57:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.495+0000 7f3956ffd700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f393c053a70 0x7f393c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.495+0000 7f3956ffd700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f393c053a70 0x7f393c055f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f394800d000 tx=0x7f3948007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.499+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39500435a0 con 0x7f39580b20a0 2022-08-01T22:57:36.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:36.623+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f39580574e0 con 0x7f39580b20a0 2022-08-01T22:57:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.343+0000 7f3954ff9700 1 -- 172.21.15.32:0/1121536212 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 173+0+0 (secure 0 0 0) 0x7f39500160d0 con 0x7f39580b20a0 2022-08-01T22:57:37.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f393c053a70 msgr2=0x7f393c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f393c053a70 0x7f393c055f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f394800d000 tx=0x7f3948007b90).stop 2022-08-01T22:57:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 msgr2=0x7f3958065b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 0x7f3958065b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f39580652e0 tx=0x7f39500091d0).stop 2022-08-01T22:57:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 shutdown_connections 2022-08-01T22:57:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f393c053a70 0x7f393c055f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39580b20a0 0x7f3958065b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39580b0250 0x7f39580656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 --2- 172.21.15.32:0/1121536212 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39580a9d00 0x7f39581434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 >> 172.21.15.32:0/1121536212 conn(0x7f3958060950 msgr2=0x7f39580b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 shutdown_connections 2022-08-01T22:57:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.347+0000 7f395d5d9700 1 -- 172.21.15.32:0/1121536212 wait complete. 2022-08-01T22:57:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T22:57:37.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T22:57:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 Processor -- start 2022-08-01T22:57:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 -- start start 2022-08-01T22:57:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd41401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd40b0270 0x7f9fd41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd4067a10 con 0x7f9fd40ac880 2022-08-01T22:57:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd40ad740 con 0x7f9fd40b0270 2022-08-01T22:57:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fdaef5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4067cb0 con 0x7f9fd40b2710 2022-08-01T22:57:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd40b0270 0x7f9fd41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd40b0270 0x7f9fd41406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42296/0 (socket says 172.21.15.32:42296) 2022-08-01T22:57:37.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 -- 172.21.15.32:0/3413643734 learned_addr learned my addr 172.21.15.32:0/3413643734 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:37.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd9ef3700 1 --2- 172.21.15.32:0/3413643734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd41401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 -- 172.21.15.32:0/3413643734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 msgr2=0x7f9fd41401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:37.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 --2- 172.21.15.32:0/3413643734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd41401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 -- 172.21.15.32:0/3413643734 >> v1:172.21.15.55:6789/0 conn(0x7f9fd40ac880 legacy=0x7f9fd4143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 -- 172.21.15.32:0/3413643734 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4147160 con 0x7f9fd40b0270 2022-08-01T22:57:37.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.475+0000 7f9fd96f2700 1 --2- 172.21.15.32:0/3413643734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd41406d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f9fd0002f70 tx=0x7f9fd0006eb0).ready entity=mon.0 client_cookie=6bc35163991d280b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:37.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fd8ef1700 1 -- 172.21.15.32:0/3413643734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd000d570 con 0x7f9fd40b0270 2022-08-01T22:57:37.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fd8ef1700 1 -- 172.21.15.32:0/3413643734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0000e90 con 0x7f9fd40b0270 2022-08-01T22:57:37.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fd8ef1700 1 -- 172.21.15.32:0/3413643734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd000d900 con 0x7f9fd40b0270 2022-08-01T22:57:37.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- 172.21.15.32:0/3413643734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 msgr2=0x7f9fd41406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/3413643734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd41406d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f9fd0002f70 tx=0x7f9fd0006eb0).stop 2022-08-01T22:57:37.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- 172.21.15.32:0/3413643734 shutdown_connections 2022-08-01T22:57:37.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/3413643734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd41406d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/3413643734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd41401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- 172.21.15.32:0/3413643734 >> 172.21.15.32:0/3413643734 conn(0x7f9fd4060310 msgr2=0x7f9fd40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:37.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- 172.21.15.32:0/3413643734 shutdown_connections 2022-08-01T22:57:37.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- 172.21.15.32:0/3413643734 wait complete. 2022-08-01T22:57:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 Processor -- start 2022-08-01T22:57:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- start start 2022-08-01T22:57:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd40ac880 0x7f9fd4142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd4146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd4140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4146e40 con 0x7f9fd40b0270 2022-08-01T22:57:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4156b10 con 0x7f9fd40ac880 2022-08-01T22:57:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.479+0000 7f9fdaef5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4156600 con 0x7f9fd40b2710 2022-08-01T22:57:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd4146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd9ef3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd40ac880 0x7f9fd4142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd4146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42302/0 (socket says 172.21.15.32:42302) 2022-08-01T22:57:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 -- 172.21.15.32:0/2049761054 learned_addr learned my addr 172.21.15.32:0/2049761054 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fda6f4700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd4140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 -- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 msgr2=0x7f9fd4140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd4140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 -- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd40ac880 msgr2=0x7f9fd4142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd40ac880 0x7f9fd4142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 -- 172.21.15.32:0/2049761054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd000b1e0 con 0x7f9fd40b0270 2022-08-01T22:57:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fd96f2700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd4146590 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f9fd4143fd0 tx=0x7f9fd0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd0011610 con 0x7f9fd40b0270 2022-08-01T22:57:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0011770 con 0x7f9fd40b0270 2022-08-01T22:57:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd0011ad0 con 0x7f9fd40b0270 2022-08-01T22:57:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd4156870 con 0x7f9fd40b0270 2022-08-01T22:57:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd4152a80 con 0x7f9fd40b0270 2022-08-01T22:57:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd40ac110 con 0x7f9fd40b0270 2022-08-01T22:57:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.483+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9fd0025030 con 0x7f9fd40b0270 2022-08-01T22:57:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.487+0000 7f9fc2ffd700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9fb4053a70 0x7f9fb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.487+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9fd006cda0 con 0x7f9fd40b0270 2022-08-01T22:57:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.487+0000 7f9fd9ef3700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9fb4053a70 0x7f9fb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.487+0000 7f9fd9ef3700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9fb4053a70 0x7f9fb4055f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9fc400b000 tx=0x7f9fc4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:37.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.491+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9fd40ac110 con 0x7f9fd40b0270 2022-08-01T22:57:37.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.619+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9fd40ac110 con 0x7f9fd40b0270 2022-08-01T22:57:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fc2ffd700 1 -- 172.21.15.32:0/2049761054 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9fd40ac110 con 0x7f9fd40b0270 2022-08-01T22:57:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9fb4053a70 msgr2=0x7f9fb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9fb4053a70 0x7f9fb4055f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9fc400b000 tx=0x7f9fc4006df0).stop 2022-08-01T22:57:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 msgr2=0x7f9fd4146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd4146590 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f9fd4143fd0 tx=0x7f9fd0015000).stop 2022-08-01T22:57:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 shutdown_connections 2022-08-01T22:57:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9fb4053a70 0x7f9fb4055f30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9fd40b2710 0x7f9fd4140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40b0270 0x7f9fd4146590 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 --2- 172.21.15.32:0/2049761054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd40ac880 0x7f9fd4142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.627+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 >> 172.21.15.32:0/2049761054 conn(0x7f9fd4060310 msgr2=0x7f9fd40adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.631+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 shutdown_connections 2022-08-01T22:57:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.631+0000 7f9fdaef5700 1 -- 172.21.15.32:0/2049761054 wait complete. 2022-08-01T22:57:37.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-08-01T22:57:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 Processor -- start 2022-08-01T22:57:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- start start 2022-08-01T22:57:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f8c0b6a90 0x7f5f8c144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f8c05ff60 0x7f5f8c14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c0b6e70 con 0x7f5f8c0aa210 2022-08-01T22:57:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0ac6e0 con 0x7f5f8c05ff60 2022-08-01T22:57:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0ac980 con 0x7f5f8c0b6a90 2022-08-01T22:57:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f8c05ff60 0x7f5f8c14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f83fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5f8c0aa210 0x7f5f8c144e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58086/0 (socket says 172.21.15.32:58086) 2022-08-01T22:57:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f8c05ff60 0x7f5f8c14ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42308/0 (socket says 172.21.15.32:42308) 2022-08-01T22:57:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f83fff700 1 -- 172.21.15.32:0/2075957976 learned_addr learned my addr 172.21.15.32:0/2075957976 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f8bfff700 1 --2- 172.21.15.32:0/2075957976 >> v2:172.21.15.55:3300/0 conn(0x7f5f8c0b6a90 0x7f5f8c144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 -- 172.21.15.32:0/2075957976 >> v2:172.21.15.55:3300/0 conn(0x7f5f8c0b6a90 msgr2=0x7f5f8c144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 --2- 172.21.15.32:0/2075957976 >> v2:172.21.15.55:3300/0 conn(0x7f5f8c0b6a90 0x7f5f8c144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 -- 172.21.15.32:0/2075957976 >> v1:172.21.15.32:6789/0 conn(0x7f5f8c0aa210 legacy=0x7f5f8c144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 -- 172.21.15.32:0/2075957976 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0acc20 con 0x7f5f8c05ff60 2022-08-01T22:57:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f8bfff700 1 --2- 172.21.15.32:0/2075957976 >> v2:172.21.15.55:3300/0 conn(0x7f5f8c0b6a90 0x7f5f8c144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f909d0700 1 --2- 172.21.15.32:0/2075957976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c05ff60 0x7f5f8c14ad70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f5f840032f0 tx=0x7f5f840091d0).ready entity=mon.0 client_cookie=777757dd1a773fab server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f8b7fe700 1 -- 172.21.15.32:0/2075957976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f84009ea0 con 0x7f5f8c05ff60 2022-08-01T22:57:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f8b7fe700 1 -- 172.21.15.32:0/2075957976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8400b870 con 0x7f5f8c05ff60 2022-08-01T22:57:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f8b7fe700 1 -- 172.21.15.32:0/2075957976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f8400b9e0 con 0x7f5f8c05ff60 2022-08-01T22:57:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- 172.21.15.32:0/2075957976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c05ff60 msgr2=0x7f5f8c14ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 --2- 172.21.15.32:0/2075957976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c05ff60 0x7f5f8c14ad70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f5f840032f0 tx=0x7f5f840091d0).stop 2022-08-01T22:57:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- 172.21.15.32:0/2075957976 shutdown_connections 2022-08-01T22:57:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 --2- 172.21.15.32:0/2075957976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c05ff60 0x7f5f8c14ad70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 --2- 172.21.15.32:0/2075957976 >> v2:172.21.15.55:3300/0 conn(0x7f5f8c0b6a90 0x7f5f8c144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- 172.21.15.32:0/2075957976 >> 172.21.15.32:0/2075957976 conn(0x7f5f8c0b3610 msgr2=0x7f5f8c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- 172.21.15.32:0/2075957976 shutdown_connections 2022-08-01T22:57:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.747+0000 7f5f92433700 1 -- 172.21.15.32:0/2075957976 wait complete. 2022-08-01T22:57:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.751+0000 7f5f92433700 1 Processor -- start 2022-08-01T22:57:37.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.751+0000 7f5f92433700 1 -- start start 2022-08-01T22:57:37.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5f8c05ff60 0x7f5f8c0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 0x7f5f8c141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f8c0b6a90 0x7f5f8c0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0674f0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c148cb0 con 0x7f5f8c0b6a90 2022-08-01T22:57:37.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0ae840 con 0x7f5f8c05ff60 2022-08-01T22:57:37.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 0x7f5f8c141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 0x7f5f8c141a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42316/0 (socket says 172.21.15.32:42316) 2022-08-01T22:57:37.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 -- 172.21.15.32:0/2211031568 learned_addr learned my addr 172.21.15.32:0/2211031568 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:37.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f8bfff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5f8c05ff60 0x7f5f8c0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 -- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5f8c05ff60 msgr2=0x7f5f8c0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5f8c05ff60 0x7f5f8c0aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:37.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 -- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f8c0b6a90 msgr2=0x7f5f8c0af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:37.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f8c0b6a90 0x7f5f8c0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T22:57:37.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 -- 172.21.15.32:0/2211031568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f84009470 con 0x7f5f8c0aa210 2022-08-01T22:57:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f8bfff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5f8c05ff60 0x7f5f8c0aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f83fff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 0x7f5f8c141a70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f5f8c0b5f50 tx=0x7f5f74007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7400ce30 con 0x7f5f8c0aa210 2022-08-01T22:57:37.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f740072e0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c0aeae0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f740057e0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.755+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c0613d0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.759+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5f74010070 con 0x7f5f8c0aa210 2022-08-01T22:57:37.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.759+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c148eb0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.759+0000 7f5f89ffb700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5f70053950 0x7f5f70055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:37.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.759+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5f7406b420 con 0x7f5f8c0aa210 2022-08-01T22:57:37.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.759+0000 7f5f8bfff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5f70053950 0x7f5f70055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:37.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.759+0000 7f5f8bfff700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5f70053950 0x7f5f70055e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f5f7c00c000 tx=0x7f5f7c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:37.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.763+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f740423b0 con 0x7f5f8c0aa210 2022-08-01T22:57:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:37.927+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f5f8c0acec0 con 0x7f5f8c0aa210 2022-08-01T22:57:38.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.359+0000 7f5f89ffb700 1 -- 172.21.15.32:0/2211031568 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v96) v1 ==== 139+0+0 (secure 0 0 0) 0x7f5f74018070 con 0x7f5f8c0aa210 2022-08-01T22:57:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5f70053950 msgr2=0x7f5f70055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5f70053950 0x7f5f70055e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f5f7c00c000 tx=0x7f5f7c008040).stop 2022-08-01T22:57:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 msgr2=0x7f5f8c141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 0x7f5f8c141a70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f5f8c0b5f50 tx=0x7f5f74007c00).stop 2022-08-01T22:57:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 shutdown_connections 2022-08-01T22:57:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5f70053950 0x7f5f70055e10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f8c0b6a90 0x7f5f8c0af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T22:57:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f8c0aa210 0x7f5f8c141a70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 --2- 172.21.15.32:0/2211031568 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5f8c05ff60 0x7f5f8c0aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 >> 172.21.15.32:0/2211031568 conn(0x7f5f8c0b3610 msgr2=0x7f5f8c060760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 shutdown_connections 2022-08-01T22:57:38.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.363+0000 7f5f92433700 1 -- 172.21.15.32:0/2211031568 wait complete. 2022-08-01T22:57:38.386 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T22:57:38.387 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T22:57:38.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-08-01T22:57:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 Processor -- start 2022-08-01T22:57:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 -- start start 2022-08-01T22:57:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d340b2710 0x7f0d3414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d3414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d340a7250 con 0x7f0d340afd60 2022-08-01T22:57:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d340a7480 con 0x7f0d340b2710 2022-08-01T22:57:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d3926c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340a7720 con 0x7f0d340b01a0 2022-08-01T22:57:38.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d340b2710 0x7f0d3414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d340b2710 0x7f0d3414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42324/0 (socket says 172.21.15.32:42324) 2022-08-01T22:57:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 -- 172.21.15.32:0/2826331692 learned_addr learned my addr 172.21.15.32:0/2826331692 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d337fe700 1 --2- 172.21.15.32:0/2826331692 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d3414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 -- 172.21.15.32:0/2826331692 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 msgr2=0x7f0d3414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 --2- 172.21.15.32:0/2826331692 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d3414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 -- 172.21.15.32:0/2826331692 >> v1:172.21.15.55:6789/0 conn(0x7f0d340afd60 legacy=0x7f0d340a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d32ffd700 1 -- 172.21.15.32:0/2826331692 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649508186 0 0) 0x7f0d340a7250 con 0x7f0d340afd60 2022-08-01T22:57:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.495+0000 7f0d33fff700 1 -- 172.21.15.32:0/2826331692 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340a79c0 con 0x7f0d340b2710 2022-08-01T22:57:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d33fff700 1 --2- 172.21.15.32:0/2826331692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d3414afd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f0d28002f70 tx=0x7f0d28006eb0).ready entity=mon.0 client_cookie=6fd260c87a584e67 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d32ffd700 1 -- 172.21.15.32:0/2826331692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2800d570 con 0x7f0d340b2710 2022-08-01T22:57:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d32ffd700 1 -- 172.21.15.32:0/2826331692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28000e90 con 0x7f0d340b2710 2022-08-01T22:57:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d32ffd700 1 -- 172.21.15.32:0/2826331692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2800d8c0 con 0x7f0d340b2710 2022-08-01T22:57:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 -- 172.21.15.32:0/2826331692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 msgr2=0x7f0d3414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 --2- 172.21.15.32:0/2826331692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d3414afd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f0d28002f70 tx=0x7f0d28006eb0).stop 2022-08-01T22:57:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 -- 172.21.15.32:0/2826331692 shutdown_connections 2022-08-01T22:57:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 --2- 172.21.15.32:0/2826331692 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d3414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 --2- 172.21.15.32:0/2826331692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d3414afd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 -- 172.21.15.32:0/2826331692 >> 172.21.15.32:0/2826331692 conn(0x7f0d34060310 msgr2=0x7f0d340b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 -- 172.21.15.32:0/2826331692 shutdown_connections 2022-08-01T22:57:38.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 -- 172.21.15.32:0/2826331692 wait complete. 2022-08-01T22:57:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 Processor -- start 2022-08-01T22:57:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.499+0000 7f0d3926c700 1 -- start start 2022-08-01T22:57:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d340afd60 0x7f0d340af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d340b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d340b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:38.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3414cf10 con 0x7f0d340b2710 2022-08-01T22:57:38.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340b3f60 con 0x7f0d340afd60 2022-08-01T22:57:38.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340aa730 con 0x7f0d340b01a0 2022-08-01T22:57:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d340b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d340b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42332/0 (socket says 172.21.15.32:42332) 2022-08-01T22:57:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 -- 172.21.15.32:0/397754581 learned_addr learned my addr 172.21.15.32:0/397754581 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d33fff700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d340afd60 0x7f0d340af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d337fe700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d340b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 -- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 msgr2=0x7f0d340b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d340b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 -- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d340afd60 msgr2=0x7f0d340af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d340afd60 0x7f0d340af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 -- 172.21.15.32:0/397754581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2800b1e0 con 0x7f0d340b2710 2022-08-01T22:57:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d38a6b700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d340b3a70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f0d2c000ed0 tx=0x7f0d2c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2c012400 con 0x7f0d340b2710 2022-08-01T22:57:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c012560 con 0x7f0d340b2710 2022-08-01T22:57:38.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340aa9d0 con 0x7f0d340b2710 2022-08-01T22:57:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d2c0128c0 con 0x7f0d340b2710 2022-08-01T22:57:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d340aaf10 con 0x7f0d340b2710 2022-08-01T22:57:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.503+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340ab0d0 con 0x7f0d340b2710 2022-08-01T22:57:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.507+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0d2c019030 con 0x7f0d340b2710 2022-08-01T22:57:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.507+0000 7f0d317fa700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d14053b30 0x7f0d14055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.507+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0d2c06b080 con 0x7f0d340b2710 2022-08-01T22:57:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.507+0000 7f0d33fff700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d14053b30 0x7f0d14055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.507+0000 7f0d33fff700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d14053b30 0x7f0d14055ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0d2801c000 tx=0x7f0d28015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.515+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d2c042120 con 0x7f0d340b2710 2022-08-01T22:57:38.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:38.679+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f0d340574e0 con 0x7f0d340b2710 2022-08-01T22:57:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.359+0000 7f0d317fa700 1 -- 172.21.15.32:0/397754581 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v97) v1 ==== 147+0+0 (secure 0 0 0) 0x7f0d2c045740 con 0x7f0d340b2710 2022-08-01T22:57:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.363+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d14053b30 msgr2=0x7f0d14055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.363+0000 7f0d3926c700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d14053b30 0x7f0d14055ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0d2801c000 tx=0x7f0d28015000).stop 2022-08-01T22:57:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.363+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 msgr2=0x7f0d340b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.363+0000 7f0d3926c700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d340b3a70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f0d2c000ed0 tx=0x7f0d2c007a40).stop 2022-08-01T22:57:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 shutdown_connections 2022-08-01T22:57:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d14053b30 0x7f0d14055ff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d340b2710 0x7f0d340b3a70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0d340b01a0 0x7f0d340b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 --2- 172.21.15.32:0/397754581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d340afd60 0x7f0d340af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 >> 172.21.15.32:0/397754581 conn(0x7f0d34060310 msgr2=0x7f0d34149d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 shutdown_connections 2022-08-01T22:57:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.367+0000 7f0d3926c700 1 -- 172.21.15.32:0/397754581 wait complete. 2022-08-01T22:57:39.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T22:57:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.495+0000 7f6414d88700 1 Processor -- start 2022-08-01T22:57:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.495+0000 7f6414d88700 1 -- start start 2022-08-01T22:57:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100b2710 0x7f641014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64100b01a0 0x7f641014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64100a7250 con 0x7f64100afd60 2022-08-01T22:57:39.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64100a7480 con 0x7f64100b01a0 2022-08-01T22:57:39.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64100a7720 con 0x7f64100b2710 2022-08-01T22:57:39.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64100b01a0 0x7f641014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64100afd60 0x7f64100a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58112/0 (socket says 172.21.15.32:58112) 2022-08-01T22:57:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640ffff700 1 -- 172.21.15.32:0/1113440857 learned_addr learned my addr 172.21.15.32:0/1113440857 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640f7fe700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100b2710 0x7f641014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640effd700 1 -- 172.21.15.32:0/1113440857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100b2710 msgr2=0x7f641014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640effd700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100b2710 0x7f641014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640effd700 1 -- 172.21.15.32:0/1113440857 >> v1:172.21.15.32:6789/0 conn(0x7f64100afd60 legacy=0x7f64100a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640e7fc700 1 -- 172.21.15.32:0/1113440857 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790364686 0 0) 0x7f64100a7250 con 0x7f64100afd60 2022-08-01T22:57:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640effd700 1 -- 172.21.15.32:0/1113440857 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64100a79c0 con 0x7f64100b01a0 2022-08-01T22:57:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640f7fe700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100b2710 0x7f641014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:57:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640effd700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f641014b4c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f6400002f70 tx=0x7f6400006df0).ready entity=mon.0 client_cookie=35a98f18c73de292 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640e7fc700 1 -- 172.21.15.32:0/1113440857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000d570 con 0x7f64100b01a0 2022-08-01T22:57:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640e7fc700 1 -- 172.21.15.32:0/1113440857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6400000e90 con 0x7f64100b01a0 2022-08-01T22:57:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f640e7fc700 1 -- 172.21.15.32:0/1113440857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000d8c0 con 0x7f64100b01a0 2022-08-01T22:57:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- 172.21.15.32:0/1113440857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 msgr2=0x7f641014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f641014b4c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f6400002f70 tx=0x7f6400006df0).stop 2022-08-01T22:57:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- 172.21.15.32:0/1113440857 shutdown_connections 2022-08-01T22:57:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f641014b4c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 --2- 172.21.15.32:0/1113440857 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100b2710 0x7f641014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- 172.21.15.32:0/1113440857 >> 172.21.15.32:0/1113440857 conn(0x7f6410060310 msgr2=0x7f64100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- 172.21.15.32:0/1113440857 shutdown_connections 2022-08-01T22:57:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- 172.21.15.32:0/1113440857 wait complete. 2022-08-01T22:57:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 Processor -- start 2022-08-01T22:57:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.499+0000 7f6414d88700 1 -- start start 2022-08-01T22:57:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100afd60 0x7f64100678f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f64100b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64100b2710 0x7f64100aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64100b3290 con 0x7f64100b01a0 2022-08-01T22:57:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64100b3530 con 0x7f64100b2710 2022-08-01T22:57:39.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64100a9350 con 0x7f64100afd60 2022-08-01T22:57:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f64100b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f64100b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42342/0 (socket says 172.21.15.32:42342) 2022-08-01T22:57:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 -- 172.21.15.32:0/3743050249 learned_addr learned my addr 172.21.15.32:0/3743050249 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640f7fe700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100afd60 0x7f64100678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640ffff700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64100b2710 0x7f64100aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 -- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100afd60 msgr2=0x7f64100678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100afd60 0x7f64100678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 -- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64100b2710 msgr2=0x7f64100aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64100b2710 0x7f64100aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 -- 172.21.15.32:0/3743050249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640000b180 con 0x7f64100b01a0 2022-08-01T22:57:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640f7fe700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100afd60 0x7f64100678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:57:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640effd700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f64100b5ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f6400002f70 tx=0x7f6400015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000d8f0 con 0x7f64100b01a0 2022-08-01T22:57:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6400011ce0 con 0x7f64100b01a0 2022-08-01T22:57:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64100a95f0 con 0x7f64100b01a0 2022-08-01T22:57:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640001e650 con 0x7f64100b01a0 2022-08-01T22:57:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64100b3a40 con 0x7f64100b01a0 2022-08-01T22:57:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64100574e0 con 0x7f64100b01a0 2022-08-01T22:57:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.503+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6400025030 con 0x7f64100b01a0 2022-08-01T22:57:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.507+0000 7f640cff9700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63f00539e0 0x7f63f0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.507+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f640006d250 con 0x7f64100b01a0 2022-08-01T22:57:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.507+0000 7f640f7fe700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63f00539e0 0x7f63f0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.507+0000 7f640f7fe700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63f00539e0 0x7f63f0055ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6404008000 tx=0x7f6404008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.511+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f64000441e0 con 0x7f64100b01a0 2022-08-01T22:57:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.639+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6410069a70 con 0x7f64100b01a0 2022-08-01T22:57:39.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f640cff9700 1 -- 172.21.15.32:0/3743050249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6400023020 con 0x7f64100b01a0 2022-08-01T22:57:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63f00539e0 msgr2=0x7f63f0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63f00539e0 0x7f63f0055ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6404008000 tx=0x7f6404008040).stop 2022-08-01T22:57:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 msgr2=0x7f64100b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f64100b5ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f6400002f70 tx=0x7f6400015000).stop 2022-08-01T22:57:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 shutdown_connections 2022-08-01T22:57:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63f00539e0 0x7f63f0055ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64100b2710 0x7f64100aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64100b01a0 0x7f64100b5ba0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 --2- 172.21.15.32:0/3743050249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64100afd60 0x7f64100678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 >> 172.21.15.32:0/3743050249 conn(0x7f6410060310 msgr2=0x7f6410144cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 shutdown_connections 2022-08-01T22:57:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.647+0000 7f6414d88700 1 -- 172.21.15.32:0/3743050249 wait complete. 2022-08-01T22:57:39.669 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T22:57:39.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:39.691 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:39.714 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:39.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:57:39.757 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T22:57:39.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:39.758 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:39.759 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:39.766 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T22:57:39.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:57:39.769 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T22:57:39.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:39.770 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:39.771 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:57:39.780 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T22:57:39.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:57:39.781 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T22:57:39.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T22:57:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8b3f3700 1 Processor -- start 2022-08-01T22:57:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8b3f3700 1 -- start start 2022-08-01T22:57:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8b3f3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c840b5d80 0x7f7c840b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8b3f3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c84144240 con 0x7f7c840b4550 2022-08-01T22:57:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8b3f3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c840b6fe0 con 0x7f7c840b6710 2022-08-01T22:57:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8b3f3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84143d00 con 0x7f7c840b5d80 2022-08-01T22:57:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8898e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c840b4550 0x7f7c8412bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58126/0 (socket says 172.21.15.32:58126) 2022-08-01T22:57:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8898e700 1 -- 172.21.15.32:0/3087600380 learned_addr learned my addr 172.21.15.32:0/3087600380 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c89990700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c840b5d80 0x7f7c840b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212479316 0 0) 0x7f7c84144240 con 0x7f7c840b4550 2022-08-01T22:57:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c003260 con 0x7f7c840b4550 2022-08-01T22:57:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378588837 0 0) 0x7f7c6c003260 con 0x7f7c840b4550 2022-08-01T22:57:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c840b5d80 msgr2=0x7f7c840b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c840b5d80 0x7f7c840b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 >> v1:172.21.15.55:6789/0 conn(0x7f7c840b6710 legacy=0x7f7c84061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84143fa0 con 0x7f7c840b4550 2022-08-01T22:57:39.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f7c74001660 con 0x7f7c840b4550 2022-08-01T22:57:39.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c89990700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c840b5d80 0x7f7c840b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:39.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 >> v1:172.21.15.32:6789/0 conn(0x7f7c840b4550 legacy=0x7f7c8412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c6c003780 0x7f7c6c003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6c004090 0x7f7c6c004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c004d10 0x7f7c6c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84143d00 con 0x7f7c6c004090 2022-08-01T22:57:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c00bc40 con 0x7f7c6c004d10 2022-08-01T22:57:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c00bd80 con 0x7f7c6c003780 2022-08-01T22:57:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8918f700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c6c003780 0x7f7c6c003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8898e700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6c004090 0x7f7c6c004560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c89990700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c004d10 0x7f7c6c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8918f700 1 -- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c004d10 msgr2=0x7f7c6c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8918f700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c004d10 0x7f7c6c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8918f700 1 -- 172.21.15.32:0/3087600380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6c004090 msgr2=0x7f7c6c004560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8918f700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6c004090 0x7f7c6c004560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8918f700 1 -- 172.21.15.32:0/3087600380 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c003260 con 0x7f7c6c003780 2022-08-01T22:57:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.887+0000 7f7c8898e700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c6c004090 0x7f7c6c004560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8918f700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c6c003780 0x7f7c6c003b60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7c80001020 tx=0x7f7c800079e0).ready entity=mon.2 client_cookie=427e5b3c61375e25 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c80007cf0 con 0x7f7c6c003780 2022-08-01T22:57:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c80007030 con 0x7f7c6c003780 2022-08-01T22:57:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c73fff700 1 -- 172.21.15.32:0/3087600380 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c80007390 con 0x7f7c6c003780 2022-08-01T22:57:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c6c003780 msgr2=0x7f7c6c003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/3087600380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c6c003780 0x7f7c6c003b60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7c80001020 tx=0x7f7c800079e0).stop 2022-08-01T22:57:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c89990700 1 -- 172.21.15.32:0/3087600380 reap_dead start 2022-08-01T22:57:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/3087600380 shutdown_connections 2022-08-01T22:57:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/3087600380 >> 172.21.15.32:0/3087600380 conn(0x7f7c8405c620 msgr2=0x7f7c8405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/3087600380 shutdown_connections 2022-08-01T22:57:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/3087600380 wait complete. 2022-08-01T22:57:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 Processor -- start 2022-08-01T22:57:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- start start 2022-08-01T22:57:39.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c003780 0x7f7c840adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 0x7f7c84142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c84153030 0x7f7c84142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84152650 con 0x7f7c840afd30 2022-08-01T22:57:39.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c841528f0 con 0x7f7c6c003780 2022-08-01T22:57:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8b3f3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84152340 con 0x7f7c84153030 2022-08-01T22:57:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8898e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 0x7f7c84142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8898e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 0x7f7c84142230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42364/0 (socket says 172.21.15.32:42364) 2022-08-01T22:57:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8898e700 1 -- 172.21.15.32:0/1299930259 learned_addr learned my addr 172.21.15.32:0/1299930259 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c8918f700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c003780 0x7f7c840adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.891+0000 7f7c89990700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c84153030 0x7f7c84142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8898e700 1 -- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c84153030 msgr2=0x7f7c84142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8898e700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c84153030 0x7f7c84142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8898e700 1 -- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c003780 msgr2=0x7f7c840adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:39.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8898e700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c003780 0x7f7c840adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:39.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8898e700 1 -- 172.21.15.32:0/1299930259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84141c00 con 0x7f7c840afd30 2022-08-01T22:57:39.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8898e700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 0x7f7c84142230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f7c7400d000 tx=0x7f7c84144130).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:39.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c74011850 con 0x7f7c840afd30 2022-08-01T22:57:39.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c740119b0 con 0x7f7c840afd30 2022-08-01T22:57:39.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c84141eb0 con 0x7f7c840afd30 2022-08-01T22:57:39.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c74011d10 con 0x7f7c840afd30 2022-08-01T22:57:39.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c840b0560 con 0x7f7c840afd30 2022-08-01T22:57:39.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c841524b0 con 0x7f7c840afd30 2022-08-01T22:57:39.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7c7401b030 con 0x7f7c840afd30 2022-08-01T22:57:39.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c727fc700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c64053b00 0x7f7c64055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:39.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7c7406b890 con 0x7f7c840afd30 2022-08-01T22:57:39.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.895+0000 7f7c8918f700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c64053b00 0x7f7c64055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:39.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.899+0000 7f7c8918f700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c64053b00 0x7f7c64055fc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f7c8001b000 tx=0x7f7c80015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:39.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:39.903+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c841524b0 con 0x7f7c840afd30 2022-08-01T22:57:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.027+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7c84152a60 con 0x7f7c840afd30 2022-08-01T22:57:40.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.031+0000 7f7c727fc700 1 -- 172.21.15.32:0/1299930259 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v38) v1 ==== 81+0+29219 (secure 0 0 0) 0x7f7c84152a60 con 0x7f7c840afd30 2022-08-01T22:57:40.035 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:40.036 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:57:40.036 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 38, 2022-08-01T22:57:40.036 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T22:57:40.037 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T22:57:40.037 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T22:57:37.341729+0000", 2022-08-01T22:57:40.037 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:57:40.037 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:57:40.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T22:57:40.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T22:57:40.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T22:57:40.038 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T22:57:40.038 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T22:57:40.039 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T22:57:40.039 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T22:57:40.039 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:40.039 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T22:57:40.040 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T22:57:40.040 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T22:57:40.040 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T22:57:40.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T22:57:40.040 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 6, 2022-08-01T22:57:40.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T22:57:40.041 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T22:57:40.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T22:57:40.041 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T22:57:40.042 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T22:57:40.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.042 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T22:57:40.042 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T22:57:40.043 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T22:57:40.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:57:40.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:57:40.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:57:40.044 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:57:40.044 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:57:40.044 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:57:40.044 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:57:40.044 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:57:40.045 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T22:57:40.045 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T22:57:40.045 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T22:57:40.045 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T22:57:40.045 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T22:57:40.046 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:57:40.046 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:57:40.046 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:57:40.046 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:57:40.047 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:57:40.047 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:57:40.047 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:57:40.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.047 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T22:57:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:57:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:57:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:57:40.048 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:57:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:57:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:57:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:57:40.049 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:57:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:57:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:57:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:57:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:57:40.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:57:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:57:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:57:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:57:40.051 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:57:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:57:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:57:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:57:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:57:40.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:57:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:57:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:57:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:57:40.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:57:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:57:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:57:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:57:40.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:57:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:57:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:57:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:57:40.055 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:57:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:57:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:57:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:57:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:57:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T22:57:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:57:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T22:57:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T22:57:40.058 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T22:57:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T22:57:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:57:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:57:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:57:40.059 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:57:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:57:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:57:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:57:40.060 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:57:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:57:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:57:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:57:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:57:40.061 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:57:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:57:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:57:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:57:40.062 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:57:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:57:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:57:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:57:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.063 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T22:57:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:57:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:57:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:57:40.064 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:57:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:57:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:57:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:57:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:57:40.065 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:57:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:57:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:57:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:57:40.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:57:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:57:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:57:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:57:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:57:40.067 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:57:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:57:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:57:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:57:40.068 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:57:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:57:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:57:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:57:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:57:40.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:57:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:57:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:57:40.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:57:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:57:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:57:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:57:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:57:40.071 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:57:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:57:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:57:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:57:40.072 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:57:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T22:57:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T22:57:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.074 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:57:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T22:57:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.075 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T22:57:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T22:57:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T22:57:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T22:57:40.082 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.083 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T22:57:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T22:57:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.087 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T22:57:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T22:57:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:57:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T22:57:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T22:57:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T22:57:40.090 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T22:57:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:57:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T22:57:40.092 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.093 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.094 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T22:57:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T22:57:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.097 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T22:57:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T22:57:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T22:57:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T22:57:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T22:57:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T22:57:40.106 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:57:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T22:57:40.107 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T22:57:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T22:57:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T22:57:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.108 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:57:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T22:57:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.110 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.111 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2022-08-01T22:57:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.112 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T22:57:40.113 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.114 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T22:57:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.116 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T22:57:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T22:57:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T22:57:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T22:57:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.121 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T22:57:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T22:57:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:57:40.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T22:57:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T22:57:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T22:57:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T22:57:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:57:40.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T22:57:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.128 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:40.129 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T22:57:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T22:57:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T22:57:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T22:57:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T22:57:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T22:57:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.139 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T22:57:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T22:57:40.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:57:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.142 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T22:57:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T22:57:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T22:57:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T22:57:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.143 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:57:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T22:57:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.145 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.146 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.146 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.146 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.146 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.147 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:40.147 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.147 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.147 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.148 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T22:57:40.148 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T22:57:40.150 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.151 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T22:57:40.153 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T22:57:40.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.155 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.155 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T22:57:40.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T22:57:40.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.158 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.159 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T22:57:40.160 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T22:57:40.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:57:40.162 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.162 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.162 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T22:57:40.163 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T22:57:40.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T22:57:40.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T22:57:40.164 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.164 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.164 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:57:40.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T22:57:40.165 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.166 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.166 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.166 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.167 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.167 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.167 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:40.167 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.168 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.168 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T22:57:40.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.170 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T22:57:40.170 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.171 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T22:57:40.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T22:57:40.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.174 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T22:57:40.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T22:57:40.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T22:57:40.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T22:57:40.180 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:57:40.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.181 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T22:57:40.181 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T22:57:40.181 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T22:57:40.181 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T22:57:40.181 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.182 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.182 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:57:40.183 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T22:57:40.183 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.184 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.184 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.184 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.184 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.184 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.185 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2022-08-01T22:57:40.185 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.185 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.186 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T22:57:40.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T22:57:40.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.190 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T22:57:40.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.191 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T22:57:40.191 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.192 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.192 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.193 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T22:57:40.193 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T22:57:40.194 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.195 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.195 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T22:57:40.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.197 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T22:57:40.198 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:57:40.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.198 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T22:57:40.199 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T22:57:40.199 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T22:57:40.199 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T22:57:40.199 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.200 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.200 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.201 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:57:40.201 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T22:57:40.201 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:57:40.201 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:57:40.202 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:57:40.202 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:57:40.202 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:57:40.202 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:57:40.202 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:57:40.203 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 33, 2022-08-01T22:57:40.203 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:57:40.203 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:57:40.203 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:57:40.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T22:57:40.204 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T22:57:40.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.207 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:57:40.207 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.208 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T22:57:40.208 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.209 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T22:57:40.209 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.210 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:57:40.210 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.211 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T22:57:40.211 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T22:57:40.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.214 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:57:40.214 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:57:40.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:57:40.214 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T22:57:40.215 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:57:40.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T22:57:40.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:57:40.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.217 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T22:57:40.217 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T22:57:40.217 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T22:57:40.217 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T22:57:40.218 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:57:40.218 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:57:40.218 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:57:40.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:40.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.219 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:40.219 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T22:57:40.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.219 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:57:40.220 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.220 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.220 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.220 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.221 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.221 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T22:57:40.221 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.222 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:57:40.222 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.222 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.222 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.223 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.223 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.223 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T22:57:40.223 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.224 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:57:40.224 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.225 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.225 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.225 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.225 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T22:57:40.226 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.226 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:57:40.227 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.227 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.227 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.227 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.228 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T22:57:40.228 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.229 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:57:40.229 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.229 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.229 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.230 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.230 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.230 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T22:57:40.230 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.231 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:57:40.231 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.231 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.232 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.232 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.232 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.232 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T22:57:40.232 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.233 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:57:40.233 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.233 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.234 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.234 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.234 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.234 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T22:57:40.235 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:40.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:57:40.236 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:57:40.236 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:57:40.236 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:57:40.236 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:57:40.237 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:57:40.237 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T22:57:40.237 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:57:40.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.237 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:57:40.238 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T22:57:40.238 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T22:57:40.238 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T22:57:40.238 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T22:57:40.239 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T22:57:40.239 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6833/3560980402": "2022-08-02T22:57:33.428916+0000", 2022-08-01T22:57:40.239 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6832/3560980402": "2022-08-02T22:57:33.428916+0000" 2022-08-01T22:57:40.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.240 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T22:57:40.240 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T22:57:40.240 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T22:57:40.240 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:57:40.240 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T22:57:40.241 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T22:57:40.241 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:57:40.241 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T22:57:40.241 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:57:40.242 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:57:40.242 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T22:57:40.242 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T22:57:40.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T22:57:40.243 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:57:40.243 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:57:40.243 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T22:57:40.243 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:57:40.244 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:57:40.244 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T22:57:40.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:40.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:40.244 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T22:57:40.245 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T22:57:40.245 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T22:57:40.245 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T22:57:40.245 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T22:57:40.246 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:57:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c64053b00 msgr2=0x7f7c64055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c64053b00 0x7f7c64055fc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f7c8001b000 tx=0x7f7c80015000).stop 2022-08-01T22:57:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 msgr2=0x7f7c84142230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 0x7f7c84142230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f7c7400d000 tx=0x7f7c84144130).stop 2022-08-01T22:57:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 shutdown_connections 2022-08-01T22:57:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c64053b00 0x7f7c64055fc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7c84153030 0x7f7c84142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c840afd30 0x7f7c84142230 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 --2- 172.21.15.32:0/1299930259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c6c003780 0x7f7c840adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 >> 172.21.15.32:0/1299930259 conn(0x7f7c8405c620 msgr2=0x7f7c84128710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 shutdown_connections 2022-08-01T22:57:40.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.035+0000 7f7c8b3f3700 1 -- 172.21.15.32:0/1299930259 wait complete. 2022-08-01T22:57:40.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 Processor -- start 2022-08-01T22:57:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 -- start start 2022-08-01T22:57:40.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c680b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c680b6af0 0x7f5c6812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c68144240 con 0x7f5c680b6710 2022-08-01T22:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c68143d00 con 0x7f5c680b6af0 2022-08-01T22:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6f53f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68143fa0 con 0x7f5c680b4550 2022-08-01T22:57:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c680b6af0 0x7f5c6812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c680b6af0 0x7f5c6812bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52160/0 (socket says 172.21.15.32:52160) 2022-08-01T22:57:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 -- 172.21.15.32:0/3082573529 learned_addr learned my addr 172.21.15.32:0/3082573529 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6cada700 1 --2- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c680b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6d2db700 1 --1- 172.21.15.32:0/3082573529 >> v1:172.21.15.55:6789/0 conn(0x7f5c680b6710 0x7f5c68061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:50818/0 (socket says 172.21.15.32:50818) 2022-08-01T22:57:40.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 -- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 msgr2=0x7f5c680b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:40.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 --2- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c680b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 -- 172.21.15.32:0/3082573529 >> v1:172.21.15.55:6789/0 conn(0x7f5c680b6710 legacy=0x7f5c68061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 -- 172.21.15.32:0/3082573529 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68061780 con 0x7f5c680b6af0 2022-08-01T22:57:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c6dadc700 1 --2- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6af0 0x7f5c6812bb90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5c600032f0 tx=0x7f5c60009290).ready entity=mon.1 client_cookie=e08667bc2df2e8c9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c57fff700 1 -- 172.21.15.32:0/3082573529 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c6000b8a0 con 0x7f5c680b6af0 2022-08-01T22:57:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.363+0000 7f5c57fff700 1 -- 172.21.15.32:0/3082573529 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6000ba00 con 0x7f5c680b6af0 2022-08-01T22:57:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c57fff700 1 -- 172.21.15.32:0/3082573529 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c6000bb70 con 0x7f5c680b6af0 2022-08-01T22:57:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6af0 msgr2=0x7f5c6812bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6af0 0x7f5c6812bb90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5c600032f0 tx=0x7f5c60009290).stop 2022-08-01T22:57:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- 172.21.15.32:0/3082573529 shutdown_connections 2022-08-01T22:57:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6af0 0x7f5c6812bb90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/3082573529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c680b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- 172.21.15.32:0/3082573529 >> 172.21.15.32:0/3082573529 conn(0x7f5c6805c620 msgr2=0x7f5c6805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- 172.21.15.32:0/3082573529 shutdown_connections 2022-08-01T22:57:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- 172.21.15.32:0/3082573529 wait complete. 2022-08-01T22:57:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 Processor -- start 2022-08-01T22:57:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- start start 2022-08-01T22:57:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c681449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6710 0x7f5c68141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 0x7f5c681559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b3e00 con 0x7f5c680b6af0 2022-08-01T22:57:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b3430 con 0x7f5c680b6710 2022-08-01T22:57:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6f53f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6812b130 con 0x7f5c680b4550 2022-08-01T22:57:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6dadc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 0x7f5c681559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6dadc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 0x7f5c681559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42380/0 (socket says 172.21.15.32:42380) 2022-08-01T22:57:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6dadc700 1 -- 172.21.15.32:0/554624287 learned_addr learned my addr 172.21.15.32:0/554624287 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6cada700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6710 0x7f5c68141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.367+0000 7f5c6d2db700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c681449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6dadc700 1 -- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 msgr2=0x7f5c681449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6dadc700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c681449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6dadc700 1 -- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6710 msgr2=0x7f5c68141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6dadc700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6710 0x7f5c68141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6dadc700 1 -- 172.21.15.32:0/554624287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009530 con 0x7f5c680b6af0 2022-08-01T22:57:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6cada700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6710 0x7f5c68141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6dadc700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 0x7f5c681559d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f5c681269b0 tx=0x7f5c60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c6000b8a0 con 0x7f5c680b6af0 2022-08-01T22:57:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60021440 con 0x7f5c680b6af0 2022-08-01T22:57:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6812ae90 con 0x7f5c680b6af0 2022-08-01T22:57:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c600217a0 con 0x7f5c680b6af0 2022-08-01T22:57:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68126550 con 0x7f5c680b6af0 2022-08-01T22:57:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c68156060 con 0x7f5c680b6af0 2022-08-01T22:57:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.371+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c60027020 con 0x7f5c680b6af0 2022-08-01T22:57:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.375+0000 7f5c567fc700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c48053aa0 0x7f5c48055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.375+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5c6006d190 con 0x7f5c680b6af0 2022-08-01T22:57:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.375+0000 7f5c6d2db700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c48053aa0 0x7f5c48055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.375+0000 7f5c6d2db700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c48053aa0 0x7f5c48055f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f5c6400d000 tx=0x7f5c64007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.383+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c60044270 con 0x7f5c680b6af0 2022-08-01T22:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.547+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c68061a20 con 0x7f5c680b6af0 2022-08-01T22:57:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c567fc700 1 -- 172.21.15.32:0/554624287 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+1555 (secure 0 0 0) 0x7f5c60015090 con 0x7f5c680b6af0 2022-08-01T22:57:40.554 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:40.555 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/589932339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":589932339},{"type":"v1","addr":"172.21.15.32:6833","nonce":589932339}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[]} 2022-08-01T22:57:40.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c48053aa0 msgr2=0x7f5c48055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c48053aa0 0x7f5c48055f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f5c6400d000 tx=0x7f5c64007b80).stop 2022-08-01T22:57:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 msgr2=0x7f5c681559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 0x7f5c681559d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f5c681269b0 tx=0x7f5c60016000).stop 2022-08-01T22:57:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 shutdown_connections 2022-08-01T22:57:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c48053aa0 0x7f5c48055f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c680b6af0 0x7f5c681559d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c680b6710 0x7f5c68141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 --2- 172.21.15.32:0/554624287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c680b4550 0x7f5c681449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 >> 172.21.15.32:0/554624287 conn(0x7f5c6805c620 msgr2=0x7f5c681422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 shutdown_connections 2022-08-01T22:57:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.551+0000 7f5c6f53f700 1 -- 172.21.15.32:0/554624287 wait complete. 2022-08-01T22:57:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 14 2022-08-01T22:57:40.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:40.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 Processor -- start 2022-08-01T22:57:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 -- start start 2022-08-01T22:57:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4550042530 0x7f4550005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550040c90 0x7f45500400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4550041640 con 0x7f455003f590 2022-08-01T22:57:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455003e450 con 0x7f4550042530 2022-08-01T22:57:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455003f1a0 con 0x7f4550040c90 2022-08-01T22:57:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4556d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f455003f590 0x7f45500d8950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58162/0 (socket says 172.21.15.32:58162) 2022-08-01T22:57:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4556d9d700 1 -- 172.21.15.32:0/657609474 learned_addr learned my addr 172.21.15.32:0/657609474 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4555d9b700 1 --2- 172.21.15.32:0/657609474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550040c90 0x7f45500400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455659c700 1 --2- 172.21.15.32:0/657609474 >> v2:172.21.15.55:3300/0 conn(0x7f4550042530 0x7f4550005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455559a700 1 -- 172.21.15.32:0/657609474 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377350025 0 0) 0x7f4550041640 con 0x7f455003f590 2022-08-01T22:57:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455559a700 1 -- 172.21.15.32:0/657609474 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4538003260 con 0x7f455003f590 2022-08-01T22:57:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4555d9b700 1 -- 172.21.15.32:0/657609474 >> v2:172.21.15.55:3300/0 conn(0x7f4550042530 msgr2=0x7f4550005660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4555d9b700 1 --2- 172.21.15.32:0/657609474 >> v2:172.21.15.55:3300/0 conn(0x7f4550042530 0x7f4550005660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4555d9b700 1 -- 172.21.15.32:0/657609474 >> v1:172.21.15.32:6789/0 conn(0x7f455003f590 legacy=0x7f45500d8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4555d9b700 1 -- 172.21.15.32:0/657609474 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4550042910 con 0x7f4550040c90 2022-08-01T22:57:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f4555d9b700 1 --2- 172.21.15.32:0/657609474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550040c90 0x7f45500400c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4540002f70 tx=0x7f4540006df0).ready entity=mon.2 client_cookie=4fda6b9cda35cae4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.671+0000 7f455559a700 1 -- 172.21.15.32:0/657609474 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f454000d4a0 con 0x7f4550040c90 2022-08-01T22:57:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455559a700 1 -- 172.21.15.32:0/657609474 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540000e90 con 0x7f4550040c90 2022-08-01T22:57:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455559a700 1 -- 172.21.15.32:0/657609474 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f454000d7f0 con 0x7f4550040c90 2022-08-01T22:57:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 -- 172.21.15.32:0/657609474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550040c90 msgr2=0x7f45500400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 --2- 172.21.15.32:0/657609474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550040c90 0x7f45500400c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4540002f70 tx=0x7f4540006df0).stop 2022-08-01T22:57:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 -- 172.21.15.32:0/657609474 shutdown_connections 2022-08-01T22:57:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 --2- 172.21.15.32:0/657609474 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550040c90 0x7f45500400c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 --2- 172.21.15.32:0/657609474 >> v2:172.21.15.55:3300/0 conn(0x7f4550042530 0x7f4550005660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 -- 172.21.15.32:0/657609474 >> 172.21.15.32:0/657609474 conn(0x7f45500462f0 msgr2=0x7f4550046700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 -- 172.21.15.32:0/657609474 shutdown_connections 2022-08-01T22:57:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 -- 172.21.15.32:0/657609474 wait complete. 2022-08-01T22:57:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.675+0000 7f455759e700 1 Processor -- start 2022-08-01T22:57:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 -- start start 2022-08-01T22:57:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455003f590 0x7f455003b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4550040c90 0x7f45500a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550042530 0x7f45500ea870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455003a160 con 0x7f455003f590 2022-08-01T22:57:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455003a680 con 0x7f4550040c90 2022-08-01T22:57:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500a1280 con 0x7f4550042530 2022-08-01T22:57:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455003f590 0x7f455003b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455003f590 0x7f455003b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42390/0 (socket says 172.21.15.32:42390) 2022-08-01T22:57:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455659c700 1 -- 172.21.15.32:0/1984754017 learned_addr learned my addr 172.21.15.32:0/1984754017 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550042530 0x7f45500ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4555d9b700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4550040c90 0x7f45500a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 -- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4550040c90 msgr2=0x7f45500a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4550040c90 0x7f45500a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 -- 172.21.15.32:0/1984754017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455003f590 msgr2=0x7f455003b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455003f590 0x7f455003b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 -- 172.21.15.32:0/1984754017 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454000b180 con 0x7f4550042530 2022-08-01T22:57:40.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f4556d9d700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550042530 0x7f45500ea870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f45500a09a0 tx=0x7f455805f750).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:40.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4558069c60 con 0x7f4550042530 2022-08-01T22:57:40.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558069dc0 con 0x7f4550042530 2022-08-01T22:57:40.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455003ad80 con 0x7f4550042530 2022-08-01T22:57:40.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4558068620 con 0x7f4550042530 2022-08-01T22:57:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.679+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455003aae0 con 0x7f4550042530 2022-08-01T22:57:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.683+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455003b1a0 con 0x7f4550042530 2022-08-01T22:57:40.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.683+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4558068780 con 0x7f4550042530 2022-08-01T22:57:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.683+0000 7f454f7fe700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f453c0539e0 0x7f453c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.683+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f45580da070 con 0x7f4550042530 2022-08-01T22:57:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.683+0000 7f455659c700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f453c0539e0 0x7f453c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.683+0000 7f455659c700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f453c0539e0 0x7f453c055ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f454800b000 tx=0x7f4548006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.691+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45580b1000 con 0x7f4550042530 2022-08-01T22:57:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.855+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4550041870 con 0x7f4550042530 2022-08-01T22:57:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f454f7fe700 1 -- 172.21.15.32:0/1984754017 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+1555 (secure 0 0 0) 0x7f45580b4620 con 0x7f4550042530 2022-08-01T22:57:40.863 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:40.863 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/901881548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":901881548},{"type":"v1","addr":"172.21.15.55:6835","nonce":901881548}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1146314083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1146314083},{"type":"v1","addr":"172.21.15.32:6835","nonce":1146314083}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/589932339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":589932339},{"type":"v1","addr":"172.21.15.32:6833","nonce":589932339}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[]} 2022-08-01T22:57:40.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f453c0539e0 msgr2=0x7f453c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f453c0539e0 0x7f453c055ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f454800b000 tx=0x7f4548006df0).stop 2022-08-01T22:57:40.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550042530 msgr2=0x7f45500ea870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:40.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550042530 0x7f45500ea870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f45500a09a0 tx=0x7f455805f750).stop 2022-08-01T22:57:40.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 shutdown_connections 2022-08-01T22:57:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f453c0539e0 0x7f453c055ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4550042530 0x7f45500ea870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4550040c90 0x7f45500a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 --2- 172.21.15.32:0/1984754017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455003f590 0x7f455003b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:40.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 >> 172.21.15.32:0/1984754017 conn(0x7f45500462f0 msgr2=0x7f4550005be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:40.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 shutdown_connections 2022-08-01T22:57:40.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:40.859+0000 7f455759e700 1 -- 172.21.15.32:0/1984754017 wait complete. 2022-08-01T22:57:40.869 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 14 2022-08-01T22:57:40.876 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T22:57:40.876 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T22:57:40.876 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T22:57:40.877 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:57:40.877 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T22:57:40.877 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T22:57:40.878 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T22:57:40.878 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:57:40.878 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T22:57:40.879 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T22:57:40.879 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T22:57:40.879 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:57:40.889 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T22:57:40.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T22:57:40.891 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T22:57:40.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T22:57:40.894 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T22:57:40.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T22:57:40.897 INFO:tasks.ceph.mds.b:Started 2022-08-01T22:57:40.897 DEBUG:teuthology.parallel:result is None 2022-08-01T22:57:40.931 INFO:tasks.ceph.mds.a:Started 2022-08-01T22:57:40.932 DEBUG:teuthology.parallel:result is None 2022-08-01T22:57:40.932 INFO:tasks.ceph.mds.c:Started 2022-08-01T22:57:40.933 DEBUG:teuthology.parallel:result is None 2022-08-01T22:57:40.933 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T22:57:40.963 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T22:57:40.995 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T22:57:41.007 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T22:57:41.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 Processor -- start 2022-08-01T22:57:41.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 -- start start 2022-08-01T22:57:41.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 0x7fa324143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3240aad90 0x7fa32413d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa32405f840 0x7fa32413d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32413dda0 con 0x7fa3241437f0 2022-08-01T22:57:41.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3240a8cf0 con 0x7fa32405f840 2022-08-01T22:57:41.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa328865700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240a8f50 con 0x7fa3240aad90 2022-08-01T22:57:41.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa32259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 0x7fa324143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa321d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3240aad90 0x7fa32413d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa32259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 0x7fa324143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42422/0 (socket says 172.21.15.32:42422) 2022-08-01T22:57:41.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa32259c700 1 -- 172.21.15.32:0/2638838191 learned_addr learned my addr 172.21.15.32:0/2638838191 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:41.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.027+0000 7fa322d9d700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.55:3300/0 conn(0x7fa32405f840 0x7fa32413d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa321d9b700 1 -- 172.21.15.32:0/2638838191 >> v2:172.21.15.55:3300/0 conn(0x7fa32405f840 msgr2=0x7fa32413d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:41.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa321d9b700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.55:3300/0 conn(0x7fa32405f840 0x7fa32413d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa321d9b700 1 -- 172.21.15.32:0/2638838191 >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 msgr2=0x7fa324143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:41.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa321d9b700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 0x7fa324143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa321d9b700 1 -- 172.21.15.32:0/2638838191 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3241441d0 con 0x7fa3240aad90 2022-08-01T22:57:41.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa322d9d700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.55:3300/0 conn(0x7fa32405f840 0x7fa32413d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:41.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa32259c700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 0x7fa324143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa321d9b700 1 --2- 172.21.15.32:0/2638838191 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3240aad90 0x7fa32413d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa314002f70 tx=0x7fa314006df0).ready entity=mon.2 client_cookie=c2cff90c3c9279c9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa32159a700 1 -- 172.21.15.32:0/2638838191 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31400d4a0 con 0x7fa3240aad90 2022-08-01T22:57:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa32159a700 1 -- 172.21.15.32:0/2638838191 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa314000e90 con 0x7fa3240aad90 2022-08-01T22:57:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa32159a700 1 -- 172.21.15.32:0/2638838191 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31400d7f0 con 0x7fa3240aad90 2022-08-01T22:57:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 -- 172.21.15.32:0/2638838191 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3240aad90 msgr2=0x7fa32413d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:41.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 --2- 172.21.15.32:0/2638838191 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3240aad90 0x7fa32413d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa314002f70 tx=0x7fa314006df0).stop 2022-08-01T22:57:41.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 -- 172.21.15.32:0/2638838191 shutdown_connections 2022-08-01T22:57:41.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.55:3300/0 conn(0x7fa32405f840 0x7fa32413d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 --2- 172.21.15.32:0/2638838191 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3240aad90 0x7fa32413d3c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 --2- 172.21.15.32:0/2638838191 >> v2:172.21.15.32:3300/0 conn(0x7fa3241437f0 0x7fa324143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 -- 172.21.15.32:0/2638838191 >> 172.21.15.32:0/2638838191 conn(0x7fa3240b5570 msgr2=0x7fa3240b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:41.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 -- 172.21.15.32:0/2638838191 shutdown_connections 2022-08-01T22:57:41.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 -- 172.21.15.32:0/2638838191 wait complete. 2022-08-01T22:57:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.031+0000 7fa328865700 1 Processor -- start 2022-08-01T22:57:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 -- start start 2022-08-01T22:57:41.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa32405f840 0x7fa3240b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240aad90 0x7fa32414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3241437f0 0x7fa32414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32413aa80 con 0x7fa3240aad90 2022-08-01T22:57:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32414f510 con 0x7fa3241437f0 2022-08-01T22:57:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32413c2c0 con 0x7fa32405f840 2022-08-01T22:57:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa321d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240aad90 0x7fa32414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa321d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240aad90 0x7fa32414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42430/0 (socket says 172.21.15.32:42430) 2022-08-01T22:57:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa321d9b700 1 -- 172.21.15.32:0/2570610519 learned_addr learned my addr 172.21.15.32:0/2570610519 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa32259c700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa32405f840 0x7fa3240b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3241437f0 0x7fa32414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 -- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa32405f840 msgr2=0x7fa3240b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa32405f840 0x7fa3240b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 -- 172.21.15.32:0/2570610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240aad90 msgr2=0x7fa32414f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:41.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240aad90 0x7fa32414f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:41.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 -- 172.21.15.32:0/2570610519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c009470 con 0x7fa3241437f0 2022-08-01T22:57:41.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa322d9d700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3241437f0 0x7fa32414fdf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fa31c00c000 tx=0x7fa31c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:41.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31c00ee60 con 0x7fa3241437f0 2022-08-01T22:57:41.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00c850 con 0x7fa3241437f0 2022-08-01T22:57:41.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31c00cbb0 con 0x7fa3241437f0 2022-08-01T22:57:41.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c00b1e0 con 0x7fa3241437f0 2022-08-01T22:57:41.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.035+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31400b180 con 0x7fa3241437f0 2022-08-01T22:57:41.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.039+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32413c8c0 con 0x7fa3241437f0 2022-08-01T22:57:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.039+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa31c022070 con 0x7fa3241437f0 2022-08-01T22:57:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.039+0000 7fa31b7fe700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa308053aa0 0x7fa308055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.039+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa31c06c770 con 0x7fa3241437f0 2022-08-01T22:57:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.039+0000 7fa32259c700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa308053aa0 0x7fa308055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.039+0000 7fa32259c700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa308053aa0 0x7fa308055f60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fa30c00b000 tx=0x7fa30c006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.047+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa31c043700 con 0x7fa3241437f0 2022-08-01T22:57:41.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:41.183+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa3240574e0 con 0x7fa3241437f0 2022-08-01T22:57:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.115+0000 7fa31b7fe700 1 -- 172.21.15.32:0/2570610519 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa31c016070 con 0x7fa3241437f0 2022-08-01T22:57:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa308053aa0 msgr2=0x7fa308055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa308053aa0 0x7fa308055f60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fa30c00b000 tx=0x7fa30c006eb0).stop 2022-08-01T22:57:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3241437f0 msgr2=0x7fa32414fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3241437f0 0x7fa32414fdf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fa31c00c000 tx=0x7fa31c0091d0).stop 2022-08-01T22:57:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 shutdown_connections 2022-08-01T22:57:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa308053aa0 0x7fa308055f60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3241437f0 0x7fa32414fdf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240aad90 0x7fa32414f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 --2- 172.21.15.32:0/2570610519 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa32405f840 0x7fa3240b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 >> 172.21.15.32:0/2570610519 conn(0x7fa3240b5570 msgr2=0x7fa3240afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 shutdown_connections 2022-08-01T22:57:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.119+0000 7fa328865700 1 -- 172.21.15.32:0/2570610519 wait complete. 2022-08-01T22:57:42.132 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T22:57:42.140 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T22:57:42.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T22:57:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89d7d0700 1 Processor -- start 2022-08-01T22:57:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89d7d0700 1 -- start start 2022-08-01T22:57:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89d7d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8980617f0 0x7fa8980b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89d7d0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa898061bd0 con 0x7fa898144d20 2022-08-01T22:57:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89d7d0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8981451a0 con 0x7fa8980b3f90 2022-08-01T22:57:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89d7d0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980a9fd0 con 0x7fa8980617f0 2022-08-01T22:57:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa897fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8980617f0 0x7fa8980b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89cfcf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa898144d20 0x7fa89814aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58216/0 (socket says 172.21.15.32:58216) 2022-08-01T22:57:42.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa89cfcf700 1 -- 172.21.15.32:0/2421740547 learned_addr learned my addr 172.21.15.32:0/2421740547 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:42.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa897fff700 1 -- 172.21.15.32:0/2421740547 >> v1:172.21.15.55:6789/0 conn(0x7fa8980b3f90 legacy=0x7fa898144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa897fff700 1 -- 172.21.15.32:0/2421740547 >> v1:172.21.15.32:6789/0 conn(0x7fa898144d20 legacy=0x7fa89814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa896ffd700 1 -- 172.21.15.32:0/2421740547 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754678234 0 0) 0x7fa898061bd0 con 0x7fa898144d20 2022-08-01T22:57:42.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa897fff700 1 -- 172.21.15.32:0/2421740547 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980a88e0 con 0x7fa8980617f0 2022-08-01T22:57:42.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa897fff700 1 --2- 172.21.15.32:0/2421740547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa8980b4640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fa888002f70 tx=0x7fa888006df0).ready entity=mon.0 client_cookie=7ebdee72d3ed907e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.243+0000 7fa896ffd700 1 -- 172.21.15.32:0/2421740547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa88800d610 con 0x7fa8980617f0 2022-08-01T22:57:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa896ffd700 1 -- 172.21.15.32:0/2421740547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888000e90 con 0x7fa8980617f0 2022-08-01T22:57:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa896ffd700 1 -- 172.21.15.32:0/2421740547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa88800d960 con 0x7fa8980617f0 2022-08-01T22:57:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- 172.21.15.32:0/2421740547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 msgr2=0x7fa8980b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/2421740547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa8980b4640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fa888002f70 tx=0x7fa888006df0).stop 2022-08-01T22:57:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- 172.21.15.32:0/2421740547 shutdown_connections 2022-08-01T22:57:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/2421740547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa8980b4640 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- 172.21.15.32:0/2421740547 >> 172.21.15.32:0/2421740547 conn(0x7fa898066f70 msgr2=0x7fa898067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- 172.21.15.32:0/2421740547 shutdown_connections 2022-08-01T22:57:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- 172.21.15.32:0/2421740547 wait complete. 2022-08-01T22:57:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 Processor -- start 2022-08-01T22:57:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- start start 2022-08-01T22:57:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa898157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8980b3f90 0x7fa898065b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa898144d20 0x7fa898062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89814b5e0 con 0x7fa8980617f0 2022-08-01T22:57:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980b03c0 con 0x7fa8980b3f90 2022-08-01T22:57:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89d7d0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa898065950 con 0x7fa898144d20 2022-08-01T22:57:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa897fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa898157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa897fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa898157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42446/0 (socket says 172.21.15.32:42446) 2022-08-01T22:57:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa897fff700 1 -- 172.21.15.32:0/47019249 learned_addr learned my addr 172.21.15.32:0/47019249 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa8977fe700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8980b3f90 0x7fa898065b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.247+0000 7fa89cfcf700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa898144d20 0x7fa898062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa897fff700 1 -- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa898144d20 msgr2=0x7fa898062e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa897fff700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa898144d20 0x7fa898062e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa897fff700 1 -- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8980b3f90 msgr2=0x7fa898065b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa897fff700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8980b3f90 0x7fa898065b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa897fff700 1 -- 172.21.15.32:0/47019249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88800b180 con 0x7fa8980617f0 2022-08-01T22:57:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa8977fe700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8980b3f90 0x7fa898065b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa897fff700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa898157fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa888002f70 tx=0x7fa888015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa88800d290 con 0x7fa8980617f0 2022-08-01T22:57:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888011b80 con 0x7fa8980617f0 2022-08-01T22:57:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89814b810 con 0x7fa8980617f0 2022-08-01T22:57:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa88801e460 con 0x7fa8980617f0 2022-08-01T22:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980b4690 con 0x7fa8980617f0 2022-08-01T22:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8980a8b80 con 0x7fa8980617f0 2022-08-01T22:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.251+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa888025030 con 0x7fa8980617f0 2022-08-01T22:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.255+0000 7fa8957fa700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa878053a70 0x7fa878055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.255+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa88806d0b0 con 0x7fa8980617f0 2022-08-01T22:57:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.255+0000 7fa8977fe700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa878053a70 0x7fa878055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.255+0000 7fa8977fe700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa878053a70 0x7fa878055f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fa88c00d000 tx=0x7fa88c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.259+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa8880441b0 con 0x7fa8980617f0 2022-08-01T22:57:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.423+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa8980574e0 con 0x7fa8980617f0 2022-08-01T22:57:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa8957fa700 1 -- 172.21.15.32:0/47019249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fa8880477d0 con 0x7fa8980617f0 2022-08-01T22:57:42.431 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:42.431 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:57:42.432 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T22:57:42.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.432 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T22:57:42.432 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T22:57:42.432 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.433 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:57:42.433 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.434 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.435 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T22:57:42.435 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T22:57:42.435 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:57:42.436 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.436 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.437 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T22:57:42.437 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T22:57:42.438 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.438 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:57:42.438 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.438 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.438 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.439 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T22:57:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T22:57:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T22:57:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T22:57:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T22:57:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T22:57:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T22:57:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T22:57:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T22:57:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T22:57:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.450 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T22:57:42.450 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T22:57:42.450 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.450 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.451 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.451 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.451 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.452 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T22:57:42.452 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T22:57:42.452 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.452 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.453 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.453 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.454 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T22:57:42.454 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T22:57:42.454 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.454 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.455 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.455 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.456 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T22:57:42.456 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T22:57:42.456 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.456 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:57:42.457 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T22:57:42.457 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T22:57:42.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.458 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T22:57:42.458 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T22:57:42.458 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.458 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:57:42.459 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T22:57:42.459 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T22:57:42.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.460 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T22:57:42.460 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T22:57:42.460 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.460 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:57:42.461 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:57:42.461 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:57:42.461 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.462 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T22:57:42.462 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T22:57:42.462 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.463 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:57:42.463 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T22:57:42.463 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:42.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:42.464 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T22:57:42.464 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T22:57:42.464 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:57:42.465 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:57:42.465 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T22:57:42.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:57:42.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:42.466 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:57:42.466 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:57:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa878053a70 msgr2=0x7fa878055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa878053a70 0x7fa878055f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fa88c00d000 tx=0x7fa88c009040).stop 2022-08-01T22:57:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 msgr2=0x7fa898157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa898157fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fa888002f70 tx=0x7fa888015000).stop 2022-08-01T22:57:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 shutdown_connections 2022-08-01T22:57:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa878053a70 0x7fa878055f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa898144d20 0x7fa898062e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8980b3f90 0x7fa898065b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 --2- 172.21.15.32:0/47019249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8980617f0 0x7fa898157fa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 >> 172.21.15.32:0/47019249 conn(0x7fa898066f70 msgr2=0x7fa898143ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 shutdown_connections 2022-08-01T22:57:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.427+0000 7fa89d7d0700 1 -- 172.21.15.32:0/47019249 wait complete. 2022-08-01T22:57:42.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:57:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 Processor -- start 2022-08-01T22:57:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- start start 2022-08-01T22:57:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf40b1f40 0x7fcdf40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdf4148530 0x7fcdf40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40b04e0 con 0x7fcdf40a7250 2022-08-01T22:57:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40a9250 con 0x7fcdf4148530 2022-08-01T22:57:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40b0780 con 0x7fcdf40b1f40 2022-08-01T22:57:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdf4148530 0x7fcdf40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdf4148530 0x7fcdf40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42452/0 (socket says 172.21.15.32:42452) 2022-08-01T22:57:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 -- 172.21.15.32:0/429338133 learned_addr learned my addr 172.21.15.32:0/429338133 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf259c700 1 --2- 172.21.15.32:0/429338133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf40b1f40 0x7fcdf40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 -- 172.21.15.32:0/429338133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf40b1f40 msgr2=0x7fcdf40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 --2- 172.21.15.32:0/429338133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf40b1f40 0x7fcdf40b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 -- 172.21.15.32:0/429338133 >> v1:172.21.15.32:6789/0 conn(0x7fcdf40a7250 legacy=0x7fcdf41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 -- 172.21.15.32:0/429338133 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40b0a20 con 0x7fcdf4148530 2022-08-01T22:57:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf1d9b700 1 --2- 172.21.15.32:0/429338133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf4148530 0x7fcdf40b2320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fcdec0032f0 tx=0x7fcdec009290).ready entity=mon.0 client_cookie=55ce9f198925687c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf159a700 1 -- 172.21.15.32:0/429338133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdec00b8a0 con 0x7fcdf4148530 2022-08-01T22:57:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf159a700 1 -- 172.21.15.32:0/429338133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec00ba00 con 0x7fcdf4148530 2022-08-01T22:57:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf159a700 1 -- 172.21.15.32:0/429338133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdec00bb70 con 0x7fcdf4148530 2022-08-01T22:57:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- 172.21.15.32:0/429338133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf4148530 msgr2=0x7fcdf40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/429338133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf4148530 0x7fcdf40b2320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fcdec0032f0 tx=0x7fcdec009290).stop 2022-08-01T22:57:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- 172.21.15.32:0/429338133 shutdown_connections 2022-08-01T22:57:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/429338133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf4148530 0x7fcdf40b2320 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/429338133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf40b1f40 0x7fcdf40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- 172.21.15.32:0/429338133 >> 172.21.15.32:0/429338133 conn(0x7fcdf405f840 msgr2=0x7fcdf405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- 172.21.15.32:0/429338133 shutdown_connections 2022-08-01T22:57:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.575+0000 7fcdf8a33700 1 -- 172.21.15.32:0/429338133 wait complete. 2022-08-01T22:57:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 Processor -- start 2022-08-01T22:57:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 -- start start 2022-08-01T22:57:42.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 0x7fcdf40ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf40b1f40 0x7fcdf40ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf4148530 0x7fcdf40ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf414b5f0 con 0x7fcdf40a7250 2022-08-01T22:57:42.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf414ac50 con 0x7fcdf40b1f40 2022-08-01T22:57:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf414b820 con 0x7fcdf4148530 2022-08-01T22:57:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 0x7fcdf40ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 0x7fcdf40ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42458/0 (socket says 172.21.15.32:42458) 2022-08-01T22:57:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 -- 172.21.15.32:0/1798528426 learned_addr learned my addr 172.21.15.32:0/1798528426 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf1d9b700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf40b1f40 0x7fcdf40ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf2d9d700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf4148530 0x7fcdf40ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 -- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf4148530 msgr2=0x7fcdf40ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf4148530 0x7fcdf40ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 -- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf40b1f40 msgr2=0x7fcdf40ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf40b1f40 0x7fcdf40ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 -- 172.21.15.32:0/1798528426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec009530 con 0x7fcdf40a7250 2022-08-01T22:57:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf259c700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 0x7fcdf40ae300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fcddc00a000 tx=0x7fcddc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcddc011780 con 0x7fcdf40a7250 2022-08-01T22:57:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc0118e0 con 0x7fcdf40a7250 2022-08-01T22:57:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcddc011c40 con 0x7fcdf40a7250 2022-08-01T22:57:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40aed20 con 0x7fcdf40a7250 2022-08-01T22:57:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.579+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf414a950 con 0x7fcdf40a7250 2022-08-01T22:57:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.583+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf40b0cc0 con 0x7fcdf40a7250 2022-08-01T22:57:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.583+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcddc012420 con 0x7fcdf40a7250 2022-08-01T22:57:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.583+0000 7fcdeb7fe700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.583+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fcddc06bd20 con 0x7fcdf40a7250 2022-08-01T22:57:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.583+0000 7fcdf1d9b700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.583+0000 7fcdf1d9b700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fcdec019000 tx=0x7fcdec016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.591+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcddc042cb0 con 0x7fcdf40a7250 2022-08-01T22:57:42.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.715+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdf40574e0 con 0x7fcdf40a7250 2022-08-01T22:57:42.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.715+0000 7fcdeb7fe700 1 -- 172.21.15.32:0/1798528426 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13414 (secure 0 0 0) 0x7fcddc0462d0 con 0x7fcdf40a7250 2022-08-01T22:57:42.722 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:42.723 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:57:42.102502+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:57:42.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 msgr2=0x7fcdd8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fcdec019000 tx=0x7fcdec016000).stop 2022-08-01T22:57:42.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 msgr2=0x7fcdf40ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 0x7fcdf40ae300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fcddc00a000 tx=0x7fcddc008040).stop 2022-08-01T22:57:42.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 shutdown_connections 2022-08-01T22:57:42.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf4148530 0x7fcdf40ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf40b1f40 0x7fcdf40ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 --2- 172.21.15.32:0/1798528426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf40a7250 0x7fcdf40ae300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 >> 172.21.15.32:0/1798528426 conn(0x7fcdf405f840 msgr2=0x7fcdf4144930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 shutdown_connections 2022-08-01T22:57:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.719+0000 7fcdf8a33700 1 -- 172.21.15.32:0/1798528426 wait complete. 2022-08-01T22:57:42.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T22:57:42.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:57:42.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae5d5f6700 1 Processor -- start 2022-08-01T22:57:42.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae5d5f6700 1 -- start start 2022-08-01T22:57:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae5d5f6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae580b1460 0x7fae580b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae5d5f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580a7960 con 0x7fae580b2050 2022-08-01T22:57:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae5d5f6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580b07e0 con 0x7fae580b2430 2022-08-01T22:57:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae5d5f6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae580b18d0 con 0x7fae580b1460 2022-08-01T22:57:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae56ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fae580b2050 0x7fae580a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58240/0 (socket says 172.21.15.32:58240) 2022-08-01T22:57:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae56ffd700 1 -- 172.21.15.32:0/4117033279 learned_addr learned my addr 172.21.15.32:0/4117033279 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae567fc700 1 --2- 172.21.15.32:0/4117033279 >> v2:172.21.15.55:3300/0 conn(0x7fae580b1460 0x7fae580b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559962558 0 0) 0x7fae580a7960 con 0x7fae580b2050 2022-08-01T22:57:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae38003260 con 0x7fae580b2050 2022-08-01T22:57:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957950961 0 0) 0x7fae580b07e0 con 0x7fae580b2430 2022-08-01T22:57:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae580a7960 con 0x7fae580b2430 2022-08-01T22:57:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2207100559 0 0) 0x7fae38003260 con 0x7fae580b2050 2022-08-01T22:57:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 >> v2:172.21.15.55:3300/0 conn(0x7fae580b1460 msgr2=0x7fae580b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:42.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 --2- 172.21.15.32:0/4117033279 >> v2:172.21.15.55:3300/0 conn(0x7fae580b1460 0x7fae580b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 >> v1:172.21.15.55:6789/0 conn(0x7fae580b2430 legacy=0x7fae581413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580b0a80 con 0x7fae580b2050 2022-08-01T22:57:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fae40001960 con 0x7fae580b2050 2022-08-01T22:57:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 >> v1:172.21.15.32:6789/0 conn(0x7fae580b2050 legacy=0x7fae580a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae38003930 0x7fae38003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae38004500 0x7fae38004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004e70 0x7fae3800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b18d0 con 0x7fae38004500 2022-08-01T22:57:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3800bd20 con 0x7fae38004e70 2022-08-01T22:57:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3800bee0 con 0x7fae38003930 2022-08-01T22:57:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae567fc700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae38004500 0x7fae38004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae577fe700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004e70 0x7fae3800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae56ffd700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae38003930 0x7fae38003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae567fc700 1 -- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae38003930 msgr2=0x7fae38003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae567fc700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae38003930 0x7fae38003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae567fc700 1 -- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004e70 msgr2=0x7fae3800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.843+0000 7fae567fc700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004e70 0x7fae3800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae567fc700 1 -- 172.21.15.32:0/4117033279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae38003260 con 0x7fae38004500 2022-08-01T22:57:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae577fe700 1 -- 172.21.15.32:0/4117033279 reap_dead start 2022-08-01T22:57:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae56ffd700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae38003930 0x7fae38003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:57:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae567fc700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae38004500 0x7fae38004980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fae4800c000 tx=0x7fae48006df0).ready entity=mon.0 client_cookie=95d456c7f7cbd30b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4800e830 con 0x7fae38004500 2022-08-01T22:57:42.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4800e990 con 0x7fae38004500 2022-08-01T22:57:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae55ffb700 1 -- 172.21.15.32:0/4117033279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae4800ecf0 con 0x7fae38004500 2022-08-01T22:57:42.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 -- 172.21.15.32:0/4117033279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae38004500 msgr2=0x7fae38004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae38004500 0x7fae38004980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fae4800c000 tx=0x7fae48006df0).stop 2022-08-01T22:57:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 -- 172.21.15.32:0/4117033279 shutdown_connections 2022-08-01T22:57:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/4117033279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae38004500 0x7fae38004980 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 -- 172.21.15.32:0/4117033279 >> 172.21.15.32:0/4117033279 conn(0x7fae5805f840 msgr2=0x7fae5805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 -- 172.21.15.32:0/4117033279 shutdown_connections 2022-08-01T22:57:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.847+0000 7fae5d5f6700 1 -- 172.21.15.32:0/4117033279 wait complete. 2022-08-01T22:57:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 Processor -- start 2022-08-01T22:57:42.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 -- start start 2022-08-01T22:57:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004500 0x7fae5814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae580aa6e0 0x7fae58148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 0x7fae5815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580af090 con 0x7fae580abd10 2022-08-01T22:57:42.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580aad10 con 0x7fae38004500 2022-08-01T22:57:42.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580ab1c0 con 0x7fae580aa6e0 2022-08-01T22:57:42.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 0x7fae5815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 0x7fae5815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42482/0 (socket says 172.21.15.32:42482) 2022-08-01T22:57:42.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 -- 172.21.15.32:0/2050440114 learned_addr learned my addr 172.21.15.32:0/2050440114 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:42.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae567fc700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae580aa6e0 0x7fae58148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae56ffd700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004500 0x7fae5814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 -- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae580aa6e0 msgr2=0x7fae58148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae580aa6e0 0x7fae58148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 -- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004500 msgr2=0x7fae5814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004500 0x7fae5814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 -- 172.21.15.32:0/2050440114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580ab460 con 0x7fae580abd10 2022-08-01T22:57:42.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae577fe700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 0x7fae5815a5c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fae500011d0 tx=0x7fae50009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5000eeb0 con 0x7fae580abd10 2022-08-01T22:57:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50014430 con 0x7fae580abd10 2022-08-01T22:57:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae580ab700 con 0x7fae580abd10 2022-08-01T22:57:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae50014790 con 0x7fae580abd10 2022-08-01T22:57:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.851+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae580ac1c0 con 0x7fae580abd10 2022-08-01T22:57:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.855+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae580aaac0 con 0x7fae580abd10 2022-08-01T22:57:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.855+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fae500148f0 con 0x7fae580abd10 2022-08-01T22:57:42.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.855+0000 7fae4ffff700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae3c053a70 0x7fae3c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:42.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.855+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fae5006ba40 con 0x7fae580abd10 2022-08-01T22:57:42.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.855+0000 7fae56ffd700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae3c053a70 0x7fae3c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:42.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.859+0000 7fae56ffd700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae3c053a70 0x7fae3c055f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fae4000e000 tx=0x7fae40009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:42.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.863+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fae50042b40 con 0x7fae580abd10 2022-08-01T22:57:42.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.987+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fae580b0d20 con 0x7fae580abd10 2022-08-01T22:57:42.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.987+0000 7fae4ffff700 1 -- 172.21.15.32:0/2050440114 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v40) v1 ==== 110+0+10 (secure 0 0 0) 0x7fae50046160 con 0x7fae580abd10 2022-08-01T22:57:42.994 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:57:42.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae3c053a70 msgr2=0x7fae3c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae3c053a70 0x7fae3c055f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fae4000e000 tx=0x7fae40009040).stop 2022-08-01T22:57:42.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 msgr2=0x7fae5815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:42.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 0x7fae5815a5c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fae500011d0 tx=0x7fae50009040).stop 2022-08-01T22:57:42.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 shutdown_connections 2022-08-01T22:57:42.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae3c053a70 0x7fae3c055f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae580abd10 0x7fae5815a5c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae580aa6e0 0x7fae58148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 --2- 172.21.15.32:0/2050440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38004500 0x7fae5814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:42.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 >> 172.21.15.32:0/2050440114 conn(0x7fae5805f840 msgr2=0x7fae580af850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:42.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 shutdown_connections 2022-08-01T22:57:42.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:42.991+0000 7fae5d5f6700 1 -- 172.21.15.32:0/2050440114 wait complete. 2022-08-01T22:57:43.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:57:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 Processor -- start 2022-08-01T22:57:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 -- start start 2022-08-01T22:57:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f394c144ad0 0x7f394c0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f394c0aa050 0x7f394c0b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c144eb0 con 0x7f394c0b0ef0 2022-08-01T22:57:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c0ac390 con 0x7f394c144ad0 2022-08-01T22:57:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f39541bb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c145110 con 0x7f394c0aa050 2022-08-01T22:57:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f394c144ad0 0x7f394c0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f394c144ad0 0x7f394c0abea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42486/0 (socket says 172.21.15.32:42486) 2022-08-01T22:57:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 -- 172.21.15.32:0/4219333078 learned_addr learned my addr 172.21.15.32:0/4219333078 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951756700 1 --2- 172.21.15.32:0/4219333078 >> v2:172.21.15.55:3300/0 conn(0x7f394c0aa050 0x7f394c0b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 -- 172.21.15.32:0/4219333078 >> v2:172.21.15.55:3300/0 conn(0x7f394c0aa050 msgr2=0x7f394c0b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 --2- 172.21.15.32:0/4219333078 >> v2:172.21.15.55:3300/0 conn(0x7f394c0aa050 0x7f394c0b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 -- 172.21.15.32:0/4219333078 >> v1:172.21.15.55:6789/0 conn(0x7f394c0b0ef0 legacy=0x7f394c13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 -- 172.21.15.32:0/4219333078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c1453b0 con 0x7f394c144ad0 2022-08-01T22:57:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3951f57700 1 --2- 172.21.15.32:0/4219333078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0abea0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f393c002f70 tx=0x7f393c006eb0).ready entity=mon.0 client_cookie=8c5e1de0bc80dfeb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3950f55700 1 -- 172.21.15.32:0/4219333078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393c00d5c0 con 0x7f394c144ad0 2022-08-01T22:57:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3950f55700 1 -- 172.21.15.32:0/4219333078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c000e90 con 0x7f394c144ad0 2022-08-01T22:57:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.115+0000 7f3950f55700 1 -- 172.21.15.32:0/4219333078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393c00d950 con 0x7f394c144ad0 2022-08-01T22:57:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- 172.21.15.32:0/4219333078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 msgr2=0x7f394c0abea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 --2- 172.21.15.32:0/4219333078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0abea0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f393c002f70 tx=0x7f393c006eb0).stop 2022-08-01T22:57:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- 172.21.15.32:0/4219333078 shutdown_connections 2022-08-01T22:57:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 --2- 172.21.15.32:0/4219333078 >> v2:172.21.15.55:3300/0 conn(0x7f394c0aa050 0x7f394c0b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 --2- 172.21.15.32:0/4219333078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0abea0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- 172.21.15.32:0/4219333078 >> 172.21.15.32:0/4219333078 conn(0x7f394c0b2b00 msgr2=0x7f394c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- 172.21.15.32:0/4219333078 shutdown_connections 2022-08-01T22:57:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- 172.21.15.32:0/4219333078 wait complete. 2022-08-01T22:57:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 Processor -- start 2022-08-01T22:57:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- start start 2022-08-01T22:57:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f394c0aa050 0x7f394c1433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f394c0b0ef0 0x7f394c065fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0664a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c14ee80 con 0x7f394c144ad0 2022-08-01T22:57:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c14f120 con 0x7f394c0b0ef0 2022-08-01T22:57:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f39541bb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c1402c0 con 0x7f394c0aa050 2022-08-01T22:57:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42496/0 (socket says 172.21.15.32:42496) 2022-08-01T22:57:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 -- 172.21.15.32:0/3256168106 learned_addr learned my addr 172.21.15.32:0/3256168106 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3951756700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f394c0b0ef0 0x7f394c065fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3951f57700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f394c0aa050 0x7f394c1433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 -- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f394c0aa050 msgr2=0x7f394c1433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f394c0aa050 0x7f394c1433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 -- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f394c0b0ef0 msgr2=0x7f394c065fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f394c0b0ef0 0x7f394c065fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 -- 172.21.15.32:0/3256168106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c00b1e0 con 0x7f394c144ad0 2022-08-01T22:57:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3951f57700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f394c0aa050 0x7f394c1433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f3952758700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0664a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f394c061b70 tx=0x7f3944007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.119+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f394400e4a0 con 0x7f394c144ad0 2022-08-01T22:57:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394400e600 con 0x7f394c144ad0 2022-08-01T22:57:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c140560 con 0x7f394c144ad0 2022-08-01T22:57:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f394400e960 con 0x7f394c144ad0 2022-08-01T22:57:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c066990 con 0x7f394c144ad0 2022-08-01T22:57:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c13ff20 con 0x7f394c144ad0 2022-08-01T22:57:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3944019030 con 0x7f394c144ad0 2022-08-01T22:57:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f393affd700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f392c053980 0x7f392c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f394406b440 con 0x7f394c144ad0 2022-08-01T22:57:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f3951f57700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f392c053980 0x7f392c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.123+0000 7f3951f57700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f392c053980 0x7f392c055e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f393c01a000 tx=0x7f393c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.131+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f394c13ff20 con 0x7f394c144ad0 2022-08-01T22:57:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.255+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f394c13ff20 con 0x7f394c144ad0 2022-08-01T22:57:43.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.255+0000 7f393affd700 1 -- 172.21.15.32:0/3256168106 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f394c13ff20 con 0x7f394c144ad0 2022-08-01T22:57:43.261 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:57:43.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f392c053980 msgr2=0x7f392c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f392c053980 0x7f392c055e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f393c01a000 tx=0x7f393c015000).stop 2022-08-01T22:57:43.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 msgr2=0x7f394c0664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0664a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f394c061b70 tx=0x7f3944007f40).stop 2022-08-01T22:57:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 shutdown_connections 2022-08-01T22:57:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f392c053980 0x7f392c055e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f394c144ad0 0x7f394c0664a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f394c0b0ef0 0x7f394c065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 --2- 172.21.15.32:0/3256168106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f394c0aa050 0x7f394c1433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 >> 172.21.15.32:0/3256168106 conn(0x7f394c0b2b00 msgr2=0x7f394c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 shutdown_connections 2022-08-01T22:57:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.259+0000 7f39541bb700 1 -- 172.21.15.32:0/3256168106 wait complete. 2022-08-01T22:57:43.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T22:57:43.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 Processor -- start 2022-08-01T22:57:43.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 -- start start 2022-08-01T22:57:43.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 0x7f4ec00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 0x7f4ec0147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec00b5a10 con 0x7f4ec0067730 2022-08-01T22:57:43.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec0145470 con 0x7f4ec00b5630 2022-08-01T22:57:43.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec6fb9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00ae180 con 0x7f4ec00b07a0 2022-08-01T22:57:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec5fb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 0x7f4ec00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec57b6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 0x7f4ec0147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec57b6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 0x7f4ec0147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:54238/0 (socket says 172.21.15.32:54238) 2022-08-01T22:57:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec57b6700 1 -- 172.21.15.32:0/3513860154 learned_addr learned my addr 172.21.15.32:0/3513860154 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.371+0000 7f4ec5fb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 0x7f4ec00af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42502/0 (socket says 172.21.15.32:42502) 2022-08-01T22:57:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec57b6700 1 -- 172.21.15.32:0/3513860154 >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 msgr2=0x7f4ec00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec57b6700 1 --2- 172.21.15.32:0/3513860154 >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 0x7f4ec00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec57b6700 1 -- 172.21.15.32:0/3513860154 >> v1:172.21.15.55:6789/0 conn(0x7f4ec0067730 legacy=0x7f4ec00acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec57b6700 1 -- 172.21.15.32:0/3513860154 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00ac990 con 0x7f4ec00b07a0 2022-08-01T22:57:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec5fb7700 1 --2- 172.21.15.32:0/3513860154 >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 0x7f4ec00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec57b6700 1 --2- 172.21.15.32:0/3513860154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 0x7f4ec0147730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4eb0002f70 tx=0x7f4eb0006eb0).ready entity=mon.2 client_cookie=b00252dd91160e42 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec4fb5700 1 -- 172.21.15.32:0/3513860154 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb000d570 con 0x7f4ec00b07a0 2022-08-01T22:57:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec4fb5700 1 -- 172.21.15.32:0/3513860154 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0000e90 con 0x7f4ec00b07a0 2022-08-01T22:57:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec4fb5700 1 -- 172.21.15.32:0/3513860154 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb000d900 con 0x7f4ec00b07a0 2022-08-01T22:57:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/3513860154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 msgr2=0x7f4ec0147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/3513860154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 0x7f4ec0147730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4eb0002f70 tx=0x7f4eb0006eb0).stop 2022-08-01T22:57:43.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/3513860154 shutdown_connections 2022-08-01T22:57:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/3513860154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec00b07a0 0x7f4ec0147730 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/3513860154 >> v2:172.21.15.32:3300/0 conn(0x7f4ec00b5630 0x7f4ec00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/3513860154 >> 172.21.15.32:0/3513860154 conn(0x7f4ec00626f0 msgr2=0x7f4ec0062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/3513860154 shutdown_connections 2022-08-01T22:57:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/3513860154 wait complete. 2022-08-01T22:57:43.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 Processor -- start 2022-08-01T22:57:43.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.375+0000 7f4ec6fb9700 1 -- start start 2022-08-01T22:57:43.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec0067730 0x7f4ec0144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 0x7f4ec0157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ec00b5630 0x7f4ec0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0065150 con 0x7f4ec00b07a0 2022-08-01T22:57:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0064970 con 0x7f4ec00b5630 2022-08-01T22:57:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0064c10 con 0x7f4ec0067730 2022-08-01T22:57:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 0x7f4ec0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 0x7f4ec0157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42508/0 (socket says 172.21.15.32:42508) 2022-08-01T22:57:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 -- 172.21.15.32:0/1121462820 learned_addr learned my addr 172.21.15.32:0/1121462820 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec67b8700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ec00b5630 0x7f4ec0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec5fb7700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec0067730 0x7f4ec0144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 -- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec0067730 msgr2=0x7f4ec0144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec0067730 0x7f4ec0144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 -- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ec00b5630 msgr2=0x7f4ec0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ec00b5630 0x7f4ec0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 -- 172.21.15.32:0/1121462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb400b180 con 0x7f4ec00b07a0 2022-08-01T22:57:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec57b6700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 0x7f4ec0157220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f4eb0002f70 tx=0x7f4eb0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb00116e0 con 0x7f4ec00b07a0 2022-08-01T22:57:43.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0011840 con 0x7f4ec00b07a0 2022-08-01T22:57:43.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb000b1e0 con 0x7f4ec00b07a0 2022-08-01T22:57:43.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb0011ba0 con 0x7f4ec00b07a0 2022-08-01T22:57:43.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec00a8e80 con 0x7f4ec00b07a0 2022-08-01T22:57:43.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.379+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec0147b40 con 0x7f4ec00b07a0 2022-08-01T22:57:43.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.383+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4eb001a040 con 0x7f4ec00b07a0 2022-08-01T22:57:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.383+0000 7f4ebeffd700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4ea00539e0 0x7f4ea0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.383+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4eb006ce00 con 0x7f4ec00b07a0 2022-08-01T22:57:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.383+0000 7f4ec5fb7700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4ea00539e0 0x7f4ea0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.383+0000 7f4ec5fb7700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4ea00539e0 0x7f4ea0055ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4eb400b000 tx=0x7f4eb4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.387+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4eb0043df0 con 0x7f4ec00b07a0 2022-08-01T22:57:43.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.519+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4ec00b1680 con 0x7f4ec00b07a0 2022-08-01T22:57:43.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.519+0000 7f4ebeffd700 1 -- 172.21.15.32:0/1121462820 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4eb0023070 con 0x7f4ec00b07a0 2022-08-01T22:57:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:43.529 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T22:57:43.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T22:57:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T22:57:43.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:57:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:57:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T22:57:43.532 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T22:57:43.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:57:43.533 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T22:57:43.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.519+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4ea00539e0 msgr2=0x7f4ea0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.519+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4ea00539e0 0x7f4ea0055ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4eb400b000 tx=0x7f4eb4006df0).stop 2022-08-01T22:57:43.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.519+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 msgr2=0x7f4ec0157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.519+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 0x7f4ec0157220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f4eb0002f70 tx=0x7f4eb0015000).stop 2022-08-01T22:57:43.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 shutdown_connections 2022-08-01T22:57:43.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4ea00539e0 0x7f4ea0055ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ec00b5630 0x7f4ec0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ec00b07a0 0x7f4ec0157220 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 --2- 172.21.15.32:0/1121462820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ec0067730 0x7f4ec0144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 >> 172.21.15.32:0/1121462820 conn(0x7f4ec00626f0 msgr2=0x7f4ec0062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 shutdown_connections 2022-08-01T22:57:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.523+0000 7f4ec6fb9700 1 -- 172.21.15.32:0/1121462820 wait complete. 2022-08-01T22:57:43.539 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T22:57:43.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T22:57:43.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.639+0000 7febc249e700 1 Processor -- start 2022-08-01T22:57:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.639+0000 7febc249e700 1 -- start start 2022-08-01T22:57:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7febbc0af550 0x7febbc0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc0aa630 con 0x7febbc144ad0 2022-08-01T22:57:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc0b15e0 con 0x7febbc0af550 2022-08-01T22:57:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0af930 con 0x7febbc060330 2022-08-01T22:57:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc1c9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7febbc144ad0 0x7febbc0a8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58290/0 (socket says 172.21.15.32:58290) 2022-08-01T22:57:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc1c9d700 1 -- 172.21.15.32:0/1811565522 learned_addr learned my addr 172.21.15.32:0/1811565522 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc149c700 1 --2- 172.21.15.32:0/1811565522 >> v2:172.21.15.55:3300/0 conn(0x7febbc0af550 0x7febbc0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc0c9b700 1 --2- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febaffff700 1 -- 172.21.15.32:0/1811565522 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428011058 0 0) 0x7febbc0aa630 con 0x7febbc144ad0 2022-08-01T22:57:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febaffff700 1 -- 172.21.15.32:0/1811565522 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba4003260 con 0x7febbc144ad0 2022-08-01T22:57:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc149c700 1 -- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 msgr2=0x7febbc1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc149c700 1 --2- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc1445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc149c700 1 -- 172.21.15.32:0/1811565522 >> v1:172.21.15.32:6789/0 conn(0x7febbc144ad0 legacy=0x7febbc0a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc149c700 1 -- 172.21.15.32:0/1811565522 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc05fd10 con 0x7febbc0af550 2022-08-01T22:57:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc149c700 1 --2- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc0af550 0x7febbc0b0e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7febb8002f70 tx=0x7febb8006df0).ready entity=mon.1 client_cookie=21bc12aefded997d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febaffff700 1 -- 172.21.15.32:0/1811565522 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb800d4a0 con 0x7febbc0af550 2022-08-01T22:57:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febaffff700 1 -- 172.21.15.32:0/1811565522 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8000e90 con 0x7febbc0af550 2022-08-01T22:57:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febaffff700 1 -- 172.21.15.32:0/1811565522 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb800d7f0 con 0x7febbc0af550 2022-08-01T22:57:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc0af550 msgr2=0x7febbc0b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 --2- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc0af550 0x7febbc0b0e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7febb8002f70 tx=0x7febb8006df0).stop 2022-08-01T22:57:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- 172.21.15.32:0/1811565522 shutdown_connections 2022-08-01T22:57:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 --2- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 --2- 172.21.15.32:0/1811565522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc0af550 0x7febbc0b0e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- 172.21.15.32:0/1811565522 >> 172.21.15.32:0/1811565522 conn(0x7febbc0612d0 msgr2=0x7febbc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- 172.21.15.32:0/1811565522 shutdown_connections 2022-08-01T22:57:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.643+0000 7febc249e700 1 -- 172.21.15.32:0/1811565522 wait complete. 2022-08-01T22:57:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.647+0000 7febc249e700 1 Processor -- start 2022-08-01T22:57:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.647+0000 7febc249e700 1 -- start start 2022-08-01T22:57:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.647+0000 7febc249e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 0x7febbc150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc144ad0 0x7febbc150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc064820 con 0x7febbc0af550 2022-08-01T22:57:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc14ff00 con 0x7febbc144ad0 2022-08-01T22:57:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc140ca0 con 0x7febbc060330 2022-08-01T22:57:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 0x7febbc150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 0x7febbc150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42522/0 (socket says 172.21.15.32:42522) 2022-08-01T22:57:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 -- 172.21.15.32:0/3567845633 learned_addr learned my addr 172.21.15.32:0/3567845633 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:43.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc149c700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc1c9d700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc144ad0 0x7febbc150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 -- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 msgr2=0x7febbc143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 -- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc144ad0 msgr2=0x7febbc150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc144ad0 0x7febbc150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 -- 172.21.15.32:0/3567845633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb800b180 con 0x7febbc0af550 2022-08-01T22:57:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc0c9b700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 0x7febbc150590 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7febb000c000 tx=0x7febb0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb0010780 con 0x7febbc0af550 2022-08-01T22:57:43.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb00108e0 con 0x7febbc0af550 2022-08-01T22:57:43.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc05f840 con 0x7febbc0af550 2022-08-01T22:57:43.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb0010c40 con 0x7febbc0af550 2022-08-01T22:57:43.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc141d30 con 0x7febbc0af550 2022-08-01T22:57:43.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.651+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc05fa70 con 0x7febbc0af550 2022-08-01T22:57:43.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.655+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7febb001c460 con 0x7febbc0af550 2022-08-01T22:57:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.655+0000 7febae7fc700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb9c053aa0 0x7feb9c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:43.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.655+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5600+0+0 (secure 0 0 0) 0x7febb006c090 con 0x7febbc0af550 2022-08-01T22:57:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.655+0000 7febc149c700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb9c053aa0 0x7feb9c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:43.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.655+0000 7febc149c700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb9c053aa0 0x7feb9c055f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7febb801a000 tx=0x7febb8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:43.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.659+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7febb0043130 con 0x7febbc0af550 2022-08-01T22:57:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:43.791+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7febbc0574e0 con 0x7febbc0af550 2022-08-01T22:57:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.667+0000 7febae7fc700 1 -- 172.21.15.32:0/3567845633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v41) v1 ==== 133+0+0 (secure 0 0 0) 0x7febb0046750 con 0x7febbc0af550 2022-08-01T22:57:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb9c053aa0 msgr2=0x7feb9c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb9c053aa0 0x7feb9c055f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7febb801a000 tx=0x7febb8014000).stop 2022-08-01T22:57:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 msgr2=0x7febbc150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 0x7febbc150590 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7febb000c000 tx=0x7febb0006df0).stop 2022-08-01T22:57:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 shutdown_connections 2022-08-01T22:57:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb9c053aa0 0x7feb9c055f60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febbc144ad0 0x7febbc150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febbc0af550 0x7febbc150590 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 --2- 172.21.15.32:0/3567845633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febbc060330 0x7febbc143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 >> 172.21.15.32:0/3567845633 conn(0x7febbc0612d0 msgr2=0x7febbc13aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 shutdown_connections 2022-08-01T22:57:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.671+0000 7febc249e700 1 -- 172.21.15.32:0/3567845633 wait complete. 2022-08-01T22:57:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T22:57:44.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T22:57:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.803+0000 7fb5ca276700 1 Processor -- start 2022-08-01T22:57:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.803+0000 7fb5ca276700 1 -- start start 2022-08-01T22:57:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5c40b08a0 0x7fb5c4142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c41433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5c40ad000 0x7fb5c405f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c40b0c80 con 0x7fb5c40ad000 2022-08-01T22:57:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c40ae810 con 0x7fb5c40b08a0 2022-08-01T22:57:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40aeab0 con 0x7fb5c40b3260 2022-08-01T22:57:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5c40ad000 0x7fb5c405f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5c40ad000 0x7fb5c405f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42532/0 (socket says 172.21.15.32:42532) 2022-08-01T22:57:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 -- 172.21.15.32:0/736370707 learned_addr learned my addr 172.21.15.32:0/736370707 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c8a73700 1 --2- 172.21.15.32:0/736370707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c41433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9274700 1 --2- 172.21.15.32:0/736370707 >> v2:172.21.15.55:3300/0 conn(0x7fb5c40b08a0 0x7fb5c4142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 -- 172.21.15.32:0/736370707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 msgr2=0x7fb5c41433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 --2- 172.21.15.32:0/736370707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c41433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 -- 172.21.15.32:0/736370707 >> v2:172.21.15.55:3300/0 conn(0x7fb5c40b08a0 msgr2=0x7fb5c4142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:44.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 --2- 172.21.15.32:0/736370707 >> v2:172.21.15.55:3300/0 conn(0x7fb5c40b08a0 0x7fb5c4142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 -- 172.21.15.32:0/736370707 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c40aed50 con 0x7fb5c40ad000 2022-08-01T22:57:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5c9a75700 1 --2- 172.21.15.32:0/736370707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40ad000 0x7fb5c405f840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fb5bc0032f0 tx=0x7fb5bc009290).ready entity=mon.0 client_cookie=3fe68fdd7444b35f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5b3fff700 1 -- 172.21.15.32:0/736370707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5bc00ba60 con 0x7fb5c40ad000 2022-08-01T22:57:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5b3fff700 1 -- 172.21.15.32:0/736370707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc00bbc0 con 0x7fb5c40ad000 2022-08-01T22:57:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5b3fff700 1 -- 172.21.15.32:0/736370707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5bc010470 con 0x7fb5c40ad000 2022-08-01T22:57:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- 172.21.15.32:0/736370707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40ad000 msgr2=0x7fb5c405f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- 172.21.15.32:0/736370707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40ad000 0x7fb5c405f840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fb5bc0032f0 tx=0x7fb5bc009290).stop 2022-08-01T22:57:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- 172.21.15.32:0/736370707 shutdown_connections 2022-08-01T22:57:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- 172.21.15.32:0/736370707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40ad000 0x7fb5c405f840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- 172.21.15.32:0/736370707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c41433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 --2- 172.21.15.32:0/736370707 >> v2:172.21.15.55:3300/0 conn(0x7fb5c40b08a0 0x7fb5c4142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- 172.21.15.32:0/736370707 >> 172.21.15.32:0/736370707 conn(0x7fb5c40612d0 msgr2=0x7fb5c4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:44.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- 172.21.15.32:0/736370707 shutdown_connections 2022-08-01T22:57:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.807+0000 7fb5ca276700 1 -- 172.21.15.32:0/736370707 wait complete. 2022-08-01T22:57:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 Processor -- start 2022-08-01T22:57:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 -- start start 2022-08-01T22:57:44.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c40ad000 0x7fb5c40aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 0x7fb5c40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c40a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c409ea40 con 0x7fb5c40b08a0 2022-08-01T22:57:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b4810 con 0x7fb5c40ad000 2022-08-01T22:57:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c409ec70 con 0x7fb5c40b3260 2022-08-01T22:57:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 0x7fb5c40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 0x7fb5c40b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42538/0 (socket says 172.21.15.32:42538) 2022-08-01T22:57:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 -- 172.21.15.32:0/3210166412 learned_addr learned my addr 172.21.15.32:0/3210166412 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c9274700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c40ad000 0x7fb5c40aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c9a75700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c40a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 -- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 msgr2=0x7fb5c40a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c40a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 -- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c40ad000 msgr2=0x7fb5c40aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c40ad000 0x7fb5c40aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 -- 172.21.15.32:0/3210166412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc009530 con 0x7fb5c40b08a0 2022-08-01T22:57:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5c8a73700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 0x7fb5c40b4320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fb5b400a000 tx=0x7fb5b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b4011590 con 0x7fb5c40b08a0 2022-08-01T22:57:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b40116f0 con 0x7fb5c40b08a0 2022-08-01T22:57:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c409ef10 con 0x7fb5c40b08a0 2022-08-01T22:57:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b4011a50 con 0x7fb5c40b08a0 2022-08-01T22:57:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.811+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c41522f0 con 0x7fb5c40b08a0 2022-08-01T22:57:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.815+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c40574e0 con 0x7fb5c40b08a0 2022-08-01T22:57:44.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.815+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5b4011bb0 con 0x7fb5c40b08a0 2022-08-01T22:57:44.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.815+0000 7fb5b27fc700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5a4053980 0x7fb5a4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:44.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.815+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fb5b406c200 con 0x7fb5c40b08a0 2022-08-01T22:57:44.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.815+0000 7fb5c9274700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5a4053980 0x7fb5a4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:44.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.819+0000 7fb5c9274700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5a4053980 0x7fb5a4055e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fb5c000c000 tx=0x7fb5c0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:44.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.823+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5b4043020 con 0x7fb5c40b08a0 2022-08-01T22:57:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:44.951+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb5c40b2e00 con 0x7fb5c40b08a0 2022-08-01T22:57:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.015+0000 7fb5b27fc700 1 -- 172.21.15.32:0/3210166412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v42) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb5b4046640 con 0x7fb5c40b08a0 2022-08-01T22:57:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5a4053980 msgr2=0x7fb5a4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5a4053980 0x7fb5a4055e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fb5c000c000 tx=0x7fb5c0008040).stop 2022-08-01T22:57:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 msgr2=0x7fb5c40b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 0x7fb5c40b4320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fb5b400a000 tx=0x7fb5b4008040).stop 2022-08-01T22:57:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 shutdown_connections 2022-08-01T22:57:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5a4053980 0x7fb5a4055e40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5c40b3260 0x7fb5c40a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5c40b08a0 0x7fb5c40b4320 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 --2- 172.21.15.32:0/3210166412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c40ad000 0x7fb5c40aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 >> 172.21.15.32:0/3210166412 conn(0x7fb5c40612d0 msgr2=0x7fb5c4140450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 shutdown_connections 2022-08-01T22:57:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.019+0000 7fb5ca276700 1 -- 172.21.15.32:0/3210166412 wait complete. 2022-08-01T22:57:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T22:57:45.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T22:57:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.131+0000 7fab46722700 1 Processor -- start 2022-08-01T22:57:45.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.131+0000 7fab46722700 1 -- start start 2022-08-01T22:57:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 0x7fab40141210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 0x7fab40141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab400aac20 con 0x7fab400b28b0 2022-08-01T22:57:45.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab40140e20 con 0x7fab400b1180 2022-08-01T22:57:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab400b2c90 con 0x7fab4013e8d0 2022-08-01T22:57:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 0x7fab40141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab45f21700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 0x7fab40141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab45f21700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 0x7fab40141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52332/0 (socket says 172.21.15.32:52332) 2022-08-01T22:57:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 0x7fab40141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:54282/0 (socket says 172.21.15.32:54282) 2022-08-01T22:57:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab45f21700 1 -- 172.21.15.32:0/2229343698 learned_addr learned my addr 172.21.15.32:0/2229343698 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:45.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 -- 172.21.15.32:0/2229343698 >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 msgr2=0x7fab40141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 --2- 172.21.15.32:0/2229343698 >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 0x7fab40141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 -- 172.21.15.32:0/2229343698 >> v1:172.21.15.55:6789/0 conn(0x7fab400b28b0 legacy=0x7fab4013e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 -- 172.21.15.32:0/2229343698 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400aa440 con 0x7fab4013e8d0 2022-08-01T22:57:45.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab45f21700 1 --2- 172.21.15.32:0/2229343698 >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 0x7fab40141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab44f1f700 1 --2- 172.21.15.32:0/2229343698 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 0x7fab40141210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fab3c002f70 tx=0x7fab3c006df0).ready entity=mon.2 client_cookie=412325aad0706c32 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab37fff700 1 -- 172.21.15.32:0/2229343698 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3c00d570 con 0x7fab4013e8d0 2022-08-01T22:57:45.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab37fff700 1 -- 172.21.15.32:0/2229343698 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c000e90 con 0x7fab4013e8d0 2022-08-01T22:57:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab37fff700 1 -- 172.21.15.32:0/2229343698 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3c00d8c0 con 0x7fab4013e8d0 2022-08-01T22:57:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- 172.21.15.32:0/2229343698 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 msgr2=0x7fab40141210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 --2- 172.21.15.32:0/2229343698 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 0x7fab40141210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fab3c002f70 tx=0x7fab3c006df0).stop 2022-08-01T22:57:45.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- 172.21.15.32:0/2229343698 shutdown_connections 2022-08-01T22:57:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 --2- 172.21.15.32:0/2229343698 >> v2:172.21.15.55:3300/0 conn(0x7fab400b1180 0x7fab40141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 --2- 172.21.15.32:0/2229343698 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab4013e8d0 0x7fab40141210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- 172.21.15.32:0/2229343698 >> 172.21.15.32:0/2229343698 conn(0x7fab40060310 msgr2=0x7fab400b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- 172.21.15.32:0/2229343698 shutdown_connections 2022-08-01T22:57:45.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 -- 172.21.15.32:0/2229343698 wait complete. 2022-08-01T22:57:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.135+0000 7fab46722700 1 Processor -- start 2022-08-01T22:57:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 -- start start 2022-08-01T22:57:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400b1180 0x7fab4013ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 0x7fab400a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 0x7fab400a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400b3470 con 0x7fab400b1180 2022-08-01T22:57:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400b36a0 con 0x7fab400ae450 2022-08-01T22:57:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40156390 con 0x7fab400b28b0 2022-08-01T22:57:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab45720700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400b1180 0x7fab4013ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 0x7fab400a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab45f21700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 0x7fab400a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab45f21700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 0x7fab400a82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52338/0 (socket says 172.21.15.32:52338) 2022-08-01T22:57:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 0x7fab400a7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:54288/0 (socket says 172.21.15.32:54288) 2022-08-01T22:57:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 -- 172.21.15.32:0/1978948271 learned_addr learned my addr 172.21.15.32:0/1978948271 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab45720700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400b1180 0x7fab4013ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42548/0 (socket says 172.21.15.32:42548) 2022-08-01T22:57:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 -- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 msgr2=0x7fab400a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 0x7fab400a82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 -- 172.21.15.32:0/1978948271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400b1180 msgr2=0x7fab4013ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400b1180 0x7fab4013ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 -- 172.21.15.32:0/1978948271 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38009530 con 0x7fab400b28b0 2022-08-01T22:57:45.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab45f21700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 0x7fab400a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab44f1f700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 0x7fab400a7e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fab4009f2c0 tx=0x7fab3c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3c016070 con 0x7fab400b28b0 2022-08-01T22:57:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c00de30 con 0x7fab400b28b0 2022-08-01T22:57:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c00b180 con 0x7fab400b28b0 2022-08-01T22:57:45.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3c013d60 con 0x7fab400b28b0 2022-08-01T22:57:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.139+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab400a87e0 con 0x7fab400b28b0 2022-08-01T22:57:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.143+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab400b38a0 con 0x7fab400b28b0 2022-08-01T22:57:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.143+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab3c025030 con 0x7fab400b28b0 2022-08-01T22:57:45.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.143+0000 7fab367fc700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab28053a70 0x7fab28055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.143+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fab3c06cf50 con 0x7fab400b28b0 2022-08-01T22:57:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.143+0000 7fab45720700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab28053a70 0x7fab28055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.143+0000 7fab45720700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab28053a70 0x7fab28055f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fab3000d000 tx=0x7fab30000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.151+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab3c043d90 con 0x7fab400b28b0 2022-08-01T22:57:45.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.315+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fab400574e0 con 0x7fab400b28b0 2022-08-01T22:57:45.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.331+0000 7fab367fc700 1 -- 172.21.15.32:0/1978948271 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v16) v1 ==== 187+0+0 (secure 0 0 0) 0x7fab3c023070 con 0x7fab400b28b0 2022-08-01T22:57:45.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.331+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab28053a70 msgr2=0x7fab28055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.331+0000 7fab46722700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab28053a70 0x7fab28055f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fab3000d000 tx=0x7fab30000bc0).stop 2022-08-01T22:57:45.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.331+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 msgr2=0x7fab400a7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.331+0000 7fab46722700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 0x7fab400a7e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fab4009f2c0 tx=0x7fab3c014000).stop 2022-08-01T22:57:45.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 shutdown_connections 2022-08-01T22:57:45.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab28053a70 0x7fab28055f30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab400ae450 0x7fab400a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab400b28b0 0x7fab400a7e00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 --2- 172.21.15.32:0/1978948271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400b1180 0x7fab4013ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 >> 172.21.15.32:0/1978948271 conn(0x7fab40060310 msgr2=0x7fab40139ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 shutdown_connections 2022-08-01T22:57:45.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.335+0000 7fab46722700 1 -- 172.21.15.32:0/1978948271 wait complete. 2022-08-01T22:57:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 7 and data pool 8 2022-08-01T22:57:45.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:57:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.443+0000 7f5b9b59e700 1 Processor -- start 2022-08-01T22:57:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.443+0000 7f5b9b59e700 1 -- start start 2022-08-01T22:57:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c0af550 0x7f5b9c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b9c05c6f0 0x7f5b9c0a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c0aa630 con 0x7f5b9c060330 2022-08-01T22:57:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9c0b15e0 con 0x7f5b9c05c6f0 2022-08-01T22:57:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0af930 con 0x7f5b9c0af550 2022-08-01T22:57:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b9c05c6f0 0x7f5b9c0a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b99d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b9c060330 0x7f5b9c1445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58344/0 (socket says 172.21.15.32:58344) 2022-08-01T22:57:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b99d9b700 1 -- 172.21.15.32:0/62776096 learned_addr learned my addr 172.21.15.32:0/62776096 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9a59c700 1 --2- 172.21.15.32:0/62776096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c0af550 0x7f5b9c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9959a700 1 -- 172.21.15.32:0/62776096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866781295 0 0) 0x7f5b9c0aa630 con 0x7f5b9c060330 2022-08-01T22:57:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9959a700 1 -- 172.21.15.32:0/62776096 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b80003260 con 0x7f5b9c060330 2022-08-01T22:57:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/62776096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c0af550 msgr2=0x7f5b9c0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9ad9d700 1 --2- 172.21.15.32:0/62776096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c0af550 0x7f5b9c0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/62776096 >> v1:172.21.15.32:6789/0 conn(0x7f5b9c060330 legacy=0x7f5b9c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/62776096 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c05fd10 con 0x7f5b9c05c6f0 2022-08-01T22:57:45.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9ad9d700 1 --2- 172.21.15.32:0/62776096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c05c6f0 0x7f5b9c0a8d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f5b940032f0 tx=0x7f5b940091d0).ready entity=mon.0 client_cookie=adc1eb0df558db78 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9959a700 1 -- 172.21.15.32:0/62776096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9400b600 con 0x7f5b9c05c6f0 2022-08-01T22:57:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9959a700 1 -- 172.21.15.32:0/62776096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9400b760 con 0x7f5b9c05c6f0 2022-08-01T22:57:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9959a700 1 -- 172.21.15.32:0/62776096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9400b8d0 con 0x7f5b9c05c6f0 2022-08-01T22:57:45.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- 172.21.15.32:0/62776096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c05c6f0 msgr2=0x7f5b9c0a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/62776096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c05c6f0 0x7f5b9c0a8d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f5b940032f0 tx=0x7f5b940091d0).stop 2022-08-01T22:57:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- 172.21.15.32:0/62776096 shutdown_connections 2022-08-01T22:57:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/62776096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c05c6f0 0x7f5b9c0a8d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/62776096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c0af550 0x7f5b9c0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- 172.21.15.32:0/62776096 >> 172.21.15.32:0/62776096 conn(0x7f5b9c0612d0 msgr2=0x7f5b9c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- 172.21.15.32:0/62776096 shutdown_connections 2022-08-01T22:57:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.447+0000 7f5b9b59e700 1 -- 172.21.15.32:0/62776096 wait complete. 2022-08-01T22:57:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 Processor -- start 2022-08-01T22:57:45.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 -- start start 2022-08-01T22:57:45.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c05c6f0 0x7f5b9c142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c060330 0x7f5b9c152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 0x7f5b9c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c1432d0 con 0x7f5b9c0af550 2022-08-01T22:57:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c152e70 con 0x7f5b9c060330 2022-08-01T22:57:45.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c153110 con 0x7f5b9c05c6f0 2022-08-01T22:57:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 0x7f5b9c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 0x7f5b9c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42578/0 (socket says 172.21.15.32:42578) 2022-08-01T22:57:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/1049305648 learned_addr learned my addr 172.21.15.32:0/1049305648 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9a59c700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c05c6f0 0x7f5b9c142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b99d9b700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c060330 0x7f5b9c152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c05c6f0 msgr2=0x7f5b9c142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c05c6f0 0x7f5b9c142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c060330 msgr2=0x7f5b9c152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c060330 0x7f5b9c152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 -- 172.21.15.32:0/1049305648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009470 con 0x7f5b9c0af550 2022-08-01T22:57:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9ad9d700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 0x7f5b9c144cd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f5b940032f0 tx=0x7f5b94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9400b600 con 0x7f5b9c0af550 2022-08-01T22:57:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b94011b60 con 0x7f5b9c0af550 2022-08-01T22:57:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c05f840 con 0x7f5b9c0af550 2022-08-01T22:57:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9401f3f0 con 0x7f5b9c0af550 2022-08-01T22:57:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.451+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c141380 con 0x7f5b9c0af550 2022-08-01T22:57:45.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.455+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c141540 con 0x7f5b9c0af550 2022-08-01T22:57:45.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.455+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b94026030 con 0x7f5b9c0af550 2022-08-01T22:57:45.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.455+0000 7f5b8b7fe700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b78053a70 0x7f5b78055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.455+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b9406cb70 con 0x7f5b9c0af550 2022-08-01T22:57:45.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.455+0000 7f5b9a59c700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b78053a70 0x7f5b78055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.459+0000 7f5b9a59c700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b78053a70 0x7f5b78055f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5b8c00b000 tx=0x7f5b8c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.463+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b940439b0 con 0x7f5b9c0af550 2022-08-01T22:57:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.583+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b9c05fa70 con 0x7f5b9c0af550 2022-08-01T22:57:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b8b7fe700 1 -- 172.21.15.32:0/1049305648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f5b94024090 con 0x7f5b9c0af550 2022-08-01T22:57:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:45.593 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:57:45.323644+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:57:43.852752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T22:57:45.012782+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:57:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b78053a70 msgr2=0x7f5b78055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b78053a70 0x7f5b78055f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5b8c00b000 tx=0x7f5b8c006df0).stop 2022-08-01T22:57:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 msgr2=0x7f5b9c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 0x7f5b9c144cd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f5b940032f0 tx=0x7f5b94016000).stop 2022-08-01T22:57:45.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 shutdown_connections 2022-08-01T22:57:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b78053a70 0x7f5b78055f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c0af550 0x7f5b9c144cd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c060330 0x7f5b9c152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 --2- 172.21.15.32:0/1049305648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b9c05c6f0 0x7f5b9c142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 >> 172.21.15.32:0/1049305648 conn(0x7f5b9c0612d0 msgr2=0x7f5b9c0ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 shutdown_connections 2022-08-01T22:57:45.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.587+0000 7f5b9b59e700 1 -- 172.21.15.32:0/1049305648 wait complete. 2022-08-01T22:57:45.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:57:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa7816a0700 1 Processor -- start 2022-08-01T22:57:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa7816a0700 1 -- start start 2022-08-01T22:57:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa7816a0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa77c05c6f0 0x7fa77c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa7816a0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0aa630 con 0x7fa77c0af550 2022-08-01T22:57:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa7816a0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c05fd10 con 0x7fa77c060330 2022-08-01T22:57:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa7816a0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0b15e0 con 0x7fa77c05c6f0 2022-08-01T22:57:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa77c0af550 0x7fa77c1445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58356/0 (socket says 172.21.15.32:58356) 2022-08-01T22:57:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77bfff700 1 -- 172.21.15.32:0/2190651882 learned_addr learned my addr 172.21.15.32:0/2190651882 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa780e9f700 1 --2- 172.21.15.32:0/2190651882 >> v2:172.21.15.55:3300/0 conn(0x7fa77c05c6f0 0x7fa77c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040242987 0 0) 0x7fa77c0aa630 con 0x7fa77c0af550 2022-08-01T22:57:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa768003260 con 0x7fa77c0af550 2022-08-01T22:57:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839516248 0 0) 0x7fa77c05fd10 con 0x7fa77c060330 2022-08-01T22:57:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa77c0aa630 con 0x7fa77c060330 2022-08-01T22:57:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1181774356 0 0) 0x7fa768003260 con 0x7fa77c0af550 2022-08-01T22:57:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 >> v2:172.21.15.55:3300/0 conn(0x7fa77c05c6f0 msgr2=0x7fa77c0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 --2- 172.21.15.32:0/2190651882 >> v2:172.21.15.55:3300/0 conn(0x7fa77c05c6f0 0x7fa77c0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 >> v1:172.21.15.55:6789/0 conn(0x7fa77c060330 legacy=0x7fa77c0a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c05f840 con 0x7fa77c0af550 2022-08-01T22:57:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fa76c0019e0 con 0x7fa77c0af550 2022-08-01T22:57:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa780e9f700 1 --2- 172.21.15.32:0/2190651882 >> v2:172.21.15.55:3300/0 conn(0x7fa77c05c6f0 0x7fa77c0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 >> v1:172.21.15.32:6789/0 conn(0x7fa77c0af550 legacy=0x7fa77c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7680038e0 0x7fa768003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004500 0x7fa768004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa768004e70 0x7fa76800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0b15e0 con 0x7fa768004e70 2022-08-01T22:57:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76800bd20 con 0x7fa7680038e0 2022-08-01T22:57:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.703+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76800bee0 con 0x7fa768004500 2022-08-01T22:57:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa768004e70 0x7fa76800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa77bfff700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7680038e0 0x7fa768003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa77b7fe700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004500 0x7fa768004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 -- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004500 msgr2=0x7fa768004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004500 0x7fa768004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 -- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7680038e0 msgr2=0x7fa768003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7680038e0 0x7fa768003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 -- 172.21.15.32:0/2190651882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768003260 con 0x7fa768004e70 2022-08-01T22:57:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 -- 172.21.15.32:0/2190651882 reap_dead start 2022-08-01T22:57:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa780e9f700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa768004e70 0x7fa76800b750 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fa77400c000 tx=0x7fa774006df0).ready entity=mon.0 client_cookie=7b940e637c2ada44 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa774016070 con 0x7fa768004e70 2022-08-01T22:57:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77400e830 con 0x7fa768004e70 2022-08-01T22:57:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa77affd700 1 -- 172.21.15.32:0/2190651882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa77400eb90 con 0x7fa768004e70 2022-08-01T22:57:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 -- 172.21.15.32:0/2190651882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa768004e70 msgr2=0x7fa76800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa768004e70 0x7fa76800b750 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fa77400c000 tx=0x7fa774006df0).stop 2022-08-01T22:57:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 -- 172.21.15.32:0/2190651882 shutdown_connections 2022-08-01T22:57:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 --2- 172.21.15.32:0/2190651882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa768004e70 0x7fa76800b750 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 -- 172.21.15.32:0/2190651882 >> 172.21.15.32:0/2190651882 conn(0x7fa77c0612d0 msgr2=0x7fa77c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 -- 172.21.15.32:0/2190651882 shutdown_connections 2022-08-01T22:57:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.707+0000 7fa7816a0700 1 -- 172.21.15.32:0/2190651882 wait complete. 2022-08-01T22:57:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 Processor -- start 2022-08-01T22:57:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 -- start start 2022-08-01T22:57:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004e70 0x7fa77c145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa77c143e00 0x7fa77c150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 0x7fa77c150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c146030 con 0x7fa77c145b40 2022-08-01T22:57:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c067420 con 0x7fa77c143e00 2022-08-01T22:57:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c067890 con 0x7fa768004e70 2022-08-01T22:57:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 0x7fa77c150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 0x7fa77c150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42596/0 (socket says 172.21.15.32:42596) 2022-08-01T22:57:45.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 -- 172.21.15.32:0/1250123201 learned_addr learned my addr 172.21.15.32:0/1250123201 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:45.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa77bfff700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004e70 0x7fa77c145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa77b7fe700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa77c143e00 0x7fa77c150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 -- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004e70 msgr2=0x7fa77c145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004e70 0x7fa77c145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 -- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa77c143e00 msgr2=0x7fa77c150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa77c143e00 0x7fa77c150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 -- 172.21.15.32:0/1250123201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c1461b0 con 0x7fa77c145b40 2022-08-01T22:57:45.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa780e9f700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 0x7fa77c150610 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fa77400c000 tx=0x7fa774018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa774022070 con 0x7fa77c145b40 2022-08-01T22:57:45.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774000e90 con 0x7fa77c145b40 2022-08-01T22:57:45.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c142ee0 con 0x7fa77c145b40 2022-08-01T22:57:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa774012df0 con 0x7fa77c145b40 2022-08-01T22:57:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.711+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c065c10 con 0x7fa77c145b40 2022-08-01T22:57:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.715+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c0675e0 con 0x7fa77c145b40 2022-08-01T22:57:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.715+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa77401d430 con 0x7fa77c145b40 2022-08-01T22:57:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.715+0000 7fa7797fa700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa75c053a70 0x7fa75c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.715+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa77406d710 con 0x7fa77c145b40 2022-08-01T22:57:45.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.715+0000 7fa77bfff700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa75c053a70 0x7fa75c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.715+0000 7fa77bfff700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa75c053a70 0x7fa75c055f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fa76c005000 tx=0x7fa77c0b05f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.719+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa774044470 con 0x7fa77c145b40 2022-08-01T22:57:45.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.847+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa77c146340 con 0x7fa77c145b40 2022-08-01T22:57:45.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7797fa700 1 -- 172.21.15.32:0/1250123201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16284 (secure 0 0 0) 0x7fa774047a90 con 0x7fa77c145b40 2022-08-01T22:57:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:45.856 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:57:45.323644+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:57:43.852752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T22:57:45.012782+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:57:45.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa75c053a70 msgr2=0x7fa75c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa75c053a70 0x7fa75c055f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fa76c005000 tx=0x7fa77c0b05f0).stop 2022-08-01T22:57:45.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 msgr2=0x7fa77c150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 0x7fa77c150610 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fa77400c000 tx=0x7fa774018000).stop 2022-08-01T22:57:45.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 shutdown_connections 2022-08-01T22:57:45.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa75c053a70 0x7fa75c055f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77c145b40 0x7fa77c150610 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa77c143e00 0x7fa77c150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 --2- 172.21.15.32:0/1250123201 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa768004e70 0x7fa77c145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 >> 172.21.15.32:0/1250123201 conn(0x7fa77c0612d0 msgr2=0x7fa77c062f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 shutdown_connections 2022-08-01T22:57:45.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.851+0000 7fa7816a0700 1 -- 172.21.15.32:0/1250123201 wait complete. 2022-08-01T22:57:45.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T22:57:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 Processor -- start 2022-08-01T22:57:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 -- start start 2022-08-01T22:57:45.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14180614b0 0x7f1418061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14181405d0 0x7f1418140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180aee80 con 0x7f14180ad750 2022-08-01T22:57:45.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180b70b0 con 0x7f14180614b0 2022-08-01T22:57:45.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1417fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180af0b0 con 0x7f14181405d0 2022-08-01T22:57:45.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14180614b0 0x7f1418061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14180614b0 0x7f1418061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42602/0 (socket says 172.21.15.32:42602) 2022-08-01T22:57:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 -- 172.21.15.32:0/1716310224 learned_addr learned my addr 172.21.15.32:0/1716310224 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f14177fe700 1 --2- 172.21.15.32:0/1716310224 >> v2:172.21.15.55:3300/0 conn(0x7f14181405d0 0x7f1418140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 -- 172.21.15.32:0/1716310224 >> v2:172.21.15.55:3300/0 conn(0x7f14181405d0 msgr2=0x7f1418140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 --2- 172.21.15.32:0/1716310224 >> v2:172.21.15.55:3300/0 conn(0x7f14181405d0 0x7f1418140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 -- 172.21.15.32:0/1716310224 >> v1:172.21.15.55:6789/0 conn(0x7f14180ad750 legacy=0x7f14180b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 -- 172.21.15.32:0/1716310224 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180af350 con 0x7f14180614b0 2022-08-01T22:57:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1416ffd700 1 --2- 172.21.15.32:0/1716310224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180614b0 0x7f1418061890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f14180aed30 tx=0x7f1408006eb0).ready entity=mon.0 client_cookie=147907bd59f28bee server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1415ffb700 1 -- 172.21.15.32:0/1716310224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f140800d610 con 0x7f14180614b0 2022-08-01T22:57:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1415ffb700 1 -- 172.21.15.32:0/1716310224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1408000e90 con 0x7f14180614b0 2022-08-01T22:57:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.983+0000 7f1415ffb700 1 -- 172.21.15.32:0/1716310224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f140800d960 con 0x7f14180614b0 2022-08-01T22:57:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- 172.21.15.32:0/1716310224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180614b0 msgr2=0x7f1418061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 --2- 172.21.15.32:0/1716310224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180614b0 0x7f1418061890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f14180aed30 tx=0x7f1408006eb0).stop 2022-08-01T22:57:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- 172.21.15.32:0/1716310224 shutdown_connections 2022-08-01T22:57:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 --2- 172.21.15.32:0/1716310224 >> v2:172.21.15.55:3300/0 conn(0x7f14181405d0 0x7f1418140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 --2- 172.21.15.32:0/1716310224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180614b0 0x7f1418061890 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- 172.21.15.32:0/1716310224 >> 172.21.15.32:0/1716310224 conn(0x7f1418066f70 msgr2=0x7f1418067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- 172.21.15.32:0/1716310224 shutdown_connections 2022-08-01T22:57:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- 172.21.15.32:0/1716310224 wait complete. 2022-08-01T22:57:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 Processor -- start 2022-08-01T22:57:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- start start 2022-08-01T22:57:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14180614b0 0x7f14180b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 0x7f1418143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14181405d0 0x7f1418155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180b3570 con 0x7f14180ad750 2022-08-01T22:57:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418153600 con 0x7f14181405d0 2022-08-01T22:57:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1417fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14180b2910 con 0x7f14180614b0 2022-08-01T22:57:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 0x7f1418143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 0x7f1418143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42610/0 (socket says 172.21.15.32:42610) 2022-08-01T22:57:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 -- 172.21.15.32:0/2606347525 learned_addr learned my addr 172.21.15.32:0/2606347525 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f1416ffd700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14180614b0 0x7f14180b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14177fe700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14181405d0 0x7f1418155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 -- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14180614b0 msgr2=0x7f14180b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14180614b0 0x7f14180b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 -- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14181405d0 msgr2=0x7f1418155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14181405d0 0x7f1418155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.987+0000 7f14167fc700 1 -- 172.21.15.32:0/2606347525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140800b1e0 con 0x7f14180ad750 2022-08-01T22:57:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f14167fc700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 0x7f1418143640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f14180ae2b0 tx=0x7f140c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f140c007030 con 0x7f14180ad750 2022-08-01T22:57:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c007190 con 0x7f14180ad750 2022-08-01T22:57:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f140c006510 con 0x7f14180ad750 2022-08-01T22:57:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14180b21b0 con 0x7f14180ad750 2022-08-01T22:57:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1418142a80 con 0x7f14180ad750 2022-08-01T22:57:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14180af5f0 con 0x7f14180ad750 2022-08-01T22:57:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f140c010030 con 0x7f14180ad750 2022-08-01T22:57:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1407fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13f4053950 0x7f13f4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f140c06bb80 con 0x7f14180ad750 2022-08-01T22:57:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1416ffd700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13f4053950 0x7f13f4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.991+0000 7f1416ffd700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13f4053950 0x7f13f4055e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f140801c000 tx=0x7f1408015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:45.995+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f140c042940 con 0x7f14180ad750 2022-08-01T22:57:46.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:46.159+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1418062290 con 0x7f14180ad750 2022-08-01T22:57:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.135+0000 7f1407fff700 1 -- 172.21.15.32:0/2606347525 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f140c045f60 con 0x7f14180ad750 2022-08-01T22:57:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13f4053950 msgr2=0x7f13f4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13f4053950 0x7f13f4055e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f140801c000 tx=0x7f1408015000).stop 2022-08-01T22:57:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 msgr2=0x7f1418143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 0x7f1418143640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f14180ae2b0 tx=0x7f140c007bb0).stop 2022-08-01T22:57:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 shutdown_connections 2022-08-01T22:57:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13f4053950 0x7f13f4055e10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14181405d0 0x7f1418155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14180ad750 0x7f1418143640 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 --2- 172.21.15.32:0/2606347525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14180614b0 0x7f14180b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 >> 172.21.15.32:0/2606347525 conn(0x7f1418066f70 msgr2=0x7f14180b8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:47.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 shutdown_connections 2022-08-01T22:57:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.139+0000 7f1417fff700 1 -- 172.21.15.32:0/2606347525 wait complete. 2022-08-01T22:57:47.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.263+0000 7f50645b7700 1 Processor -- start 2022-08-01T22:57:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.263+0000 7f50645b7700 1 -- start start 2022-08-01T22:57:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c148530 0x7f505c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c0b04e0 con 0x7f505c0b1f40 2022-08-01T22:57:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c0b0780 con 0x7f505c0a7250 2022-08-01T22:57:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0a9250 con 0x7f505c148530 2022-08-01T22:57:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5062353700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f505c0b1f40 0x7f505c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58392/0 (socket says 172.21.15.32:58392) 2022-08-01T22:57:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5062353700 1 -- 172.21.15.32:0/3768193909 learned_addr learned my addr 172.21.15.32:0/3768193909 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061b52700 1 --2- 172.21.15.32:0/3768193909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c148530 0x7f505c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25403349 0 0) 0x7f505c0b04e0 con 0x7f505c0b1f40 2022-08-01T22:57:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5044003260 con 0x7f505c0b1f40 2022-08-01T22:57:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899980787 0 0) 0x7f505c0b0780 con 0x7f505c0a7250 2022-08-01T22:57:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505c0b04e0 con 0x7f505c0a7250 2022-08-01T22:57:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061b52700 1 -- 172.21.15.32:0/3768193909 >> v1:172.21.15.55:6789/0 conn(0x7f505c0a7250 legacy=0x7f505c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061b52700 1 -- 172.21.15.32:0/3768193909 >> v1:172.21.15.32:6789/0 conn(0x7f505c0b1f40 legacy=0x7f505c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061b52700 1 -- 172.21.15.32:0/3768193909 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0b0a20 con 0x7f505c148530 2022-08-01T22:57:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061b52700 1 --2- 172.21.15.32:0/3768193909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c148530 0x7f505c0b1160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f5058002f70 tx=0x7f5058006df0).ready entity=mon.2 client_cookie=d837ce718ec92f95 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f505800d5c0 con 0x7f505c148530 2022-08-01T22:57:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5058000e90 con 0x7f505c148530 2022-08-01T22:57:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f5061351700 1 -- 172.21.15.32:0/3768193909 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f505800d910 con 0x7f505c148530 2022-08-01T22:57:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- 172.21.15.32:0/3768193909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c148530 msgr2=0x7f505c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 --2- 172.21.15.32:0/3768193909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c148530 0x7f505c0b1160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f5058002f70 tx=0x7f5058006df0).stop 2022-08-01T22:57:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- 172.21.15.32:0/3768193909 shutdown_connections 2022-08-01T22:57:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 --2- 172.21.15.32:0/3768193909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c148530 0x7f505c0b1160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- 172.21.15.32:0/3768193909 >> 172.21.15.32:0/3768193909 conn(0x7f505c05f840 msgr2=0x7f505c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- 172.21.15.32:0/3768193909 shutdown_connections 2022-08-01T22:57:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.267+0000 7f50645b7700 1 -- 172.21.15.32:0/3768193909 wait complete. 2022-08-01T22:57:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 Processor -- start 2022-08-01T22:57:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 -- start start 2022-08-01T22:57:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c0a7250 0x7f505c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f505c0b1f40 0x7f505c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 0x7f505c1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0aecc0 con 0x7f505c148530 2022-08-01T22:57:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0aea30 con 0x7f505c0b1f40 2022-08-01T22:57:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0a96e0 con 0x7f505c0a7250 2022-08-01T22:57:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 0x7f505c1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 0x7f505c1439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42626/0 (socket says 172.21.15.32:42626) 2022-08-01T22:57:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 -- 172.21.15.32:0/1365891633 learned_addr learned my addr 172.21.15.32:0/1365891633 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062353700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c0a7250 0x7f505c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5061b52700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f505c0b1f40 0x7f505c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 -- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c0a7250 msgr2=0x7f505c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c0a7250 0x7f505c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 -- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f505c0b1f40 msgr2=0x7f505c0ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f505c0b1f40 0x7f505c0ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 -- 172.21.15.32:0/1365891633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505800b180 con 0x7f505c148530 2022-08-01T22:57:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f5062b54700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 0x7f505c1439d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f5054001490 tx=0x7f5054007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5054002310 con 0x7f505c148530 2022-08-01T22:57:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054002470 con 0x7f505c148530 2022-08-01T22:57:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c0a9980 con 0x7f505c148530 2022-08-01T22:57:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5054001840 con 0x7f505c148530 2022-08-01T22:57:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.271+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c14a870 con 0x7f505c148530 2022-08-01T22:57:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.275+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c0574e0 con 0x7f505c148530 2022-08-01T22:57:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.275+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5054010030 con 0x7f505c148530 2022-08-01T22:57:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.275+0000 7f504b7fe700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f503c053aa0 0x7f503c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.275+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f505406c090 con 0x7f505c148530 2022-08-01T22:57:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.275+0000 7f5062353700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f503c053aa0 0x7f503c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.275+0000 7f5062353700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f503c053aa0 0x7f503c055f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f504c00d000 tx=0x7f504c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.283+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5054042db0 con 0x7f505c148530 2022-08-01T22:57:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.443+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f505c0679a0 con 0x7f505c148530 2022-08-01T22:57:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f504b7fe700 1 -- 172.21.15.32:0/1365891633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f50540463d0 con 0x7f505c148530 2022-08-01T22:57:47.451 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:57:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f503c053aa0 msgr2=0x7f503c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f503c053aa0 0x7f503c055f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f504c00d000 tx=0x7f504c009040).stop 2022-08-01T22:57:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 msgr2=0x7f505c1439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 0x7f505c1439d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f5054001490 tx=0x7f5054007d90).stop 2022-08-01T22:57:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 shutdown_connections 2022-08-01T22:57:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f503c053aa0 0x7f503c055f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c148530 0x7f505c1439d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f505c0b1f40 0x7f505c0ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 --2- 172.21.15.32:0/1365891633 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f505c0a7250 0x7f505c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 >> 172.21.15.32:0/1365891633 conn(0x7f505c05f840 msgr2=0x7f505c147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:47.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 shutdown_connections 2022-08-01T22:57:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.447+0000 7f50645b7700 1 -- 172.21.15.32:0/1365891633 wait complete. 2022-08-01T22:57:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:57:47.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:57:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.559+0000 7f85cef7a700 1 Processor -- start 2022-08-01T22:57:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cef7a700 1 -- start start 2022-08-01T22:57:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cef7a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f85c80af3c0 0x7f85c80601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cef7a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805c620 0x7f85c80a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cef7a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c81419f0 con 0x7f85c805f840 2022-08-01T22:57:47.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cef7a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80b1b60 con 0x7f85c80af3c0 2022-08-01T22:57:47.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cef7a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8141c20 con 0x7f85c805c620 2022-08-01T22:57:47.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cd777700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f85c805f840 0x7f85c80a8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58404/0 (socket says 172.21.15.32:58404) 2022-08-01T22:57:47.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cd777700 1 -- 172.21.15.32:0/2350032634 learned_addr learned my addr 172.21.15.32:0/2350032634 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:47.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cdf78700 1 --2- 172.21.15.32:0/2350032634 >> v2:172.21.15.55:3300/0 conn(0x7f85c80af3c0 0x7f85c80601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ce779700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805c620 0x7f85c80a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ccf76700 1 -- 172.21.15.32:0/2350032634 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717939008 0 0) 0x7f85c81419f0 con 0x7f85c805f840 2022-08-01T22:57:47.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ccf76700 1 -- 172.21.15.32:0/2350032634 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b0003260 con 0x7f85c805f840 2022-08-01T22:57:47.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ccf76700 1 -- 172.21.15.32:0/2350032634 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3849433402 0 0) 0x7f85b0003260 con 0x7f85c805f840 2022-08-01T22:57:47.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cdf78700 1 -- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805c620 msgr2=0x7f85c80a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cdf78700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805c620 0x7f85c80a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cdf78700 1 -- 172.21.15.32:0/2350032634 >> v1:172.21.15.32:6789/0 conn(0x7f85c805f840 legacy=0x7f85c80a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cdf78700 1 -- 172.21.15.32:0/2350032634 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8141ec0 con 0x7f85c80af3c0 2022-08-01T22:57:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ce779700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805c620 0x7f85c80a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85cdf78700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80601a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f85b8001800 tx=0x7f85c4006df0).ready entity=mon.1 client_cookie=75bfecb43f9c8ab4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ccf76700 1 -- 172.21.15.32:0/2350032634 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b8001660 con 0x7f85c80af3c0 2022-08-01T22:57:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ccf76700 1 -- 172.21.15.32:0/2350032634 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c4000e90 con 0x7f85c80af3c0 2022-08-01T22:57:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.563+0000 7f85ccf76700 1 -- 172.21.15.32:0/2350032634 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c4012460 con 0x7f85c80af3c0 2022-08-01T22:57:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 msgr2=0x7f85c80601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80601a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f85b8001800 tx=0x7f85c4006df0).stop 2022-08-01T22:57:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- 172.21.15.32:0/2350032634 shutdown_connections 2022-08-01T22:57:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805c620 0x7f85c80a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2350032634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80601a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- 172.21.15.32:0/2350032634 >> 172.21.15.32:0/2350032634 conn(0x7f85c8061360 msgr2=0x7f85c8062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- 172.21.15.32:0/2350032634 shutdown_connections 2022-08-01T22:57:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- 172.21.15.32:0/2350032634 wait complete. 2022-08-01T22:57:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 Processor -- start 2022-08-01T22:57:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- start start 2022-08-01T22:57:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 0x7f85c8066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805f840 0x7f85c809f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8065e10 con 0x7f85c805c620 2022-08-01T22:57:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8150450 con 0x7f85c80af3c0 2022-08-01T22:57:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cef7a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c809ee70 con 0x7f85c805f840 2022-08-01T22:57:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 0x7f85c8066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 0x7f85c8066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42638/0 (socket says 172.21.15.32:42638) 2022-08-01T22:57:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 -- 172.21.15.32:0/2777072909 learned_addr learned my addr 172.21.15.32:0/2777072909 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cd777700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805f840 0x7f85c809f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85ce779700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 -- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805f840 msgr2=0x7f85c809f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805f840 0x7f85c809f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 -- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 msgr2=0x7f85c80a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.567+0000 7f85cdf78700 1 -- 172.21.15.32:0/2777072909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c0009470 con 0x7f85c805c620 2022-08-01T22:57:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85cdf78700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 0x7f85c8066840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f85b8001800 tx=0x7f85c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c4016070 con 0x7f85c805c620 2022-08-01T22:57:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c400bec0 con 0x7f85c805c620 2022-08-01T22:57:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c400b180 con 0x7f85c805c620 2022-08-01T22:57:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c400ddf0 con 0x7f85c805c620 2022-08-01T22:57:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c809f3b0 con 0x7f85c805c620 2022-08-01T22:57:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c80b0a80 con 0x7f85c805c620 2022-08-01T22:57:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85c4025030 con 0x7f85c805c620 2022-08-01T22:57:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85b6ffd700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f85a8053980 0x7f85a8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f85c406cfe0 con 0x7f85c805c620 2022-08-01T22:57:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.571+0000 7f85cd777700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f85a8053980 0x7f85a8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.575+0000 7f85cd777700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f85a8053980 0x7f85a8055e40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f85b800d000 tx=0x7f85b8001de0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.579+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85c4043da0 con 0x7f85c805c620 2022-08-01T22:57:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.703+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85c80574e0 con 0x7f85c805c620 2022-08-01T22:57:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85b6ffd700 1 -- 172.21.15.32:0/2777072909 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f85c4023070 con 0x7f85c805c620 2022-08-01T22:57:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:47.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:57:46.326950+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:57:43.852752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T22:57:45.012782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:57:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f85a8053980 msgr2=0x7f85a8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f85a8053980 0x7f85a8055e40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f85b800d000 tx=0x7f85b8001de0).stop 2022-08-01T22:57:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 msgr2=0x7f85c8066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 0x7f85c8066840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f85b8001800 tx=0x7f85c4014000).stop 2022-08-01T22:57:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 shutdown_connections 2022-08-01T22:57:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f85a8053980 0x7f85a8055e40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c80af3c0 0x7f85c80a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f85c805f840 0x7f85c809f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 --2- 172.21.15.32:0/2777072909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85c805c620 0x7f85c8066840 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 >> 172.21.15.32:0/2777072909 conn(0x7f85c8061360 msgr2=0x7f85c813d280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 shutdown_connections 2022-08-01T22:57:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.707+0000 7f85cef7a700 1 -- 172.21.15.32:0/2777072909 wait complete. 2022-08-01T22:57:47.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T22:57:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 Processor -- start 2022-08-01T22:57:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 -- start start 2022-08-01T22:57:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fedfc0b4550 0x7fedfc0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedfc144240 con 0x7fedfc0b6710 2022-08-01T22:57:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedfc143d00 con 0x7fedfc0b4550 2022-08-01T22:57:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.831+0000 7fee01be3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc143fa0 con 0x7fedfc0b6af0 2022-08-01T22:57:47.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fedfc0b6710 0x7fedfc061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58418/0 (socket says 172.21.15.32:58418) 2022-08-01T22:57:47.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfb7fe700 1 -- 172.21.15.32:0/3256667258 learned_addr learned my addr 172.21.15.32:0/3256667258 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:47.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfaffd700 1 --2- 172.21.15.32:0/3256667258 >> v2:172.21.15.55:3300/0 conn(0x7fedfc0b4550 0x7fedfc0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfbfff700 1 --2- 172.21.15.32:0/3256667258 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfa7fc700 1 -- 172.21.15.32:0/3256667258 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396169593 0 0) 0x7fedfc144240 con 0x7fedfc0b6710 2022-08-01T22:57:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfa7fc700 1 -- 172.21.15.32:0/3256667258 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feddc003260 con 0x7fedfc0b6710 2022-08-01T22:57:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfbfff700 1 -- 172.21.15.32:0/3256667258 >> v2:172.21.15.55:3300/0 conn(0x7fedfc0b4550 msgr2=0x7fedfc0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfbfff700 1 --2- 172.21.15.32:0/3256667258 >> v2:172.21.15.55:3300/0 conn(0x7fedfc0b4550 0x7fedfc0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfbfff700 1 -- 172.21.15.32:0/3256667258 >> v1:172.21.15.32:6789/0 conn(0x7fedfc0b6710 legacy=0x7fedfc061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfbfff700 1 -- 172.21.15.32:0/3256667258 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc061780 con 0x7fedfc0b6af0 2022-08-01T22:57:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfaffd700 1 --2- 172.21.15.32:0/3256667258 >> v2:172.21.15.55:3300/0 conn(0x7fedfc0b4550 0x7fedfc0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfbfff700 1 --2- 172.21.15.32:0/3256667258 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc12bb90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fedfc1432b0 tx=0x7fedf40091d0).ready entity=mon.2 client_cookie=fe0e770bd71d6434 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfa7fc700 1 -- 172.21.15.32:0/3256667258 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedf400b600 con 0x7fedfc0b6af0 2022-08-01T22:57:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfa7fc700 1 -- 172.21.15.32:0/3256667258 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf400b760 con 0x7fedfc0b6af0 2022-08-01T22:57:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fedfa7fc700 1 -- 172.21.15.32:0/3256667258 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedf400b8d0 con 0x7fedfc0b6af0 2022-08-01T22:57:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 -- 172.21.15.32:0/3256667258 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 msgr2=0x7fedfc12bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 --2- 172.21.15.32:0/3256667258 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc12bb90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fedfc1432b0 tx=0x7fedf40091d0).stop 2022-08-01T22:57:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 -- 172.21.15.32:0/3256667258 shutdown_connections 2022-08-01T22:57:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 --2- 172.21.15.32:0/3256667258 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc12bb90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 --2- 172.21.15.32:0/3256667258 >> v2:172.21.15.55:3300/0 conn(0x7fedfc0b4550 0x7fedfc0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 -- 172.21.15.32:0/3256667258 >> 172.21.15.32:0/3256667258 conn(0x7fedfc05c620 msgr2=0x7fedfc05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 -- 172.21.15.32:0/3256667258 shutdown_connections 2022-08-01T22:57:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.835+0000 7fee01be3700 1 -- 172.21.15.32:0/3256667258 wait complete. 2022-08-01T22:57:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 Processor -- start 2022-08-01T22:57:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 -- start start 2022-08-01T22:57:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 0x7fedfc1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedfc0b6710 0x7fedfc129910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc1288e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc129f40 con 0x7fedfc0b4550 2022-08-01T22:57:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc12a1e0 con 0x7fedfc0b6710 2022-08-01T22:57:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc1523a0 con 0x7fedfc0b6af0 2022-08-01T22:57:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 0x7fedfc1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 0x7fedfc1558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42650/0 (socket says 172.21.15.32:42650) 2022-08-01T22:57:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 -- 172.21.15.32:0/1446158972 learned_addr learned my addr 172.21.15.32:0/1446158972 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfbfff700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc1288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfaffd700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedfc0b6710 0x7fedfc129910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 -- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 msgr2=0x7fedfc1288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc1288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 -- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedfc0b6710 msgr2=0x7fedfc129910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedfc0b6710 0x7fedfc129910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 -- 172.21.15.32:0/1446158972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede400b180 con 0x7fedfc0b4550 2022-08-01T22:57:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedfb7fe700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 0x7fedfc1558f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fedec001110 tx=0x7fedec007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedec0070d0 con 0x7fedfc0b4550 2022-08-01T22:57:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec007230 con 0x7fedfc0b4550 2022-08-01T22:57:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf4009470 con 0x7fedfc0b4550 2022-08-01T22:57:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedec005610 con 0x7fedfc0b4550 2022-08-01T22:57:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.839+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc1479d0 con 0x7fedfc0b4550 2022-08-01T22:57:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.843+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc061a20 con 0x7fedfc0b4550 2022-08-01T22:57:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.843+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fedec010070 con 0x7fedfc0b4550 2022-08-01T22:57:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.843+0000 7fedf8ff9700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fede0053b00 0x7fede0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.843+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fedec06b7d0 con 0x7fedfc0b4550 2022-08-01T22:57:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.843+0000 7fedfaffd700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fede0053b00 0x7fede0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.847+0000 7fedfaffd700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fede0053b00 0x7fede0055fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fede400b000 tx=0x7fede4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:47.851+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fedec042480 con 0x7fedfc0b4550 2022-08-01T22:57:48.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.015+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fedfc0574e0 con 0x7fedfc0b4550 2022-08-01T22:57:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.027+0000 7fedf8ff9700 1 -- 172.21.15.32:0/1446158972 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fedec045aa0 con 0x7fedfc0b4550 2022-08-01T22:57:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fede0053b00 msgr2=0x7fede0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fede0053b00 0x7fede0055fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fede400b000 tx=0x7fede4006df0).stop 2022-08-01T22:57:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 msgr2=0x7fedfc1558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 0x7fedfc1558f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fedec001110 tx=0x7fedec007ad0).stop 2022-08-01T22:57:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 shutdown_connections 2022-08-01T22:57:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fede0053b00 0x7fede0055fc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fedfc0b6af0 0x7fedfc1288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedfc0b6710 0x7fedfc129910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 --2- 172.21.15.32:0/1446158972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0b4550 0x7fedfc1558f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 >> 172.21.15.32:0/1446158972 conn(0x7fedfc05c620 msgr2=0x7fedfc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 shutdown_connections 2022-08-01T22:57:48.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.031+0000 7fee01be3700 1 -- 172.21.15.32:0/1446158972 wait complete. 2022-08-01T22:57:48.040 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T22:57:48.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T22:57:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.147+0000 7f1d41fa0700 1 Processor -- start 2022-08-01T22:57:48.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.147+0000 7f1d41fa0700 1 -- start start 2022-08-01T22:57:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d3c0b2730 0x7f1d3c0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c066ba0 0x7f1d3c066fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c0654c0 con 0x7f1d3c0b3190 2022-08-01T22:57:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c062190 con 0x7f1d3c0b2730 2022-08-01T22:57:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b3570 con 0x7f1d3c066ba0 2022-08-01T22:57:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d3bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d3c0b2730 0x7f1d3c0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d4179f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c066ba0 0x7f1d3c066fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d40f9e700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1d3c0b3190 0x7f1d3c060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:51106/0 (socket says 172.21.15.32:51106) 2022-08-01T22:57:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d40f9e700 1 -- 172.21.15.32:0/3989410761 learned_addr learned my addr 172.21.15.32:0/3989410761 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d4179f700 1 -- 172.21.15.32:0/3989410761 >> v2:172.21.15.55:3300/0 conn(0x7f1d3c0b2730 msgr2=0x7f1d3c0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d3b7fe700 1 -- 172.21.15.32:0/3989410761 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572020798 0 0) 0x7f1d3c0654c0 con 0x7f1d3c0b3190 2022-08-01T22:57:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d4179f700 1 --2- 172.21.15.32:0/3989410761 >> v2:172.21.15.55:3300/0 conn(0x7f1d3c0b2730 0x7f1d3c0666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d4179f700 1 -- 172.21.15.32:0/3989410761 >> v1:172.21.15.55:6789/0 conn(0x7f1d3c0b3190 legacy=0x7f1d3c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d4179f700 1 -- 172.21.15.32:0/3989410761 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c05fbf0 con 0x7f1d3c066ba0 2022-08-01T22:57:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d4179f700 1 --2- 172.21.15.32:0/3989410761 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c066ba0 0x7f1d3c066fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1d340032f0 tx=0x7f1d340091d0).ready entity=mon.2 client_cookie=ca28f69ce60d003d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d3b7fe700 1 -- 172.21.15.32:0/3989410761 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3400b6c0 con 0x7f1d3c066ba0 2022-08-01T22:57:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d3b7fe700 1 -- 172.21.15.32:0/3989410761 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400b820 con 0x7f1d3c066ba0 2022-08-01T22:57:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d3b7fe700 1 -- 172.21.15.32:0/3989410761 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3400b990 con 0x7f1d3c066ba0 2022-08-01T22:57:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- 172.21.15.32:0/3989410761 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c066ba0 msgr2=0x7f1d3c066fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/3989410761 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c066ba0 0x7f1d3c066fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1d340032f0 tx=0x7f1d340091d0).stop 2022-08-01T22:57:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- 172.21.15.32:0/3989410761 shutdown_connections 2022-08-01T22:57:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/3989410761 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c066ba0 0x7f1d3c066fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/3989410761 >> v2:172.21.15.55:3300/0 conn(0x7f1d3c0b2730 0x7f1d3c0666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- 172.21.15.32:0/3989410761 >> 172.21.15.32:0/3989410761 conn(0x7f1d3c063870 msgr2=0x7f1d3c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:48.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- 172.21.15.32:0/3989410761 shutdown_connections 2022-08-01T22:57:48.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.151+0000 7f1d41fa0700 1 -- 172.21.15.32:0/3989410761 wait complete. 2022-08-01T22:57:48.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 Processor -- start 2022-08-01T22:57:48.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 -- start start 2022-08-01T22:57:48.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c066ba0 0x7f1d3c0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2730 0x7f1d3c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 0x7f1d3c0ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b0500 con 0x7f1d3c0b3190 2022-08-01T22:57:48.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b0760 con 0x7f1d3c066ba0 2022-08-01T22:57:48.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d41fa0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b0a00 con 0x7f1d3c0b2730 2022-08-01T22:57:48.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d4179f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 0x7f1d3c0ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d4179f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 0x7f1d3c0ac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42668/0 (socket says 172.21.15.32:42668) 2022-08-01T22:57:48.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d4179f700 1 -- 172.21.15.32:0/1932962415 learned_addr learned my addr 172.21.15.32:0/1932962415 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:48.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d40f9e700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c066ba0 0x7f1d3c0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d3bfff700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2730 0x7f1d3c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d40f9e700 1 -- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2730 msgr2=0x7f1d3c1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d40f9e700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2730 0x7f1d3c1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d40f9e700 1 -- 172.21.15.32:0/1932962415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 msgr2=0x7f1d3c0ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d40f9e700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 0x7f1d3c0ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d40f9e700 1 -- 172.21.15.32:0/1932962415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009470 con 0x7f1d3c066ba0 2022-08-01T22:57:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d4179f700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 0x7f1d3c0ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.155+0000 7f1d3bfff700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2730 0x7f1d3c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d40f9e700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c066ba0 0x7f1d3c0b6fb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f1d30000ed0 tx=0x7f1d30007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d300070d0 con 0x7f1d3c066ba0 2022-08-01T22:57:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30007230 con 0x7f1d3c066ba0 2022-08-01T22:57:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d30005610 con 0x7f1d3c066ba0 2022-08-01T22:57:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c0aa3f0 con 0x7f1d3c066ba0 2022-08-01T22:57:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c143660 con 0x7f1d3c066ba0 2022-08-01T22:57:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c0aa890 con 0x7f1d3c066ba0 2022-08-01T22:57:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d30010070 con 0x7f1d3c066ba0 2022-08-01T22:57:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d39ffb700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d1c0539e0 0x7f1d1c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.159+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d3006b870 con 0x7f1d3c066ba0 2022-08-01T22:57:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.163+0000 7f1d3bfff700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d1c0539e0 0x7f1d1c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.163+0000 7f1d3bfff700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d1c0539e0 0x7f1d1c055ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1d2800b000 tx=0x7f1d28006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.171+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d3c0aa890 con 0x7f1d3c066ba0 2022-08-01T22:57:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.339+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1d3c0aa890 con 0x7f1d3c066ba0 2022-08-01T22:57:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.347+0000 7f1d39ffb700 1 -- 172.21.15.32:0/1932962415 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1d3c0aa890 con 0x7f1d3c066ba0 2022-08-01T22:57:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d1c0539e0 msgr2=0x7f1d1c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d1c0539e0 0x7f1d1c055ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1d2800b000 tx=0x7f1d28006df0).stop 2022-08-01T22:57:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c066ba0 msgr2=0x7f1d3c0b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c066ba0 0x7f1d3c0b6fb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f1d30000ed0 tx=0x7f1d30007c20).stop 2022-08-01T22:57:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 shutdown_connections 2022-08-01T22:57:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d1c0539e0 0x7f1d1c055ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b3190 0x7f1d3c0ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2730 0x7f1d3c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 --2- 172.21.15.32:0/1932962415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c066ba0 0x7f1d3c0b6fb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 >> 172.21.15.32:0/1932962415 conn(0x7f1d3c063870 msgr2=0x7f1d3c1462c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 shutdown_connections 2022-08-01T22:57:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.351+0000 7f1d41fa0700 1 -- 172.21.15.32:0/1932962415 wait complete. 2022-08-01T22:57:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T22:57:48.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.463+0000 7f0dbe4ca700 1 Processor -- start 2022-08-01T22:57:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.463+0000 7f0dbe4ca700 1 -- start start 2022-08-01T22:57:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80b3630 0x7f0db8143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0db80aa0d0 0x7f0db80af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db80b3a10 con 0x7f0db80af6f0 2022-08-01T22:57:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db81443f0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80afb80 con 0x7f0db80b3630 2022-08-01T22:57:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbca67700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0db80af6f0 0x7f0db805f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58448/0 (socket says 172.21.15.32:58448) 2022-08-01T22:57:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbca67700 1 -- 172.21.15.32:0/124067877 learned_addr learned my addr 172.21.15.32:0/124067877 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0daffff700 1 --2- 172.21.15.32:0/124067877 >> v2:172.21.15.32:3300/0 conn(0x7f0db80aa0d0 0x7f0db80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0db7fff700 1 --2- 172.21.15.32:0/124067877 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80b3630 0x7f0db8143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0db77fe700 1 -- 172.21.15.32:0/124067877 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837409775 0 0) 0x7f0db80b3a10 con 0x7f0db80af6f0 2022-08-01T22:57:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0db77fe700 1 -- 172.21.15.32:0/124067877 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d98003260 con 0x7f0db80af6f0 2022-08-01T22:57:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0daffff700 1 -- 172.21.15.32:0/124067877 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80b3630 msgr2=0x7f0db8143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0daffff700 1 --2- 172.21.15.32:0/124067877 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80b3630 0x7f0db8143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0daffff700 1 -- 172.21.15.32:0/124067877 >> v1:172.21.15.32:6789/0 conn(0x7f0db80af6f0 legacy=0x7f0db805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0daffff700 1 -- 172.21.15.32:0/124067877 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8146b40 con 0x7f0db80aa0d0 2022-08-01T22:57:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0daffff700 1 --2- 172.21.15.32:0/124067877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db80af200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f0da0002f70 tx=0x7f0da0006df0).ready entity=mon.0 client_cookie=52eb09f8be0ca8a1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0db77fe700 1 -- 172.21.15.32:0/124067877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da000d4a0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0db77fe700 1 -- 172.21.15.32:0/124067877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da0000e90 con 0x7f0db80aa0d0 2022-08-01T22:57:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0db77fe700 1 -- 172.21.15.32:0/124067877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da000d7f0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/124067877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 msgr2=0x7f0db80af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/124067877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db80af200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f0da0002f70 tx=0x7f0da0006df0).stop 2022-08-01T22:57:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/124067877 shutdown_connections 2022-08-01T22:57:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/124067877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db80af200 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/124067877 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80b3630 0x7f0db8143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.467+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/124067877 >> 172.21.15.32:0/124067877 conn(0x7f0db8060950 msgr2=0x7f0db8060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.471+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/124067877 shutdown_connections 2022-08-01T22:57:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.471+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/124067877 wait complete. 2022-08-01T22:57:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.471+0000 7f0dbe4ca700 1 Processor -- start 2022-08-01T22:57:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.471+0000 7f0dbe4ca700 1 -- start start 2022-08-01T22:57:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db8152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80af6f0 0x7f0db80aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0db80b3630 0x7f0db8144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80acf20 con 0x7f0db80aa0d0 2022-08-01T22:57:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80ac540 con 0x7f0db80b3630 2022-08-01T22:57:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80ac7e0 con 0x7f0db80af6f0 2022-08-01T22:57:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db8152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db8152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42678/0 (socket says 172.21.15.32:42678) 2022-08-01T22:57:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 -- 172.21.15.32:0/2902903044 learned_addr learned my addr 172.21.15.32:0/2902903044 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbca67700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0db80b3630 0x7f0db8144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0daffff700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80af6f0 0x7f0db80aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 -- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80af6f0 msgr2=0x7f0db80aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80af6f0 0x7f0db80aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 -- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0db80b3630 msgr2=0x7f0db8144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0db80b3630 0x7f0db8144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 -- 172.21.15.32:0/2902903044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da000b180 con 0x7f0db80aa0d0 2022-08-01T22:57:48.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db7fff700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db8152fa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f0da800c000 tx=0x7f0da8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da8010780 con 0x7f0db80aa0d0 2022-08-01T22:57:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80108e0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da8010c40 con 0x7f0db80aa0d0 2022-08-01T22:57:48.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db80aca80 con 0x7f0db80aa0d0 2022-08-01T22:57:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db80ad4f0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.475+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db8146de0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.479+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0da801c460 con 0x7f0db80aa0d0 2022-08-01T22:57:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.479+0000 7f0db5ffb700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d9c053a70 0x7f0d9c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:48.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.479+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0da806c560 con 0x7f0db80aa0d0 2022-08-01T22:57:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.479+0000 7f0daffff700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d9c053a70 0x7f0d9c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.479+0000 7f0daffff700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d9c053a70 0x7f0d9c055f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f0da001a000 tx=0x7f0da0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:48.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.487+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0da80431b0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.647+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0db80574e0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.647+0000 7f0db5ffb700 1 -- 172.21.15.32:0/2902903044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f0da80467d0 con 0x7f0db80aa0d0 2022-08-01T22:57:48.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:48.654 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:57:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d9c053a70 msgr2=0x7f0d9c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d9c053a70 0x7f0d9c055f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f0da001a000 tx=0x7f0da0014000).stop 2022-08-01T22:57:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 msgr2=0x7f0db8152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db8152fa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f0da800c000 tx=0x7f0da8006df0).stop 2022-08-01T22:57:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 shutdown_connections 2022-08-01T22:57:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0d9c053a70 0x7f0d9c055f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0db80b3630 0x7f0db8144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0db80af6f0 0x7f0db80aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 --2- 172.21.15.32:0/2902903044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db80aa0d0 0x7f0db8152fa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 >> 172.21.15.32:0/2902903044 conn(0x7f0db8060950 msgr2=0x7f0db80b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 shutdown_connections 2022-08-01T22:57:48.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:48.651+0000 7f0dbe4ca700 1 -- 172.21.15.32:0/2902903044 wait complete. 2022-08-01T22:57:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:57:48.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:57:45.324157+0000', 'modified': '2022-08-01T22:57:47.120762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5241': {'gid': 5241, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/505560818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 505560818}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 505560818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:57:48.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:57:48.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T22:57:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5241,"want_state":"up:active","state":"up:active","rank_uptime":3.7542924520000001,"mdsmap_epoch":18,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":8.0730854159999996} 2022-08-01T22:57:49.091 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5241, "mdsmap_epoch": 18, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 3.754292452, "state": "up:active", "uptime": 8.073085416, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:57:49.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:57:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2abfff700 1 Processor -- start 2022-08-01T22:57:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2abfff700 1 -- start start 2022-08-01T22:57:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2abfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2ac066dc0 0x7fa2ac14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2abfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac0654c0 con 0x7fa2ac0b3190 2022-08-01T22:57:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2abfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac0b3570 con 0x7fa2ac0b2730 2022-08-01T22:57:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2abfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac062190 con 0x7fa2ac066dc0 2022-08-01T22:57:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2ab7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2ac066dc0 0x7fa2ac14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.187+0000 7fa2aaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2ac0b3190 0x7fa2ac060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58462/0 (socket says 172.21.15.32:58462) 2022-08-01T22:57:49.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2aaffd700 1 -- 172.21.15.32:0/2122962754 learned_addr learned my addr 172.21.15.32:0/2122962754 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:49.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533498096 0 0) 0x7fa2ac0654c0 con 0x7fa2ac0b3190 2022-08-01T22:57:49.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa28c003260 con 0x7fa2ac0b3190 2022-08-01T22:57:49.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409070955 0 0) 0x7fa2ac0b3570 con 0x7fa2ac0b2730 2022-08-01T22:57:49.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac0654c0 con 0x7fa2ac0b2730 2022-08-01T22:57:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2ab7fe700 1 -- 172.21.15.32:0/2122962754 >> v1:172.21.15.55:6789/0 conn(0x7fa2ac0b2730 legacy=0x7fa2ac0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2ab7fe700 1 -- 172.21.15.32:0/2122962754 >> v1:172.21.15.32:6789/0 conn(0x7fa2ac0b3190 legacy=0x7fa2ac060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2ab7fe700 1 -- 172.21.15.32:0/2122962754 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac05fbf0 con 0x7fa2ac066dc0 2022-08-01T22:57:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2ab7fe700 1 --2- 172.21.15.32:0/2122962754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac066dc0 0x7fa2ac14eac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fa2a40032f0 tx=0x7fa2a40091d0).ready entity=mon.0 client_cookie=3b45b6c1ceacda2d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a400b600 con 0x7fa2ac066dc0 2022-08-01T22:57:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a400b760 con 0x7fa2ac066dc0 2022-08-01T22:57:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2a9ffb700 1 -- 172.21.15.32:0/2122962754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a400b8d0 con 0x7fa2ac066dc0 2022-08-01T22:57:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 -- 172.21.15.32:0/2122962754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac066dc0 msgr2=0x7fa2ac14eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 --2- 172.21.15.32:0/2122962754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac066dc0 0x7fa2ac14eac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fa2a40032f0 tx=0x7fa2a40091d0).stop 2022-08-01T22:57:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 -- 172.21.15.32:0/2122962754 shutdown_connections 2022-08-01T22:57:49.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 --2- 172.21.15.32:0/2122962754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac066dc0 0x7fa2ac14eac0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 -- 172.21.15.32:0/2122962754 >> 172.21.15.32:0/2122962754 conn(0x7fa2ac063870 msgr2=0x7fa2ac063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:49.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 -- 172.21.15.32:0/2122962754 shutdown_connections 2022-08-01T22:57:49.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.191+0000 7fa2abfff700 1 -- 172.21.15.32:0/2122962754 wait complete. 2022-08-01T22:57:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 Processor -- start 2022-08-01T22:57:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- start start 2022-08-01T22:57:49.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ac066dc0 0x7fa2ac0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 0x7fa2ac0abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ac0b3190 0x7fa2ac0ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac155bd0 con 0x7fa2ac0b2730 2022-08-01T22:57:49.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0ae040 con 0x7fa2ac0b3190 2022-08-01T22:57:49.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0ae2e0 con 0x7fa2ac066dc0 2022-08-01T22:57:49.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 0x7fa2ac0abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 0x7fa2ac0abfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42694/0 (socket says 172.21.15.32:42694) 2022-08-01T22:57:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 -- 172.21.15.32:0/3076413192 learned_addr learned my addr 172.21.15.32:0/3076413192 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aaffd700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ac066dc0 0x7fa2ac0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2ab7fe700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ac0b3190 0x7fa2ac0ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 -- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ac066dc0 msgr2=0x7fa2ac0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ac066dc0 0x7fa2ac0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 -- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ac0b3190 msgr2=0x7fa2ac0ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ac0b3190 0x7fa2ac0ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 -- 172.21.15.32:0/3076413192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4009470 con 0x7fa2ac0b2730 2022-08-01T22:57:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2aa7fc700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 0x7fa2ac0abfd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fa294001110 tx=0x7fa294007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa29400ce30 con 0x7fa2ac0b2730 2022-08-01T22:57:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400f470 con 0x7fa2ac0b2730 2022-08-01T22:57:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac0ae580 con 0x7fa2ac0b2730 2022-08-01T22:57:49.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa29400f7d0 con 0x7fa2ac0b2730 2022-08-01T22:57:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac146560 con 0x7fa2ac0b2730 2022-08-01T22:57:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.195+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac155d40 con 0x7fa2ac0b2730 2022-08-01T22:57:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.199+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa294019030 con 0x7fa2ac0b2730 2022-08-01T22:57:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.199+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa290053a70 0x7fa290055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.199+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa29406b670 con 0x7fa2ac0b2730 2022-08-01T22:57:49.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.199+0000 7fa2aaffd700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa290053a70 0x7fa290055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.199+0000 7fa2aaffd700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa290053a70 0x7fa290055f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fa29c00b000 tx=0x7fa29c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.203+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2ac155d40 con 0x7fa2ac0b2730 2022-08-01T22:57:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.331+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2ac155d10 con 0x7fa2ac0b2730 2022-08-01T22:57:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.331+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3076413192 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fa294045e60 con 0x7fa2ac0b2730 2022-08-01T22:57:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:49.338 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:57:46.326950+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:57:43.852752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T22:57:45.012782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:57:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa290053a70 msgr2=0x7fa290055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa290053a70 0x7fa290055f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fa29c00b000 tx=0x7fa29c009040).stop 2022-08-01T22:57:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 msgr2=0x7fa2ac0abfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 0x7fa2ac0abfd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fa294001110 tx=0x7fa294007ab0).stop 2022-08-01T22:57:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 shutdown_connections 2022-08-01T22:57:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa290053a70 0x7fa290055f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ac0b3190 0x7fa2ac0ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0b2730 0x7fa2ac0abfd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 --2- 172.21.15.32:0/3076413192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ac066dc0 0x7fa2ac0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 >> 172.21.15.32:0/3076413192 conn(0x7fa2ac063870 msgr2=0x7fa2ac147870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 shutdown_connections 2022-08-01T22:57:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.335+0000 7fa2abfff700 1 -- 172.21.15.32:0/3076413192 wait complete. 2022-08-01T22:57:49.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:57:43.852752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:57:45.012782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:57:49.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:57:49.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.455+0000 7f56c0ab6700 1 Processor -- start 2022-08-01T22:57:49.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.455+0000 7f56c0ab6700 1 -- start start 2022-08-01T22:57:49.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.455+0000 7f56c0ab6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56bc146ae0 0x7f56bc1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc143cf0 con 0x7f56bc0b4820 2022-08-01T22:57:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc143f90 con 0x7f56bc0b4310 2022-08-01T22:57:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc1476a0 con 0x7f56bc146ae0 2022-08-01T22:57:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56bad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56bc0b4820 0x7f56bc0b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58474/0 (socket says 172.21.15.32:58474) 2022-08-01T22:57:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56bad9d700 1 -- 172.21.15.32:0/4185935971 learned_addr learned my addr 172.21.15.32:0/4185935971 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56ba59c700 1 --2- 172.21.15.32:0/4185935971 >> v2:172.21.15.55:3300/0 conn(0x7f56bc146ae0 0x7f56bc1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703437835 0 0) 0x7f56bc143cf0 con 0x7f56bc0b4820 2022-08-01T22:57:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569c003260 con 0x7f56bc0b4820 2022-08-01T22:57:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137262137 0 0) 0x7f56bc143f90 con 0x7f56bc0b4310 2022-08-01T22:57:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc143cf0 con 0x7f56bc0b4310 2022-08-01T22:57:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56ba59c700 1 -- 172.21.15.32:0/4185935971 >> v1:172.21.15.55:6789/0 conn(0x7f56bc0b4310 legacy=0x7f56bc1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56ba59c700 1 -- 172.21.15.32:0/4185935971 >> v1:172.21.15.32:6789/0 conn(0x7f56bc0b4820 legacy=0x7f56bc0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56ba59c700 1 -- 172.21.15.32:0/4185935971 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0b0e20 con 0x7f56bc146ae0 2022-08-01T22:57:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56ba59c700 1 --2- 172.21.15.32:0/4185935971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc146ae0 0x7f56bc1403c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f56ac002f70 tx=0x7f56ac006df0).ready entity=mon.1 client_cookie=e1f8d1cba54afe0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56ac00d540 con 0x7f56bc146ae0 2022-08-01T22:57:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac000e90 con 0x7f56bc146ae0 2022-08-01T22:57:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56b9d9b700 1 -- 172.21.15.32:0/4185935971 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56ac00d890 con 0x7f56bc146ae0 2022-08-01T22:57:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- 172.21.15.32:0/4185935971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc146ae0 msgr2=0x7f56bc1403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/4185935971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc146ae0 0x7f56bc1403c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f56ac002f70 tx=0x7f56ac006df0).stop 2022-08-01T22:57:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- 172.21.15.32:0/4185935971 shutdown_connections 2022-08-01T22:57:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/4185935971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc146ae0 0x7f56bc1403c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- 172.21.15.32:0/4185935971 >> 172.21.15.32:0/4185935971 conn(0x7f56bc05c620 msgr2=0x7f56bc05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- 172.21.15.32:0/4185935971 shutdown_connections 2022-08-01T22:57:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.459+0000 7f56c0ab6700 1 -- 172.21.15.32:0/4185935971 wait complete. 2022-08-01T22:57:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 Processor -- start 2022-08-01T22:57:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 -- start start 2022-08-01T22:57:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56bc0b4310 0x7f56bc0b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc0b4820 0x7f56bc152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 0x7f56bc156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc155d90 con 0x7f56bc146ae0 2022-08-01T22:57:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc144990 con 0x7f56bc0b4820 2022-08-01T22:57:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc155ff0 con 0x7f56bc0b4310 2022-08-01T22:57:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 0x7f56bc156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 0x7f56bc156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42710/0 (socket says 172.21.15.32:42710) 2022-08-01T22:57:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 -- 172.21.15.32:0/2411252933 learned_addr learned my addr 172.21.15.32:0/2411252933 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56ba59c700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56bc0b4310 0x7f56bc0b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56b3fff700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc0b4820 0x7f56bc152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 -- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56bc0b4310 msgr2=0x7f56bc0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56bc0b4310 0x7f56bc0b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 -- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc0b4820 msgr2=0x7f56bc152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc0b4820 0x7f56bc152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 -- 172.21.15.32:0/2411252933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac00b180 con 0x7f56bc146ae0 2022-08-01T22:57:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56bad9d700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 0x7f56bc156490 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f56b4001490 tx=0x7f56b4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56b4005c70 con 0x7f56bc146ae0 2022-08-01T22:57:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b4005dd0 con 0x7f56bc146ae0 2022-08-01T22:57:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc144c30 con 0x7f56bc146ae0 2022-08-01T22:57:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56b40072c0 con 0x7f56bc146ae0 2022-08-01T22:57:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.463+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc1444f0 con 0x7f56bc146ae0 2022-08-01T22:57:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.467+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc1561b0 con 0x7f56bc146ae0 2022-08-01T22:57:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.467+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f56b401a060 con 0x7f56bc146ae0 2022-08-01T22:57:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.467+0000 7f56b37fe700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f56a00539e0 0x7f56a0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.467+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f56b406b780 con 0x7f56bc146ae0 2022-08-01T22:57:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.467+0000 7f56ba59c700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f56a00539e0 0x7f56a0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.467+0000 7f56ba59c700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f56a00539e0 0x7f56a0055ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f56ac018000 tx=0x7f56ac014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.471+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f56b4042540 con 0x7f56bc146ae0 2022-08-01T22:57:49.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.595+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f56bc0574e0 con 0x7f56bc146ae0 2022-08-01T22:57:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56b37fe700 1 -- 172.21.15.32:0/2411252933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7f56b4018020 con 0x7f56bc146ae0 2022-08-01T22:57:49.603 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:57:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f56a00539e0 msgr2=0x7f56a0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f56a00539e0 0x7f56a0055ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f56ac018000 tx=0x7f56ac014000).stop 2022-08-01T22:57:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 msgr2=0x7f56bc156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 0x7f56bc156490 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f56b4001490 tx=0x7f56b4007e00).stop 2022-08-01T22:57:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 shutdown_connections 2022-08-01T22:57:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f56a00539e0 0x7f56a0055ea0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56bc146ae0 0x7f56bc156490 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56bc0b4820 0x7f56bc152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 --2- 172.21.15.32:0/2411252933 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f56bc0b4310 0x7f56bc0b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 >> 172.21.15.32:0/2411252933 conn(0x7f56bc05c620 msgr2=0x7f56bc13b190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 shutdown_connections 2022-08-01T22:57:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.599+0000 7f56c0ab6700 1 -- 172.21.15.32:0/2411252933 wait complete. 2022-08-01T22:57:49.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:57:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.727+0000 7f724de03700 1 Processor -- start 2022-08-01T22:57:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.727+0000 7f724de03700 1 -- start start 2022-08-01T22:57:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.727+0000 7f724de03700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72480b1460 0x7f72480b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.727+0000 7f724de03700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b2430 0x7f72481413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480a7960 con 0x7f72480b2050 2022-08-01T22:57:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72480b18d0 con 0x7f72480b1460 2022-08-01T22:57:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480b07e0 con 0x7f72480b2430 2022-08-01T22:57:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7246ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72480b1460 0x7f72480b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f72477fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f72480b2050 0x7f72480a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:51162/0 (socket says 172.21.15.32:51162) 2022-08-01T22:57:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f72477fe700 1 -- 172.21.15.32:0/768872881 learned_addr learned my addr 172.21.15.32:0/768872881 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7247fff700 1 --2- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b2430 0x7f72481413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7246ffd700 1 -- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b2430 msgr2=0x7f72481413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7246ffd700 1 --2- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b2430 0x7f72481413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7246ffd700 1 -- 172.21.15.32:0/768872881 >> v1:172.21.15.55:6789/0 conn(0x7f72480b2050 legacy=0x7f72480a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7246ffd700 1 -- 172.21.15.32:0/768872881 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480b0a80 con 0x7f72480b1460 2022-08-01T22:57:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f7246ffd700 1 --2- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b1460 0x7f72480b0140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f723c002f70 tx=0x7f723c006df0).ready entity=mon.1 client_cookie=df2e281521de0c22 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f72467fc700 1 -- 172.21.15.32:0/768872881 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f723c00d570 con 0x7f72480b1460 2022-08-01T22:57:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f72467fc700 1 -- 172.21.15.32:0/768872881 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c000e90 con 0x7f72480b1460 2022-08-01T22:57:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f72467fc700 1 -- 172.21.15.32:0/768872881 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f723c00d8c0 con 0x7f72480b1460 2022-08-01T22:57:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b1460 msgr2=0x7f72480b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 --2- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b1460 0x7f72480b0140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f723c002f70 tx=0x7f723c006df0).stop 2022-08-01T22:57:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- 172.21.15.32:0/768872881 shutdown_connections 2022-08-01T22:57:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 --2- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b2430 0x7f72481413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 --2- 172.21.15.32:0/768872881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b1460 0x7f72480b0140 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- 172.21.15.32:0/768872881 >> 172.21.15.32:0/768872881 conn(0x7f724805f840 msgr2=0x7f724805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- 172.21.15.32:0/768872881 shutdown_connections 2022-08-01T22:57:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.731+0000 7f724de03700 1 -- 172.21.15.32:0/768872881 wait complete. 2022-08-01T22:57:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 Processor -- start 2022-08-01T22:57:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 -- start start 2022-08-01T22:57:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b1460 0x7f724813f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b2050 0x7f724815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 0x7f724815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724813b790 con 0x7f72480b2430 2022-08-01T22:57:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480b33a0 con 0x7f72480b2050 2022-08-01T22:57:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f724de03700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480b55c0 con 0x7f72480b1460 2022-08-01T22:57:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 0x7f724815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 0x7f724815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42724/0 (socket says 172.21.15.32:42724) 2022-08-01T22:57:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 -- 172.21.15.32:0/423058219 learned_addr learned my addr 172.21.15.32:0/423058219 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f72477fe700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b1460 0x7f724813f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7246ffd700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b2050 0x7f724815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 -- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b1460 msgr2=0x7f724813f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b1460 0x7f724813f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 -- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b2050 msgr2=0x7f724815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b2050 0x7f724815a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 -- 172.21.15.32:0/423058219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c00b180 con 0x7f72480b2430 2022-08-01T22:57:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f72477fe700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b1460 0x7f724813f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:57:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.735+0000 7f7247fff700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 0x7f724815adb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f724814b3e0 tx=0x7f7238006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7238011750 con 0x7f72480b2430 2022-08-01T22:57:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72380118b0 con 0x7f72480b2430 2022-08-01T22:57:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7238011c10 con 0x7f72480b2430 2022-08-01T22:57:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72480b5820 con 0x7f72480b2430 2022-08-01T22:57:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72480b3870 con 0x7f72480b2430 2022-08-01T22:57:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72480574e0 con 0x7f72480b2430 2022-08-01T22:57:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7238011d70 con 0x7f72480b2430 2022-08-01T22:57:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f7244ff9700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72280539e0 0x7f7228055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f723806c330 con 0x7f72480b2430 2022-08-01T22:57:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.739+0000 7f72477fe700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72280539e0 0x7f7228055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.743+0000 7f72477fe700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72280539e0 0x7f7228055ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f724000d000 tx=0x7f7240007f90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.751+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72380430f0 con 0x7f72480b2430 2022-08-01T22:57:49.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.879+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72480ac8f0 con 0x7f72480b2430 2022-08-01T22:57:49.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.879+0000 7f7244ff9700 1 -- 172.21.15.32:0/423058219 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7238046710 con 0x7f72480b2430 2022-08-01T22:57:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:57:49.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72280539e0 msgr2=0x7f7228055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72280539e0 0x7f7228055ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f724000d000 tx=0x7f7240007f90).stop 2022-08-01T22:57:49.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 msgr2=0x7f724815adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:49.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 0x7f724815adb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f724814b3e0 tx=0x7f7238006df0).stop 2022-08-01T22:57:49.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 shutdown_connections 2022-08-01T22:57:49.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72280539e0 0x7f7228055ea0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480b2430 0x7f724815adb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72480b2050 0x7f724815a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 --2- 172.21.15.32:0/423058219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72480b1460 0x7f724813f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:49.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 >> 172.21.15.32:0/423058219 conn(0x7f724805f840 msgr2=0x7f72480ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:49.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 shutdown_connections 2022-08-01T22:57:49.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:49.883+0000 7f724de03700 1 -- 172.21.15.32:0/423058219 wait complete. 2022-08-01T22:57:49.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:57:50.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c07514700 1 Processor -- start 2022-08-01T22:57:50.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c07514700 1 -- start start 2022-08-01T22:57:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c07514700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 0x7f4c000b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c07514700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c00143ab0 con 0x7f4c000b38a0 2022-08-01T22:57:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c07514700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c00143d50 con 0x7f4c000b0740 2022-08-01T22:57:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c07514700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c00143ff0 con 0x7f4c000a9da0 2022-08-01T22:57:50.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c04aaf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4c000b38a0 0x7f4c000b1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58504/0 (socket says 172.21.15.32:58504) 2022-08-01T22:57:50.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.007+0000 7f4c04aaf700 1 -- 172.21.15.32:0/3034188738 learned_addr learned my addr 172.21.15.32:0/3034188738 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:50.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c05ab1700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 0x7f4c000b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575255489 0 0) 0x7f4c00143ab0 con 0x7f4c000b38a0 2022-08-01T22:57:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be8003260 con 0x7f4c000b38a0 2022-08-01T22:57:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555192961 0 0) 0x7f4be8003260 con 0x7f4c000b38a0 2022-08-01T22:57:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 msgr2=0x7f4c000b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 0x7f4c000b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 >> v1:172.21.15.55:6789/0 conn(0x7f4c000b0740 legacy=0x7f4c0013d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c00144290 con 0x7f4c000b38a0 2022-08-01T22:57:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f4bf0001660 con 0x7f4c000b38a0 2022-08-01T22:57:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 >> v1:172.21.15.32:6789/0 conn(0x7f4c000b38a0 legacy=0x7f4c000b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4be8003780 0x7f4be8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4be8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 0x7f4be8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c00143ff0 con 0x7f4be8004500 2022-08-01T22:57:50.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be800b780 con 0x7f4be8003780 2022-08-01T22:57:50.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be800b940 con 0x7f4c000a9da0 2022-08-01T22:57:50.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4be8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c052b0700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4be8003780 0x7f4be8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c05ab1700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 0x7f4be8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 -- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 msgr2=0x7f4be8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c000a9da0 0x7f4be8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 -- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4be8003780 msgr2=0x7f4be8003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4be8003780 0x7f4be8003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 -- 172.21.15.32:0/3034188738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8003260 con 0x7f4be8004500 2022-08-01T22:57:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4be8004980 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f4bf0001370 tx=0x7f4c000b0660).ready entity=mon.0 client_cookie=2bea5e85a01f187 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf00070d0 con 0x7f4be8004500 2022-08-01T22:57:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0007230 con 0x7f4be8004500 2022-08-01T22:57:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4beffff700 1 -- 172.21.15.32:0/3034188738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf000c8f0 con 0x7f4be8004500 2022-08-01T22:57:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 -- 172.21.15.32:0/3034188738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 msgr2=0x7f4be8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 --2- 172.21.15.32:0/3034188738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4be8004980 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f4bf0001370 tx=0x7f4c000b0660).stop 2022-08-01T22:57:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c05ab1700 1 -- 172.21.15.32:0/3034188738 reap_dead start 2022-08-01T22:57:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 -- 172.21.15.32:0/3034188738 shutdown_connections 2022-08-01T22:57:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 -- 172.21.15.32:0/3034188738 >> 172.21.15.32:0/3034188738 conn(0x7f4c00060950 msgr2=0x7f4c00060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 -- 172.21.15.32:0/3034188738 shutdown_connections 2022-08-01T22:57:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 -- 172.21.15.32:0/3034188738 wait complete. 2022-08-01T22:57:50.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 Processor -- start 2022-08-01T22:57:50.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.011+0000 7f4c07514700 1 -- start start 2022-08-01T22:57:50.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4c00144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c0013e0d0 0x7f4c00145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c00146230 0x7f4c00155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0013ef10 con 0x7f4be8004500 2022-08-01T22:57:50.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0013ec70 con 0x7f4c00146230 2022-08-01T22:57:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000adef0 con 0x7f4c0013e0d0 2022-08-01T22:57:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4c00144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4c00144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42740/0 (socket says 172.21.15.32:42740) 2022-08-01T22:57:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 -- 172.21.15.32:0/1477026379 learned_addr learned my addr 172.21.15.32:0/1477026379 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c0013e0d0 0x7f4c00145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c05ab1700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c00146230 0x7f4c00155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 -- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c0013e0d0 msgr2=0x7f4c00145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c0013e0d0 0x7f4c00145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 -- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c00146230 msgr2=0x7f4c00155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c00146230 0x7f4c00155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 -- 172.21.15.32:0/1477026379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000ae190 con 0x7f4be8004500 2022-08-01T22:57:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c0013e0d0 0x7f4c00145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c052b0700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4c00144e60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4bfc00d000 tx=0x7f4bfc007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bfc011590 con 0x7f4be8004500 2022-08-01T22:57:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc0116f0 con 0x7f4be8004500 2022-08-01T22:57:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0013e8f0 con 0x7f4be8004500 2022-08-01T22:57:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bfc011a50 con 0x7f4be8004500 2022-08-01T22:57:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c001466b0 con 0x7f4be8004500 2022-08-01T22:57:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.015+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0013ddf0 con 0x7f4be8004500 2022-08-01T22:57:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.019+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4bfc01a030 con 0x7f4be8004500 2022-08-01T22:57:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.019+0000 7f4bee7fc700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4be0053b00 0x7f4be0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.019+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4bfc06baf0 con 0x7f4be8004500 2022-08-01T22:57:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.019+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4be0053b00 0x7f4be0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.019+0000 7f4c04aaf700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4be0053b00 0x7f4be0055fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f4bf0019000 tx=0x7f4bf0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.023+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4bfc042960 con 0x7f4be8004500 2022-08-01T22:57:50.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c000574e0 con 0x7f4be8004500 2022-08-01T22:57:50.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4bee7fc700 1 -- 172.21.15.32:0/1477026379 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4bfc045f80 con 0x7f4be8004500 2022-08-01T22:57:50.152 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:57:50.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4be0053b00 msgr2=0x7f4be0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4be0053b00 0x7f4be0055fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f4bf0019000 tx=0x7f4bf0015000).stop 2022-08-01T22:57:50.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 msgr2=0x7f4c00144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4c00144e60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4bfc00d000 tx=0x7f4bfc007a00).stop 2022-08-01T22:57:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 shutdown_connections 2022-08-01T22:57:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4be0053b00 0x7f4be0055fc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c00146230 0x7f4c00155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c0013e0d0 0x7f4c00145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 --2- 172.21.15.32:0/1477026379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be8004500 0x7f4c00144e60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.147+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 >> 172.21.15.32:0/1477026379 conn(0x7f4c00060950 msgr2=0x7f4c000b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:50.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.151+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 shutdown_connections 2022-08-01T22:57:50.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.151+0000 7f4c07514700 1 -- 172.21.15.32:0/1477026379 wait complete. 2022-08-01T22:57:50.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:57:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 Processor -- start 2022-08-01T22:57:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 -- start start 2022-08-01T22:57:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b3980 0x7f3c7c147f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c7c144830 0x7f3c7c144c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c0b5fd0 con 0x7f3c7c0b4310 2022-08-01T22:57:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7c0b3d60 con 0x7f3c7c144830 2022-08-01T22:57:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c8394b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0b4eb0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c7c144830 0x7f3c7c144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c816e7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b3980 0x7f3c7c147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c81ee8700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3c7c0b4310 0x7f3c7c0b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:51198/0 (socket says 172.21.15.32:51198) 2022-08-01T22:57:50.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c7c144830 0x7f3c7c144c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52534/0 (socket says 172.21.15.32:52534) 2022-08-01T22:57:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c81ee8700 1 -- 172.21.15.32:0/3455798924 learned_addr learned my addr 172.21.15.32:0/3455798924 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 -- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b3980 msgr2=0x7f3c7c147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 --2- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b3980 0x7f3c7c147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 -- 172.21.15.32:0/3455798924 >> v1:172.21.15.55:6789/0 conn(0x7f3c7c0b4310 legacy=0x7f3c7c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 -- 172.21.15.32:0/3455798924 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0b6200 con 0x7f3c7c144830 2022-08-01T22:57:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c80ee6700 1 --2- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c144c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f3c6c002f70 tx=0x7f3c6c006df0).ready entity=mon.1 client_cookie=28279e666182e99e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c73fff700 1 -- 172.21.15.32:0/3455798924 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c6c00d570 con 0x7f3c7c144830 2022-08-01T22:57:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c73fff700 1 -- 172.21.15.32:0/3455798924 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c000e90 con 0x7f3c7c144830 2022-08-01T22:57:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.271+0000 7f3c73fff700 1 -- 172.21.15.32:0/3455798924 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c6c00d8c0 con 0x7f3c7c144830 2022-08-01T22:57:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 msgr2=0x7f3c7c144c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 --2- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c144c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f3c6c002f70 tx=0x7f3c6c006df0).stop 2022-08-01T22:57:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- 172.21.15.32:0/3455798924 shutdown_connections 2022-08-01T22:57:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 --2- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c144c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 --2- 172.21.15.32:0/3455798924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b3980 0x7f3c7c147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- 172.21.15.32:0/3455798924 >> 172.21.15.32:0/3455798924 conn(0x7f3c7c05c620 msgr2=0x7f3c7c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- 172.21.15.32:0/3455798924 shutdown_connections 2022-08-01T22:57:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- 172.21.15.32:0/3455798924 wait complete. 2022-08-01T22:57:50.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 Processor -- start 2022-08-01T22:57:50.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- start start 2022-08-01T22:57:50.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 0x7f3c7c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b4310 0x7f3c7c0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c0b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0b8bc0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0af180 con 0x7f3c7c144830 2022-08-01T22:57:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c8394b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0af420 con 0x7f3c7c0b4310 2022-08-01T22:57:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 0x7f3c7c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 0x7f3c7c156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42752/0 (socket says 172.21.15.32:42752) 2022-08-01T22:57:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 -- 172.21.15.32:0/1882564078 learned_addr learned my addr 172.21.15.32:0/1882564078 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c80ee6700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b4310 0x7f3c7c0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c81ee8700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c0b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 -- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b4310 msgr2=0x7f3c7c0b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b4310 0x7f3c7c0b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 -- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 msgr2=0x7f3c7c0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c0b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.275+0000 7f3c816e7700 1 -- 172.21.15.32:0/1882564078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c00b180 con 0x7f3c7c0b3980 2022-08-01T22:57:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c816e7700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 0x7f3c7c156850 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f3c7800c000 tx=0x7f3c78006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c78010780 con 0x7f3c7c0b3980 2022-08-01T22:57:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c780108e0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c0af6c0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c78010c40 con 0x7f3c7c0b3980 2022-08-01T22:57:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c062870 con 0x7f3c7c0b3980 2022-08-01T22:57:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c0574e0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3c7801c460 con 0x7f3c7c0b3980 2022-08-01T22:57:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c727fc700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c64053a70 0x7f3c64055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.279+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c7806c3a0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.283+0000 7f3c80ee6700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c64053a70 0x7f3c64055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.283+0000 7f3c80ee6700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c64053a70 0x7f3c64055f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3c6c01a000 tx=0x7f3c6c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.287+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c78043060 con 0x7f3c7c0b3980 2022-08-01T22:57:50.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.411+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c7c0b03b0 con 0x7f3c7c0b3980 2022-08-01T22:57:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c727fc700 1 -- 172.21.15.32:0/1882564078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3c78046680 con 0x7f3c7c0b3980 2022-08-01T22:57:50.420 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:57:50.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c64053a70 msgr2=0x7f3c64055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c64053a70 0x7f3c64055f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3c6c01a000 tx=0x7f3c6c014000).stop 2022-08-01T22:57:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 msgr2=0x7f3c7c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 0x7f3c7c156850 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f3c7800c000 tx=0x7f3c78006df0).stop 2022-08-01T22:57:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 shutdown_connections 2022-08-01T22:57:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c64053a70 0x7f3c64055f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c7c144830 0x7f3c7c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c7c0b4310 0x7f3c7c0b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 --2- 172.21.15.32:0/1882564078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c0b3980 0x7f3c7c156850 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 >> 172.21.15.32:0/1882564078 conn(0x7f3c7c05c620 msgr2=0x7f3c7c14bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 shutdown_connections 2022-08-01T22:57:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.415+0000 7f3c8394b700 1 -- 172.21.15.32:0/1882564078 wait complete. 2022-08-01T22:57:50.437 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:57:50.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:57:50.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 Processor -- start 2022-08-01T22:57:50.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 -- start start 2022-08-01T22:57:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 0x7f691814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f69180a8c50 0x7f691814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69180a7250 con 0x7f69180b01a0 2022-08-01T22:57:50.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69180a7480 con 0x7f69180a8c50 2022-08-01T22:57:50.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a7720 con 0x7f69180b2710 2022-08-01T22:57:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f6915d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f69180b01a0 0x7f69180a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58534/0 (socket says 172.21.15.32:58534) 2022-08-01T22:57:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f6915d9b700 1 -- 172.21.15.32:0/4004766873 learned_addr learned my addr 172.21.15.32:0/4004766873 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:57:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691659c700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 0x7f691814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f6916d9d700 1 --2- 172.21.15.32:0/4004766873 >> v2:172.21.15.55:3300/0 conn(0x7f69180a8c50 0x7f691814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416064805 0 0) 0x7f69180a7250 con 0x7f69180b01a0 2022-08-01T22:57:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68fc003260 con 0x7f69180b01a0 2022-08-01T22:57:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559317239 0 0) 0x7f68fc003260 con 0x7f69180b01a0 2022-08-01T22:57:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 msgr2=0x7f691814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 0x7f691814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 >> v2:172.21.15.55:3300/0 conn(0x7f69180a8c50 msgr2=0x7f691814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 --2- 172.21.15.32:0/4004766873 >> v2:172.21.15.55:3300/0 conn(0x7f69180a8c50 0x7f691814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69180a79c0 con 0x7f69180b01a0 2022-08-01T22:57:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f6908001660 con 0x7f69180b01a0 2022-08-01T22:57:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 >> v1:172.21.15.32:6789/0 conn(0x7f69180b01a0 legacy=0x7f69180a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 0x7f68fc003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc003c80 0x7f68fc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc0049f0 0x7f68fc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a7480 con 0x7f68fc003c80 2022-08-01T22:57:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a7720 con 0x7f68fc0049f0 2022-08-01T22:57:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc00b750 con 0x7f69180b2710 2022-08-01T22:57:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.535+0000 7f6915d9b700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc003c80 0x7f68fc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691659c700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 0x7f68fc003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6916d9d700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc0049f0 0x7f68fc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6915d9b700 1 -- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 msgr2=0x7f68fc003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6915d9b700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180b2710 0x7f68fc003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6915d9b700 1 -- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc0049f0 msgr2=0x7f68fc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:57:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6915d9b700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc0049f0 0x7f68fc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6915d9b700 1 -- 172.21.15.32:0/4004766873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc003260 con 0x7f68fc003c80 2022-08-01T22:57:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6916d9d700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc0049f0 0x7f68fc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:57:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6915d9b700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc003c80 0x7f68fc004500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f68fc0054d0 tx=0x7f691806a5f0).ready entity=mon.0 client_cookie=649f35260e397e71 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69080071a0 con 0x7f68fc003c80 2022-08-01T22:57:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908007300 con 0x7f68fc003c80 2022-08-01T22:57:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691559a700 1 -- 172.21.15.32:0/4004766873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f690800cb70 con 0x7f68fc003c80 2022-08-01T22:57:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691759e700 1 -- 172.21.15.32:0/4004766873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc003c80 msgr2=0x7f68fc004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691759e700 1 --2- 172.21.15.32:0/4004766873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc003c80 0x7f68fc004500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f68fc0054d0 tx=0x7f691806a5f0).stop 2022-08-01T22:57:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f6916d9d700 1 -- 172.21.15.32:0/4004766873 reap_dead start 2022-08-01T22:57:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691759e700 1 -- 172.21.15.32:0/4004766873 shutdown_connections 2022-08-01T22:57:50.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691759e700 1 -- 172.21.15.32:0/4004766873 >> 172.21.15.32:0/4004766873 conn(0x7f6918060310 msgr2=0x7f69180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:50.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691759e700 1 -- 172.21.15.32:0/4004766873 shutdown_connections 2022-08-01T22:57:50.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.539+0000 7f691759e700 1 -- 172.21.15.32:0/4004766873 wait complete. 2022-08-01T22:57:50.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 Processor -- start 2022-08-01T22:57:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 -- start start 2022-08-01T22:57:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc003c80 0x7f691814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180aac70 0x7f69180ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 0x7f6918159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6918064e70 con 0x7f69180ac800 2022-08-01T22:57:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6918065110 con 0x7f68fc003c80 2022-08-01T22:57:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180653b0 con 0x7f69180aac70 2022-08-01T22:57:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 0x7f6918159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 0x7f6918159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42776/0 (socket says 172.21.15.32:42776) 2022-08-01T22:57:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 -- 172.21.15.32:0/2313918295 learned_addr learned my addr 172.21.15.32:0/2313918295 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:57:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f691659c700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc003c80 0x7f691814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6915d9b700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180aac70 0x7f69180ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 -- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180aac70 msgr2=0x7f69180ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180aac70 0x7f69180ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 -- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc003c80 msgr2=0x7f691814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc003c80 0x7f691814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 -- 172.21.15.32:0/2313918295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691814d0c0 con 0x7f69180ac800 2022-08-01T22:57:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.543+0000 7f6916d9d700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 0x7f6918159c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f691000b000 tx=0x7f6910008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6910011e60 con 0x7f69180ac800 2022-08-01T22:57:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69100145f0 con 0x7f69180ac800 2022-08-01T22:57:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6918064af0 con 0x7f69180ac800 2022-08-01T22:57:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6910014950 con 0x7f69180ac800 2022-08-01T22:57:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69180ab270 con 0x7f69180ac800 2022-08-01T22:57:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69180574e0 con 0x7f69180ac800 2022-08-01T22:57:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6910012040 con 0x7f69180ac800 2022-08-01T22:57:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f69077fe700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68f4053b00 0x7f68f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:57:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.547+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f691006c690 con 0x7f69180ac800 2022-08-01T22:57:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.551+0000 7f691659c700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68f4053b00 0x7f68f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:57:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.551+0000 7f691659c700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68f4053b00 0x7f68f4055fc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f690c00b000 tx=0x7f690c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:57:50.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.551+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6910043370 con 0x7f69180ac800 2022-08-01T22:57:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.715+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69180abf90 con 0x7f69180ac800 2022-08-01T22:57:50.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.715+0000 7f69077fe700 1 -- 172.21.15.32:0/2313918295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f6910046990 con 0x7f69180ac800 2022-08-01T22:57:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:57:50.720 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:57:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68f4053b00 msgr2=0x7f68f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68f4053b00 0x7f68f4055fc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f690c00b000 tx=0x7f690c006df0).stop 2022-08-01T22:57:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 msgr2=0x7f6918159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:57:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 0x7f6918159c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f691000b000 tx=0x7f6910008040).stop 2022-08-01T22:57:50.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 shutdown_connections 2022-08-01T22:57:50.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68f4053b00 0x7f68f4055fc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69180ac800 0x7f6918159c90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69180aac70 0x7f69180ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 --2- 172.21.15.32:0/2313918295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68fc003c80 0x7f691814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:57:50.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 >> 172.21.15.32:0/2313918295 conn(0x7f6918060310 msgr2=0x7f691814a300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:57:50.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 shutdown_connections 2022-08-01T22:57:50.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:57:50.719+0000 7f691759e700 1 -- 172.21.15.32:0/2313918295 wait complete. 2022-08-01T22:57:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:57:51.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:57:45.324157+0000', 'modified': '2022-08-01T22:57:47.120762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5241': {'gid': 5241, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/505560818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 505560818}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 505560818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:57:51.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:57:51.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T22:57:51.160 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5241,"want_state":"up:active","state":"up:active","rank_uptime":5.8230685949999996,"mdsmap_epoch":18,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":10.141860175} 2022-08-01T22:57:51.161 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5241, "mdsmap_epoch": 18, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 5.823068595, "state": "up:active", "uptime": 10.141860175, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:57:51.162 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:57:51.162 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T22:57:51.163 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T22:57:51.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.165 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:51.173 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:57:51.192 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:57:51.193 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:57:51.247 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:57:51.248 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:57:51.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.249 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:57:51.256 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:57:51.257 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:51.261 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:57:51.278 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:57:51.279 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:57:51.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T22:57:51.329+0000 7fcf79f38080 -1 init, newargv = 0x5578d45f97b0 newargc=9ceph-fuse[22526]: starting ceph client 2022-08-01T22:57:51.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr: 2022-08-01T22:57:51.342 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:57:51.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22526]: starting fuse 2022-08-01T22:57:51.343 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:57:51.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.344 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:57:51.350 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T22:57:51.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:57:51.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.351 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T22:57:51.352 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T22:57:51.352 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T22:57:51.353 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:57:51.353 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T22:57:51.354 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.354 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T22:57:51.355 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T22:57:51.355 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T22:57:51.356 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.356 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T22:57:51.357 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T22:57:51.357 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T22:57:51.358 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T22:57:51.359 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.360 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T22:57:51.361 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:57:51.361 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:57:51.362 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T22:57:51.363 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:57:51.363 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.364 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T22:57:51.364 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:57:51.403 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22526.asok 2022-08-01T22:57:51.408 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22526.asok 2022-08-01T22:57:51.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.409 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22526.asok status 2022-08-01T22:57:51.539 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T22:57:51.539 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T22:57:51.540 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:57:51.540 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:57:51.540 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T22:57:51.541 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T22:57:51.541 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T22:57:51.541 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22526", 2022-08-01T22:57:51.541 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T22:57:51.542 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:57:51.542 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T22:57:51.542 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T22:57:51.543 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 5347, 2022-08-01T22:57:51.543 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T22:57:51.544 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T22:57:51.544 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T22:57:51.544 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 5347 2022-08-01T22:57:51.545 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:57:51.545 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:57:51.545 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:57:51.545 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:57:51.546 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 666403777 2022-08-01T22:57:51.546 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T22:57:51.546 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:57:51.547 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:57:51.547 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:57:51.547 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:57:51.547 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 666403777 2022-08-01T22:57:51.548 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:57:51.548 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.5347 172.21.15.130:0/666403777", 2022-08-01T22:57:51.548 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/666403777", 2022-08-01T22:57:51.548 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T22:57:51.548 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 18, 2022-08-01T22:57:51.549 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 45, 2022-08-01T22:57:51.549 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:57:51.549 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T22:57:51.549 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T22:57:51.550 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T22:57:51.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.552 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:51.553 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:51.562 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:57:51.564 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:57:51.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.565 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:51.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:57:51.631 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:57:51.632 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:57:51.639 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:57:51.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:57:51.642 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T22:57:51.643 DEBUG:teuthology.orchestra.run.smithi130:> import sys 2022-08-01T22:57:51.644 DEBUG:teuthology.orchestra.run.smithi130:> import time 2022-08-01T22:57:51.644 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:57:51.645 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.646 DEBUG:teuthology.orchestra.run.smithi130:> n = 1000 2022-08-01T22:57:51.646 DEBUG:teuthology.orchestra.run.smithi130:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T22:57:51.647 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.648 DEBUG:teuthology.orchestra.run.smithi130:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T22:57:51.648 DEBUG:teuthology.orchestra.run.smithi130:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T22:57:51.649 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:57:51.650 DEBUG:teuthology.orchestra.run.smithi130:> for i in range(0, n): 2022-08-01T22:57:51.651 DEBUG:teuthology.orchestra.run.smithi130:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T22:57:51.651 DEBUG:teuthology.orchestra.run.smithi130:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T22:57:51.652 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content'"'"') 2022-08-01T22:57:51.653 DEBUG:teuthology.orchestra.run.smithi130:> if True: 2022-08-01T22:57:51.653 DEBUG:teuthology.orchestra.run.smithi130:> f.flush() 2022-08-01T22:57:51.654 DEBUG:teuthology.orchestra.run.smithi130:> os.fsync(f.fileno()) 2022-08-01T22:57:51.654 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:57:53.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:53.460 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:53.463 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:57:53.466 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:02.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:58:02.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 Processor -- start 2022-08-01T22:58:02.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- start start 2022-08-01T22:58:02.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1b40b6710 0x7fd1b40b7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:02.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b5cd0 0x7fd1b41481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:02.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b40b6af0 con 0x7fd1b40b0de0 2022-08-01T22:58:02.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b4147640 con 0x7fd1b40b6710 2022-08-01T22:58:02.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b40b6250 con 0x7fd1b40b5cd0 2022-08-01T22:58:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1b40b6710 0x7fd1b40b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1b40b6710 0x7fd1b40b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42782/0 (socket says 172.21.15.32:42782) 2022-08-01T22:58:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 -- 172.21.15.32:0/1648417623 learned_addr learned my addr 172.21.15.32:0/1648417623 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b37fe700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b5cd0 0x7fd1b41481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 -- 172.21.15.32:0/1648417623 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b5cd0 msgr2=0x7fd1b41481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b5cd0 0x7fd1b41481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 -- 172.21.15.32:0/1648417623 >> v1:172.21.15.55:6789/0 conn(0x7fd1b40b0de0 legacy=0x7fd1b4144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 -- 172.21.15.32:0/1648417623 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b4147830 con 0x7fd1b40b6710 2022-08-01T22:58:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b37fe700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b5cd0 0x7fd1b41481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:58:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b3fff700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b40b7010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fd1a4002f70 tx=0x7fd1a4006eb0).ready entity=mon.0 client_cookie=e282b5885cb0cbca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b2ffd700 1 -- 172.21.15.32:0/1648417623 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1a400d5c0 con 0x7fd1b40b6710 2022-08-01T22:58:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b2ffd700 1 -- 172.21.15.32:0/1648417623 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a4000e90 con 0x7fd1b40b6710 2022-08-01T22:58:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1b2ffd700 1 -- 172.21.15.32:0/1648417623 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1a400d950 con 0x7fd1b40b6710 2022-08-01T22:58:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- 172.21.15.32:0/1648417623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 msgr2=0x7fd1b40b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b40b7010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fd1a4002f70 tx=0x7fd1a4006eb0).stop 2022-08-01T22:58:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- 172.21.15.32:0/1648417623 shutdown_connections 2022-08-01T22:58:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b5cd0 0x7fd1b41481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 --2- 172.21.15.32:0/1648417623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b40b7010 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- 172.21.15.32:0/1648417623 >> 172.21.15.32:0/1648417623 conn(0x7fd1b405c620 msgr2=0x7fd1b405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- 172.21.15.32:0/1648417623 shutdown_connections 2022-08-01T22:58:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.847+0000 7fd1ba477700 1 -- 172.21.15.32:0/1648417623 wait complete. 2022-08-01T22:58:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 Processor -- start 2022-08-01T22:58:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 -- start start 2022-08-01T22:58:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b0de0 0x7fd1b41495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b40b5cd0 0x7fd1b4148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b4149bd0 con 0x7fd1b40b6710 2022-08-01T22:58:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b4062d30 con 0x7fd1b40b5cd0 2022-08-01T22:58:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b414a3f0 con 0x7fd1b40b0de0 2022-08-01T22:58:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b414add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42790/0 (socket says 172.21.15.32:42790) 2022-08-01T22:58:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 -- 172.21.15.32:0/3240430304 learned_addr learned my addr 172.21.15.32:0/3240430304 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b3fff700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b0de0 0x7fd1b41495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b37fe700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b40b5cd0 0x7fd1b4148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 -- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b0de0 msgr2=0x7fd1b41495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b0de0 0x7fd1b41495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 -- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b40b5cd0 msgr2=0x7fd1b4148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b40b5cd0 0x7fd1b4148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 -- 172.21.15.32:0/3240430304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a400b1e0 con 0x7fd1b40b6710 2022-08-01T22:58:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b8a14700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b414add0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fd1b405d420 tx=0x7fd1ac0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:02.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1ac006aa0 con 0x7fd1b40b6710 2022-08-01T22:58:02.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac006c00 con 0x7fd1b40b6710 2022-08-01T22:58:02.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b414a690 con 0x7fd1b40b6710 2022-08-01T22:58:02.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1ac007030 con 0x7fd1b40b6710 2022-08-01T22:58:02.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.851+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b414b2c0 con 0x7fd1b40b6710 2022-08-01T22:58:02.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.855+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b40574e0 con 0x7fd1b40b6710 2022-08-01T22:58:02.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.855+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd1ac010030 con 0x7fd1b40b6710 2022-08-01T22:58:02.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.855+0000 7fd1b17fa700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd194053a70 0x7fd194055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:02.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.855+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd1ac06be30 con 0x7fd1b40b6710 2022-08-01T22:58:02.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.855+0000 7fd1b3fff700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd194053a70 0x7fd194055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:02.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.855+0000 7fd1b3fff700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd194053a70 0x7fd194055f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fd1a401c000 tx=0x7fd1a4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:02.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:02.863+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd1ac042bf0 con 0x7fd1b40b6710 2022-08-01T22:58:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1b4146fd0 con 0x7fd1b40b6710 2022-08-01T22:58:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1b17fa700 1 -- 172.21.15.32:0/3240430304 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fd1ac046210 con 0x7fd1b40b6710 2022-08-01T22:58:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:58:03.031 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:58:03.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd194053a70 msgr2=0x7fd194055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd194053a70 0x7fd194055f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fd1a401c000 tx=0x7fd1a4015000).stop 2022-08-01T22:58:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 msgr2=0x7fd1b414add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b414add0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fd1b405d420 tx=0x7fd1ac0013a0).stop 2022-08-01T22:58:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 shutdown_connections 2022-08-01T22:58:03.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd194053a70 0x7fd194055f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b40b6710 0x7fd1b414add0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b40b5cd0 0x7fd1b4148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 --2- 172.21.15.32:0/3240430304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1b40b0de0 0x7fd1b41495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 >> 172.21.15.32:0/3240430304 conn(0x7fd1b405c620 msgr2=0x7fd1b40ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:03.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.027+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 shutdown_connections 2022-08-01T22:58:03.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.031+0000 7fd1ba477700 1 -- 172.21.15.32:0/3240430304 wait complete. 2022-08-01T22:58:03.037 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:58:03.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2022-08-01T22:58:03.151 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T22:58:03.151 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T22:58:03.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:58:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 Processor -- start 2022-08-01T22:58:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 -- start start 2022-08-01T22:58:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2880148530 0x7f28800b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28800a7250 0x7f28801448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28800b04e0 con 0x7f28800b1f40 2022-08-01T22:58:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28800a9250 con 0x7f28800a7250 2022-08-01T22:58:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28800b0780 con 0x7f2880148530 2022-08-01T22:58:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28800a7250 0x7f28801448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28800a7250 0x7f28801448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42800/0 (socket says 172.21.15.32:42800) 2022-08-01T22:58:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 -- 172.21.15.32:0/4023653847 learned_addr learned my addr 172.21.15.32:0/4023653847 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287f7fe700 1 --1- 172.21.15.32:0/4023653847 >> v1:172.21.15.32:6789/0 conn(0x7f28800b1f40 0x7f28800b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58572/0 (socket says 172.21.15.32:58572) 2022-08-01T22:58:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287effd700 1 --2- 172.21.15.32:0/4023653847 >> v2:172.21.15.55:3300/0 conn(0x7f2880148530 0x7f28800b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 -- 172.21.15.32:0/4023653847 >> v2:172.21.15.55:3300/0 conn(0x7f2880148530 msgr2=0x7f28800b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 --2- 172.21.15.32:0/4023653847 >> v2:172.21.15.55:3300/0 conn(0x7f2880148530 0x7f28800b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 -- 172.21.15.32:0/4023653847 >> v1:172.21.15.32:6789/0 conn(0x7f28800b1f40 legacy=0x7f28800b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 -- 172.21.15.32:0/4023653847 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28800b0a20 con 0x7f28800a7250 2022-08-01T22:58:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287ffff700 1 --2- 172.21.15.32:0/4023653847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f28801448b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f28801478c0 tx=0x7f2878006eb0).ready entity=mon.0 client_cookie=bb72595332c65a1a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287e7fc700 1 -- 172.21.15.32:0/4023653847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287800d610 con 0x7f28800a7250 2022-08-01T22:58:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287e7fc700 1 -- 172.21.15.32:0/4023653847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2878000e90 con 0x7f28800a7250 2022-08-01T22:58:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f287e7fc700 1 -- 172.21.15.32:0/4023653847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287800d9a0 con 0x7f28800a7250 2022-08-01T22:58:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 -- 172.21.15.32:0/4023653847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 msgr2=0x7f28801448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 --2- 172.21.15.32:0/4023653847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f28801448b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f28801478c0 tx=0x7f2878006eb0).stop 2022-08-01T22:58:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 -- 172.21.15.32:0/4023653847 shutdown_connections 2022-08-01T22:58:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 --2- 172.21.15.32:0/4023653847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f28801448b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.251+0000 7f288602f700 1 --2- 172.21.15.32:0/4023653847 >> v2:172.21.15.55:3300/0 conn(0x7f2880148530 0x7f28800b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- 172.21.15.32:0/4023653847 >> 172.21.15.32:0/4023653847 conn(0x7f288005f840 msgr2=0x7f288005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- 172.21.15.32:0/4023653847 shutdown_connections 2022-08-01T22:58:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- 172.21.15.32:0/4023653847 wait complete. 2022-08-01T22:58:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 Processor -- start 2022-08-01T22:58:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- start start 2022-08-01T22:58:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f288014b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f28800b1f40 0x7f2880156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2880148530 0x7f2880156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288014afc0 con 0x7f28800a7250 2022-08-01T22:58:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28800aac80 con 0x7f2880148530 2022-08-01T22:58:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f288602f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28800aaf20 con 0x7f28800b1f40 2022-08-01T22:58:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f288014b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f288014b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42802/0 (socket says 172.21.15.32:42802) 2022-08-01T22:58:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 -- 172.21.15.32:0/556932325 learned_addr learned my addr 172.21.15.32:0/556932325 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287effd700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f28800b1f40 0x7f2880156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287ffff700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2880148530 0x7f2880156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 -- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f28800b1f40 msgr2=0x7f2880156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f28800b1f40 0x7f2880156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 -- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2880148530 msgr2=0x7f2880156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2880148530 0x7f2880156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 -- 172.21.15.32:0/556932325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287800b1e0 con 0x7f28800a7250 2022-08-01T22:58:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.255+0000 7f287f7fe700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f288014b4f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f28800acff0 tx=0x7f2874007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28740070d0 con 0x7f28800a7250 2022-08-01T22:58:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874007230 con 0x7f28800a7250 2022-08-01T22:58:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28800ab1c0 con 0x7f28800a7250 2022-08-01T22:58:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2874005550 con 0x7f28800a7250 2022-08-01T22:58:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28801574b0 con 0x7f28800a7250 2022-08-01T22:58:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28800b0cc0 con 0x7f28800a7250 2022-08-01T22:58:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f287401a070 con 0x7f28800a7250 2022-08-01T22:58:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287cff9700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2860053a70 0x7f2860055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f287406b280 con 0x7f28800a7250 2022-08-01T22:58:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287effd700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2860053a70 0x7f2860055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.259+0000 7f287effd700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2860053a70 0x7f2860055f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f287000b000 tx=0x7f28700091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.263+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2874042040 con 0x7f28800a7250 2022-08-01T22:58:03.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.439+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28800574e0 con 0x7f28800a7250 2022-08-01T22:58:03.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f287cff9700 1 -- 172.21.15.32:0/556932325 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f2874045660 con 0x7f28800a7250 2022-08-01T22:58:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:58:03.448 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:58:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2860053a70 msgr2=0x7f2860055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2860053a70 0x7f2860055f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f287000b000 tx=0x7f28700091d0).stop 2022-08-01T22:58:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 msgr2=0x7f288014b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f288014b4f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f28800acff0 tx=0x7f2874007c40).stop 2022-08-01T22:58:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 shutdown_connections 2022-08-01T22:58:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2860053a70 0x7f2860055f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2880148530 0x7f2880156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f28800b1f40 0x7f2880156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 --2- 172.21.15.32:0/556932325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28800a7250 0x7f288014b4f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 >> 172.21.15.32:0/556932325 conn(0x7f288005f840 msgr2=0x7f2880144930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:03.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 shutdown_connections 2022-08-01T22:58:03.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.443+0000 7f288602f700 1 -- 172.21.15.32:0/556932325 wait complete. 2022-08-01T22:58:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:58:03.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T22:58:03.560 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T22:58:03.561 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T22:58:03.562 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:58:03.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:58:03.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 Processor -- start 2022-08-01T22:58:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 -- start start 2022-08-01T22:58:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43340614b0 0x7f4334061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f43340b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43341405d0 0x7f4334140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43340aee80 con 0x7f43340614b0 2022-08-01T22:58:03.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43340b70b0 con 0x7f43341405d0 2022-08-01T22:58:03.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4339580700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43340af0b0 con 0x7f43340ad750 2022-08-01T22:58:03.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43340614b0 0x7f4334061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43340614b0 0x7f4334061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42810/0 (socket says 172.21.15.32:42810) 2022-08-01T22:58:03.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 -- 172.21.15.32:0/2872865458 learned_addr learned my addr 172.21.15.32:0/2872865458 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:03.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f43337fe700 1 --2- 172.21.15.32:0/2872865458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f43340b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4338d7f700 1 --2- 172.21.15.32:0/2872865458 >> v2:172.21.15.55:3300/0 conn(0x7f43341405d0 0x7f4334140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 -- 172.21.15.32:0/2872865458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 msgr2=0x7f43340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:03.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 --2- 172.21.15.32:0/2872865458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f43340b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 -- 172.21.15.32:0/2872865458 >> v2:172.21.15.55:3300/0 conn(0x7f43341405d0 msgr2=0x7f4334140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:03.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 --2- 172.21.15.32:0/2872865458 >> v2:172.21.15.55:3300/0 conn(0x7f43341405d0 0x7f4334140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 -- 172.21.15.32:0/2872865458 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340af350 con 0x7f43340614b0 2022-08-01T22:58:03.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.683+0000 7f4333fff700 1 --2- 172.21.15.32:0/2872865458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f4334061890 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f43240032f0 tx=0x7f4324009290).ready entity=mon.0 client_cookie=7e531cd88bcb2b8d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:03.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4332ffd700 1 -- 172.21.15.32:0/2872865458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432400b680 con 0x7f43340614b0 2022-08-01T22:58:03.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4332ffd700 1 -- 172.21.15.32:0/2872865458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432400b7e0 con 0x7f43340614b0 2022-08-01T22:58:03.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4332ffd700 1 -- 172.21.15.32:0/2872865458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432400b950 con 0x7f43340614b0 2022-08-01T22:58:03.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 -- 172.21.15.32:0/2872865458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 msgr2=0x7f4334061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 --2- 172.21.15.32:0/2872865458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f4334061890 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f43240032f0 tx=0x7f4324009290).stop 2022-08-01T22:58:03.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 -- 172.21.15.32:0/2872865458 shutdown_connections 2022-08-01T22:58:03.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 --2- 172.21.15.32:0/2872865458 >> v2:172.21.15.55:3300/0 conn(0x7f43341405d0 0x7f4334140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 --2- 172.21.15.32:0/2872865458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f43340b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 --2- 172.21.15.32:0/2872865458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f4334061890 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 -- 172.21.15.32:0/2872865458 >> 172.21.15.32:0/2872865458 conn(0x7f4334066f70 msgr2=0x7f4334067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:03.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 -- 172.21.15.32:0/2872865458 shutdown_connections 2022-08-01T22:58:03.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 -- 172.21.15.32:0/2872865458 wait complete. 2022-08-01T22:58:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 Processor -- start 2022-08-01T22:58:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.687+0000 7f4339580700 1 -- start start 2022-08-01T22:58:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f433405d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f433409ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43341405d0 0x7f4334152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433409f470 con 0x7f43340614b0 2022-08-01T22:58:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43341565c0 con 0x7f43341405d0 2022-08-01T22:58:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4334156860 con 0x7f43340ad750 2022-08-01T22:58:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f433405d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f433405d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42816/0 (socket says 172.21.15.32:42816) 2022-08-01T22:58:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 -- 172.21.15.32:0/1558717604 learned_addr learned my addr 172.21.15.32:0/1558717604 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4338d7f700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43341405d0 0x7f4334152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f43337fe700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f433409ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 -- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 msgr2=0x7f433409ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f433409ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 -- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43341405d0 msgr2=0x7f4334152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43341405d0 0x7f4334152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 -- 172.21.15.32:0/1558717604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324009530 con 0x7f43340614b0 2022-08-01T22:58:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4333fff700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f433405d0d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f43240032f0 tx=0x7f4324017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:03.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432400b680 con 0x7f43340614b0 2022-08-01T22:58:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4324011e50 con 0x7f43340614b0 2022-08-01T22:58:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433409eae0 con 0x7f43340614b0 2022-08-01T22:58:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432401b810 con 0x7f43340614b0 2022-08-01T22:58:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43340b0370 con 0x7f43340614b0 2022-08-01T22:58:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.691+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43341563f0 con 0x7f43340614b0 2022-08-01T22:58:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.695+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f432400f070 con 0x7f43340614b0 2022-08-01T22:58:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.695+0000 7f43317fa700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4314053a70 0x7f4314055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.695+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f432406d4f0 con 0x7f43340614b0 2022-08-01T22:58:03.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.695+0000 7f43337fe700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4314053a70 0x7f4314055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:03.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.695+0000 7f43337fe700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4314053a70 0x7f4314055f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f432800b000 tx=0x7f4328006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:03.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.699+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43240442b0 con 0x7f43340614b0 2022-08-01T22:58:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43340574e0 con 0x7f43340614b0 2022-08-01T22:58:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f43317fa700 1 -- 172.21.15.32:0/1558717604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f432401f020 con 0x7f43340614b0 2022-08-01T22:58:03.867 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:58:03.867 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:58:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4314053a70 msgr2=0x7f4314055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4314053a70 0x7f4314055f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f432800b000 tx=0x7f4328006df0).stop 2022-08-01T22:58:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 msgr2=0x7f433405d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f433405d0d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f43240032f0 tx=0x7f4324017000).stop 2022-08-01T22:58:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 shutdown_connections 2022-08-01T22:58:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4314053a70 0x7f4314055f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43341405d0 0x7f4334152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43340ad750 0x7f433409ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 --2- 172.21.15.32:0/1558717604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43340614b0 0x7f433405d0d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:03.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 >> 172.21.15.32:0/1558717604 conn(0x7f4334066f70 msgr2=0x7f4334146880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:03.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 shutdown_connections 2022-08-01T22:58:03.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.863+0000 7f4339580700 1 -- 172.21.15.32:0/1558717604 wait complete. 2022-08-01T22:58:03.872 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:58:03.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 2022-08-01T22:58:03.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.987+0000 7f66b6108700 1 Processor -- start 2022-08-01T22:58:03.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.987+0000 7f66b6108700 1 -- start start 2022-08-01T22:58:03.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b6108700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66b00af550 0x7f66b00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:03.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b6108700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b00aa630 con 0x7f66b005c6f0 2022-08-01T22:58:03.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b6108700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b005fd10 con 0x7f66b0060330 2022-08-01T22:58:03.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b6108700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b00b15e0 con 0x7f66b00af550 2022-08-01T22:58:03.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5907700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66b005c6f0 0x7f66b00a8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58602/0 (socket says 172.21.15.32:58602) 2022-08-01T22:58:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5907700 1 -- 172.21.15.32:0/4213170329 learned_addr learned my addr 172.21.15.32:0/4213170329 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:58:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5106700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66b00af550 0x7f66b00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f669ffff700 1 -- 172.21.15.32:0/4213170329 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942702304 0 0) 0x7f66b00aa630 con 0x7f66b005c6f0 2022-08-01T22:58:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f669ffff700 1 -- 172.21.15.32:0/4213170329 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6698003260 con 0x7f66b005c6f0 2022-08-01T22:58:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5106700 1 -- 172.21.15.32:0/4213170329 >> v1:172.21.15.55:6789/0 conn(0x7f66b0060330 legacy=0x7f66b01445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5106700 1 -- 172.21.15.32:0/4213170329 >> v1:172.21.15.32:6789/0 conn(0x7f66b005c6f0 legacy=0x7f66b00a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5106700 1 -- 172.21.15.32:0/4213170329 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b005f840 con 0x7f66b00af550 2022-08-01T22:58:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f66b5106700 1 --2- 172.21.15.32:0/4213170329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b00b0e40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f66ac002f70 tx=0x7f66ac006df0).ready entity=mon.0 client_cookie=c04d5414a864212b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f669ffff700 1 -- 172.21.15.32:0/4213170329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66ac00d5c0 con 0x7f66b00af550 2022-08-01T22:58:04.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f669ffff700 1 -- 172.21.15.32:0/4213170329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac000e90 con 0x7f66b00af550 2022-08-01T22:58:04.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.991+0000 7f669ffff700 1 -- 172.21.15.32:0/4213170329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66ac00d910 con 0x7f66b00af550 2022-08-01T22:58:04.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 -- 172.21.15.32:0/4213170329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 msgr2=0x7f66b00b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 --2- 172.21.15.32:0/4213170329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b00b0e40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f66ac002f70 tx=0x7f66ac006df0).stop 2022-08-01T22:58:04.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 -- 172.21.15.32:0/4213170329 shutdown_connections 2022-08-01T22:58:04.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 --2- 172.21.15.32:0/4213170329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b00b0e40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 -- 172.21.15.32:0/4213170329 >> 172.21.15.32:0/4213170329 conn(0x7f66b00612d0 msgr2=0x7f66b0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:04.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 -- 172.21.15.32:0/4213170329 shutdown_connections 2022-08-01T22:58:04.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 -- 172.21.15.32:0/4213170329 wait complete. 2022-08-01T22:58:04.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 Processor -- start 2022-08-01T22:58:04.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.995+0000 7f66b6108700 1 -- start start 2022-08-01T22:58:04.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66b005c6f0 0x7f66b0148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66b0060330 0x7f66b0153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b0150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0066120 con 0x7f66b00af550 2022-08-01T22:58:04.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b014ff10 con 0x7f66b0060330 2022-08-01T22:58:04.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0066b20 con 0x7f66b005c6f0 2022-08-01T22:58:04.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b0150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b0150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42832/0 (socket says 172.21.15.32:42832) 2022-08-01T22:58:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 -- 172.21.15.32:0/3473258223 learned_addr learned my addr 172.21.15.32:0/3473258223 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b4905700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66b0060330 0x7f66b0153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5106700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66b005c6f0 0x7f66b0148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 -- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66b005c6f0 msgr2=0x7f66b0148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66b005c6f0 0x7f66b0148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 -- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66b0060330 msgr2=0x7f66b0153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66b0060330 0x7f66b0153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 -- 172.21.15.32:0/3473258223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac00b180 con 0x7f66b00af550 2022-08-01T22:58:04.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b4905700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66b0060330 0x7f66b0153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:58:04.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b5907700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b0150300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f66a8001490 tx=0x7f66a8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f669e7fc700 1 -- 172.21.15.32:0/3473258223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a8005ba0 con 0x7f66b00af550 2022-08-01T22:58:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f669e7fc700 1 -- 172.21.15.32:0/3473258223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a8005d00 con 0x7f66b00af550 2022-08-01T22:58:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f669e7fc700 1 -- 172.21.15.32:0/3473258223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a80072c0 con 0x7f66b00af550 2022-08-01T22:58:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0066d50 con 0x7f66b00af550 2022-08-01T22:58:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b0067430 con 0x7f66b00af550 2022-08-01T22:58:04.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:03.999+0000 7f669e7fc700 1 -- 172.21.15.32:0/3473258223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f66a8010070 con 0x7f66b00af550 2022-08-01T22:58:04.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f669e7fc700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6690053a70 0x7f6690055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f669e7fc700 1 -- 172.21.15.32:0/3473258223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f66a806bba0 con 0x7f66b00af550 2022-08-01T22:58:04.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b5106700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6690053a70 0x7f6690055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b5106700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6690053a70 0x7f6690055f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f66ac015000 tx=0x7f66ac014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 Processor -- start 2022-08-01T22:58:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 -- start start 2022-08-01T22:58:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 0x2141a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2141ef0 0x2142370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x209d320 con 0x2173310 2022-08-01T22:58:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x20b0c50 con 0x2141ef0 2022-08-01T22:58:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x20911f0 con 0x2177250 2022-08-01T22:58:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 0x2141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 0x2141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42840/0 (socket says 172.21.15.32:42840) 2022-08-01T22:58:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 -- 172.21.15.32:0/2017752865 learned_addr learned my addr 172.21.15.32:0/2017752865 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b5907700 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2141ef0 0x2142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b5106700 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 -- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 msgr2=0x2174830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 -- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2141ef0 msgr2=0x2142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2141ef0 0x2142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.003+0000 7f66b4905700 1 -- 172.21.15.32:0/2017752865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20e8ec0 con 0x2173310 2022-08-01T22:58:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b5907700 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2141ef0 0x2142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:58:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 0x2141a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x20fc120 tx=0x7f66a0009040).ready entity=mon.0 client_cookie=cec5736d4f27cabd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b6108700 1 -- 172.21.15.32:0/2017752865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a000f440 con 0x2173310 2022-08-01T22:58:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b6108700 1 -- 172.21.15.32:0/2017752865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a000d030 con 0x2173310 2022-08-01T22:58:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b6108700 1 -- 172.21.15.32:0/2017752865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a000f790 con 0x2173310 2022-08-01T22:58:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- 172.21.15.32:0/2017752865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 msgr2=0x2141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 0x2141a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x20fc120 tx=0x7f66a0009040).stop 2022-08-01T22:58:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- 172.21.15.32:0/2017752865 shutdown_connections 2022-08-01T22:58:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2141ef0 0x2142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2173310 0x2141a00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- 172.21.15.32:0/2017752865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- 172.21.15.32:0/2017752865 >> 172.21.15.32:0/2017752865 conn(0x213cf50 msgr2=0x21767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- 172.21.15.32:0/2017752865 shutdown_connections 2022-08-01T22:58:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- 172.21.15.32:0/2017752865 wait complete. 2022-08-01T22:58:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- start start 2022-08-01T22:58:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 Processor -- start 2022-08-01T22:58:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x2140e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x21412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x21443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2145ef0 con 0x2177250 2022-08-01T22:58:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x213c920 con 0x217a810 2022-08-01T22:58:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x2185ad0 con 0x2173310 2022-08-01T22:58:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66c15c1740 10 client.0 ll_register_callbacks cb 0x2096310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T22:58:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x21412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x21412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42846/0 (socket says 172.21.15.32:42846) 2022-08-01T22:58:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 -- 172.21.15.32:0/240997405 learned_addr learned my addr 172.21.15.32:0/240997405 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b5106700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x2140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b5907700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x21443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 -- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 msgr2=0x2140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x2140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 -- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 msgr2=0x21443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x21443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.007+0000 7f66b4905700 1 -- 172.21.15.32:0/240997405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b005fd10 con 0x2177250 2022-08-01T22:58:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66b5106700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x2140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:58:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66b5907700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x21443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:58:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66b4905700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x21412f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x20fc120 tx=0x7f66a0017000).ready entity=mon.0 client_cookie=5686f65f93177575 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T22:58:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 1 -- 172.21.15.32:0/240997405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a0007880 con 0x2177250 2022-08-01T22:58:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 1 -- 172.21.15.32:0/240997405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f66a0013630 con 0x2177250 2022-08-01T22:58:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20a6c70 con 0x2177250 2022-08-01T22:58:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 1 -- 172.21.15.32:0/240997405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a00137f0 con 0x2177250 2022-08-01T22:58:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 10 client.5424 fetch_fsmap learned FSMap version 18 2022-08-01T22:58:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x2083950 con 0x2177250 2022-08-01T22:58:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 1 -- 172.21.15.32:0/240997405 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20a6c70 con 0x2177250 2022-08-01T22:58:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 1 -- 172.21.15.32:0/240997405 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 18) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f66a0011030 con 0x2177250 2022-08-01T22:58:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 10 client.5424 fetch_fsmap finished waiting for FSMap version 18 2022-08-01T22:58:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 10 client.5424 resolve_mds: resolved ID 'c' to GID 5241 2022-08-01T22:58:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 20 client.5424 populate_metadata read hostname 'smithi032' 2022-08-01T22:58:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21855f0 0x2188180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 4 client.5424 mds_command: new command op to 5241 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T22:58:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 --> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x209efe0 con 0x21855f0 2022-08-01T22:58:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66b5106700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21855f0 0x2188180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 10 client.5424 ms_handle_connect on v2:172.21.15.32:6834/505560818 2022-08-01T22:58:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66b5106700 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21855f0 0x2188180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3595dcec2d5f9c21 server_cookie=3581a6a9cfc3e553 in_seq=0 out_seq=0 2022-08-01T22:58:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 1 -- 172.21.15.32:0/240997405 <== mds.0 v2:172.21.15.32:6834/505560818 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x209efe0 con 0x21855f0 2022-08-01T22:58:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 10 client.5424 handle_command_reply: tid=0 2022-08-01T22:58:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 client.5424 shutdown 2022-08-01T22:58:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 msgr2=0x21412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x21412f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x20fc120 tx=0x7f66a0017000).stop 2022-08-01T22:58:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 shutdown_connections 2022-08-01T22:58:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21855f0 0x2188180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x21443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f668c7f8700 0 client.5424 ms_handle_reset on v2:172.21.15.32:6834/505560818 2022-08-01T22:58:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x21412f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 --2- 172.21.15.32:0/240997405 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x2140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 >> 172.21.15.32:0/240997405 conn(0x2141ef0 msgr2=0x2174250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 shutdown_connections 2022-08-01T22:58:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 1 -- 172.21.15.32:0/240997405 wait complete. 2022-08-01T22:58:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.011+0000 7f66c15c1740 20 client.5424 trim_cache size 0 max 16384 2022-08-01T22:58:04.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.027+0000 7f66c15c1740 1 Processor -- start 2022-08-01T22:58:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.027+0000 7f66c15c1740 1 -- start start 2022-08-01T22:58:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2173310 0x2174190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 0x2140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2172f30 con 0x217a810 2022-08-01T22:58:04.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x2144240 con 0x2173310 2022-08-01T22:58:04.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x2179ae0 con 0x2177250 2022-08-01T22:58:04.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 0x2140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 0x2140fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42854/0 (socket says 172.21.15.32:42854) 2022-08-01T22:58:04.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 -- 172.21.15.32:0/3478531011 learned_addr learned my addr 172.21.15.32:0/3478531011 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:04.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5106700 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2173310 0x2174190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b4905700 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 -- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 msgr2=0x2174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 -- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2173310 msgr2=0x2174190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2173310 0x2174190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 -- 172.21.15.32:0/3478531011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20a0550 con 0x217a810 2022-08-01T22:58:04.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5106700 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2173310 0x2174190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:58:04.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66b5907700 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 0x2140fa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x2145c70 tx=0x7f66a8048040).ready entity=mon.0 client_cookie=401aede64f86bed6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f668e7fc700 1 -- 172.21.15.32:0/3478531011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a80460c0 con 0x217a810 2022-08-01T22:58:04.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f668e7fc700 1 -- 172.21.15.32:0/3478531011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a804c070 con 0x217a810 2022-08-01T22:58:04.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f668e7fc700 1 -- 172.21.15.32:0/3478531011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a8045a70 con 0x217a810 2022-08-01T22:58:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- 172.21.15.32:0/3478531011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 msgr2=0x2140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 0x2140fa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x2145c70 tx=0x7f66a8048040).stop 2022-08-01T22:58:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- 172.21.15.32:0/3478531011 shutdown_connections 2022-08-01T22:58:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x217a810 0x2140fa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2177250 0x2174680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- 172.21.15.32:0/3478531011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x2173310 0x2174190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- 172.21.15.32:0/3478531011 >> 172.21.15.32:0/3478531011 conn(0x2141ef0 msgr2=0x2172820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- 172.21.15.32:0/3478531011 shutdown_connections 2022-08-01T22:58:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- 172.21.15.32:0/3478531011 wait complete. 2022-08-01T22:58:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- start start 2022-08-01T22:58:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 Processor -- start 2022-08-01T22:58:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x217d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x213f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x213fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x21407d0 con 0x2177250 2022-08-01T22:58:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x2142410 con 0x217a810 2022-08-01T22:58:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x2195490 con 0x2173310 2022-08-01T22:58:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.031+0000 7f66c15c1740 10 client.0 ll_register_callbacks cb 0x21464d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T22:58:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x213f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x213f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42860/0 (socket says 172.21.15.32:42860) 2022-08-01T22:58:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 -- 172.21.15.32:0/1618333298 learned_addr learned my addr 172.21.15.32:0/1618333298 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b5106700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x217d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b5907700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x213fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 -- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 msgr2=0x217d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x217d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 -- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 msgr2=0x213fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x213fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 -- 172.21.15.32:0/1618333298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b0067430 con 0x2177250 2022-08-01T22:58:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b4905700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x213f9e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f66a0018000 tx=0x7f66a001d000).ready entity=mon.0 client_cookie=ca7aab43cd1c6c31 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T22:58:04.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 1 -- 172.21.15.32:0/1618333298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a0007850 con 0x2177250 2022-08-01T22:58:04.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 1 -- 172.21.15.32:0/1618333298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f66a0007040 con 0x2177250 2022-08-01T22:58:04.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66a000d030 con 0x2177250 2022-08-01T22:58:04.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 1 -- 172.21.15.32:0/1618333298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a0007200 con 0x2177250 2022-08-01T22:58:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 10 client.5430 fetch_fsmap learned FSMap version 18 2022-08-01T22:58:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x20a8e50 con 0x2177250 2022-08-01T22:58:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 1 -- 172.21.15.32:0/1618333298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a000d030 con 0x2177250 2022-08-01T22:58:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 1 -- 172.21.15.32:0/1618333298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 18) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f66a0011030 con 0x2177250 2022-08-01T22:58:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 10 client.5430 fetch_fsmap finished waiting for FSMap version 18 2022-08-01T22:58:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 10 client.5430 resolve_mds: resolved ID 'c' to GID 5241 2022-08-01T22:58:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 20 client.5430 populate_metadata read hostname 'smithi032' 2022-08-01T22:58:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21dcbe0 0x21d5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 4 client.5430 mds_command: new command op to 5241 tid=0[{"prefix": "cache drop"}] 2022-08-01T22:58:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 --> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1f94590 con 0x21dcbe0 2022-08-01T22:58:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b5106700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21dcbe0 0x21d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f668d7fa700 10 client.5430 ms_handle_connect on v2:172.21.15.32:6834/505560818 2022-08-01T22:58:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:04.035+0000 7f66b5106700 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21dcbe0 0x21d5cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d643269dcc2f6ff1 server_cookie=238c2f8f82dcd5be in_seq=0 out_seq=0 2022-08-01T22:58:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f668d7fa700 1 -- 172.21.15.32:0/1618333298 <== mds.0 v2:172.21.15.32:6834/505560818 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1f94590 con 0x21dcbe0 2022-08-01T22:58:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f668d7fa700 10 client.5430 handle_command_reply: tid=0 2022-08-01T22:58:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 client.5430 shutdown 2022-08-01T22:58:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 msgr2=0x213f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x213f9e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f66a0018000 tx=0x7f66a001d000).stop 2022-08-01T22:58:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 shutdown_connections 2022-08-01T22:58:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:6834/505560818,v1:172.21.15.32:6835/505560818] conn(0x21dcbe0 0x21d5cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x217a810 0x213fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x2177250 0x213f9e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 --2- 172.21.15.32:0/1618333298 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x2173310 0x217d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f668d7fa700 0 client.5430 ms_handle_reset on v2:172.21.15.32:6834/505560818 2022-08-01T22:58:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 >> 172.21.15.32:0/1618333298 conn(0x2141ef0 msgr2=0x2174190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 shutdown_connections 2022-08-01T22:58:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 1 -- 172.21.15.32:0/1618333298 wait complete. 2022-08-01T22:58:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.235+0000 7f66c15c1740 20 client.5430 trim_cache size 0 max 16384 2022-08-01T22:58:15.250 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:58:15.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2022-08-01T22:58:15.251 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2022-08-01T22:58:15.252 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(0) Success", 2022-08-01T22:58:15.252 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 220 2022-08-01T22:58:15.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:58:15.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2022-08-01T22:58:15.254 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2022-08-01T22:58:15.254 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2022-08-01T22:58:15.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:58:15.255 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2022-08-01T22:58:15.255 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 0 2022-08-01T22:58:15.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:58:15.256 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2022-08-01T22:58:15.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2022-08-01T22:58:15.257 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 19975, 2022-08-01T22:58:15.258 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 3098720 2022-08-01T22:58:15.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:58:15.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:58:15.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 11.199450544999999 2022-08-01T22:58:15.260 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:58:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6690053a70 msgr2=0x7f6690055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6690053a70 0x7f6690055f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f66ac015000 tx=0x7f66ac014000).stop 2022-08-01T22:58:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 msgr2=0x7f66b0150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b0150300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f66a8001490 tx=0x7f66a8007e00).stop 2022-08-01T22:58:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 shutdown_connections 2022-08-01T22:58:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6690053a70 0x7f6690055f30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b00af550 0x7f66b0150300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66b0060330 0x7f66b0153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 --2- 172.21.15.32:0/3473258223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66b005c6f0 0x7f66b0148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 >> 172.21.15.32:0/3473258223 conn(0x7f66b00612d0 msgr2=0x7f66b0140920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 shutdown_connections 2022-08-01T22:58:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.239+0000 7f66b6108700 1 -- 172.21.15.32:0/3473258223 wait complete. 2022-08-01T22:58:15.268 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-08-01T22:58:15.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:15.270 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:58:15.271 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:15.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:58:15.282 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T22:58:15.282 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T22:58:15.283 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:58:15.284 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T22:58:15.285 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:15.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:15.311 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:58:15.311 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:15.359 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T22:58:15.360 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T22:58:15.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:58:15.361 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:15.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:58:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 Processor -- start 2022-08-01T22:58:15.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- start start 2022-08-01T22:58:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c0b1f40 0x7f9d1c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d1c148530 0x7f9d1c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c0b04e0 con 0x7f9d1c0a7250 2022-08-01T22:58:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0a9250 con 0x7f9d1c148530 2022-08-01T22:58:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0b0780 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d1c148530 0x7f9d1c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d1c148530 0x7f9d1c0b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42870/0 (socket says 172.21.15.32:42870) 2022-08-01T22:58:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/4263601501 learned_addr learned my addr 172.21.15.32:0/4263601501 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:15.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1bfff700 1 --2- 172.21.15.32:0/4263601501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c0b1f40 0x7f9d1c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/4263601501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c0b1f40 msgr2=0x7f9d1c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 --2- 172.21.15.32:0/4263601501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c0b1f40 0x7f9d1c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/4263601501 >> v1:172.21.15.55:6789/0 conn(0x7f9d1c0a7250 legacy=0x7f9d1c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/4263601501 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0b0a20 con 0x7f9d1c148530 2022-08-01T22:58:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1b7fe700 1 --2- 172.21.15.32:0/4263601501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c148530 0x7f9d1c0b2320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f9d10002f70 tx=0x7f9d10006eb0).ready entity=mon.0 client_cookie=6c6540aaa55da837 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1affd700 1 -- 172.21.15.32:0/4263601501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1000d5c0 con 0x7f9d1c148530 2022-08-01T22:58:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1affd700 1 -- 172.21.15.32:0/4263601501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10000e90 con 0x7f9d1c148530 2022-08-01T22:58:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d1affd700 1 -- 172.21.15.32:0/4263601501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1000d950 con 0x7f9d1c148530 2022-08-01T22:58:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- 172.21.15.32:0/4263601501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c148530 msgr2=0x7f9d1c0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 --2- 172.21.15.32:0/4263601501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c148530 0x7f9d1c0b2320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f9d10002f70 tx=0x7f9d10006eb0).stop 2022-08-01T22:58:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- 172.21.15.32:0/4263601501 shutdown_connections 2022-08-01T22:58:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 --2- 172.21.15.32:0/4263601501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c148530 0x7f9d1c0b2320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 --2- 172.21.15.32:0/4263601501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c0b1f40 0x7f9d1c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- 172.21.15.32:0/4263601501 >> 172.21.15.32:0/4263601501 conn(0x7f9d1c05f840 msgr2=0x7f9d1c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- 172.21.15.32:0/4263601501 shutdown_connections 2022-08-01T22:58:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.471+0000 7f9d2294c700 1 -- 172.21.15.32:0/4263601501 wait complete. 2022-08-01T22:58:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 Processor -- start 2022-08-01T22:58:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 -- start start 2022-08-01T22:58:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c0a7250 0x7f9d1c144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 0x7f9d1c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c148530 0x7f9d1c0ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1548e0 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c149210 con 0x7f9d1c0a7250 2022-08-01T22:58:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d2294c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c15a2e0 con 0x7f9d1c148530 2022-08-01T22:58:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.475+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 0x7f9d1c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 0x7f9d1c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42876/0 (socket says 172.21.15.32:42876) 2022-08-01T22:58:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/1256482004 learned_addr learned my addr 172.21.15.32:0/1256482004 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1bfff700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c0a7250 0x7f9d1c144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d20ee9700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c148530 0x7f9d1c0ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c148530 msgr2=0x7f9d1c0ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c148530 0x7f9d1c0ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c0a7250 msgr2=0x7f9d1c144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c0a7250 0x7f9d1c144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 -- 172.21.15.32:0/1256482004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1000b1e0 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d1b7fe700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 0x7f9d1c15aa20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f9d10002f70 tx=0x7f9d10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d10011610 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10011770 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c15a580 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d10011ad0 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c15af10 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c15a710 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.479+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d1001a040 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.483+0000 7f9d197fa700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cfc0539e0 0x7f9cfc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.483+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9d1006cd40 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.483+0000 7f9d1bfff700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cfc0539e0 0x7f9cfc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.483+0000 7f9d1bfff700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cfc0539e0 0x7f9cfc055ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f9d0c00d000 tx=0x7f9d0c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.487+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d10043bc0 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.611+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d1c0b0cc0 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d197fa700 1 -- 172.21.15.32:0/1256482004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f9d10047120 con 0x7f9d1c0b1f40 2022-08-01T22:58:15.620 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:58:15.620 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:57:46.326950+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:57:43.852752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T22:57:45.012782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:58:15.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cfc0539e0 msgr2=0x7f9cfc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cfc0539e0 0x7f9cfc055ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f9d0c00d000 tx=0x7f9d0c008040).stop 2022-08-01T22:58:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 msgr2=0x7f9d1c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 0x7f9d1c15aa20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f9d10002f70 tx=0x7f9d10015000).stop 2022-08-01T22:58:15.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 shutdown_connections 2022-08-01T22:58:15.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cfc0539e0 0x7f9cfc055ea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d1c148530 0x7f9d1c0ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1c0b1f40 0x7f9d1c15aa20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 --2- 172.21.15.32:0/1256482004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c0a7250 0x7f9d1c144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 >> 172.21.15.32:0/1256482004 conn(0x7f9d1c05f840 msgr2=0x7f9d1c0b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 shutdown_connections 2022-08-01T22:58:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.615+0000 7f9d2294c700 1 -- 172.21.15.32:0/1256482004 wait complete. 2022-08-01T22:58:15.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:57:43.852752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:57:45.012782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:58:15.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:58:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 Processor -- start 2022-08-01T22:58:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 -- start start 2022-08-01T22:58:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35a80614b0 0x7f35a8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35a80ad750 0x7f35a80b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a80aee80 con 0x7f35a80ad750 2022-08-01T22:58:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a80b70b0 con 0x7f35a80614b0 2022-08-01T22:58:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35af1df700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80af0b0 con 0x7f35a81405d0 2022-08-01T22:58:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35ad9dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35a80ad750 0x7f35a80b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35ad9dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35a80ad750 0x7f35a80b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42886/0 (socket says 172.21.15.32:42886) 2022-08-01T22:58:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35ad9dc700 1 -- 172.21.15.32:0/3340904589 learned_addr learned my addr 172.21.15.32:0/3340904589 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35ae9de700 1 --2- 172.21.15.32:0/3340904589 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.743+0000 7f35ae1dd700 1 --2- 172.21.15.32:0/3340904589 >> v2:172.21.15.55:3300/0 conn(0x7f35a80614b0 0x7f35a8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad9dc700 1 -- 172.21.15.32:0/3340904589 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 msgr2=0x7f35a8140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:15.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad9dc700 1 --2- 172.21.15.32:0/3340904589 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a8140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad9dc700 1 -- 172.21.15.32:0/3340904589 >> v2:172.21.15.55:3300/0 conn(0x7f35a80614b0 msgr2=0x7f35a8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad9dc700 1 --2- 172.21.15.32:0/3340904589 >> v2:172.21.15.55:3300/0 conn(0x7f35a80614b0 0x7f35a8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad9dc700 1 -- 172.21.15.32:0/3340904589 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80af350 con 0x7f35a80ad750 2022-08-01T22:58:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ae1dd700 1 --2- 172.21.15.32:0/3340904589 >> v2:172.21.15.55:3300/0 conn(0x7f35a80614b0 0x7f35a8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:58:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad9dc700 1 --2- 172.21.15.32:0/3340904589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a80b9ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f35a80aed30 tx=0x7f3598006eb0).ready entity=mon.0 client_cookie=94d7e31cffa1e19a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad1db700 1 -- 172.21.15.32:0/3340904589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359800e610 con 0x7f35a80ad750 2022-08-01T22:58:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad1db700 1 -- 172.21.15.32:0/3340904589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598000e90 con 0x7f35a80ad750 2022-08-01T22:58:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35ad1db700 1 -- 172.21.15.32:0/3340904589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359800e9a0 con 0x7f35a80ad750 2022-08-01T22:58:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 -- 172.21.15.32:0/3340904589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 msgr2=0x7f35a80b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 --2- 172.21.15.32:0/3340904589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a80b9ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f35a80aed30 tx=0x7f3598006eb0).stop 2022-08-01T22:58:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 -- 172.21.15.32:0/3340904589 shutdown_connections 2022-08-01T22:58:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 --2- 172.21.15.32:0/3340904589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a80b9ff0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 --2- 172.21.15.32:0/3340904589 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a8140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 --2- 172.21.15.32:0/3340904589 >> v2:172.21.15.55:3300/0 conn(0x7f35a80614b0 0x7f35a8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 -- 172.21.15.32:0/3340904589 >> 172.21.15.32:0/3340904589 conn(0x7f35a8066f70 msgr2=0x7f35a8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 -- 172.21.15.32:0/3340904589 shutdown_connections 2022-08-01T22:58:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 -- 172.21.15.32:0/3340904589 wait complete. 2022-08-01T22:58:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 Processor -- start 2022-08-01T22:58:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.747+0000 7f35af1df700 1 -- start start 2022-08-01T22:58:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a80614b0 0x7f35a80b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a8152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a80abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80b2f70 con 0x7f35a80ad750 2022-08-01T22:58:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80b01c0 con 0x7f35a80614b0 2022-08-01T22:58:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80afac0 con 0x7f35a81405d0 2022-08-01T22:58:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a8152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a8152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42890/0 (socket says 172.21.15.32:42890) 2022-08-01T22:58:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ae9de700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a80abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 -- 172.21.15.32:0/177278036 learned_addr learned my addr 172.21.15.32:0/177278036 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ae1dd700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a80614b0 0x7f35a80b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 -- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 msgr2=0x7f35a80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a80abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 -- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a80614b0 msgr2=0x7f35a80b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a80614b0 0x7f35a80b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 -- 172.21.15.32:0/177278036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a400b180 con 0x7f35a80ad750 2022-08-01T22:58:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35ad9dc700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a8152e80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f35a80b0300 tx=0x7f3598015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35980116e0 con 0x7f35a80ad750 2022-08-01T22:58:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598011840 con 0x7f35a80ad750 2022-08-01T22:58:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359800b1e0 con 0x7f35a80ad750 2022-08-01T22:58:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3598011ba0 con 0x7f35a80ad750 2022-08-01T22:58:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a80b0ab0 con 0x7f35a80ad750 2022-08-01T22:58:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.751+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a80b0920 con 0x7f35a80ad750 2022-08-01T22:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.755+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f359801a040 con 0x7f35a80ad750 2022-08-01T22:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.755+0000 7f35977fe700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3588053980 0x7f3588055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.755+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f359806cdd0 con 0x7f35a80ad750 2022-08-01T22:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.755+0000 7f35ae1dd700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3588053980 0x7f3588055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.755+0000 7f35ae1dd700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3588053980 0x7f3588055e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f35a400b000 tx=0x7f35a4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.763+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3598043b90 con 0x7f35a80ad750 2022-08-01T22:58:15.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.887+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f35a80af5f0 con 0x7f35a80ad750 2022-08-01T22:58:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.887+0000 7f35977fe700 1 -- 172.21.15.32:0/177278036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7f35980471b0 con 0x7f35a80ad750 2022-08-01T22:58:15.891 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:58:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.887+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3588053980 msgr2=0x7f3588055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.887+0000 7f35af1df700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3588053980 0x7f3588055e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f35a400b000 tx=0x7f35a4006df0).stop 2022-08-01T22:58:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.887+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 msgr2=0x7f35a8152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.887+0000 7f35af1df700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a8152e80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f35a80b0300 tx=0x7f3598015000).stop 2022-08-01T22:58:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 shutdown_connections 2022-08-01T22:58:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3588053980 0x7f3588055e40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35a81405d0 0x7f35a80abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35a80ad750 0x7f35a8152e80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 --2- 172.21.15.32:0/177278036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35a80614b0 0x7f35a80b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:15.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 >> 172.21.15.32:0/177278036 conn(0x7f35a8066f70 msgr2=0x7f35a8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:15.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 shutdown_connections 2022-08-01T22:58:15.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:15.891+0000 7f35af1df700 1 -- 172.21.15.32:0/177278036 wait complete. 2022-08-01T22:58:15.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:58:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42dd6d0700 1 Processor -- start 2022-08-01T22:58:16.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42dd6d0700 1 -- start start 2022-08-01T22:58:16.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42dd6d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 0x7f42d80b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42dd6d0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80a7960 con 0x7f42d80b1460 2022-08-01T22:58:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42dd6d0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80b07e0 con 0x7f42d80b2050 2022-08-01T22:58:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42dd6d0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b18d0 con 0x7f42d8141af0 2022-08-01T22:58:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42ce7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42d80b1460 0x7f42d81413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58672/0 (socket says 172.21.15.32:58672) 2022-08-01T22:58:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42ce7fc700 1 -- 172.21.15.32:0/934709859 learned_addr learned my addr 172.21.15.32:0/934709859 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:58:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d77fe700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 0x7f42d80b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912061178 0 0) 0x7f42d80a7960 con 0x7f42d80b1460 2022-08-01T22:58:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b8003260 con 0x7f42d80b1460 2022-08-01T22:58:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325872579 0 0) 0x7f42d80b07e0 con 0x7f42d80b2050 2022-08-01T22:58:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d80a7960 con 0x7f42d80b2050 2022-08-01T22:58:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1087412563 0 0) 0x7f42b8003260 con 0x7f42d80b1460 2022-08-01T22:58:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 msgr2=0x7f42d80b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 0x7f42d80b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 >> v1:172.21.15.55:6789/0 conn(0x7f42d80b2050 legacy=0x7f42d80a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80b0a80 con 0x7f42d80b1460 2022-08-01T22:58:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f42c0001660 con 0x7f42d80b1460 2022-08-01T22:58:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 >> v1:172.21.15.32:6789/0 conn(0x7f42d80b1460 legacy=0x7f42d81413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 0x7f42b8003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b8003e20 0x7f42b8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42b80049f0 0x7f42b8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b18d0 con 0x7f42b8003e20 2022-08-01T22:58:16.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b800b800 con 0x7f42b80049f0 2022-08-01T22:58:16.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b800b9c0 con 0x7f42d8141af0 2022-08-01T22:58:16.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42ce7fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b8003e20 0x7f42b8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d6ffd700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 0x7f42b8003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.011+0000 7f42d77fe700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42b80049f0 0x7f42b8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42ce7fc700 1 -- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 msgr2=0x7f42b8003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:16.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42ce7fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42d8141af0 0x7f42b8003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42ce7fc700 1 -- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42b80049f0 msgr2=0x7f42b8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:16.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42ce7fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42b80049f0 0x7f42b8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42ce7fc700 1 -- 172.21.15.32:0/934709859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b8003260 con 0x7f42b8003e20 2022-08-01T22:58:16.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42d77fe700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42b80049f0 0x7f42b8004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:58:16.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42ce7fc700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b8003e20 0x7f42b8004500 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f42c0001430 tx=0x7f42d80b2b90).ready entity=mon.0 client_cookie=3f0ba38f2e1079f3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42c00071a0 con 0x7f42b8003e20 2022-08-01T22:58:16.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0007300 con 0x7f42b8003e20 2022-08-01T22:58:16.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42d67fc700 1 -- 172.21.15.32:0/934709859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42c000c8f0 con 0x7f42b8003e20 2022-08-01T22:58:16.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 -- 172.21.15.32:0/934709859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b8003e20 msgr2=0x7f42b8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/934709859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b8003e20 0x7f42b8004500 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f42c0001430 tx=0x7f42d80b2b90).stop 2022-08-01T22:58:16.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42d77fe700 1 -- 172.21.15.32:0/934709859 reap_dead start 2022-08-01T22:58:16.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 -- 172.21.15.32:0/934709859 shutdown_connections 2022-08-01T22:58:16.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 -- 172.21.15.32:0/934709859 >> 172.21.15.32:0/934709859 conn(0x7f42d805f840 msgr2=0x7f42d805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 -- 172.21.15.32:0/934709859 shutdown_connections 2022-08-01T22:58:16.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 -- 172.21.15.32:0/934709859 wait complete. 2022-08-01T22:58:16.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 Processor -- start 2022-08-01T22:58:16.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.015+0000 7f42dd6d0700 1 -- start start 2022-08-01T22:58:16.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42b8003e20 0x7f42d81493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42d80ac020 0x7f42d815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 0x7f42d8149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80abd30 con 0x7f42d815a710 2022-08-01T22:58:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d814a8c0 con 0x7f42d80ac020 2022-08-01T22:58:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b3640 con 0x7f42b8003e20 2022-08-01T22:58:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 0x7f42d8149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 0x7f42d8149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42912/0 (socket says 172.21.15.32:42912) 2022-08-01T22:58:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 -- 172.21.15.32:0/3363910914 learned_addr learned my addr 172.21.15.32:0/3363910914 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d6ffd700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42b8003e20 0x7f42d81493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42ce7fc700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42d80ac020 0x7f42d815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 -- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42b8003e20 msgr2=0x7f42d81493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42b8003e20 0x7f42d81493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 -- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42d80ac020 msgr2=0x7f42d815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42d80ac020 0x7f42d815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 -- 172.21.15.32:0/3363910914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80b38e0 con 0x7f42d815a710 2022-08-01T22:58:16.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d77fe700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 0x7f42d8149b30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f42d000b000 tx=0x7f42d00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42d0019070 con 0x7f42d815a710 2022-08-01T22:58:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000f9e0 con 0x7f42d815a710 2022-08-01T22:58:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d80b3b80 con 0x7f42d815a710 2022-08-01T22:58:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42d000fd40 con 0x7f42d815a710 2022-08-01T22:58:16.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d81498c0 con 0x7f42d815a710 2022-08-01T22:58:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d80b3d40 con 0x7f42d815a710 2022-08-01T22:58:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f42d0014440 con 0x7f42d815a710 2022-08-01T22:58:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.019+0000 7f42d4ff9700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42bc053980 0x7f42bc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.023+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f42d006cbd0 con 0x7f42d815a710 2022-08-01T22:58:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.023+0000 7f42d6ffd700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42bc053980 0x7f42bc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.023+0000 7f42d6ffd700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42bc053980 0x7f42bc055e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f42c8003000 tx=0x7f42c8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.031+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f42d0043820 con 0x7f42d815a710 2022-08-01T22:58:16.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.155+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f42d80b0d20 con 0x7f42d815a710 2022-08-01T22:58:16.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.155+0000 7f42d4ff9700 1 -- 172.21.15.32:0/3363910914 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f42d0046e40 con 0x7f42d815a710 2022-08-01T22:58:16.161 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:58:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42bc053980 msgr2=0x7f42bc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42bc053980 0x7f42bc055e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f42c8003000 tx=0x7f42c8009040).stop 2022-08-01T22:58:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 msgr2=0x7f42d8149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 0x7f42d8149b30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f42d000b000 tx=0x7f42d00091d0).stop 2022-08-01T22:58:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 shutdown_connections 2022-08-01T22:58:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42bc053980 0x7f42bc055e40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d815a710 0x7f42d8149b30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42d80ac020 0x7f42d815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 --2- 172.21.15.32:0/3363910914 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42b8003e20 0x7f42d81493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 >> 172.21.15.32:0/3363910914 conn(0x7f42d805f840 msgr2=0x7f42d813fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 shutdown_connections 2022-08-01T22:58:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.159+0000 7f42dd6d0700 1 -- 172.21.15.32:0/3363910914 wait complete. 2022-08-01T22:58:16.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:58:16.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 Processor -- start 2022-08-01T22:58:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- start start 2022-08-01T22:58:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99300b4820 0x7f99300b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9930143cf0 con 0x7f9930146ae0 2022-08-01T22:58:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99301476a0 con 0x7f99300b4820 2022-08-01T22:58:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9930143f90 con 0x7f99300b4310 2022-08-01T22:58:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99300b4820 0x7f99300b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99300b4820 0x7f99300b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42920/0 (socket says 172.21.15.32:42920) 2022-08-01T22:58:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 -- 172.21.15.32:0/2698456508 learned_addr learned my addr 172.21.15.32:0/2698456508 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992dd9b700 1 --2- 172.21.15.32:0/2698456508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 -- 172.21.15.32:0/2698456508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 msgr2=0x7f9930140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 --2- 172.21.15.32:0/2698456508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 -- 172.21.15.32:0/2698456508 >> v1:172.21.15.55:6789/0 conn(0x7f9930146ae0 legacy=0x7f99301403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 -- 172.21.15.32:0/2698456508 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300b0e20 con 0x7f99300b4820 2022-08-01T22:58:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992ed9d700 1 --2- 172.21.15.32:0/2698456508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99300b5060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f99280032f0 tx=0x7f9928009290).ready entity=mon.0 client_cookie=6d85d07f2260fa4c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992d59a700 1 -- 172.21.15.32:0/2698456508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992800b8a0 con 0x7f99300b4820 2022-08-01T22:58:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992d59a700 1 -- 172.21.15.32:0/2698456508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800ba00 con 0x7f99300b4820 2022-08-01T22:58:16.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f992d59a700 1 -- 172.21.15.32:0/2698456508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992800bb70 con 0x7f99300b4820 2022-08-01T22:58:16.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- 172.21.15.32:0/2698456508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 msgr2=0x7f99300b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 --2- 172.21.15.32:0/2698456508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99300b5060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f99280032f0 tx=0x7f9928009290).stop 2022-08-01T22:58:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- 172.21.15.32:0/2698456508 shutdown_connections 2022-08-01T22:58:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 --2- 172.21.15.32:0/2698456508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99300b5060 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 --2- 172.21.15.32:0/2698456508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- 172.21.15.32:0/2698456508 >> 172.21.15.32:0/2698456508 conn(0x7f993005c620 msgr2=0x7f993005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- 172.21.15.32:0/2698456508 shutdown_connections 2022-08-01T22:58:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.283+0000 7f99348e4700 1 -- 172.21.15.32:0/2698456508 wait complete. 2022-08-01T22:58:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 Processor -- start 2022-08-01T22:58:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 -- start start 2022-08-01T22:58:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99301444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9930146ae0 0x7f9930152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930152680 con 0x7f99300b4820 2022-08-01T22:58:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300adfe0 con 0x7f9930146ae0 2022-08-01T22:58:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.287+0000 7f99348e4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300add40 con 0x7f99300b4310 2022-08-01T22:58:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99301444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99301444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42924/0 (socket says 172.21.15.32:42924) 2022-08-01T22:58:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 -- 172.21.15.32:0/4010183056 learned_addr learned my addr 172.21.15.32:0/4010183056 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992e59c700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992ed9d700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9930146ae0 0x7f9930152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 -- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 msgr2=0x7f9930142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 -- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9930146ae0 msgr2=0x7f9930152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9930146ae0 0x7f9930152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 -- 172.21.15.32:0/4010183056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928009530 con 0x7f99300b4820 2022-08-01T22:58:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f992dd9b700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99301444d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f99300ae170 tx=0x7f9924006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9924010780 con 0x7f99300b4820 2022-08-01T22:58:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240108e0 con 0x7f99300b4820 2022-08-01T22:58:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99301528e0 con 0x7f99300b4820 2022-08-01T22:58:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9924010c40 con 0x7f99300b4820 2022-08-01T22:58:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99300ae920 con 0x7f99300b4820 2022-08-01T22:58:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300574e0 con 0x7f99300b4820 2022-08-01T22:58:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f992401c460 con 0x7f99300b4820 2022-08-01T22:58:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.291+0000 7f991f7fe700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f990c053a70 0x7f990c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.295+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f992406c3e0 con 0x7f99300b4820 2022-08-01T22:58:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.295+0000 7f992e59c700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f990c053a70 0x7f990c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.295+0000 7f992e59c700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f990c053a70 0x7f990c055f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f992000d000 tx=0x7f9920007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.299+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9924043140 con 0x7f99300b4820 2022-08-01T22:58:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9930069a70 con 0x7f99300b4820 2022-08-01T22:58:16.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f991f7fe700 1 -- 172.21.15.32:0/4010183056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9924046760 con 0x7f99300b4820 2022-08-01T22:58:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:58:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f990c053a70 msgr2=0x7f990c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f990c053a70 0x7f990c055f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f992000d000 tx=0x7f9920007b90).stop 2022-08-01T22:58:16.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 msgr2=0x7f99301444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99301444d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f99300ae170 tx=0x7f9924006df0).stop 2022-08-01T22:58:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 shutdown_connections 2022-08-01T22:58:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f990c053a70 0x7f990c055f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9930146ae0 0x7f9930152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99300b4820 0x7f99301444d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 --2- 172.21.15.32:0/4010183056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99300b4310 0x7f9930142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.427+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 >> 172.21.15.32:0/4010183056 conn(0x7f993005c620 msgr2=0x7f993013b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.431+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 shutdown_connections 2022-08-01T22:58:16.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.431+0000 7f99348e4700 1 -- 172.21.15.32:0/4010183056 wait complete. 2022-08-01T22:58:16.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:58:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.551+0000 7fe163fff700 1 Processor -- start 2022-08-01T22:58:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.551+0000 7fe163fff700 1 -- start start 2022-08-01T22:58:16.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.551+0000 7fe163fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1640aadb0 0x7fe1640b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.551+0000 7fe163fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1640b2510 con 0x7fe1640b31f0 2022-08-01T22:58:16.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.551+0000 7fe163fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1641443e0 con 0x7fe1640b7500 2022-08-01T22:58:16.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.551+0000 7fe163fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1640b3700 con 0x7fe1640aadb0 2022-08-01T22:58:16.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe15a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1640b31f0 0x7fe164064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58706/0 (socket says 172.21.15.32:58706) 2022-08-01T22:58:16.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe15a7fc700 1 -- 172.21.15.32:0/3129932077 learned_addr learned my addr 172.21.15.32:0/3129932077 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:58:16.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1637fe700 1 --2- 172.21.15.32:0/3129932077 >> v2:172.21.15.55:3300/0 conn(0x7fe1640aadb0 0x7fe1640b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1627fc700 1 -- 172.21.15.32:0/3129932077 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792681153 0 0) 0x7fe1640b2510 con 0x7fe1640b31f0 2022-08-01T22:58:16.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1627fc700 1 -- 172.21.15.32:0/3129932077 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe148003260 con 0x7fe1640b31f0 2022-08-01T22:58:16.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1637fe700 1 -- 172.21.15.32:0/3129932077 >> v1:172.21.15.55:6789/0 conn(0x7fe1640b7500 legacy=0x7fe16413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1637fe700 1 -- 172.21.15.32:0/3129932077 >> v1:172.21.15.32:6789/0 conn(0x7fe1640b31f0 legacy=0x7fe164064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1637fe700 1 -- 172.21.15.32:0/3129932077 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640b2740 con 0x7fe1640aadb0 2022-08-01T22:58:16.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1637fe700 1 --2- 172.21.15.32:0/3129932077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640b78e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fe15c0032f0 tx=0x7fe15c0091d0).ready entity=mon.1 client_cookie=a1a9838bf9531bd6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1627fc700 1 -- 172.21.15.32:0/3129932077 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15c00b6c0 con 0x7fe1640aadb0 2022-08-01T22:58:16.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1627fc700 1 -- 172.21.15.32:0/3129932077 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00b820 con 0x7fe1640aadb0 2022-08-01T22:58:16.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe1627fc700 1 -- 172.21.15.32:0/3129932077 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe15c00b990 con 0x7fe1640aadb0 2022-08-01T22:58:16.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 -- 172.21.15.32:0/3129932077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 msgr2=0x7fe1640b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 --2- 172.21.15.32:0/3129932077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640b78e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fe15c0032f0 tx=0x7fe15c0091d0).stop 2022-08-01T22:58:16.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 -- 172.21.15.32:0/3129932077 shutdown_connections 2022-08-01T22:58:16.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 --2- 172.21.15.32:0/3129932077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640b78e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 -- 172.21.15.32:0/3129932077 >> 172.21.15.32:0/3129932077 conn(0x7fe16405f840 msgr2=0x7fe16405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 -- 172.21.15.32:0/3129932077 shutdown_connections 2022-08-01T22:58:16.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.555+0000 7fe163fff700 1 -- 172.21.15.32:0/3129932077 wait complete. 2022-08-01T22:58:16.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 Processor -- start 2022-08-01T22:58:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 -- start start 2022-08-01T22:58:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 0x7fe1640b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1640b7500 0x7fe1640b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640ad220 con 0x7fe1640b31f0 2022-08-01T22:58:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640ae720 con 0x7fe1640aadb0 2022-08-01T22:58:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe163fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640af6f0 con 0x7fe1640b7500 2022-08-01T22:58:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 0x7fe1640b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 0x7fe1640b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42938/0 (socket says 172.21.15.32:42938) 2022-08-01T22:58:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 -- 172.21.15.32:0/1361945294 learned_addr learned my addr 172.21.15.32:0/1361945294 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe162ffd700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe1637fe700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1640b7500 0x7fe1640b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 -- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1640b7500 msgr2=0x7fe1640b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1640b7500 0x7fe1640b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 -- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 msgr2=0x7fe1640aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.559+0000 7fe15a7fc700 1 -- 172.21.15.32:0/1361945294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009470 con 0x7fe1640b31f0 2022-08-01T22:58:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe15a7fc700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 0x7fe1640b1640 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fe14c001110 tx=0x7fe14c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14c0070d0 con 0x7fe1640b31f0 2022-08-01T22:58:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c007230 con 0x7fe1640b31f0 2022-08-01T22:58:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1640ae280 con 0x7fe1640b31f0 2022-08-01T22:58:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14c005610 con 0x7fe1640b31f0 2022-08-01T22:58:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1640ad450 con 0x7fe1640b31f0 2022-08-01T22:58:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1640acff0 con 0x7fe1640b31f0 2022-08-01T22:58:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe14c010070 con 0x7fe1640b31f0 2022-08-01T22:58:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe160ff9700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053a70 0x7fe13c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.563+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe14c06b790 con 0x7fe1640b31f0 2022-08-01T22:58:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.567+0000 7fe162ffd700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053a70 0x7fe13c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.567+0000 7fe162ffd700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053a70 0x7fe13c055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fe15400d000 tx=0x7fe154007be0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.567+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe14c042440 con 0x7fe1640b31f0 2022-08-01T22:58:16.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1640ae410 con 0x7fe1640b31f0 2022-08-01T22:58:16.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe160ff9700 1 -- 172.21.15.32:0/1361945294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe14c045a60 con 0x7fe1640b31f0 2022-08-01T22:58:16.700 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:58:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053a70 msgr2=0x7fe13c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053a70 0x7fe13c055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fe15400d000 tx=0x7fe154007be0).stop 2022-08-01T22:58:16.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 msgr2=0x7fe1640b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 0x7fe1640b1640 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fe14c001110 tx=0x7fe14c007ad0).stop 2022-08-01T22:58:16.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 shutdown_connections 2022-08-01T22:58:16.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053a70 0x7fe13c055f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1640b7500 0x7fe1640b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1640b31f0 0x7fe1640b1640 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 --2- 172.21.15.32:0/1361945294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1640aadb0 0x7fe1640aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 >> 172.21.15.32:0/1361945294 conn(0x7fe16405f840 msgr2=0x7fe16405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 shutdown_connections 2022-08-01T22:58:16.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.695+0000 7fe163fff700 1 -- 172.21.15.32:0/1361945294 wait complete. 2022-08-01T22:58:16.712 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:58:16.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:58:16.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 Processor -- start 2022-08-01T22:58:16.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 -- start start 2022-08-01T22:58:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b01a0 0x7faa580a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa580afd60 0x7faa580a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa580a7250 con 0x7faa580b2710 2022-08-01T22:58:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa580a7480 con 0x7faa580afd60 2022-08-01T22:58:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa57fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa580a7720 con 0x7faa580b01a0 2022-08-01T22:58:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa577fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa580afd60 0x7faa580a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa56ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa580b2710 0x7faa5814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58720/0 (socket says 172.21.15.32:58720) 2022-08-01T22:58:16.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa56ffd700 1 -- 172.21.15.32:0/355361539 learned_addr learned my addr 172.21.15.32:0/355361539 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:58:16.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.807+0000 7faa577fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa580afd60 0x7faa580a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42948/0 (socket says 172.21.15.32:42948) 2022-08-01T22:58:16.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa567fc700 1 --2- 172.21.15.32:0/355361539 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b01a0 0x7faa580a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa577fe700 1 -- 172.21.15.32:0/355361539 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b01a0 msgr2=0x7faa580a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:58:16.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa577fe700 1 --2- 172.21.15.32:0/355361539 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b01a0 0x7faa580a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa577fe700 1 -- 172.21.15.32:0/355361539 >> v1:172.21.15.32:6789/0 conn(0x7faa580b2710 legacy=0x7faa5814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa577fe700 1 -- 172.21.15.32:0/355361539 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580a79c0 con 0x7faa580afd60 2022-08-01T22:58:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa577fe700 1 --2- 172.21.15.32:0/355361539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580afd60 0x7faa580a8a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7faa50002f70 tx=0x7faa50006df0).ready entity=mon.0 client_cookie=f9834b874f9dfdbe server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa55ffb700 1 -- 172.21.15.32:0/355361539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa5000d570 con 0x7faa580afd60 2022-08-01T22:58:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa55ffb700 1 -- 172.21.15.32:0/355361539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa50000e90 con 0x7faa580afd60 2022-08-01T22:58:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa55ffb700 1 -- 172.21.15.32:0/355361539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa5000d8c0 con 0x7faa580afd60 2022-08-01T22:58:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 -- 172.21.15.32:0/355361539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580afd60 msgr2=0x7faa580a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 --2- 172.21.15.32:0/355361539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580afd60 0x7faa580a8a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7faa50002f70 tx=0x7faa50006df0).stop 2022-08-01T22:58:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 -- 172.21.15.32:0/355361539 shutdown_connections 2022-08-01T22:58:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 --2- 172.21.15.32:0/355361539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580afd60 0x7faa580a8a30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 --2- 172.21.15.32:0/355361539 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b01a0 0x7faa580a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 -- 172.21.15.32:0/355361539 >> 172.21.15.32:0/355361539 conn(0x7faa58060310 msgr2=0x7faa580b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 -- 172.21.15.32:0/355361539 shutdown_connections 2022-08-01T22:58:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 -- 172.21.15.32:0/355361539 wait complete. 2022-08-01T22:58:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 Processor -- start 2022-08-01T22:58:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.811+0000 7faa57fff700 1 -- start start 2022-08-01T22:58:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa580afd60 0x7faa58143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 0x7faa5814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b2710 0x7faa5814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58156650 con 0x7faa580b01a0 2022-08-01T22:58:16.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5814caa0 con 0x7faa580afd60 2022-08-01T22:58:16.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa580b34b0 con 0x7faa580b2710 2022-08-01T22:58:16.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 0x7faa5814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 0x7faa5814c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42952/0 (socket says 172.21.15.32:42952) 2022-08-01T22:58:16.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 -- 172.21.15.32:0/2617703985 learned_addr learned my addr 172.21.15.32:0/2617703985 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:58:16.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa56ffd700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa580afd60 0x7faa58143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa577fe700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b2710 0x7faa5814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 -- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b2710 msgr2=0x7faa5814c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b2710 0x7faa5814c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 -- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa580afd60 msgr2=0x7faa58143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa580afd60 0x7faa58143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:16.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 -- 172.21.15.32:0/2617703985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5000b180 con 0x7faa580b01a0 2022-08-01T22:58:16.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa567fc700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 0x7faa5814c0c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7faa5814cbe0 tx=0x7faa48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa48013620 con 0x7faa580b01a0 2022-08-01T22:58:16.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa48013780 con 0x7faa580b01a0 2022-08-01T22:58:16.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa58156840 con 0x7faa580b01a0 2022-08-01T22:58:16.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa48013ae0 con 0x7faa580b01a0 2022-08-01T22:58:16.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa58144510 con 0x7faa580b01a0 2022-08-01T22:58:16.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa580574e0 con 0x7faa580b01a0 2022-08-01T22:58:16.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.815+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faa48013c40 con 0x7faa580b01a0 2022-08-01T22:58:16.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.819+0000 7faa47fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa34053b30 0x7faa34055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:58:16.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.819+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faa4806bf50 con 0x7faa580b01a0 2022-08-01T22:58:16.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.819+0000 7faa56ffd700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa34053b30 0x7faa34055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:58:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.819+0000 7faa56ffd700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa34053b30 0x7faa34055ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7faa4c00d000 tx=0x7faa4c007be0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:58:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.827+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa48042cb0 con 0x7faa580b01a0 2022-08-01T22:58:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.987+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa58069a70 con 0x7faa580b01a0 2022-08-01T22:58:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa47fff700 1 -- 172.21.15.32:0/2617703985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7faa480462d0 con 0x7faa580b01a0 2022-08-01T22:58:16.995 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:58:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:58:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa34053b30 msgr2=0x7faa34055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa34053b30 0x7faa34055ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7faa4c00d000 tx=0x7faa4c007be0).stop 2022-08-01T22:58:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 msgr2=0x7faa5814c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:58:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 0x7faa5814c0c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7faa5814cbe0 tx=0x7faa48008040).stop 2022-08-01T22:58:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 shutdown_connections 2022-08-01T22:58:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faa34053b30 0x7faa34055ff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faa580b2710 0x7faa5814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa580b01a0 0x7faa5814c0c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 --2- 172.21.15.32:0/2617703985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa580afd60 0x7faa58143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:58:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 >> 172.21.15.32:0/2617703985 conn(0x7faa58060310 msgr2=0x7faa580af480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:58:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 shutdown_connections 2022-08-01T22:58:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:58:16.991+0000 7faa57fff700 1 -- 172.21.15.32:0/2617703985 wait complete. 2022-08-01T22:58:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:58:17.310 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:57:45.324157+0000', 'modified': '2022-08-01T22:57:47.120762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5241': {'gid': 5241, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/505560818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 505560818}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 505560818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:58:17.311 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:58:17.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T22:58:17.434 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5241,"want_state":"up:active","state":"up:active","rank_uptime":32.097301950000002,"mdsmap_epoch":18,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":36.416096940999999} 2022-08-01T22:58:17.435 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5241, "mdsmap_epoch": 18, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 32.09730195, "state": "up:active", "uptime": 36.416096941, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:58:17.435 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:58:17.436 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T22:58:17.437 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T22:58:17.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.438 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:17.447 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:58:17.505 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:58:17.506 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:58:17.518 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:58:17.519 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:58:17.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.520 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:58:17.571 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:58:17.572 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:17.575 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:58:17.592 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:58:17.593 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:58:17.650 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:58:17.652 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:58:17.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.655 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:58:17.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T22:58:17.649+0000 7f4c7ccd3080 -1 init, newargv = 0x559e57dc7980 newargc=9 2022-08-01T22:58:17.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22602]: starting ceph client 2022-08-01T22:58:17.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22602]: starting fuse 2022-08-01T22:58:17.664 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T22:58:17.665 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:58:17.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.667 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T22:58:17.667 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T22:58:17.668 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T22:58:17.669 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:58:17.670 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T22:58:17.670 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:58:17.671 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T22:58:17.672 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T22:58:17.673 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T22:58:17.674 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:58:17.674 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T22:58:17.675 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T22:58:17.676 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T22:58:17.677 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T22:58:17.678 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:58:17.678 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T22:58:17.679 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:58:17.680 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:58:17.681 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T22:58:17.682 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:58:17.683 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:58:17.684 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T22:58:17.685 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:58:17.728 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22602.asok 2022-08-01T22:58:17.733 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22602.asok 2022-08-01T22:58:17.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.734 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22602.asok status 2022-08-01T22:58:17.870 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T22:58:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T22:58:17.871 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:58:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:58:17.872 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T22:58:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T22:58:17.873 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T22:58:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22602", 2022-08-01T22:58:17.874 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T22:58:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:58:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T22:58:17.875 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T22:58:17.876 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 5147, 2022-08-01T22:58:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T22:58:17.877 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T22:58:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T22:58:17.878 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 5147 2022-08-01T22:58:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:58:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:58:17.879 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:58:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:58:17.880 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1760061730 2022-08-01T22:58:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T22:58:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:58:17.881 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:58:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:58:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:58:17.882 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1760061730 2022-08-01T22:58:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:58:17.883 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.5147 172.21.15.130:0/1760061730", 2022-08-01T22:58:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/1760061730", 2022-08-01T22:58:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T22:58:17.884 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 18, 2022-08-01T22:58:17.885 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 45, 2022-08-01T22:58:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:58:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T22:58:17.886 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T22:58:17.887 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T22:58:17.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.889 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:58:17.890 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:17.899 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:58:17.900 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:58:17.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:58:17.901 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:58:23.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:23.571 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:23.574 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:23.576 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:53.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:53.683 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:53.686 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:58:53.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:06.015 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T22:59:06.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:06.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:06.058 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:06.079 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:06.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T22:59:06.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:06.101 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:06.102 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:06.132 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:59:06.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:59:06.134 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:59:06.134 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T22:59:06.135 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:06.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22602]: fuse finished with error 0 and tester_r 0 2022-08-01T22:59:06.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:06.152 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:06.152 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:06.159 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T22:59:06.160 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T22:59:06.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:59:06.278 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:06.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:06.287 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:06.287 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:59:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T22:59:06.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:59:06.296 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T22:59:06.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:59:06.297 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:59:06.304 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T22:59:06.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:59:06.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T22:59:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 Processor -- start 2022-08-01T22:59:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- start start 2022-08-01T22:59:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02d80b6710 0x7f02d80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f02d8061ec0 0x7f02d8062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d8144240 con 0x7f02d80b6710 2022-08-01T22:59:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d80b6fe0 con 0x7f02d8061ec0 2022-08-01T22:59:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8143d00 con 0x7f02d80b4550 2022-08-01T22:59:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02d80b6710 0x7f02d80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02d80b6710 0x7f02d80b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42960/0 (socket says 172.21.15.32:42960) 2022-08-01T22:59:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 -- 172.21.15.32:0/908580025 learned_addr learned my addr 172.21.15.32:0/908580025 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dde29700 1 --2- 172.21.15.32:0/908580025 >> v2:172.21.15.55:3300/0 conn(0x7f02d8061ec0 0x7f02d8062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dce27700 1 --2- 172.21.15.32:0/908580025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 -- 172.21.15.32:0/908580025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 msgr2=0x7f02d80b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 --2- 172.21.15.32:0/908580025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d80b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 -- 172.21.15.32:0/908580025 >> v2:172.21.15.55:3300/0 conn(0x7f02d8061ec0 msgr2=0x7f02d8062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 --2- 172.21.15.32:0/908580025 >> v2:172.21.15.55:3300/0 conn(0x7f02d8061ec0 0x7f02d8062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 -- 172.21.15.32:0/908580025 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8143fa0 con 0x7f02d80b6710 2022-08-01T22:59:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dde29700 1 --2- 172.21.15.32:0/908580025 >> v2:172.21.15.55:3300/0 conn(0x7f02d8061ec0 0x7f02d8062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02dd628700 1 --2- 172.21.15.32:0/908580025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d80b5d80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f02cc002f70 tx=0x7f02cc006eb0).ready entity=mon.0 client_cookie=e704686a08b86d7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02cbfff700 1 -- 172.21.15.32:0/908580025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02cc00d570 con 0x7f02d80b6710 2022-08-01T22:59:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02cbfff700 1 -- 172.21.15.32:0/908580025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc000e90 con 0x7f02d80b6710 2022-08-01T22:59:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02cbfff700 1 -- 172.21.15.32:0/908580025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02cc00d900 con 0x7f02d80b6710 2022-08-01T22:59:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- 172.21.15.32:0/908580025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 msgr2=0x7f02d80b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- 172.21.15.32:0/908580025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d80b5d80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f02cc002f70 tx=0x7f02cc006eb0).stop 2022-08-01T22:59:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- 172.21.15.32:0/908580025 shutdown_connections 2022-08-01T22:59:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- 172.21.15.32:0/908580025 >> v2:172.21.15.55:3300/0 conn(0x7f02d8061ec0 0x7f02d8062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- 172.21.15.32:0/908580025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d80b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 --2- 172.21.15.32:0/908580025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d80b5d80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- 172.21.15.32:0/908580025 >> 172.21.15.32:0/908580025 conn(0x7f02d805c620 msgr2=0x7f02d805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- 172.21.15.32:0/908580025 shutdown_connections 2022-08-01T22:59:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.415+0000 7f02df88c700 1 -- 172.21.15.32:0/908580025 wait complete. 2022-08-01T22:59:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 Processor -- start 2022-08-01T22:59:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 -- start start 2022-08-01T22:59:06.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02d8061ec0 0x7f02d8126420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d8140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d8140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8152330 con 0x7f02d80b6710 2022-08-01T22:59:06.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8144640 con 0x7f02d8061ec0 2022-08-01T22:59:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02df88c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8129910 con 0x7f02d80b4550 2022-08-01T22:59:06.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d8140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dd628700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02d8061ec0 0x7f02d8126420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d8140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42968/0 (socket says 172.21.15.32:42968) 2022-08-01T22:59:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dd628700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02d8061ec0 0x7f02d8126420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52750/0 (socket says 172.21.15.32:52750) 2022-08-01T22:59:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 -- 172.21.15.32:0/2091741688 learned_addr learned my addr 172.21.15.32:0/2091741688 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dce27700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d8140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 -- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 msgr2=0x7f02d8140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d8140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 -- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02d8061ec0 msgr2=0x7f02d8126420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02d8061ec0 0x7f02d8126420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.419+0000 7f02dde29700 1 -- 172.21.15.32:0/2091741688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc00b1e0 con 0x7f02d80b6710 2022-08-01T22:59:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02dde29700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d8140d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f02d0006cc0 tx=0x7f02d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02d0011780 con 0x7f02d80b6710 2022-08-01T22:59:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d00118e0 con 0x7f02d80b6710 2022-08-01T22:59:06.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d8141250 con 0x7f02d80b6710 2022-08-01T22:59:06.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02d0011c40 con 0x7f02d80b6710 2022-08-01T22:59:06.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8129de0 con 0x7f02d80b6710 2022-08-01T22:59:06.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d812a220 con 0x7f02d80b6710 2022-08-01T22:59:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f02d0012420 con 0x7f02d80b6710 2022-08-01T22:59:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02ca7fc700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c0053b00 0x7f02c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f02d006c2c0 con 0x7f02d80b6710 2022-08-01T22:59:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.423+0000 7f02dd628700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c0053b00 0x7f02c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.427+0000 7f02dd628700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c0053b00 0x7f02c0055fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f02cc01c000 tx=0x7f02cc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.431+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02d0042e90 con 0x7f02d80b6710 2022-08-01T22:59:06.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.563+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f02d8061780 con 0x7f02d80b6710 2022-08-01T22:59:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.563+0000 7f02ca7fc700 1 -- 172.21.15.32:0/2091741688 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f02d00464b0 con 0x7f02d80b6710 2022-08-01T22:59:06.569 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:06.569 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:59:06.569 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 45, 2022-08-01T22:59:06.569 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T22:59:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T22:59:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T22:57:46.326950+0000", 2022-08-01T22:59:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:59:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:59:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T22:59:06.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T22:59:06.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T22:59:06.571 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T22:59:06.571 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T22:59:06.572 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T22:59:06.572 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T22:59:06.572 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:06.573 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T22:59:06.573 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T22:59:06.573 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T22:59:06.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T22:59:06.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T22:59:06.574 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 8, 2022-08-01T22:59:06.574 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T22:59:06.575 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T22:59:06.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T22:59:06.575 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T22:59:06.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T22:59:06.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T22:59:06.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T22:59:06.576 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T22:59:06.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:59:06.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:59:06.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:59:06.577 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:59:06.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:59:06.577 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:59:06.578 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:59:06.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:59:06.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T22:59:06.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T22:59:06.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T22:59:06.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T22:59:06.579 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T22:59:06.579 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:59:06.579 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:59:06.580 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:59:06.580 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:59:06.580 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:59:06.581 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:59:06.581 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:59:06.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.581 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T22:59:06.581 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:59:06.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:59:06.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:59:06.582 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:59:06.582 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:59:06.583 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:59:06.583 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:59:06.583 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:59:06.583 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:59:06.583 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:59:06.584 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:59:06.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:59:06.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:59:06.584 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:59:06.584 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:59:06.585 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:59:06.585 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:59:06.585 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:59:06.585 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:59:06.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:59:06.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:59:06.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:59:06.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:59:06.588 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:59:06.588 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:59:06.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:59:06.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.589 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:59:06.590 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:59:06.590 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:59:06.590 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:59:06.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:59:06.591 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:59:06.591 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:59:06.592 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:59:06.592 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:59:06.593 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:59:06.593 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:59:06.593 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:59:06.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T22:59:06.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.594 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:59:06.594 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T22:59:06.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T22:59:06.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T22:59:06.595 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T22:59:06.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:59:06.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:59:06.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:59:06.596 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:59:06.596 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:59:06.597 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:59:06.597 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:59:06.597 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:59:06.597 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:59:06.597 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:59:06.598 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:59:06.598 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:59:06.598 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:59:06.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:59:06.599 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:59:06.599 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:59:06.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:59:06.599 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:59:06.599 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:59:06.600 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:59:06.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.600 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T22:59:06.600 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:59:06.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:59:06.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:59:06.601 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:59:06.601 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:59:06.601 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:59:06.602 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:59:06.602 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:59:06.602 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:59:06.602 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:59:06.602 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:59:06.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:59:06.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:59:06.603 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:59:06.603 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:59:06.604 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:59:06.604 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:59:06.604 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:59:06.604 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:59:06.604 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:59:06.605 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:59:06.605 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:59:06.605 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:59:06.605 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:59:06.606 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:59:06.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:59:06.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.606 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:59:06.606 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:59:06.607 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:59:06.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:59:06.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:59:06.607 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:59:06.607 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:59:06.608 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:59:06.608 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:59:06.608 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:59:06.608 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:59:06.609 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:59:06.609 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:59:06.609 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T22:59:06.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.610 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2022-08-01T22:59:06.610 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T22:59:06.611 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:57:43.852752+0000", 2022-08-01T22:59:06.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:59:06.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:59:06.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:59:06.612 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:59:06.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:59:06.613 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:59:06.613 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:59:06.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:59:06.614 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:59:06.614 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:59:06.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:59:06.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:59:06.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:59:06.616 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:59:06.616 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:59:06.616 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:59:06.616 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:59:06.617 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:59:06.617 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:59:06.618 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:59:06.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.618 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "44", 2022-08-01T22:59:06.619 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:59:06.619 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:59:06.619 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:59:06.620 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:59:06.620 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:59:06.620 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:59:06.621 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:59:06.621 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:59:06.621 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:59:06.622 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:59:06.622 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:59:06.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:59:06.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:59:06.623 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:59:06.623 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:59:06.624 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:59:06.624 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:59:06.625 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:59:06.625 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:59:06.625 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:59:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:59:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:59:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:59:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:59:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:59:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:59:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:59:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:59:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:59:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:59:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:59:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:59:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:59:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:59:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:59:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:59:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:59:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:59:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T22:59:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T22:59:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T22:59:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:59:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T22:59:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T22:59:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 8, 2022-08-01T22:59:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T22:59:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:57:45.012782+0000", 2022-08-01T22:59:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:59:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:59:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:59:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:59:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:59:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:59:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:59:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:59:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:59:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:59:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:59:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:59:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:59:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:59:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:59:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:59:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:59:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:59:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:59:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:59:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "45", 2022-08-01T22:59:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:59:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:59:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:59:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:59:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:59:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:59:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:59:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:59:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:59:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:59:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:59:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:59:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:59:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:59:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:59:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:59:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:59:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:59:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:59:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:59:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:59:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:59:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:59:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:59:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:59:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:59:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:59:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:59:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:59:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:59:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:59:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:59:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:59:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:59:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:59:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:59:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:59:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:59:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:59:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T22:59:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T22:59:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T22:59:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:59:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T22:59:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T22:59:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T22:59:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T22:59:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T22:59:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T22:59:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T22:59:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T22:59:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T22:59:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:59:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T22:59:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2022-08-01T22:59:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T22:59:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T22:59:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T22:59:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T22:59:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T22:59:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T22:59:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T22:59:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T22:59:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T22:59:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T22:59:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T22:59:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T22:59:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:59:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T22:59:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2022-08-01T22:59:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T22:59:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T22:59:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T22:59:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T22:59:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T22:59:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T22:59:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T22:59:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T22:59:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T22:59:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T22:59:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T22:59:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T22:59:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:59:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T22:59:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T22:59:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T22:59:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T22:59:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T22:59:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T22:59:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T22:59:06.720 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.721 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.721 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.722 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T22:59:06.722 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.723 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T22:59:06.723 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:06.723 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.723 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.724 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T22:59:06.724 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T22:59:06.724 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T22:59:06.724 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T22:59:06.725 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.725 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.725 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.726 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:59:06.726 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T22:59:06.726 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.727 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.727 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.727 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.727 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.727 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.728 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.728 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:06.728 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.728 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.728 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T22:59:06.729 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T22:59:06.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.731 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.731 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T22:59:06.732 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.733 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T22:59:06.733 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.734 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.734 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T22:59:06.735 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T22:59:06.736 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.737 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.738 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T22:59:06.739 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.739 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T22:59:06.740 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:06.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.740 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T22:59:06.741 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T22:59:06.741 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T22:59:06.741 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T22:59:06.741 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.742 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.742 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.742 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.743 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:59:06.743 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T22:59:06.743 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.743 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.743 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.744 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.744 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.744 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.744 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.745 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:06.745 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.745 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.745 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T22:59:06.746 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T22:59:06.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.748 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T22:59:06.749 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T22:59:06.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.751 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T22:59:06.752 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T22:59:06.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.755 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T22:59:06.756 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T22:59:06.757 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:06.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.758 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T22:59:06.758 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T22:59:06.758 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T22:59:06.759 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T22:59:06.759 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.759 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.759 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.759 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.760 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:59:06.760 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T22:59:06.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.761 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.761 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.761 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.762 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.762 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.762 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:06.762 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.763 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.763 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T22:59:06.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T22:59:06.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.766 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T22:59:06.767 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T22:59:06.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T22:59:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.771 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T22:59:06.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.772 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T22:59:06.773 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T22:59:06.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:06.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.775 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T22:59:06.775 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T22:59:06.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T22:59:06.776 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T22:59:06.776 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.776 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.777 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.777 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:59:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T22:59:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:06.778 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:06.779 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 41, 2022-08-01T22:59:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:06.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T22:59:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T22:59:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T22:59:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T22:59:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T22:59:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T22:59:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:06.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T22:59:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T22:59:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.792 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T22:59:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T22:59:06.793 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T22:59:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T22:59:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:06.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T22:59:06.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:59:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.796 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T22:59:06.797 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:59:06.799 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.800 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T22:59:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:59:06.803 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.804 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.805 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T22:59:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:59:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.807 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T22:59:06.808 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:59:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.809 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.810 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T22:59:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:59:06.811 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T22:59:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.813 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:59:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.814 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T22:59:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:59:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:06.816 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T22:59:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.817 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T22:59:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T22:59:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T22:59:06.818 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T22:59:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T22:59:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T22:59:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T22:59:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T22:59:06.819 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:59:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T22:59:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T22:59:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:59:06.820 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T22:59:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:59:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:59:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T22:59:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T22:59:06.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T22:59:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:59:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:59:06.822 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T22:59:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:59:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:59:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T22:59:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:06.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T22:59:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T22:59:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T22:59:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T22:59:06.824 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T22:59:06.825 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:59:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c0053b00 msgr2=0x7f02c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c0053b00 0x7f02c0055fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f02cc01c000 tx=0x7f02cc015000).stop 2022-08-01T22:59:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 msgr2=0x7f02d8140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d8140d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f02d0006cc0 tx=0x7f02d0008040).stop 2022-08-01T22:59:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 shutdown_connections 2022-08-01T22:59:06.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c0053b00 0x7f02c0055fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d80b6710 0x7f02d8140d60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02d80b4550 0x7f02d8140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 --2- 172.21.15.32:0/2091741688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02d8061ec0 0x7f02d8126420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 >> 172.21.15.32:0/2091741688 conn(0x7f02d805c620 msgr2=0x7f02d8141400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:06.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 shutdown_connections 2022-08-01T22:59:06.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.567+0000 7f02df88c700 1 -- 172.21.15.32:0/2091741688 wait complete. 2022-08-01T22:59:06.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 Processor -- start 2022-08-01T22:59:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 -- start start 2022-08-01T22:59:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90fc148530 0x7f90fc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90fc0a7250 0x7f90fc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc0b04e0 con 0x7f90fc0b1f40 2022-08-01T22:59:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc0a9250 con 0x7f90fc0a7250 2022-08-01T22:59:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc0b0780 con 0x7f90fc148530 2022-08-01T22:59:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90fc0a7250 0x7f90fc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90fc0a7250 0x7f90fc1448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42976/0 (socket says 172.21.15.32:42976) 2022-08-01T22:59:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 -- 172.21.15.32:0/3023551064 learned_addr learned my addr 172.21.15.32:0/3023551064 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f90fb7fe700 1 --2- 172.21.15.32:0/3023551064 >> v2:172.21.15.55:3300/0 conn(0x7f90fc148530 0x7f90fc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f90faffd700 1 -- 172.21.15.32:0/3023551064 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503411984 0 0) 0x7f90fc0b04e0 con 0x7f90fc0b1f40 2022-08-01T22:59:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 -- 172.21.15.32:0/3023551064 >> v2:172.21.15.55:3300/0 conn(0x7f90fc148530 msgr2=0x7f90fc0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:06.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 --2- 172.21.15.32:0/3023551064 >> v2:172.21.15.55:3300/0 conn(0x7f90fc148530 0x7f90fc0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 -- 172.21.15.32:0/3023551064 >> v1:172.21.15.55:6789/0 conn(0x7f90fc0b1f40 legacy=0x7f90fc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 -- 172.21.15.32:0/3023551064 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc0b0a20 con 0x7f90fc0a7250 2022-08-01T22:59:06.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f9100c79700 1 --2- 172.21.15.32:0/3023551064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc1448b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f90fc1478c0 tx=0x7f90f4006eb0).ready entity=mon.0 client_cookie=c4196d5eb2545274 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:06.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f90faffd700 1 -- 172.21.15.32:0/3023551064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90f400d610 con 0x7f90fc0a7250 2022-08-01T22:59:06.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f90faffd700 1 -- 172.21.15.32:0/3023551064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4000e90 con 0x7f90fc0a7250 2022-08-01T22:59:06.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f90faffd700 1 -- 172.21.15.32:0/3023551064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90f400d960 con 0x7f90fc0a7250 2022-08-01T22:59:06.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 -- 172.21.15.32:0/3023551064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 msgr2=0x7f90fc1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.939+0000 7f91026dc700 1 --2- 172.21.15.32:0/3023551064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc1448b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f90fc1478c0 tx=0x7f90f4006eb0).stop 2022-08-01T22:59:06.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- 172.21.15.32:0/3023551064 shutdown_connections 2022-08-01T22:59:06.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 --2- 172.21.15.32:0/3023551064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc1448b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 --2- 172.21.15.32:0/3023551064 >> v2:172.21.15.55:3300/0 conn(0x7f90fc148530 0x7f90fc0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- 172.21.15.32:0/3023551064 >> 172.21.15.32:0/3023551064 conn(0x7f90fc05f840 msgr2=0x7f90fc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:06.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- 172.21.15.32:0/3023551064 shutdown_connections 2022-08-01T22:59:06.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- 172.21.15.32:0/3023551064 wait complete. 2022-08-01T22:59:06.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 Processor -- start 2022-08-01T22:59:06.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- start start 2022-08-01T22:59:06.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f90fc0b1f40 0x7f90fc0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90fc148530 0x7f90fc0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0af030 con 0x7f90fc0a7250 2022-08-01T22:59:06.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc149ee0 con 0x7f90fc148530 2022-08-01T22:59:06.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f91026dc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0ab1a0 con 0x7f90fc0b1f40 2022-08-01T22:59:06.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc14a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42978/0 (socket says 172.21.15.32:42978) 2022-08-01T22:59:06.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 -- 172.21.15.32:0/2906025559 learned_addr learned my addr 172.21.15.32:0/2906025559 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:06.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fb7fe700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f90fc0b1f40 0x7f90fc0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f9100c79700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90fc148530 0x7f90fc0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 -- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f90fc0b1f40 msgr2=0x7f90fc0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f90fc0b1f40 0x7f90fc0ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 -- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90fc148530 msgr2=0x7f90fc0abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90fc148530 0x7f90fc0abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:06.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 -- 172.21.15.32:0/2906025559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f400b1e0 con 0x7f90fc0a7250 2022-08-01T22:59:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.943+0000 7f90fbfff700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc14a8e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f90fc0aa240 tx=0x7f90e4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90e4012400 con 0x7f90fc0a7250 2022-08-01T22:59:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4012560 con 0x7f90fc0a7250 2022-08-01T22:59:06.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc14a110 con 0x7f90fc0a7250 2022-08-01T22:59:06.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90e40128c0 con 0x7f90fc0a7250 2022-08-01T22:59:06.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc14b7f0 con 0x7f90fc0a7250 2022-08-01T22:59:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc0ab2e0 con 0x7f90fc0a7250 2022-08-01T22:59:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.947+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f90e4019030 con 0x7f90fc0a7250 2022-08-01T22:59:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.951+0000 7f90f97fa700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f90e00539e0 0x7f90e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.951+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f90e406b280 con 0x7f90fc0a7250 2022-08-01T22:59:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.951+0000 7f90fb7fe700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f90e00539e0 0x7f90e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.951+0000 7f90fb7fe700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f90e00539e0 0x7f90e0055ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f90ec00b000 tx=0x7f90ec0091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:06.955+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f90e40420f0 con 0x7f90fc0a7250 2022-08-01T22:59:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.115+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90fc0b0cc0 con 0x7f90fc0a7250 2022-08-01T22:59:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.115+0000 7f90f97fa700 1 -- 172.21.15.32:0/2906025559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f90e4045710 con 0x7f90fc0a7250 2022-08-01T22:59:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:07.120 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:57:47.120762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5241":{"gid":5241,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/505560818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":505560818},{"type":"v1","addr":"172.21.15.32:6835","nonce":505560818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:59:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f90e00539e0 msgr2=0x7f90e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f90e00539e0 0x7f90e0055ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f90ec00b000 tx=0x7f90ec0091d0).stop 2022-08-01T22:59:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 msgr2=0x7f90fc14a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc14a8e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f90fc0aa240 tx=0x7f90e4007ac0).stop 2022-08-01T22:59:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 shutdown_connections 2022-08-01T22:59:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f90e00539e0 0x7f90e0055ea0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90fc148530 0x7f90fc0abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f90fc0b1f40 0x7f90fc0ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 --2- 172.21.15.32:0/2906025559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90fc0a7250 0x7f90fc14a8e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 >> 172.21.15.32:0/2906025559 conn(0x7f90fc05f840 msgr2=0x7f90fc0b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 shutdown_connections 2022-08-01T22:59:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.119+0000 7f91026dc700 1 -- 172.21.15.32:0/2906025559 wait complete. 2022-08-01T22:59:07.125 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2022-08-01T22:59:07.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T22:59:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f6784934700 1 Processor -- start 2022-08-01T22:59:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f6784934700 1 -- start start 2022-08-01T22:59:07.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f6784934700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f67800b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:07.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f6784934700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67800b2720 con 0x7f6780063c00 2022-08-01T22:59:07.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f6784934700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6780148330 con 0x7f67800b2340 2022-08-01T22:59:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f6784934700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6780061e10 con 0x7f6780060de0 2022-08-01T22:59:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f67800b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677f59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6780063c00 0x7f67801447c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58762/0 (socket says 172.21.15.32:58762) 2022-08-01T22:59:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677f59e700 1 -- 172.21.15.32:0/2029092865 learned_addr learned my addr 172.21.15.32:0/2029092865 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088773527 0 0) 0x7f6780148330 con 0x7f67800b2340 2022-08-01T22:59:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6760003260 con 0x7f67800b2340 2022-08-01T22:59:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823394135 0 0) 0x7f67800b2720 con 0x7f6780063c00 2022-08-01T22:59:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6780148330 con 0x7f6780063c00 2022-08-01T22:59:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677e59c700 1 -- 172.21.15.32:0/2029092865 >> v1:172.21.15.55:6789/0 conn(0x7f67800b2340 legacy=0x7f6780060190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677e59c700 1 -- 172.21.15.32:0/2029092865 >> v1:172.21.15.32:6789/0 conn(0x7f6780063c00 legacy=0x7f67801447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677e59c700 1 -- 172.21.15.32:0/2029092865 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678005fa50 con 0x7f6780060de0 2022-08-01T22:59:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677e59c700 1 --2- 172.21.15.32:0/2029092865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f67800b2c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f67780032f0 tx=0x7f67780091d0).ready entity=mon.2 client_cookie=bffea2c24ce765da server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677800b640 con 0x7f6780060de0 2022-08-01T22:59:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677800b7a0 con 0x7f6780060de0 2022-08-01T22:59:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.231+0000 7f677dd9b700 1 -- 172.21.15.32:0/2029092865 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677800b910 con 0x7f6780060de0 2022-08-01T22:59:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- 172.21.15.32:0/2029092865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 msgr2=0x7f67800b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 --2- 172.21.15.32:0/2029092865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f67800b2c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f67780032f0 tx=0x7f67780091d0).stop 2022-08-01T22:59:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- 172.21.15.32:0/2029092865 shutdown_connections 2022-08-01T22:59:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 --2- 172.21.15.32:0/2029092865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f67800b2c50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- 172.21.15.32:0/2029092865 >> 172.21.15.32:0/2029092865 conn(0x7f678005c620 msgr2=0x7f678005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- 172.21.15.32:0/2029092865 shutdown_connections 2022-08-01T22:59:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- 172.21.15.32:0/2029092865 wait complete. 2022-08-01T22:59:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 Processor -- start 2022-08-01T22:59:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- start start 2022-08-01T22:59:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f6780110590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6780063c00 0x7f67800aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 0x7f67800af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67800adb60 con 0x7f67800b2340 2022-08-01T22:59:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780157340 con 0x7f6780063c00 2022-08-01T22:59:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67801575d0 con 0x7f6780060de0 2022-08-01T22:59:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 0x7f67800af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 0x7f67800af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42994/0 (socket says 172.21.15.32:42994) 2022-08-01T22:59:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 -- 172.21.15.32:0/2023338065 learned_addr learned my addr 172.21.15.32:0/2023338065 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677e59c700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6780063c00 0x7f67800aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677ed9d700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f6780110590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:07.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 -- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 msgr2=0x7f6780110590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f6780110590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 -- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6780063c00 msgr2=0x7f67800aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6780063c00 0x7f67800aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 -- 172.21.15.32:0/2023338065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778009470 con 0x7f67800b2340 2022-08-01T22:59:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677e59c700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6780063c00 0x7f67800aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f677f59e700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 0x7f67800af300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f6770001110 tx=0x7f6770007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67700070d0 con 0x7f67800b2340 2022-08-01T22:59:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6770007230 con 0x7f67800b2340 2022-08-01T22:59:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6780157870 con 0x7f67800b2340 2022-08-01T22:59:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.235+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6770005610 con 0x7f67800b2340 2022-08-01T22:59:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67800ae470 con 0x7f67800b2340 2022-08-01T22:59:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678005fcf0 con 0x7f67800b2340 2022-08-01T22:59:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6770010070 con 0x7f67800b2340 2022-08-01T22:59:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f6777fff700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6764053b00 0x7f6764055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f677006b720 con 0x7f67800b2340 2022-08-01T22:59:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f677ed9d700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6764053b00 0x7f6764055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.239+0000 7f677ed9d700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6764053b00 0x7f6764055fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f676800b000 tx=0x7f6768009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.243+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67700423e0 con 0x7f67800b2340 2022-08-01T22:59:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:07.403+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f67800574e0 con 0x7f67800b2340 2022-08-01T22:59:08.139 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T22:59:08.139+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T22:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.147+0000 7f6777fff700 1 -- 172.21.15.32:0/2023338065 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6770018070 con 0x7f67800b2340 2022-08-01T22:59:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6764053b00 msgr2=0x7f6764055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6764053b00 0x7f6764055fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f676800b000 tx=0x7f6768009040).stop 2022-08-01T22:59:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 msgr2=0x7f67800af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 0x7f67800af300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f6770001110 tx=0x7f6770007a20).stop 2022-08-01T22:59:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 shutdown_connections 2022-08-01T22:59:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6764053b00 0x7f6764055fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67800b2340 0x7f67800af300 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6780063c00 0x7f67800aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 --2- 172.21.15.32:0/2023338065 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6780060de0 0x7f6780110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 >> 172.21.15.32:0/2023338065 conn(0x7f678005c620 msgr2=0x7f678005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 shutdown_connections 2022-08-01T22:59:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.151+0000 7f6784934700 1 -- 172.21.15.32:0/2023338065 wait complete. 2022-08-01T22:59:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T22:59:08.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:08.193 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T22:59:08.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.275+0000 7f4f0322a700 1 Processor -- start 2022-08-01T22:59:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.275+0000 7f4f0322a700 1 -- start start 2022-08-01T22:59:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4efc0b6a90 0x7f4efc0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0aa760 0x7f4efc13c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc0b6e70 con 0x7f4efc060aa0 2022-08-01T22:59:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4efc061370 con 0x7f4efc0b6a90 2022-08-01T22:59:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc061610 con 0x7f4efc0aa760 2022-08-01T22:59:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4efc0b6a90 0x7f4efc0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4efc0b6a90 0x7f4efc0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43010/0 (socket says 172.21.15.32:43010) 2022-08-01T22:59:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 -- 172.21.15.32:0/506269748 learned_addr learned my addr 172.21.15.32:0/506269748 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4efbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4efc060aa0 0x7f4efc0aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58786/0 (socket says 172.21.15.32:58786) 2022-08-01T22:59:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f017c7700 1 --2- 172.21.15.32:0/506269748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0aa760 0x7f4efc13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 -- 172.21.15.32:0/506269748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0aa760 msgr2=0x7f4efc13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 --2- 172.21.15.32:0/506269748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0aa760 0x7f4efc13c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 -- 172.21.15.32:0/506269748 >> v1:172.21.15.32:6789/0 conn(0x7f4efc060aa0 legacy=0x7f4efc0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 -- 172.21.15.32:0/506269748 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc0618b0 con 0x7f4efc0b6a90 2022-08-01T22:59:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f00fc6700 1 --2- 172.21.15.32:0/506269748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b6a90 0x7f4efc0acee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f4ef0002f70 tx=0x7f4ef0006e40).ready entity=mon.0 client_cookie=d84436caebc52601 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4efb7fe700 1 -- 172.21.15.32:0/506269748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ef000bd20 con 0x7f4efc0b6a90 2022-08-01T22:59:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4efb7fe700 1 -- 172.21.15.32:0/506269748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef0000e90 con 0x7f4efc0b6a90 2022-08-01T22:59:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4efb7fe700 1 -- 172.21.15.32:0/506269748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ef00155e0 con 0x7f4efc0b6a90 2022-08-01T22:59:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- 172.21.15.32:0/506269748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b6a90 msgr2=0x7f4efc0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 --2- 172.21.15.32:0/506269748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b6a90 0x7f4efc0acee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f4ef0002f70 tx=0x7f4ef0006e40).stop 2022-08-01T22:59:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- 172.21.15.32:0/506269748 shutdown_connections 2022-08-01T22:59:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 --2- 172.21.15.32:0/506269748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0aa760 0x7f4efc13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 --2- 172.21.15.32:0/506269748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b6a90 0x7f4efc0acee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- 172.21.15.32:0/506269748 >> 172.21.15.32:0/506269748 conn(0x7f4efc0b3610 msgr2=0x7f4efc0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- 172.21.15.32:0/506269748 shutdown_connections 2022-08-01T22:59:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.279+0000 7f4f0322a700 1 -- 172.21.15.32:0/506269748 wait complete. 2022-08-01T22:59:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 Processor -- start 2022-08-01T22:59:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 -- start start 2022-08-01T22:59:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 0x7f4efc0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4efc0aa760 0x7f4efc0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0b6a90 0x7f4efc0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc06af00 con 0x7f4efc060aa0 2022-08-01T22:59:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0ace90 con 0x7f4efc0aa760 2022-08-01T22:59:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f0322a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0adbe0 con 0x7f4efc0b6a90 2022-08-01T22:59:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f00fc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 0x7f4efc0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f00fc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 0x7f4efc0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43016/0 (socket says 172.21.15.32:43016) 2022-08-01T22:59:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f00fc6700 1 -- 172.21.15.32:0/3171111304 learned_addr learned my addr 172.21.15.32:0/3171111304 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4efbfff700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4efc0aa760 0x7f4efc0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.283+0000 7f4f017c7700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0b6a90 0x7f4efc0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f00fc6700 1 -- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0b6a90 msgr2=0x7f4efc0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f00fc6700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0b6a90 0x7f4efc0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f00fc6700 1 -- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4efc0aa760 msgr2=0x7f4efc0ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f00fc6700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4efc0aa760 0x7f4efc0ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f00fc6700 1 -- 172.21.15.32:0/3171111304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef000d4d0 con 0x7f4efc060aa0 2022-08-01T22:59:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f00fc6700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 0x7f4efc0b5280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f4efc13dc80 tx=0x7f4ef0018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ef0009030 con 0x7f4efc060aa0 2022-08-01T22:59:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef000a040 con 0x7f4efc060aa0 2022-08-01T22:59:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc0ad0c0 con 0x7f4efc060aa0 2022-08-01T22:59:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ef0015e00 con 0x7f4efc060aa0 2022-08-01T22:59:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc0ae040 con 0x7f4efc060aa0 2022-08-01T22:59:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ef0025030 con 0x7f4efc060aa0 2022-08-01T22:59:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc0ae200 con 0x7f4efc060aa0 2022-08-01T22:59:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4ef9ffb700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4edc053a10 0x7f4edc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f4ef006d210 con 0x7f4efc060aa0 2022-08-01T22:59:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.287+0000 7f4efbfff700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4edc053a10 0x7f4edc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.291+0000 7f4efbfff700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4edc053a10 0x7f4edc055ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f4ee800b000 tx=0x7f4ee8000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.291+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ef0043f80 con 0x7f4efc060aa0 2022-08-01T22:59:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.451+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4efc061b50 con 0x7f4efc060aa0 2022-08-01T22:59:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.451+0000 7f4ef9ffb700 1 -- 172.21.15.32:0/3171111304 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2130 (secure 0 0 0) 0x7f4ef0023090 con 0x7f4efc060aa0 2022-08-01T22:59:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:57:45.324157+0000","modified":"2022-08-01T22:59:08.141465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T22:59:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4edc053a10 msgr2=0x7f4edc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4edc053a10 0x7f4edc055ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f4ee800b000 tx=0x7f4ee8000bc0).stop 2022-08-01T22:59:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 msgr2=0x7f4efc0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 0x7f4efc0b5280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f4efc13dc80 tx=0x7f4ef0018000).stop 2022-08-01T22:59:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 shutdown_connections 2022-08-01T22:59:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4edc053a10 0x7f4edc055ed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4efc0b6a90 0x7f4efc0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4efc0aa760 0x7f4efc0ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 --2- 172.21.15.32:0/3171111304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc060aa0 0x7f4efc0b5280 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 >> 172.21.15.32:0/3171111304 conn(0x7f4efc0b3610 msgr2=0x7f4efc0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 shutdown_connections 2022-08-01T22:59:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.455+0000 7f4f0322a700 1 -- 172.21.15.32:0/3171111304 wait complete. 2022-08-01T22:59:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2022-08-01T22:59:08.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T22:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 Processor -- start 2022-08-01T22:59:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- start start 2022-08-01T22:59:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ea80b2710 0x7f5ea81401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea80b0270 0x7f5ea81406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea8067a10 con 0x7f5ea80ac880 2022-08-01T22:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea80ad740 con 0x7f5ea80b0270 2022-08-01T22:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea8067cb0 con 0x7f5ea80b2710 2022-08-01T22:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5eaca43700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ea80ac880 0x7f5ea8143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58798/0 (socket says 172.21.15.32:58798) 2022-08-01T22:59:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5eaca43700 1 -- 172.21.15.32:0/1668847658 learned_addr learned my addr 172.21.15.32:0/1668847658 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea80b0270 0x7f5ea81406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea7fff700 1 --2- 172.21.15.32:0/1668847658 >> v2:172.21.15.55:3300/0 conn(0x7f5ea80b2710 0x7f5ea81401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea6ffd700 1 -- 172.21.15.32:0/1668847658 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757042866 0 0) 0x7f5ea8067a10 con 0x7f5ea80ac880 2022-08-01T22:59:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea6ffd700 1 -- 172.21.15.32:0/1668847658 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e94003260 con 0x7f5ea80ac880 2022-08-01T22:59:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea6ffd700 1 -- 172.21.15.32:0/1668847658 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147311737 0 0) 0x7f5e94003260 con 0x7f5ea80ac880 2022-08-01T22:59:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea77fe700 1 -- 172.21.15.32:0/1668847658 >> v2:172.21.15.55:3300/0 conn(0x7f5ea80b2710 msgr2=0x7f5ea81401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea77fe700 1 --2- 172.21.15.32:0/1668847658 >> v2:172.21.15.55:3300/0 conn(0x7f5ea80b2710 0x7f5ea81401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea77fe700 1 -- 172.21.15.32:0/1668847658 >> v1:172.21.15.32:6789/0 conn(0x7f5ea80ac880 legacy=0x7f5ea8143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea77fe700 1 -- 172.21.15.32:0/1668847658 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8147160 con 0x7f5ea80b0270 2022-08-01T22:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea77fe700 1 --2- 172.21.15.32:0/1668847658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b0270 0x7f5ea81406d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f5ea0001b80 tx=0x7f5e9c006df0).ready entity=mon.0 client_cookie=e9734e21fc53bd6d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea6ffd700 1 -- 172.21.15.32:0/1668847658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea00019e0 con 0x7f5ea80b0270 2022-08-01T22:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea6ffd700 1 -- 172.21.15.32:0/1668847658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c000e90 con 0x7f5ea80b0270 2022-08-01T22:59:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ea6ffd700 1 -- 172.21.15.32:0/1668847658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9c012460 con 0x7f5ea80b0270 2022-08-01T22:59:08.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- 172.21.15.32:0/1668847658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b0270 msgr2=0x7f5ea81406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 --2- 172.21.15.32:0/1668847658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b0270 0x7f5ea81406d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f5ea0001b80 tx=0x7f5e9c006df0).stop 2022-08-01T22:59:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- 172.21.15.32:0/1668847658 shutdown_connections 2022-08-01T22:59:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 --2- 172.21.15.32:0/1668847658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b0270 0x7f5ea81406d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 --2- 172.21.15.32:0/1668847658 >> v2:172.21.15.55:3300/0 conn(0x7f5ea80b2710 0x7f5ea81401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.623+0000 7f5ead244700 1 -- 172.21.15.32:0/1668847658 >> 172.21.15.32:0/1668847658 conn(0x7f5ea8060310 msgr2=0x7f5ea80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 -- 172.21.15.32:0/1668847658 shutdown_connections 2022-08-01T22:59:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 -- 172.21.15.32:0/1668847658 wait complete. 2022-08-01T22:59:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 Processor -- start 2022-08-01T22:59:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 -- start start 2022-08-01T22:59:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea80ac880 0x7f5ea813e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea80b0270 0x7f5ea81559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 0x7f5ea8155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea81527d0 con 0x7f5ea80b2710 2022-08-01T22:59:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8152a70 con 0x7f5ea80ac880 2022-08-01T22:59:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5ead244700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8152d10 con 0x7f5ea80b0270 2022-08-01T22:59:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5eaca43700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 0x7f5ea8155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5eaca43700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 0x7f5ea8155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43034/0 (socket says 172.21.15.32:43034) 2022-08-01T22:59:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.627+0000 7f5eaca43700 1 -- 172.21.15.32:0/1683379561 learned_addr learned my addr 172.21.15.32:0/1683379561 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea7fff700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea80ac880 0x7f5ea813e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea77fe700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea80b0270 0x7f5ea81559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5eaca43700 1 -- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea80b0270 msgr2=0x7f5ea81559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5eaca43700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea80b0270 0x7f5ea81559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5eaca43700 1 -- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea80ac880 msgr2=0x7f5ea813e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5eaca43700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea80ac880 0x7f5ea813e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5eaca43700 1 -- 172.21.15.32:0/1683379561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c00b180 con 0x7f5ea80b2710 2022-08-01T22:59:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea77fe700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea80b0270 0x7f5ea81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5eaca43700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 0x7f5ea8155ec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f5ea0001c40 tx=0x7f5ea000caf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea0004ab0 con 0x7f5ea80b2710 2022-08-01T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea0004c10 con 0x7f5ea80b2710 2022-08-01T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea0007170 con 0x7f5ea80b2710 2022-08-01T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea813e7e0 con 0x7f5ea80b2710 2022-08-01T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea813f0c0 con 0x7f5ea80b2710 2022-08-01T22:59:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea8152630 con 0x7f5ea80b2710 2022-08-01T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.631+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5ea0010070 con 0x7f5ea80b2710 2022-08-01T22:59:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.635+0000 7f5ea57fa700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e880539e0 0x7f5e88055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.635+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5ea006bcb0 con 0x7f5ea80b2710 2022-08-01T22:59:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.635+0000 7f5ea7fff700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e880539e0 0x7f5e88055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.635+0000 7f5ea7fff700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e880539e0 0x7f5e88055ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5e9800b000 tx=0x7f5e98006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.639+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ea8152630 con 0x7f5ea80b2710 2022-08-01T22:59:08.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:08.799+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5ea8152630 con 0x7f5ea80b2710 2022-08-01T22:59:09.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ea57fa700 1 -- 172.21.15.32:0/1683379561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5ea8152630 con 0x7f5ea80b2710 2022-08-01T22:59:09.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e880539e0 msgr2=0x7f5e88055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:09.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e880539e0 0x7f5e88055ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f5e9800b000 tx=0x7f5e98006df0).stop 2022-08-01T22:59:09.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 msgr2=0x7f5ea8155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:09.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 0x7f5ea8155ec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f5ea0001c40 tx=0x7f5ea000caf0).stop 2022-08-01T22:59:09.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 shutdown_connections 2022-08-01T22:59:09.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e880539e0 0x7f5e88055ea0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea80b2710 0x7f5ea8155ec0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea80b0270 0x7f5ea81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 --2- 172.21.15.32:0/1683379561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea80ac880 0x7f5ea813e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.155+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 >> 172.21.15.32:0/1683379561 conn(0x7f5ea8060310 msgr2=0x7f5ea805c340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:09.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.159+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 shutdown_connections 2022-08-01T22:59:09.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.159+0000 7f5ead244700 1 -- 172.21.15.32:0/1683379561 wait complete. 2022-08-01T22:59:09.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T22:59:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.291+0000 7f1b851df700 1 Processor -- start 2022-08-01T22:59:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.291+0000 7f1b851df700 1 -- start start 2022-08-01T22:59:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b800b4550 0x7f1b800b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b80144240 con 0x7f1b800b6710 2022-08-01T22:59:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b80143d00 con 0x7f1b800b6af0 2022-08-01T22:59:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b80143fa0 con 0x7f1b800b4550 2022-08-01T22:59:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b800b4550 0x7f1b800b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b800b6710 0x7f1b80061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58814/0 (socket says 172.21.15.32:58814) 2022-08-01T22:59:09.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7ed9d700 1 -- 172.21.15.32:0/1282540926 learned_addr learned my addr 172.21.15.32:0/1282540926 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:09.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7e59c700 1 -- 172.21.15.32:0/1282540926 >> v1:172.21.15.55:6789/0 conn(0x7f1b800b6af0 legacy=0x7f1b8012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:09.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7e59c700 1 -- 172.21.15.32:0/1282540926 >> v1:172.21.15.32:6789/0 conn(0x7f1b800b6710 legacy=0x7f1b80061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:09.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7e59c700 1 -- 172.21.15.32:0/1282540926 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80061780 con 0x7f1b800b4550 2022-08-01T22:59:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7e59c700 1 --2- 172.21.15.32:0/1282540926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b800b5d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f1b801432b0 tx=0x7f1b74006df0).ready entity=mon.0 client_cookie=6d021141116ef7d9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7dd9b700 1 -- 172.21.15.32:0/1282540926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b7400d610 con 0x7f1b800b4550 2022-08-01T22:59:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7dd9b700 1 -- 172.21.15.32:0/1282540926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b74000e90 con 0x7f1b800b4550 2022-08-01T22:59:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b7dd9b700 1 -- 172.21.15.32:0/1282540926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b7400d960 con 0x7f1b800b4550 2022-08-01T22:59:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- 172.21.15.32:0/1282540926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 msgr2=0x7f1b800b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 --2- 172.21.15.32:0/1282540926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b800b5d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f1b801432b0 tx=0x7f1b74006df0).stop 2022-08-01T22:59:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- 172.21.15.32:0/1282540926 shutdown_connections 2022-08-01T22:59:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 --2- 172.21.15.32:0/1282540926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b800b5d80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- 172.21.15.32:0/1282540926 >> 172.21.15.32:0/1282540926 conn(0x7f1b8005c620 msgr2=0x7f1b8005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- 172.21.15.32:0/1282540926 shutdown_connections 2022-08-01T22:59:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- 172.21.15.32:0/1282540926 wait complete. 2022-08-01T22:59:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 Processor -- start 2022-08-01T22:59:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.295+0000 7f1b851df700 1 -- start start 2022-08-01T22:59:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b80127740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:09.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1b800b6710 0x7f1b800b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:09.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b800b6af0 0x7f1b800b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:09.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80140440 con 0x7f1b800b4550 2022-08-01T22:59:09.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80128100 con 0x7f1b800b6af0 2022-08-01T22:59:09.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80140670 con 0x7f1b800b6710 2022-08-01T22:59:09.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b80127740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:09.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b80127740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43046/0 (socket says 172.21.15.32:43046) 2022-08-01T22:59:09.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 -- 172.21.15.32:0/2233537097 learned_addr learned my addr 172.21.15.32:0/2233537097 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:09.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7f59e700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b800b6af0 0x7f1b800b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:09.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7e59c700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1b800b6710 0x7f1b800b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:09.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 -- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1b800b6710 msgr2=0x7f1b800b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:09.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1b800b6710 0x7f1b800b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 -- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b800b6af0 msgr2=0x7f1b800b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:09.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b800b6af0 0x7f1b800b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:09.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 -- 172.21.15.32:0/2233537097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7400b180 con 0x7f1b800b4550 2022-08-01T22:59:09.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b7ed9d700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b80127740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f1b800b1e10 tx=0x7f1b70007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:09.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b700070d0 con 0x7f1b800b4550 2022-08-01T22:59:09.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b801261f0 con 0x7f1b800b4550 2022-08-01T22:59:09.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70007230 con 0x7f1b800b4550 2022-08-01T22:59:09.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b70005610 con 0x7f1b800b4550 2022-08-01T22:59:09.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.299+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b80140d60 con 0x7f1b800b4550 2022-08-01T22:59:09.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.303+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b70010070 con 0x7f1b800b4550 2022-08-01T22:59:09.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.303+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b80126050 con 0x7f1b800b4550 2022-08-01T22:59:09.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.303+0000 7f1b6ffff700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b5c053a70 0x7f1b5c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:09.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.303+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1b7006b720 con 0x7f1b800b4550 2022-08-01T22:59:09.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.303+0000 7f1b7e59c700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b5c053a70 0x7f1b5c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:09.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.303+0000 7f1b7e59c700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b5c053a70 0x7f1b5c055f30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f1b7401c000 tx=0x7f1b74015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:09.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.311+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b80126050 con 0x7f1b800b4550 2022-08-01T22:59:09.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:09.435+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b80126050 con 0x7f1b800b4550 2022-08-01T22:59:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.227+0000 7f1b6ffff700 1 -- 172.21.15.32:0/2233537097 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1b80126050 con 0x7f1b800b4550 2022-08-01T22:59:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b5c053a70 msgr2=0x7f1b5c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b5c053a70 0x7f1b5c055f30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f1b7401c000 tx=0x7f1b74015000).stop 2022-08-01T22:59:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 msgr2=0x7f1b80127740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b80127740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f1b800b1e10 tx=0x7f1b70007c40).stop 2022-08-01T22:59:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 shutdown_connections 2022-08-01T22:59:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b5c053a70 0x7f1b5c055f30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b800b6af0 0x7f1b800b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1b800b6710 0x7f1b800b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 --2- 172.21.15.32:0/2233537097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b800b4550 0x7f1b80127740 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0x7f1b800b1e10 tx=0x7f1b70007c40).stop 2022-08-01T22:59:10.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 >> 172.21.15.32:0/2233537097 conn(0x7f1b8005c620 msgr2=0x7f1b801432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:10.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 shutdown_connections 2022-08-01T22:59:10.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.231+0000 7f1b851df700 1 -- 172.21.15.32:0/2233537097 wait complete. 2022-08-01T22:59:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T22:59:10.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T22:59:10.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.367+0000 7f2aff59e700 1 Processor -- start 2022-08-01T22:59:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.367+0000 7f2aff59e700 1 -- start start 2022-08-01T22:59:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.367+0000 7f2aff59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000b1a80 0x7f2b000b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.367+0000 7f2aff59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b000a9da0 0x7f2b00147e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.367+0000 7f2aff59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b000b3570 con 0x7f2b000b3190 2022-08-01T22:59:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.367+0000 7f2aff59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b000b2980 con 0x7f2b000a9da0 2022-08-01T22:59:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000b0e00 con 0x7f2b000b1a80 2022-08-01T22:59:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b000a9da0 0x7f2b00147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b000a9da0 0x7f2b00147e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43056/0 (socket says 172.21.15.32:43056) 2022-08-01T22:59:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 -- 172.21.15.32:0/835800701 learned_addr learned my addr 172.21.15.32:0/835800701 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afdd9b700 1 --2- 172.21.15.32:0/835800701 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000b1a80 0x7f2b000b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:10.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 -- 172.21.15.32:0/835800701 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000b1a80 msgr2=0x7f2b000b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:10.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afd59a700 1 -- 172.21.15.32:0/835800701 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890264301 0 0) 0x7f2b000b3570 con 0x7f2b000b3190 2022-08-01T22:59:10.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 --2- 172.21.15.32:0/835800701 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000b1a80 0x7f2b000b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 -- 172.21.15.32:0/835800701 >> v1:172.21.15.55:6789/0 conn(0x7f2b000b3190 legacy=0x7f2b000656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:10.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 -- 172.21.15.32:0/835800701 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000b10a0 con 0x7f2b000a9da0 2022-08-01T22:59:10.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afed9d700 1 --2- 172.21.15.32:0/835800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000a9da0 0x7f2b00147e30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f2af80032f0 tx=0x7f2af8009290).ready entity=mon.0 client_cookie=33a3fc9a57aacc6e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:10.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afd59a700 1 -- 172.21.15.32:0/835800701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af800b8a0 con 0x7f2b000a9da0 2022-08-01T22:59:10.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afd59a700 1 -- 172.21.15.32:0/835800701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af800ba00 con 0x7f2b000a9da0 2022-08-01T22:59:10.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2afd59a700 1 -- 172.21.15.32:0/835800701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af800bb70 con 0x7f2b000a9da0 2022-08-01T22:59:10.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 -- 172.21.15.32:0/835800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000a9da0 msgr2=0x7f2b00147e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:10.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 --2- 172.21.15.32:0/835800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000a9da0 0x7f2b00147e30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f2af80032f0 tx=0x7f2af8009290).stop 2022-08-01T22:59:10.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 -- 172.21.15.32:0/835800701 shutdown_connections 2022-08-01T22:59:10.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 --2- 172.21.15.32:0/835800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000a9da0 0x7f2b00147e30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 --2- 172.21.15.32:0/835800701 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000b1a80 0x7f2b000b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 -- 172.21.15.32:0/835800701 >> 172.21.15.32:0/835800701 conn(0x7f2b00063870 msgr2=0x7f2b00063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:10.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 -- 172.21.15.32:0/835800701 shutdown_connections 2022-08-01T22:59:10.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.371+0000 7f2aff59e700 1 -- 172.21.15.32:0/835800701 wait complete. 2022-08-01T22:59:10.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 Processor -- start 2022-08-01T22:59:10.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 -- start start 2022-08-01T22:59:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000a9da0 0x7f2b00148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 0x7f2b000af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000b3190 0x7f2b00156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000ad280 con 0x7f2b000b1a80 2022-08-01T22:59:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000aee10 con 0x7f2b000b3190 2022-08-01T22:59:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aff59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000ac220 con 0x7f2b000a9da0 2022-08-01T22:59:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 0x7f2b000af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 0x7f2b000af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43062/0 (socket says 172.21.15.32:43062) 2022-08-01T22:59:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 -- 172.21.15.32:0/3132596650 learned_addr learned my addr 172.21.15.32:0/3132596650 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afed9d700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000b3190 0x7f2b00156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afe59c700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000a9da0 0x7f2b00148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 -- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000a9da0 msgr2=0x7f2b00148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000a9da0 0x7f2b00148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 -- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000b3190 msgr2=0x7f2b00156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000b3190 0x7f2b00156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 -- 172.21.15.32:0/3132596650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af8009530 con 0x7f2b000b1a80 2022-08-01T22:59:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2afdd9b700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 0x7f2b000af4e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f2af400c000 tx=0x7f2af4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.375+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af4010590 con 0x7f2b000b1a80 2022-08-01T22:59:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af40106f0 con 0x7f2b000b1a80 2022-08-01T22:59:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b000ac4c0 con 0x7f2b000b1a80 2022-08-01T22:59:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af4010a50 con 0x7f2b000b1a80 2022-08-01T22:59:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b00149950 con 0x7f2b000b1a80 2022-08-01T22:59:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b000574e0 con 0x7f2b000b1a80 2022-08-01T22:59:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2af4010bb0 con 0x7f2b000b1a80 2022-08-01T22:59:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aef7fe700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2adc053a70 0x7f2adc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f2af406be30 con 0x7f2b000b1a80 2022-08-01T22:59:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.379+0000 7f2afe59c700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2adc053a70 0x7f2adc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.383+0000 7f2afe59c700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2adc053a70 0x7f2adc055f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f2af000d000 tx=0x7f2af0007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.387+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2af4042d50 con 0x7f2b000b1a80 2022-08-01T22:59:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:10.515+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2b00069a70 con 0x7f2b000b1a80 2022-08-01T22:59:11.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.239+0000 7f2aef7fe700 1 -- 172.21.15.32:0/3132596650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2af4046370 con 0x7f2b000b1a80 2022-08-01T22:59:11.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2adc053a70 msgr2=0x7f2adc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2adc053a70 0x7f2adc055f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f2af000d000 tx=0x7f2af0007a20).stop 2022-08-01T22:59:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 msgr2=0x7f2b000af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 0x7f2b000af4e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f2af400c000 tx=0x7f2af4006df0).stop 2022-08-01T22:59:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 shutdown_connections 2022-08-01T22:59:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2adc053a70 0x7f2adc055f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000b3190 0x7f2b00156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b000b1a80 0x7f2b000af4e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 --2- 172.21.15.32:0/3132596650 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2b000a9da0 0x7f2b00148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 >> 172.21.15.32:0/3132596650 conn(0x7f2b00063870 msgr2=0x7f2b000648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 shutdown_connections 2022-08-01T22:59:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.247+0000 7f2aff59e700 1 -- 172.21.15.32:0/3132596650 wait complete. 2022-08-01T22:59:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T22:59:11.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T22:59:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.363+0000 7ff18359e700 1 Processor -- start 2022-08-01T22:59:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.363+0000 7ff18359e700 1 -- start start 2022-08-01T22:59:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1840ad750 0x7ff184061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840aee80 con 0x7ff1841405d0 2022-08-01T22:59:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840b70b0 con 0x7ff1840614b0 2022-08-01T22:59:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840af0b0 con 0x7ff1840ad750 2022-08-01T22:59:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1840ad750 0x7ff184061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1840ad750 0x7ff184061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43068/0 (socket says 172.21.15.32:43068) 2022-08-01T22:59:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 -- 172.21.15.32:0/4000680564 learned_addr learned my addr 172.21.15.32:0/4000680564 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:11.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff182d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff1841405d0 0x7ff1841466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58846/0 (socket says 172.21.15.32:58846) 2022-08-01T22:59:11.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 -- 172.21.15.32:0/4000680564 >> v1:172.21.15.55:6789/0 conn(0x7ff1840614b0 legacy=0x7ff1840b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:11.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 -- 172.21.15.32:0/4000680564 >> v1:172.21.15.32:6789/0 conn(0x7ff1841405d0 legacy=0x7ff1841466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 -- 172.21.15.32:0/4000680564 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840af350 con 0x7ff1840ad750 2022-08-01T22:59:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff181d9b700 1 --2- 172.21.15.32:0/4000680564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840ad750 0x7ff184061890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff1740032f0 tx=0x7ff174009290).ready entity=mon.0 client_cookie=7ab82eb2b924dede server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18159a700 1 -- 172.21.15.32:0/4000680564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff17400b8a0 con 0x7ff1840ad750 2022-08-01T22:59:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18159a700 1 -- 172.21.15.32:0/4000680564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17400ba00 con 0x7ff1840ad750 2022-08-01T22:59:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18159a700 1 -- 172.21.15.32:0/4000680564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff17400bb70 con 0x7ff1840ad750 2022-08-01T22:59:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- 172.21.15.32:0/4000680564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840ad750 msgr2=0x7ff184061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 --2- 172.21.15.32:0/4000680564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840ad750 0x7ff184061890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff1740032f0 tx=0x7ff174009290).stop 2022-08-01T22:59:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- 172.21.15.32:0/4000680564 shutdown_connections 2022-08-01T22:59:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 --2- 172.21.15.32:0/4000680564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840ad750 0x7ff184061890 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- 172.21.15.32:0/4000680564 >> 172.21.15.32:0/4000680564 conn(0x7ff184066f70 msgr2=0x7ff184067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:11.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- 172.21.15.32:0/4000680564 shutdown_connections 2022-08-01T22:59:11.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.367+0000 7ff18359e700 1 -- 172.21.15.32:0/4000680564 wait complete. 2022-08-01T22:59:11.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 Processor -- start 2022-08-01T22:59:11.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 -- start start 2022-08-01T22:59:11.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 0x7ff184143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1840ad750 0x7ff1840afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1841405d0 0x7ff184141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184153080 con 0x7ff1840614b0 2022-08-01T22:59:11.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184144040 con 0x7ff1840ad750 2022-08-01T22:59:11.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841532e0 con 0x7ff1841405d0 2022-08-01T22:59:11.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 0x7ff184143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 0x7ff184143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43074/0 (socket says 172.21.15.32:43074) 2022-08-01T22:59:11.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff18259c700 1 -- 172.21.15.32:0/2778438579 learned_addr learned my addr 172.21.15.32:0/2778438579 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:11.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.371+0000 7ff181d9b700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1840ad750 0x7ff1840afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff182d9d700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1841405d0 0x7ff184141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18259c700 1 -- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1841405d0 msgr2=0x7ff184141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18259c700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1841405d0 0x7ff184141000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18259c700 1 -- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1840ad750 msgr2=0x7ff1840afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18259c700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1840ad750 0x7ff1840afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18259c700 1 -- 172.21.15.32:0/2778438579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff174009530 con 0x7ff1840614b0 2022-08-01T22:59:11.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff181d9b700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1840ad750 0x7ff1840afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:11.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18259c700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 0x7ff184143670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7ff1840af970 tx=0x7ff16c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:11.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16c00ce30 con 0x7ff1840614b0 2022-08-01T22:59:11.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c007240 con 0x7ff1840614b0 2022-08-01T22:59:11.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1841442a0 con 0x7ff1840614b0 2022-08-01T22:59:11.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16c006700 con 0x7ff1840614b0 2022-08-01T22:59:11.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1840b0590 con 0x7ff1840614b0 2022-08-01T22:59:11.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff184144430 con 0x7ff1840614b0 2022-08-01T22:59:11.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff16c010030 con 0x7ff1840614b0 2022-08-01T22:59:11.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff17b7fe700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff168053950 0x7ff168055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.375+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff16c06b9e0 con 0x7ff1840614b0 2022-08-01T22:59:11.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.379+0000 7ff181d9b700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff168053950 0x7ff168055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.379+0000 7ff181d9b700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff168053950 0x7ff168055e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7ff174019000 tx=0x7ff174016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:11.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.383+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff16c042a80 con 0x7ff1840614b0 2022-08-01T22:59:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.519+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff1840af5f0 con 0x7ff1840614b0 2022-08-01T22:59:11.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.527+0000 7ff17b7fe700 1 -- 172.21.15.32:0/2778438579 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff16c0460a0 con 0x7ff1840614b0 2022-08-01T22:59:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff168053950 msgr2=0x7ff168055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff168053950 0x7ff168055e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7ff174019000 tx=0x7ff174016000).stop 2022-08-01T22:59:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 msgr2=0x7ff184143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 0x7ff184143670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7ff1840af970 tx=0x7ff16c007c10).stop 2022-08-01T22:59:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 shutdown_connections 2022-08-01T22:59:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff168053950 0x7ff168055e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1841405d0 0x7ff184141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1840ad750 0x7ff1840afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 --2- 172.21.15.32:0/2778438579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840614b0 0x7ff184143670 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 >> 172.21.15.32:0/2778438579 conn(0x7ff184066f70 msgr2=0x7ff18413eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 shutdown_connections 2022-08-01T22:59:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.531+0000 7ff18359e700 1 -- 172.21.15.32:0/2778438579 wait complete. 2022-08-01T22:59:11.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-08-01T22:59:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 Processor -- start 2022-08-01T22:59:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 -- start start 2022-08-01T22:59:11.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4eb0060290 0x7f4eb00b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4eb00b4dd0 0x7f4eb014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb00aad40 con 0x7f4eb0067490 2022-08-01T22:59:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb0060670 con 0x7f4eb0060290 2022-08-01T22:59:11.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb83bb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb0067030 con 0x7f4eb00b4dd0 2022-08-01T22:59:11.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4eb0060290 0x7f4eb00b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4eb0060290 0x7f4eb00b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43084/0 (socket says 172.21.15.32:43084) 2022-08-01T22:59:11.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 -- 172.21.15.32:0/1945319590 learned_addr learned my addr 172.21.15.32:0/1945319590 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:11.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6958700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4eb00b4dd0 0x7f4eb014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb5155700 1 -- 172.21.15.32:0/1945319590 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29492162 0 0) 0x7f4eb00aad40 con 0x7f4eb0067490 2022-08-01T22:59:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 -- 172.21.15.32:0/1945319590 >> v2:172.21.15.55:3300/0 conn(0x7f4eb00b4dd0 msgr2=0x7f4eb014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 --2- 172.21.15.32:0/1945319590 >> v2:172.21.15.55:3300/0 conn(0x7f4eb00b4dd0 0x7f4eb014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 -- 172.21.15.32:0/1945319590 >> v1:172.21.15.55:6789/0 conn(0x7f4eb0067490 legacy=0x7f4eb0143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 -- 172.21.15.32:0/1945319590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00b3540 con 0x7f4eb0060290 2022-08-01T22:59:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb6157700 1 --2- 172.21.15.32:0/1945319590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0060290 0x7f4eb00b48e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f4ea0002f70 tx=0x7f4ea0006eb0).ready entity=mon.0 client_cookie=25025ac0671f24c2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb5155700 1 -- 172.21.15.32:0/1945319590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea000d5c0 con 0x7f4eb0060290 2022-08-01T22:59:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb5155700 1 -- 172.21.15.32:0/1945319590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea0000e90 con 0x7f4eb0060290 2022-08-01T22:59:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.651+0000 7f4eb5155700 1 -- 172.21.15.32:0/1945319590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea000d910 con 0x7f4eb0060290 2022-08-01T22:59:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- 172.21.15.32:0/1945319590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0060290 msgr2=0x7f4eb00b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/1945319590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0060290 0x7f4eb00b48e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f4ea0002f70 tx=0x7f4ea0006eb0).stop 2022-08-01T22:59:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- 172.21.15.32:0/1945319590 shutdown_connections 2022-08-01T22:59:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/1945319590 >> v2:172.21.15.55:3300/0 conn(0x7f4eb00b4dd0 0x7f4eb014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/1945319590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0060290 0x7f4eb00b48e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- 172.21.15.32:0/1945319590 >> 172.21.15.32:0/1945319590 conn(0x7f4eb005d0c0 msgr2=0x7f4eb005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- 172.21.15.32:0/1945319590 shutdown_connections 2022-08-01T22:59:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- 172.21.15.32:0/1945319590 wait complete. 2022-08-01T22:59:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 Processor -- start 2022-08-01T22:59:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- start start 2022-08-01T22:59:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4eb0060290 0x7f4eb0150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 0x7f4eb00b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4eb00b4dd0 0x7f4eb01543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb0153cb0 con 0x7f4eb0067490 2022-08-01T22:59:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb0146410 con 0x7f4eb00b4dd0 2022-08-01T22:59:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb83bb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb0153f10 con 0x7f4eb0060290 2022-08-01T22:59:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb5956700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 0x7f4eb00b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb5956700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 0x7f4eb00b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43090/0 (socket says 172.21.15.32:43090) 2022-08-01T22:59:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb5956700 1 -- 172.21.15.32:0/2663266508 learned_addr learned my addr 172.21.15.32:0/2663266508 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb6958700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4eb00b4dd0 0x7f4eb01543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.655+0000 7f4eb6157700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4eb0060290 0x7f4eb0150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb5956700 1 -- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4eb0060290 msgr2=0x7f4eb0150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb5956700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4eb0060290 0x7f4eb0150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb5956700 1 -- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4eb00b4dd0 msgr2=0x7f4eb01543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb5956700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4eb00b4dd0 0x7f4eb01543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb5956700 1 -- 172.21.15.32:0/2663266508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea000b1e0 con 0x7f4eb0067490 2022-08-01T22:59:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb6157700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4eb0060290 0x7f4eb0150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb5956700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 0x7f4eb00b0290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f4ea8001250 tx=0x7f4ea8007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea8010510 con 0x7f4eb0067490 2022-08-01T22:59:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8010670 con 0x7f4eb0067490 2022-08-01T22:59:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb01466b0 con 0x7f4eb0067490 2022-08-01T22:59:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea80107e0 con 0x7f4eb0067490 2022-08-01T22:59:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb0146090 con 0x7f4eb0067490 2022-08-01T22:59:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb00574e0 con 0x7f4eb0067490 2022-08-01T22:59:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.659+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ea8010070 con 0x7f4eb0067490 2022-08-01T22:59:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.663+0000 7f4e9f7fe700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e90053aa0 0x7f4e90055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.663+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4ea806bae0 con 0x7f4eb0067490 2022-08-01T22:59:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.663+0000 7f4eb6157700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e90053aa0 0x7f4e90055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.663+0000 7f4eb6157700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e90053aa0 0x7f4e90055f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f4ea001a000 tx=0x7f4ea0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.667+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ea8042a70 con 0x7f4eb0067490 2022-08-01T22:59:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:11.831+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f4eb0154080 con 0x7f4eb0067490 2022-08-01T22:59:12.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.243+0000 7f4e9f7fe700 1 -- 172.21.15.32:0/2663266508 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v154) v1 ==== 144+0+0 (secure 0 0 0) 0x7f4ea8046090 con 0x7f4eb0067490 2022-08-01T22:59:12.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e90053aa0 msgr2=0x7f4e90055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:12.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e90053aa0 0x7f4e90055f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f4ea001a000 tx=0x7f4ea0015000).stop 2022-08-01T22:59:12.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 msgr2=0x7f4eb00b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 0x7f4eb00b0290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f4ea8001250 tx=0x7f4ea8007e10).stop 2022-08-01T22:59:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 shutdown_connections 2022-08-01T22:59:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e90053aa0 0x7f4e90055f60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4eb00b4dd0 0x7f4eb01543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eb0067490 0x7f4eb00b0290 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 --2- 172.21.15.32:0/2663266508 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4eb0060290 0x7f4eb0150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 >> 172.21.15.32:0/2663266508 conn(0x7f4eb005d0c0 msgr2=0x7f4eb00af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 shutdown_connections 2022-08-01T22:59:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.247+0000 7f4eb83bb700 1 -- 172.21.15.32:0/2663266508 wait complete. 2022-08-01T22:59:12.267 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T22:59:12.267 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T22:59:12.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-08-01T22:59:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 Processor -- start 2022-08-01T22:59:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 -- start start 2022-08-01T22:59:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fec060aa0 0x7f8fec0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8fec0aa760 0x7f8fec13c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0b6e70 con 0x7f8fec0b6a90 2022-08-01T22:59:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec061370 con 0x7f8fec060aa0 2022-08-01T22:59:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8ff0f06700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec061610 con 0x7f8fec0aa760 2022-08-01T22:59:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fec060aa0 0x7f8fec0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fec060aa0 0x7f8fec0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43098/0 (socket says 172.21.15.32:43098) 2022-08-01T22:59:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 -- 172.21.15.32:0/4116017040 learned_addr learned my addr 172.21.15.32:0/4116017040 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:12.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fead9d700 1 --2- 172.21.15.32:0/4116017040 >> v2:172.21.15.55:3300/0 conn(0x7f8fec0aa760 0x7f8fec13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 -- 172.21.15.32:0/4116017040 >> v2:172.21.15.55:3300/0 conn(0x7f8fec0aa760 msgr2=0x7f8fec13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 --2- 172.21.15.32:0/4116017040 >> v2:172.21.15.55:3300/0 conn(0x7f8fec0aa760 0x7f8fec13c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 -- 172.21.15.32:0/4116017040 >> v1:172.21.15.55:6789/0 conn(0x7f8fec0b6a90 legacy=0x7f8fec0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe959a700 1 -- 172.21.15.32:0/4116017040 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74720487 0 0) 0x7f8fec0b6e70 con 0x7f8fec0b6a90 2022-08-01T22:59:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 -- 172.21.15.32:0/4116017040 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0618b0 con 0x7f8fec060aa0 2022-08-01T22:59:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe9d9b700 1 --2- 172.21.15.32:0/4116017040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec0acee0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f8fd4002f70 tx=0x7f8fd4006eb0).ready entity=mon.0 client_cookie=1ec9bf542339ed29 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe959a700 1 -- 172.21.15.32:0/4116017040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd400d5c0 con 0x7f8fec060aa0 2022-08-01T22:59:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe959a700 1 -- 172.21.15.32:0/4116017040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd4000e90 con 0x7f8fec060aa0 2022-08-01T22:59:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.371+0000 7f8fe959a700 1 -- 172.21.15.32:0/4116017040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd400d910 con 0x7f8fec060aa0 2022-08-01T22:59:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- 172.21.15.32:0/4116017040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 msgr2=0x7f8fec0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/4116017040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec0acee0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f8fd4002f70 tx=0x7f8fd4006eb0).stop 2022-08-01T22:59:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- 172.21.15.32:0/4116017040 shutdown_connections 2022-08-01T22:59:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/4116017040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec0acee0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/4116017040 >> v2:172.21.15.55:3300/0 conn(0x7f8fec0aa760 0x7f8fec13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- 172.21.15.32:0/4116017040 >> 172.21.15.32:0/4116017040 conn(0x7f8fec0b3610 msgr2=0x7f8fec0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- 172.21.15.32:0/4116017040 shutdown_connections 2022-08-01T22:59:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- 172.21.15.32:0/4116017040 wait complete. 2022-08-01T22:59:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 Processor -- start 2022-08-01T22:59:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- start start 2022-08-01T22:59:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec13da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8fec0aa760 0x7f8fec130130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fec0b6a90 0x7f8fec0af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0b5d80 con 0x7f8fec060aa0 2022-08-01T22:59:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0b56b0 con 0x7f8fec0b6a90 2022-08-01T22:59:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8ff0f06700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec13e430 con 0x7f8fec0aa760 2022-08-01T22:59:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec13da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43102/0 (socket says 172.21.15.32:43102) 2022-08-01T22:59:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fea59c700 1 -- 172.21.15.32:0/720195440 learned_addr learned my addr 172.21.15.32:0/720195440 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fead9d700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fec0b6a90 0x7f8fec0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fe9d9b700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8fec0aa760 0x7f8fec130130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fea59c700 1 -- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8fec0aa760 msgr2=0x7f8fec130130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.375+0000 7f8fea59c700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8fec0aa760 0x7f8fec130130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fea59c700 1 -- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fec0b6a90 msgr2=0x7f8fec0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fea59c700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fec0b6a90 0x7f8fec0af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fea59c700 1 -- 172.21.15.32:0/720195440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd400b1e0 con 0x7f8fec060aa0 2022-08-01T22:59:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fea59c700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec13da50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f8fdc001250 tx=0x7f8fdc007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fdc00e4a0 con 0x7f8fec060aa0 2022-08-01T22:59:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc00e600 con 0x7f8fec060aa0 2022-08-01T22:59:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec13ede0 con 0x7f8fec060aa0 2022-08-01T22:59:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fdc00e960 con 0x7f8fec060aa0 2022-08-01T22:59:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec13f080 con 0x7f8fec060aa0 2022-08-01T22:59:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec0ad5c0 con 0x7f8fec060aa0 2022-08-01T22:59:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8fdc019030 con 0x7f8fec060aa0 2022-08-01T22:59:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe37fe700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fc4053b00 0x7f8fc4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8fdc06b3f0 con 0x7f8fec060aa0 2022-08-01T22:59:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.379+0000 7f8fe9d9b700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fc4053b00 0x7f8fc4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.383+0000 7f8fe9d9b700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fc4053b00 0x7f8fc4055fc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8fd401c000 tx=0x7f8fd4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.387+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8fdc042540 con 0x7f8fec060aa0 2022-08-01T22:59:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:12.555+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f8fec061b50 con 0x7f8fec060aa0 2022-08-01T22:59:13.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.259+0000 7f8fe37fe700 1 -- 172.21.15.32:0/720195440 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v155) v1 ==== 155+0+0 (secure 0 0 0) 0x7f8fdc017070 con 0x7f8fec060aa0 2022-08-01T22:59:13.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fc4053b00 msgr2=0x7f8fc4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fc4053b00 0x7f8fc4055fc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8fd401c000 tx=0x7f8fd4015000).stop 2022-08-01T22:59:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 msgr2=0x7f8fec13da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec13da50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f8fdc001250 tx=0x7f8fdc007e30).stop 2022-08-01T22:59:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 shutdown_connections 2022-08-01T22:59:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fc4053b00 0x7f8fc4055fc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fec0b6a90 0x7f8fec0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8fec0aa760 0x7f8fec130130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 --2- 172.21.15.32:0/720195440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec060aa0 0x7f8fec13da50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 >> 172.21.15.32:0/720195440 conn(0x7f8fec0b3610 msgr2=0x7f8fec12e480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 shutdown_connections 2022-08-01T22:59:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.263+0000 7f8ff0f06700 1 -- 172.21.15.32:0/720195440 wait complete. 2022-08-01T22:59:13.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T22:59:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 Processor -- start 2022-08-01T22:59:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 -- start start 2022-08-01T22:59:13.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 0x7f2654142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26540b3260 0x7f26541433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26540b0c80 con 0x7f26540ad000 2022-08-01T22:59:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26540ae810 con 0x7f26540b3260 2022-08-01T22:59:13.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2659572700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26540aeab0 con 0x7f26540b08a0 2022-08-01T22:59:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2658d71700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26540ad000 0x7f265405f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58888/0 (socket says 172.21.15.32:58888) 2022-08-01T22:59:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2658d71700 1 -- 172.21.15.32:0/624056554 learned_addr learned my addr 172.21.15.32:0/624056554 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f26537fe700 1 --2- 172.21.15.32:0/624056554 >> v2:172.21.15.32:3300/0 conn(0x7f26540b3260 0x7f26541433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2653fff700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 0x7f2654142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345835985 0 0) 0x7f26540b0c80 con 0x7f26540ad000 2022-08-01T22:59:13.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.395+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2640003260 con 0x7f26540ad000 2022-08-01T22:59:13.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501719188 0 0) 0x7f2640003260 con 0x7f26540ad000 2022-08-01T22:59:13.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 msgr2=0x7f2654142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:13.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 0x7f2654142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 >> v2:172.21.15.32:3300/0 conn(0x7f26540b3260 msgr2=0x7f26541433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 --2- 172.21.15.32:0/624056554 >> v2:172.21.15.32:3300/0 conn(0x7f26540b3260 0x7f26541433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540aed50 con 0x7f26540ad000 2022-08-01T22:59:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f264c0019e0 con 0x7f26540ad000 2022-08-01T22:59:13.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 >> v1:172.21.15.32:6789/0 conn(0x7f26540ad000 legacy=0x7f265405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2640003750 0x7f2640003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 0x7f2640004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26400049f0 0x7f2640004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540ae810 con 0x7f26400049f0 2022-08-01T22:59:13.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540aeab0 con 0x7f2640003750 2022-08-01T22:59:13.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264000c860 con 0x7f26540b08a0 2022-08-01T22:59:13.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f26537fe700 1 --2- 172.21.15.32:0/624056554 >> v2:172.21.15.32:3300/0 conn(0x7f26540b3260 0x7f26541433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:13.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26400049f0 0x7f2640004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2653fff700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2640003750 0x7f2640003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f26537fe700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 0x7f2640004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 -- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 msgr2=0x7f2640004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26540b08a0 0x7f2640004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 -- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2640003750 msgr2=0x7f2640003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2640003750 0x7f2640003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 -- 172.21.15.32:0/624056554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2640003260 con 0x7f26400049f0 2022-08-01T22:59:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26400049f0 0x7f2640004e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f264c0016f0 tx=0x7f26540a95b0).ready entity=mon.0 client_cookie=c36c012fc1d6ff2c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:13.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f264c00cc20 con 0x7f26400049f0 2022-08-01T22:59:13.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c00cd80 con 0x7f26400049f0 2022-08-01T22:59:13.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2652ffd700 1 -- 172.21.15.32:0/624056554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f264c005b80 con 0x7f26400049f0 2022-08-01T22:59:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2659572700 1 -- 172.21.15.32:0/624056554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26400049f0 msgr2=0x7f2640004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2659572700 1 --2- 172.21.15.32:0/624056554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26400049f0 0x7f2640004e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f264c0016f0 tx=0x7f26540a95b0).stop 2022-08-01T22:59:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2658d71700 1 -- 172.21.15.32:0/624056554 reap_dead start 2022-08-01T22:59:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2659572700 1 -- 172.21.15.32:0/624056554 shutdown_connections 2022-08-01T22:59:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2659572700 1 -- 172.21.15.32:0/624056554 >> 172.21.15.32:0/624056554 conn(0x7f26540612d0 msgr2=0x7f2654062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2659572700 1 -- 172.21.15.32:0/624056554 shutdown_connections 2022-08-01T22:59:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.399+0000 7f2659572700 1 -- 172.21.15.32:0/624056554 wait complete. 2022-08-01T22:59:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 Processor -- start 2022-08-01T22:59:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 -- start start 2022-08-01T22:59:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26400049f0 0x7f265413f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 0x7f2654137da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654138290 0x7f265413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540a9be0 con 0x7f26540ab370 2022-08-01T22:59:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540aa3f0 con 0x7f2654138290 2022-08-01T22:59:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26540a9e10 con 0x7f26400049f0 2022-08-01T22:59:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 0x7f2654137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 0x7f2654137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43124/0 (socket says 172.21.15.32:43124) 2022-08-01T22:59:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 -- 172.21.15.32:0/41790507 learned_addr learned my addr 172.21.15.32:0/41790507 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2653fff700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26400049f0 0x7f265413f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2658d71700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654138290 0x7f265413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 -- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26400049f0 msgr2=0x7f265413f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26400049f0 0x7f265413f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 -- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654138290 msgr2=0x7f265413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654138290 0x7f265413c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 -- 172.21.15.32:0/41790507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540aa150 con 0x7f26540ab370 2022-08-01T22:59:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2653fff700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26400049f0 0x7f265413f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2658d71700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654138290 0x7f265413c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26537fe700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 0x7f2654137da0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f264800b000 tx=0x7f2648008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2648010e90 con 0x7f26540ab370 2022-08-01T22:59:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264800e8d0 con 0x7f26540ab370 2022-08-01T22:59:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265413e4e0 con 0x7f26540ab370 2022-08-01T22:59:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f264800ec30 con 0x7f26540ab370 2022-08-01T22:59:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.403+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265413ea70 con 0x7f26540ab370 2022-08-01T22:59:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.407+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265413e670 con 0x7f26540ab370 2022-08-01T22:59:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.407+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2648024070 con 0x7f26540ab370 2022-08-01T22:59:13.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.407+0000 7f26517fa700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2634053980 0x7f2634055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.407+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f264806c820 con 0x7f26540ab370 2022-08-01T22:59:13.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.407+0000 7f2653fff700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2634053980 0x7f2634055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.407+0000 7f2653fff700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2634053980 0x7f2634055e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f264400b000 tx=0x7f2644008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.415+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f26480438c0 con 0x7f26540ab370 2022-08-01T22:59:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.543+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f26540574e0 con 0x7f26540ab370 2022-08-01T22:59:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.551+0000 7f26517fa700 1 -- 172.21.15.32:0/41790507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2648046ee0 con 0x7f26540ab370 2022-08-01T22:59:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2634053980 msgr2=0x7f2634055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2634053980 0x7f2634055e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f264400b000 tx=0x7f2644008040).stop 2022-08-01T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 msgr2=0x7f2654137da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 0x7f2654137da0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f264800b000 tx=0x7f2648008040).stop 2022-08-01T22:59:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 shutdown_connections 2022-08-01T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2634053980 0x7f2634055e40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654138290 0x7f265413c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26540ab370 0x7f2654137da0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 --2- 172.21.15.32:0/41790507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f26400049f0 0x7f265413f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 >> 172.21.15.32:0/41790507 conn(0x7f26540612d0 msgr2=0x7f2654140860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 shutdown_connections 2022-08-01T22:59:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.555+0000 7f2659572700 1 -- 172.21.15.32:0/41790507 wait complete. 2022-08-01T22:59:13.571 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T22:59:13.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:13.585 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:13.607 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:13.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T22:59:13.653 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T22:59:13.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:13.654 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:13.655 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:13.677 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T22:59:13.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:59:13.679 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T22:59:13.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:13.680 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:13.680 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T22:59:13.700 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T22:59:13.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:59:13.702 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T22:59:13.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T22:59:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 Processor -- start 2022-08-01T22:59:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- start start 2022-08-01T22:59:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6924148530 0x7f69240b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f69240a7250 0x7f69241448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69240b04e0 con 0x7f69240b1f40 2022-08-01T22:59:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69240a9250 con 0x7f6924148530 2022-08-01T22:59:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69240b0780 con 0x7f69240a7250 2022-08-01T22:59:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6924148530 0x7f69240b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6924148530 0x7f69240b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43132/0 (socket says 172.21.15.32:43132) 2022-08-01T22:59:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 -- 172.21.15.32:0/468769683 learned_addr learned my addr 172.21.15.32:0/468769683 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6929dba700 1 --2- 172.21.15.32:0/468769683 >> v2:172.21.15.55:3300/0 conn(0x7f69240a7250 0x7f69241448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 -- 172.21.15.32:0/468769683 >> v2:172.21.15.55:3300/0 conn(0x7f69240a7250 msgr2=0x7f69241448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 --2- 172.21.15.32:0/468769683 >> v2:172.21.15.55:3300/0 conn(0x7f69240a7250 0x7f69241448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 -- 172.21.15.32:0/468769683 >> v1:172.21.15.55:6789/0 conn(0x7f69240b1f40 legacy=0x7f69240b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 -- 172.21.15.32:0/468769683 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240b0a20 con 0x7f6924148530 2022-08-01T22:59:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f6928db8700 1 --2- 172.21.15.32:0/468769683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6924148530 0x7f69240b1160 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f69241478c0 tx=0x7f6914006eb0).ready entity=mon.0 client_cookie=2f21d52defdedcf1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f691bfff700 1 -- 172.21.15.32:0/468769683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f691400d610 con 0x7f6924148530 2022-08-01T22:59:13.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f691bfff700 1 -- 172.21.15.32:0/468769683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6914000e90 con 0x7f6924148530 2022-08-01T22:59:13.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f691bfff700 1 -- 172.21.15.32:0/468769683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f691400d960 con 0x7f6924148530 2022-08-01T22:59:13.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- 172.21.15.32:0/468769683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6924148530 msgr2=0x7f69240b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 --2- 172.21.15.32:0/468769683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6924148530 0x7f69240b1160 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f69241478c0 tx=0x7f6914006eb0).stop 2022-08-01T22:59:13.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- 172.21.15.32:0/468769683 shutdown_connections 2022-08-01T22:59:13.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 --2- 172.21.15.32:0/468769683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6924148530 0x7f69240b1160 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 --2- 172.21.15.32:0/468769683 >> v2:172.21.15.55:3300/0 conn(0x7f69240a7250 0x7f69241448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- 172.21.15.32:0/468769683 >> 172.21.15.32:0/468769683 conn(0x7f692405f840 msgr2=0x7f692405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:13.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- 172.21.15.32:0/468769683 shutdown_connections 2022-08-01T22:59:13.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.811+0000 7f692b81d700 1 -- 172.21.15.32:0/468769683 wait complete. 2022-08-01T22:59:13.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 Processor -- start 2022-08-01T22:59:13.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 -- start start 2022-08-01T22:59:13.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69240a7250 0x7f69240b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 0x7f692415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6924142bf0 0x7f6924143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924143570 con 0x7f69240b1f40 2022-08-01T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692415b0a0 con 0x7f69240a7250 2022-08-01T22:59:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f692b81d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69240a9fd0 con 0x7f6924142bf0 2022-08-01T22:59:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 0x7f692415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f69295b9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69240a7250 0x7f69240b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f69295b9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69240a7250 0x7f69240b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:52922/0 (socket says 172.21.15.32:52922) 2022-08-01T22:59:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f69295b9700 1 -- 172.21.15.32:0/148694636 learned_addr learned my addr 172.21.15.32:0/148694636 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 0x7f692415abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43140/0 (socket says 172.21.15.32:43140) 2022-08-01T22:59:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6929dba700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6924142bf0 0x7f6924143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 -- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6924142bf0 msgr2=0x7f6924143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6924142bf0 0x7f6924143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 -- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69240a7250 msgr2=0x7f69240b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69240a7250 0x7f69240b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 -- 172.21.15.32:0/148694636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691400b1e0 con 0x7f69240b1f40 2022-08-01T22:59:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.815+0000 7f6928db8700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 0x7f692415abb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f69241478c0 tx=0x7f6914015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6914011790 con 0x7f69240b1f40 2022-08-01T22:59:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69140118f0 con 0x7f69240b1f40 2022-08-01T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69240aa270 con 0x7f69240b1f40 2022-08-01T22:59:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6914011c50 con 0x7f69240b1f40 2022-08-01T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69240aa7b0 con 0x7f69240b1f40 2022-08-01T22:59:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6914025030 con 0x7f69240b1f40 2022-08-01T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69240a9db0 con 0x7f69240b1f40 2022-08-01T22:59:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f691a7fc700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f690c053a10 0x7f690c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f691406cd40 con 0x7f69240b1f40 2022-08-01T22:59:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.819+0000 7f69295b9700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f690c053a10 0x7f690c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.823+0000 7f69295b9700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f690c053a10 0x7f690c055ed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f692000d000 tx=0x7f6920009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.827+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6914043de0 con 0x7f69240b1f40 2022-08-01T22:59:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.955+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f69240aabf0 con 0x7f69240b1f40 2022-08-01T22:59:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f691a7fc700 1 -- 172.21.15.32:0/148694636 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+29217 (secure 0 0 0) 0x7f6914023090 con 0x7f69240b1f40 2022-08-01T22:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 48, 2022-08-01T22:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T22:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T22:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T22:59:11.227907+0000", 2022-08-01T22:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T22:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T22:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T22:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T22:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T22:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T22:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T22:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T22:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T22:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T22:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T22:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T22:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T22:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 8, 2022-08-01T22:59:13.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T22:59:13.967 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T22:59:13.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T22:59:13.967 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T22:59:13.968 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T22:59:13.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:13.968 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T22:59:13.968 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T22:59:13.968 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T22:59:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:59:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:59:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:59:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:59:13.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:59:13.970 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:59:13.970 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:59:13.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:59:13.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T22:59:13.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T22:59:13.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T22:59:13.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T22:59:13.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T22:59:13.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:59:13.973 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:59:13.973 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:59:13.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:59:13.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:59:13.973 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:59:13.974 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:59:13.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:13.974 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T22:59:13.974 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:59:13.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:59:13.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:59:13.975 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:59:13.975 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:59:13.976 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:59:13.976 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:59:13.976 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:59:13.976 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:59:13.977 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:59:13.977 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:59:13.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:59:13.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:59:13.978 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:59:13.978 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:59:13.978 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:59:13.979 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:59:13.979 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:59:13.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:59:13.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:59:13.980 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:59:13.980 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:59:13.980 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:59:13.980 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:59:13.980 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:59:13.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:59:13.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:13.981 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:59:13.981 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:59:13.981 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:59:13.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:59:13.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:59:13.982 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:59:13.982 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:59:13.983 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:59:13.983 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:59:13.983 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:59:13.983 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:59:13.983 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T22:59:13.984 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T22:59:13.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:13.984 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:59:13.984 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T22:59:13.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:13.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:13.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:13.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T22:59:13.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T22:59:13.986 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T22:59:13.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T22:59:13.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T22:59:13.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T22:59:13.987 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T22:59:13.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T22:59:13.987 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T22:59:13.987 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T22:59:13.988 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T22:59:13.988 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T22:59:13.988 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T22:59:13.988 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T22:59:13.988 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T22:59:13.989 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T22:59:13.989 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T22:59:13.989 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T22:59:13.989 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T22:59:13.989 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T22:59:13.990 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T22:59:13.990 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T22:59:13.990 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T22:59:13.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:13.991 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T22:59:13.991 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T22:59:13.991 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T22:59:13.991 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T22:59:13.991 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T22:59:13.992 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T22:59:13.992 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T22:59:13.992 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T22:59:13.992 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T22:59:13.993 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T22:59:13.993 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T22:59:13.993 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T22:59:13.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T22:59:13.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T22:59:13.994 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T22:59:13.994 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T22:59:13.994 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T22:59:13.994 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T22:59:13.995 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T22:59:13.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T22:59:13.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T22:59:13.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T22:59:13.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T22:59:13.996 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T22:59:13.996 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T22:59:13.996 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T22:59:13.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T22:59:13.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:13.997 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T22:59:13.997 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T22:59:13.997 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T22:59:13.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T22:59:13.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T22:59:13.998 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T22:59:13.998 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T22:59:13.998 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T22:59:13.999 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T22:59:13.999 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T22:59:13.999 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T22:59:13.999 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T22:59:14.000 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T22:59:14.000 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T22:59:14.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.001 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:14.001 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T22:59:14.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.001 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:59:14.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T22:59:14.002 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.002 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.002 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.002 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.003 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.003 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.003 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.003 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:14.003 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.004 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.004 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.004 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T22:59:14.005 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T22:59:14.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.008 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T22:59:14.009 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T22:59:14.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.011 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.011 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T22:59:14.012 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.013 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T22:59:14.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.013 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.014 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.014 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.015 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T22:59:14.015 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T22:59:14.016 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T22:59:14.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.017 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T22:59:14.017 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T22:59:14.017 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T22:59:14.018 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T22:59:14.018 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.018 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.018 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.019 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:59:14.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T22:59:14.020 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.020 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.020 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.021 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.021 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.021 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.021 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 43, 2022-08-01T22:59:14.022 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.022 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.022 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.022 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.023 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T22:59:14.023 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T22:59:14.024 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T22:59:14.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T22:59:14.027 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.028 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.028 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T22:59:14.029 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.030 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T22:59:14.031 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.032 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T22:59:14.033 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.034 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T22:59:14.034 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T22:59:14.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.035 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T22:59:14.035 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T22:59:14.035 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T22:59:14.035 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T22:59:14.036 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.036 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.036 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.037 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:59:14.037 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T22:59:14.037 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.038 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.038 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.038 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.038 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.038 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.039 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.039 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2022-08-01T22:59:14.039 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.039 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.039 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.040 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.040 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T22:59:14.041 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.041 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T22:59:14.042 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.042 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.043 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T22:59:14.044 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T22:59:14.045 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.046 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T22:59:14.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T22:59:14.048 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.049 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T22:59:14.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T22:59:14.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T22:59:14.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.052 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T22:59:14.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T22:59:14.052 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T22:59:14.052 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T22:59:14.053 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.053 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.053 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.054 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:59:14.054 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T22:59:14.054 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.055 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.055 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.055 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.055 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.055 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.056 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.056 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:14.056 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.056 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.057 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T22:59:14.058 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T22:59:14.059 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.060 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T22:59:14.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.062 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T22:59:14.062 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.064 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T22:59:14.064 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.065 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T22:59:14.065 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.066 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T22:59:14.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.068 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T22:59:14.068 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T22:59:14.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.069 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T22:59:14.069 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T22:59:14.070 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T22:59:14.070 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T22:59:14.070 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.070 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.071 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.072 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:59:14.072 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T22:59:14.072 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.072 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.073 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.073 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.073 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.073 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.073 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.074 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:14.074 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.074 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.074 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.074 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.075 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T22:59:14.075 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.076 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T22:59:14.076 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.077 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.077 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T22:59:14.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T22:59:14.080 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.081 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.081 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.082 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T22:59:14.082 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T22:59:14.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.084 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.085 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.087 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T22:59:14.087 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.088 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T22:59:14.088 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T22:59:14.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.089 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.089 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T22:59:14.089 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T22:59:14.089 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T22:59:14.090 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T22:59:14.090 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.090 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.090 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.091 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.091 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:59:14.091 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T22:59:14.092 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.092 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.092 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.092 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.093 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.093 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.093 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.093 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:14.093 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T22:59:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.096 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T22:59:14.096 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.097 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.097 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.098 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T22:59:14.098 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T22:59:14.099 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.100 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.100 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.100 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.101 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T22:59:14.101 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.102 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T22:59:14.102 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.103 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.103 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.103 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.104 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T22:59:14.104 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.105 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T22:59:14.105 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T22:59:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.106 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T22:59:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T22:59:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T22:59:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T22:59:14.107 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.108 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:59:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T22:59:14.109 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.110 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 42, 2022-08-01T22:59:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.111 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.112 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T22:59:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T22:59:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T22:59:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T22:59:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.117 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T22:59:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.120 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T22:59:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.121 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T22:59:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.123 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T22:59:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T22:59:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.124 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T22:59:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T22:59:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T22:59:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T22:59:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:59:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T22:59:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T22:59:14.127 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T22:59:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T22:59:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T22:59:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T22:59:14.128 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T22:59:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T22:59:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 41, 2022-08-01T22:59:14.129 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T22:59:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T22:59:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T22:59:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T22:59:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T22:59:14.133 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.134 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T22:59:14.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T22:59:14.137 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T22:59:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T22:59:14.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T22:59:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T22:59:14.142 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.143 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T22:59:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T22:59:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T22:59:14.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T22:59:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T22:59:14.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T22:59:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T22:59:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T22:59:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T22:59:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T22:59:14.147 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T22:59:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T22:59:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T22:59:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T22:59:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:14.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T22:59:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T22:59:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.150 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T22:59:14.151 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T22:59:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.153 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T22:59:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T22:59:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.155 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T22:59:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T22:59:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.157 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T22:59:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T22:59:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.159 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T22:59:14.160 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T22:59:14.161 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.162 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T22:59:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T22:59:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.164 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.165 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T22:59:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:14.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T22:59:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T22:59:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T22:59:14.167 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T22:59:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T22:59:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T22:59:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T22:59:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T22:59:14.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T22:59:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T22:59:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T22:59:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T22:59:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T22:59:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T22:59:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/505560818": "2022-08-02T22:59:07.410946+0000", 2022-08-01T22:59:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/505560818": "2022-08-02T22:59:07.410946+0000" 2022-08-01T22:59:14.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T22:59:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T22:59:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T22:59:14.171 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:59:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T22:59:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T22:59:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:59:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T22:59:14.172 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:59:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:59:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T22:59:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T22:59:14.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T22:59:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T22:59:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T22:59:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T22:59:14.174 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T22:59:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T22:59:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T22:59:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:14.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T22:59:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T22:59:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T22:59:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T22:59:14.176 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T22:59:14.177 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:59:14.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f690c053a10 msgr2=0x7f690c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f690c053a10 0x7f690c055ed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f692000d000 tx=0x7f6920009040).stop 2022-08-01T22:59:14.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 msgr2=0x7f692415abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 0x7f692415abb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f69241478c0 tx=0x7f6914015000).stop 2022-08-01T22:59:14.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 shutdown_connections 2022-08-01T22:59:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f690c053a10 0x7f690c055ed0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6924142bf0 0x7f6924143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69240b1f40 0x7f692415abb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 --2- 172.21.15.32:0/148694636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69240a7250 0x7f69240b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 >> 172.21.15.32:0/148694636 conn(0x7f692405f840 msgr2=0x7f6924143f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 shutdown_connections 2022-08-01T22:59:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:13.959+0000 7f692b81d700 1 -- 172.21.15.32:0/148694636 wait complete. 2022-08-01T22:59:14.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:14.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 Processor -- start 2022-08-01T22:59:14.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- start start 2022-08-01T22:59:14.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21e80ad750 0x7f21e8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e80aee80 con 0x7f21e81405d0 2022-08-01T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e80b70b0 con 0x7f21e80614b0 2022-08-01T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e80af0b0 con 0x7f21e80ad750 2022-08-01T22:59:14.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21dffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21e80ad750 0x7f21e8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21e7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21e81405d0 0x7f21e81466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58922/0 (socket says 172.21.15.32:58922) 2022-08-01T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21e7fff700 1 -- 172.21.15.32:0/2218682633 learned_addr learned my addr 172.21.15.32:0/2218682633 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21dffff700 1 -- 172.21.15.32:0/2218682633 >> v1:172.21.15.55:6789/0 conn(0x7f21e80614b0 legacy=0x7f21e80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21dffff700 1 -- 172.21.15.32:0/2218682633 >> v1:172.21.15.32:6789/0 conn(0x7f21e81405d0 legacy=0x7f21e81466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21e6ffd700 1 -- 172.21.15.32:0/2218682633 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605524101 0 0) 0x7f21e80aee80 con 0x7f21e81405d0 2022-08-01T22:59:14.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21dffff700 1 -- 172.21.15.32:0/2218682633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80af350 con 0x7f21e80ad750 2022-08-01T22:59:14.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21dffff700 1 --2- 172.21.15.32:0/2218682633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e80ad750 0x7f21e8061890 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f21d0002f70 tx=0x7f21d0006df0).ready entity=mon.0 client_cookie=348e2922f0258653 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21e6ffd700 1 -- 172.21.15.32:0/2218682633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21d000d610 con 0x7f21e80ad750 2022-08-01T22:59:14.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21e6ffd700 1 -- 172.21.15.32:0/2218682633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d0000e90 con 0x7f21e80ad750 2022-08-01T22:59:14.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21e6ffd700 1 -- 172.21.15.32:0/2218682633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21d000d960 con 0x7f21e80ad750 2022-08-01T22:59:14.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- 172.21.15.32:0/2218682633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e80ad750 msgr2=0x7f21e8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 --2- 172.21.15.32:0/2218682633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e80ad750 0x7f21e8061890 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f21d0002f70 tx=0x7f21d0006df0).stop 2022-08-01T22:59:14.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- 172.21.15.32:0/2218682633 shutdown_connections 2022-08-01T22:59:14.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 --2- 172.21.15.32:0/2218682633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e80ad750 0x7f21e8061890 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- 172.21.15.32:0/2218682633 >> 172.21.15.32:0/2218682633 conn(0x7f21e8066f70 msgr2=0x7f21e8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:14.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- 172.21.15.32:0/2218682633 shutdown_connections 2022-08-01T22:59:14.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.291+0000 7f21eca7d700 1 -- 172.21.15.32:0/2218682633 wait complete. 2022-08-01T22:59:14.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 Processor -- start 2022-08-01T22:59:14.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 -- start start 2022-08-01T22:59:14.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21e80614b0 0x7f21e8141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21e80ad750 0x7f21e8152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 0x7f21e8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80b9790 con 0x7f21e81405d0 2022-08-01T22:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8152550 con 0x7f21e80614b0 2022-08-01T22:59:14.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21eca7d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80b3e60 con 0x7f21e80ad750 2022-08-01T22:59:14.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 0x7f21e8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 0x7f21e8155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43154/0 (socket says 172.21.15.32:43154) 2022-08-01T22:59:14.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 -- 172.21.15.32:0/430433983 learned_addr learned my addr 172.21.15.32:0/430433983 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:14.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e77fe700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21e80614b0 0x7f21e8141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21dffff700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21e80ad750 0x7f21e8152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 -- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21e80ad750 msgr2=0x7f21e8152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21e80ad750 0x7f21e8152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 -- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21e80614b0 msgr2=0x7f21e8141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21e80614b0 0x7f21e8141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 -- 172.21.15.32:0/430433983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d000b180 con 0x7f21e81405d0 2022-08-01T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e77fe700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21e80614b0 0x7f21e8141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21dffff700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21e80ad750 0x7f21e8152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e7fff700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 0x7f21e8155aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f21e0000f40 tx=0x7f21e0001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.295+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e0005ba0 con 0x7f21e81405d0 2022-08-01T22:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e0005d00 con 0x7f21e81405d0 2022-08-01T22:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e00072c0 con 0x7f21e81405d0 2022-08-01T22:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e8152740 con 0x7f21e81405d0 2022-08-01T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e80b13b0 con 0x7f21e81405d0 2022-08-01T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e80af5f0 con 0x7f21e81405d0 2022-08-01T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f21e0010070 con 0x7f21e81405d0 2022-08-01T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e57fa700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21cc053a70 0x7f21cc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f21e006b710 con 0x7f21e81405d0 2022-08-01T22:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e77fe700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21cc053a70 0x7f21cc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.299+0000 7f21e77fe700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21cc053a70 0x7f21cc055f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f21d800b000 tx=0x7f21d8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.307+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f21e00427b0 con 0x7f21e81405d0 2022-08-01T22:59:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.467+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e80574e0 con 0x7f21e81405d0 2022-08-01T22:59:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21e57fa700 1 -- 172.21.15.32:0/430433983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f21e0045dd0 con 0x7f21e81405d0 2022-08-01T22:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3603298978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3603298978},{"type":"v1","addr":"172.21.15.32:6835","nonce":3603298978}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[]} 2022-08-01T22:59:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21cc053a70 msgr2=0x7f21cc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21cc053a70 0x7f21cc055f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f21d800b000 tx=0x7f21d8009040).stop 2022-08-01T22:59:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 msgr2=0x7f21e8155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 0x7f21e8155aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f21e0000f40 tx=0x7f21e0001360).stop 2022-08-01T22:59:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 shutdown_connections 2022-08-01T22:59:14.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21cc053a70 0x7f21cc055f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e81405d0 0x7f21e8155aa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21e80ad750 0x7f21e8152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 --2- 172.21.15.32:0/430433983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21e80614b0 0x7f21e8141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.471+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 >> 172.21.15.32:0/430433983 conn(0x7f21e8066f70 msgr2=0x7f21e8146240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.475+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 shutdown_connections 2022-08-01T22:59:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.475+0000 7f21eca7d700 1 -- 172.21.15.32:0/430433983 wait complete. 2022-08-01T22:59:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2022-08-01T22:59:14.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 Processor -- start 2022-08-01T22:59:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- start start 2022-08-01T22:59:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19400b1f40 0x7f19400b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f19400a7250 0x7f19401448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19400b04e0 con 0x7f1940148530 2022-08-01T22:59:14.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19400a9250 con 0x7f19400b1f40 2022-08-01T22:59:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19400b0780 con 0x7f19400a7250 2022-08-01T22:59:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19400b1f40 0x7f19400b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19400b1f40 0x7f19400b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43158/0 (socket says 172.21.15.32:43158) 2022-08-01T22:59:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 -- 172.21.15.32:0/1227252357 learned_addr learned my addr 172.21.15.32:0/1227252357 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946917700 1 --2- 172.21.15.32:0/1227252357 >> v2:172.21.15.55:3300/0 conn(0x7f19400a7250 0x7f19401448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 -- 172.21.15.32:0/1227252357 >> v2:172.21.15.55:3300/0 conn(0x7f19400a7250 msgr2=0x7f19401448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 --2- 172.21.15.32:0/1227252357 >> v2:172.21.15.55:3300/0 conn(0x7f19400a7250 0x7f19401448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 -- 172.21.15.32:0/1227252357 >> v1:172.21.15.32:6789/0 conn(0x7f1940148530 legacy=0x7f19400b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 -- 172.21.15.32:0/1227252357 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19400b0a20 con 0x7f19400b1f40 2022-08-01T22:59:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1946116700 1 --2- 172.21.15.32:0/1227252357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b1f40 0x7f19400b1160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f19401478c0 tx=0x7f193c006eb0).ready entity=mon.0 client_cookie=d800a19742e2aa59 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1945114700 1 -- 172.21.15.32:0/1227252357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193c00d610 con 0x7f19400b1f40 2022-08-01T22:59:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1945114700 1 -- 172.21.15.32:0/1227252357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c000e90 con 0x7f19400b1f40 2022-08-01T22:59:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f1945114700 1 -- 172.21.15.32:0/1227252357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193c00d960 con 0x7f19400b1f40 2022-08-01T22:59:14.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- 172.21.15.32:0/1227252357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b1f40 msgr2=0x7f19400b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 --2- 172.21.15.32:0/1227252357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b1f40 0x7f19400b1160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f19401478c0 tx=0x7f193c006eb0).stop 2022-08-01T22:59:14.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- 172.21.15.32:0/1227252357 shutdown_connections 2022-08-01T22:59:14.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 --2- 172.21.15.32:0/1227252357 >> v2:172.21.15.55:3300/0 conn(0x7f19400a7250 0x7f19401448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 --2- 172.21.15.32:0/1227252357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19400b1f40 0x7f19400b1160 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- 172.21.15.32:0/1227252357 >> 172.21.15.32:0/1227252357 conn(0x7f194005f840 msgr2=0x7f194005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:14.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- 172.21.15.32:0/1227252357 shutdown_connections 2022-08-01T22:59:14.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.595+0000 7f194837a700 1 -- 172.21.15.32:0/1227252357 wait complete. 2022-08-01T22:59:14.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 Processor -- start 2022-08-01T22:59:14.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 -- start start 2022-08-01T22:59:14.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19400a7250 0x7f19400af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19400b1f40 0x7f194015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 0x7f194015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19400636e0 con 0x7f1940148530 2022-08-01T22:59:14.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940063980 con 0x7f19400b1f40 2022-08-01T22:59:14.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1940063c20 con 0x7f19400a7250 2022-08-01T22:59:14.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 0x7f194015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 0x7f194015a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43168/0 (socket says 172.21.15.32:43168) 2022-08-01T22:59:14.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946116700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19400a7250 0x7f19400af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 -- 172.21.15.32:0/2393130655 learned_addr learned my addr 172.21.15.32:0/2393130655 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:14.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1945915700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19400b1f40 0x7f194015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 -- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19400a7250 msgr2=0x7f19400af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19400a7250 0x7f19400af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 -- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19400b1f40 msgr2=0x7f194015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19400b1f40 0x7f194015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 -- 172.21.15.32:0/2393130655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c00b1e0 con 0x7f1940148530 2022-08-01T22:59:14.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f1946917700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 0x7f194015a6a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f193800a000 tx=0x7f1938008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1938013620 con 0x7f1940148530 2022-08-01T22:59:14.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938013780 con 0x7f1940148530 2022-08-01T22:59:14.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1940063ec0 con 0x7f1940148530 2022-08-01T22:59:14.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1938013ae0 con 0x7f1940148530 2022-08-01T22:59:14.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.599+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19400a98f0 con 0x7f1940148530 2022-08-01T22:59:14.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.603+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19400b0cc0 con 0x7f1940148530 2022-08-01T22:59:14.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.603+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1938013c40 con 0x7f1940148530 2022-08-01T22:59:14.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.603+0000 7f192f7fe700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1920053a70 0x7f1920055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.603+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f193806bc10 con 0x7f1940148530 2022-08-01T22:59:14.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.603+0000 7f1946116700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1920053a70 0x7f1920055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.607+0000 7f1946116700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1920053a70 0x7f1920055f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f193c01c000 tx=0x7f193c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.611+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1938042ba0 con 0x7f1940148530 2022-08-01T22:59:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.775+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19400574e0 con 0x7f1940148530 2022-08-01T22:59:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.775+0000 7f192f7fe700 1 -- 172.21.15.32:0/2393130655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f193800f040 con 0x7f1940148530 2022-08-01T22:59:14.779 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:14.780 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1545587613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1545587613},{"type":"v1","addr":"172.21.15.55:6835","nonce":1545587613}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1822200822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1822200822},{"type":"v1","addr":"172.21.15.32:6833","nonce":1822200822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3603298978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3603298978},{"type":"v1","addr":"172.21.15.32:6835","nonce":3603298978}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[]} 2022-08-01T22:59:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1920053a70 msgr2=0x7f1920055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1920053a70 0x7f1920055f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f193c01c000 tx=0x7f193c015000).stop 2022-08-01T22:59:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 msgr2=0x7f194015a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 0x7f194015a6a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f193800a000 tx=0x7f1938008040).stop 2022-08-01T22:59:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 shutdown_connections 2022-08-01T22:59:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1920053a70 0x7f1920055f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1940148530 0x7f194015a6a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19400b1f40 0x7f194015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 --2- 172.21.15.32:0/2393130655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19400a7250 0x7f19400af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 >> 172.21.15.32:0/2393130655 conn(0x7f194005f840 msgr2=0x7f1940143fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 shutdown_connections 2022-08-01T22:59:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.779+0000 7f194837a700 1 -- 172.21.15.32:0/2393130655 wait complete. 2022-08-01T22:59:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2022-08-01T22:59:14.793 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T22:59:14.793 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T22:59:14.794 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T22:59:14.794 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:59:14.794 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T22:59:14.795 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T22:59:14.795 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T22:59:14.795 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:59:14.796 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T22:59:14.796 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T22:59:14.796 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T22:59:14.797 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T22:59:14.808 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T22:59:14.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T22:59:14.811 INFO:tasks.ceph.mds.a:Started 2022-08-01T22:59:14.812 DEBUG:teuthology.parallel:result is None 2022-08-01T22:59:14.813 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T22:59:14.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T22:59:14.815 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T22:59:14.815 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T22:59:14.817 INFO:tasks.ceph.mds.c:Started 2022-08-01T22:59:14.817 DEBUG:teuthology.parallel:result is None 2022-08-01T22:59:14.818 INFO:tasks.ceph.mds.b:Started 2022-08-01T22:59:14.818 DEBUG:teuthology.parallel:result is None 2022-08-01T22:59:14.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T22:59:14.882 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T22:59:14.885 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T22:59:14.908 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T22:59:14.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 Processor -- start 2022-08-01T22:59:14.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 -- start start 2022-08-01T22:59:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c400b2e80 0x7f6c400b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c400b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c400b2400 con 0x7f6c40064cd0 2022-08-01T22:59:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c400623c0 con 0x7f6c400b2e80 2022-08-01T22:59:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c46b3c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400650b0 con 0x7f6c40144140 2022-08-01T22:59:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c45b3a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c40064cd0 0x7f6c40143a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58976/0 (socket says 172.21.15.32:58976) 2022-08-01T22:59:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c45b3a700 1 -- 172.21.15.32:0/368456497 learned_addr learned my addr 172.21.15.32:0/368456497 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c4633b700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c400b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c45339700 1 --2- 172.21.15.32:0/368456497 >> v2:172.21.15.55:3300/0 conn(0x7f6c400b2e80 0x7f6c400b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171839464 0 0) 0x7f6c400b2400 con 0x7f6c40064cd0 2022-08-01T22:59:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.923+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c28003260 con 0x7f6c40064cd0 2022-08-01T22:59:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1449582724 0 0) 0x7f6c28003260 con 0x7f6c40064cd0 2022-08-01T22:59:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 msgr2=0x7f6c400b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:14.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c400b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 >> v2:172.21.15.55:3300/0 conn(0x7f6c400b2e80 msgr2=0x7f6c400b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 --2- 172.21.15.32:0/368456497 >> v2:172.21.15.55:3300/0 conn(0x7f6c400b2e80 0x7f6c400b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400658f0 con 0x7f6c40064cd0 2022-08-01T22:59:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f6c3c001660 con 0x7f6c40064cd0 2022-08-01T22:59:14.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 >> v1:172.21.15.32:6789/0 conn(0x7f6c40064cd0 legacy=0x7f6c40143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c28003750 0x7f6c28003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c28004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c280049f0 0x7f6c28004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400623c0 con 0x7f6c280049f0 2022-08-01T22:59:14.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400650b0 con 0x7f6c28003750 2022-08-01T22:59:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2800b750 con 0x7f6c40144140 2022-08-01T22:59:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c280049f0 0x7f6c28004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c45b3a700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c28003750 0x7f6c28003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c45339700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c28004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 -- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 msgr2=0x7f6c28004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c28004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 -- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c28003750 msgr2=0x7f6c28003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c28003750 0x7f6c28003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 -- 172.21.15.32:0/368456497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28003260 con 0x7f6c280049f0 2022-08-01T22:59:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c45339700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c40144140 0x7f6c28004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c280049f0 0x7f6c28004e70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f6c3800c000 tx=0x7f6c380091d0).ready entity=mon.0 client_cookie=45ff887dfeb73cd2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c3800e6e0 con 0x7f6c280049f0 2022-08-01T22:59:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3800e840 con 0x7f6c280049f0 2022-08-01T22:59:14.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c44b38700 1 -- 172.21.15.32:0/368456497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c3800eba0 con 0x7f6c280049f0 2022-08-01T22:59:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c46b3c700 1 -- 172.21.15.32:0/368456497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c280049f0 msgr2=0x7f6c28004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/368456497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c280049f0 0x7f6c28004e70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f6c3800c000 tx=0x7f6c380091d0).stop 2022-08-01T22:59:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c4633b700 1 -- 172.21.15.32:0/368456497 reap_dead start 2022-08-01T22:59:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c46b3c700 1 -- 172.21.15.32:0/368456497 shutdown_connections 2022-08-01T22:59:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c46b3c700 1 -- 172.21.15.32:0/368456497 >> 172.21.15.32:0/368456497 conn(0x7f6c40062d70 msgr2=0x7f6c4005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c46b3c700 1 -- 172.21.15.32:0/368456497 shutdown_connections 2022-08-01T22:59:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.927+0000 7f6c46b3c700 1 -- 172.21.15.32:0/368456497 wait complete. 2022-08-01T22:59:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 Processor -- start 2022-08-01T22:59:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 -- start start 2022-08-01T22:59:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c280049f0 0x7f6c400b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 0x7f6c400b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c4013e9c0 0x7f6c4013ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40144580 con 0x7f6c400b1660 2022-08-01T22:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400b1300 con 0x7f6c280049f0 2022-08-01T22:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401447b0 con 0x7f6c4013e9c0 2022-08-01T22:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 0x7f6c400b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45b3a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c280049f0 0x7f6c400b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 0x7f6c400b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43218/0 (socket says 172.21.15.32:43218) 2022-08-01T22:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 -- 172.21.15.32:0/3477479652 learned_addr learned my addr 172.21.15.32:0/3477479652 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c4633b700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c4013e9c0 0x7f6c4013ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 -- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c4013e9c0 msgr2=0x7f6c4013ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c4013e9c0 0x7f6c4013ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 -- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c280049f0 msgr2=0x7f6c400b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c280049f0 0x7f6c400b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 -- 172.21.15.32:0/3477479652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c40144a50 con 0x7f6c400b1660 2022-08-01T22:59:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45b3a700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c280049f0 0x7f6c400b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c45339700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 0x7f6c400b0c60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f6c30006000 tx=0x7f6c30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c30014470 con 0x7f6c400b1660 2022-08-01T22:59:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c300145d0 con 0x7f6c400b1660 2022-08-01T22:59:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c30014930 con 0x7f6c400b1660 2022-08-01T22:59:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4013f330 con 0x7f6c400b1660 2022-08-01T22:59:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.931+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c40146290 con 0x7f6c400b1660 2022-08-01T22:59:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.935+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c30014a90 con 0x7f6c400b1660 2022-08-01T22:59:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c400b1470 con 0x7f6c400b1660 2022-08-01T22:59:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.935+0000 7f6c2effd700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c20053a70 0x7f6c20055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.935+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6c3006c490 con 0x7f6c400b1660 2022-08-01T22:59:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.935+0000 7f6c45b3a700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c20053a70 0x7f6c20055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.935+0000 7f6c45b3a700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c20053a70 0x7f6c20055f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f6c3c005000 tx=0x7f6c3c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:14.943+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c400b1470 con 0x7f6c400b1660 2022-08-01T22:59:15.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.079+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6c400b1470 con 0x7f6c400b1660 2022-08-01T22:59:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.931+0000 7f6c2effd700 1 -- 172.21.15.32:0/3477479652 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c400b1470 con 0x7f6c400b1660 2022-08-01T22:59:15.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c20053a70 msgr2=0x7f6c20055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c20053a70 0x7f6c20055f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f6c3c005000 tx=0x7f6c3c009040).stop 2022-08-01T22:59:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 msgr2=0x7f6c400b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 0x7f6c400b0c60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f6c30006000 tx=0x7f6c30008040).stop 2022-08-01T22:59:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 shutdown_connections 2022-08-01T22:59:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c20053a70 0x7f6c20055f30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c4013e9c0 0x7f6c4013ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400b1660 0x7f6c400b0c60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 --2- 172.21.15.32:0/3477479652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c280049f0 0x7f6c400b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 >> 172.21.15.32:0/3477479652 conn(0x7f6c40062d70 msgr2=0x7f6c4005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 shutdown_connections 2022-08-01T22:59:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:15.935+0000 7f6c46b3c700 1 -- 172.21.15.32:0/3477479652 wait complete. 2022-08-01T22:59:15.942 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T22:59:15.952 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T22:59:15.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T22:59:16.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 Processor -- start 2022-08-01T22:59:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- start start 2022-08-01T22:59:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4900b0630 0x7ff4900adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900b2c80 con 0x7ff4900aa2b0 2022-08-01T22:59:16.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900b2900 con 0x7ff4900b0a10 2022-08-01T22:59:16.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4900b2eb0 con 0x7ff4900b0630 2022-08-01T22:59:16.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4900b0630 0x7ff4900adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4900aa2b0 0x7ff4900ae3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59000/0 (socket says 172.21.15.32:59000) 2022-08-01T22:59:16.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48f7fe700 1 -- 172.21.15.32:0/364486375 learned_addr learned my addr 172.21.15.32:0/364486375 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:16.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48ffff700 1 -- 172.21.15.32:0/364486375 >> v1:172.21.15.55:6789/0 conn(0x7ff4900b0a10 legacy=0x7ff490140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48ffff700 1 -- 172.21.15.32:0/364486375 >> v1:172.21.15.32:6789/0 conn(0x7ff4900aa2b0 legacy=0x7ff4900ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48ffff700 1 -- 172.21.15.32:0/364486375 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900a9d00 con 0x7ff4900b0630 2022-08-01T22:59:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48ffff700 1 --2- 172.21.15.32:0/364486375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0630 0x7ff4900adec0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7ff478002f70 tx=0x7ff478006df0).ready entity=mon.0 client_cookie=3bfe2e7b69436309 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48effd700 1 -- 172.21.15.32:0/364486375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47800d610 con 0x7ff4900b0630 2022-08-01T22:59:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48effd700 1 -- 172.21.15.32:0/364486375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478000e90 con 0x7ff4900b0630 2022-08-01T22:59:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff48effd700 1 -- 172.21.15.32:0/364486375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff47800d960 con 0x7ff4900b0630 2022-08-01T22:59:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- 172.21.15.32:0/364486375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0630 msgr2=0x7ff4900adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 --2- 172.21.15.32:0/364486375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0630 0x7ff4900adec0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7ff478002f70 tx=0x7ff478006df0).stop 2022-08-01T22:59:16.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- 172.21.15.32:0/364486375 shutdown_connections 2022-08-01T22:59:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 --2- 172.21.15.32:0/364486375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0630 0x7ff4900adec0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- 172.21.15.32:0/364486375 >> 172.21.15.32:0/364486375 conn(0x7ff490060950 msgr2=0x7ff490060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- 172.21.15.32:0/364486375 shutdown_connections 2022-08-01T22:59:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.063+0000 7ff49668e700 1 -- 172.21.15.32:0/364486375 wait complete. 2022-08-01T22:59:16.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.067+0000 7ff49668e700 1 Processor -- start 2022-08-01T22:59:16.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.067+0000 7ff49668e700 1 -- start start 2022-08-01T22:59:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4900aa2b0 0x7ff49005fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4900b0630 0x7ff490064c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 0x7ff490065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900642b0 con 0x7ff4900b0a10 2022-08-01T22:59:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490063fb0 con 0x7ff4900b0630 2022-08-01T22:59:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff490065780 con 0x7ff4900aa2b0 2022-08-01T22:59:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 0x7ff490065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 0x7ff490065180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43236/0 (socket says 172.21.15.32:43236) 2022-08-01T22:59:16.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 -- 172.21.15.32:0/513463760 learned_addr learned my addr 172.21.15.32:0/513463760 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:16.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48ffff700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4900aa2b0 0x7ff49005fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48f7fe700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4900b0630 0x7ff490064c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 -- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4900aa2b0 msgr2=0x7ff49005fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4900aa2b0 0x7ff49005fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 -- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4900b0630 msgr2=0x7ff490064c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4900b0630 0x7ff490064c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 -- 172.21.15.32:0/513463760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47800b180 con 0x7ff4900b0a10 2022-08-01T22:59:16.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48ffff700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4900aa2b0 0x7ff49005fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:16.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff494c2b700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 0x7ff490065180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7ff4900b7900 tx=0x7ff488007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff488006aa0 con 0x7ff4900b0a10 2022-08-01T22:59:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff488006c00 con 0x7ff4900b0a10 2022-08-01T22:59:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff490063d10 con 0x7ff4900b0a10 2022-08-01T22:59:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff488007030 con 0x7ff4900b0a10 2022-08-01T22:59:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff490060510 con 0x7ff4900b0a10 2022-08-01T22:59:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4900574e0 con 0x7ff4900b0a10 2022-08-01T22:59:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.071+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff488010030 con 0x7ff4900b0a10 2022-08-01T22:59:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.075+0000 7ff48d7fa700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff474053b00 0x7ff474055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.075+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff48806bac0 con 0x7ff4900b0a10 2022-08-01T22:59:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.075+0000 7ff48ffff700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff474053b00 0x7ff474055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.075+0000 7ff48ffff700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff474053b00 0x7ff474055fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7ff47801a000 tx=0x7ff478015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.079+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff488042bc0 con 0x7ff4900b0a10 2022-08-01T22:59:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.247+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff490067aa0 con 0x7ff4900b0a10 2022-08-01T22:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.247+0000 7ff48d7fa700 1 -- 172.21.15.32:0/513463760 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+4255 (secure 0 0 0) 0x7ff4880461e0 con 0x7ff4900b0a10 2022-08-01T22:59:16.251 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:16.251 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:59:16.252 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T22:59:16.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.252 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T22:59:16.252 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T22:59:16.252 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.253 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:59:16.253 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.253 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.253 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.254 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T22:59:16.255 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T22:59:16.255 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.255 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:59:16.255 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.255 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.256 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.257 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T22:59:16.257 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T22:59:16.257 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.257 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:59:16.257 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.258 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.258 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T22:59:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T22:59:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T22:59:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T22:59:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T22:59:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T22:59:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T22:59:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T22:59:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T22:59:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T22:59:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T22:59:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T22:59:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T22:59:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T22:59:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T22:59:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T22:59:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T22:59:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T22:59:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:59:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T22:59:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T22:59:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T22:59:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T22:59:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T22:59:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T22:59:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T22:59:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T22:59:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T22:59:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:59:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T22:59:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T22:59:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T22:59:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T22:59:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:59:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T22:59:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T22:59:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T22:59:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T22:59:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T22:59:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T22:59:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T22:59:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T22:59:16.287 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:59:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff474053b00 msgr2=0x7ff474055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff474053b00 0x7ff474055fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7ff47801a000 tx=0x7ff478015000).stop 2022-08-01T22:59:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 msgr2=0x7ff490065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 0x7ff490065180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7ff4900b7900 tx=0x7ff488007f40).stop 2022-08-01T22:59:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 shutdown_connections 2022-08-01T22:59:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff474053b00 0x7ff474055fc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4900b0a10 0x7ff490065180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4900b0630 0x7ff490064c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 --2- 172.21.15.32:0/513463760 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4900aa2b0 0x7ff49005fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 >> 172.21.15.32:0/513463760 conn(0x7ff490060950 msgr2=0x7ff4900b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 shutdown_connections 2022-08-01T22:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.251+0000 7ff49668e700 1 -- 172.21.15.32:0/513463760 wait complete. 2022-08-01T22:59:16.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:59:16.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0eb59e700 1 Processor -- start 2022-08-01T22:59:16.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0eb59e700 1 -- start start 2022-08-01T22:59:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0eb59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc0ec0b2710 0x7fc0ec14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0eb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec0a7250 con 0x7fc0ec0b01a0 2022-08-01T22:59:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0eb59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec0a7480 con 0x7fc0ec0a8c50 2022-08-01T22:59:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0eb59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec0a7720 con 0x7fc0ec0b2710 2022-08-01T22:59:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc0ec0b01a0 0x7fc0ec0a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59018/0 (socket says 172.21.15.32:59018) 2022-08-01T22:59:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e9d9b700 1 -- 172.21.15.32:0/1602073199 learned_addr learned my addr 172.21.15.32:0/1602073199 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0ea59c700 1 --2- 172.21.15.32:0/1602073199 >> v2:172.21.15.55:3300/0 conn(0x7fc0ec0b2710 0x7fc0ec14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408152665 0 0) 0x7fc0ec0a7250 con 0x7fc0ec0b01a0 2022-08-01T22:59:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d0003260 con 0x7fc0ec0b01a0 2022-08-01T22:59:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2293969503 0 0) 0x7fc0d0003260 con 0x7fc0ec0b01a0 2022-08-01T22:59:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 >> v2:172.21.15.55:3300/0 conn(0x7fc0ec0b2710 msgr2=0x7fc0ec14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:16.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 --2- 172.21.15.32:0/1602073199 >> v2:172.21.15.55:3300/0 conn(0x7fc0ec0b2710 0x7fc0ec14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 >> v1:172.21.15.55:6789/0 conn(0x7fc0ec0a8c50 legacy=0x7fc0ec151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0a79c0 con 0x7fc0ec0b01a0 2022-08-01T22:59:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fc0e0001660 con 0x7fc0ec0b01a0 2022-08-01T22:59:16.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 >> v1:172.21.15.32:6789/0 conn(0x7fc0ec0b01a0 legacy=0x7fc0ec0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0003750 0x7fc0d0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0d0004500 0x7fc0d0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0d0004e70 0x7fc0d000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0a7720 con 0x7fc0d0004e70 2022-08-01T22:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d000bd20 con 0x7fc0d0003750 2022-08-01T22:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d000bee0 con 0x7fc0d0004500 2022-08-01T22:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.387+0000 7fc0ead9d700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0d0004e70 0x7fc0d000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ea59c700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0003750 0x7fc0d0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0e9d9b700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0d0004500 0x7fc0d0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 -- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0d0004500 msgr2=0x7fc0d0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:16.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0d0004500 0x7fc0d0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 -- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0003750 msgr2=0x7fc0d0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:16.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0003750 0x7fc0d0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 -- 172.21.15.32:0/1602073199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0003260 con 0x7fc0d0004e70 2022-08-01T22:59:16.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0e9d9b700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0d0004500 0x7fc0d0004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:16.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 -- 172.21.15.32:0/1602073199 reap_dead start 2022-08-01T22:59:16.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0ead9d700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0d0004e70 0x7fc0d000b750 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fc0ec061c10 tx=0x7fc0dc007a00).ready entity=mon.0 client_cookie=ced077c4758f90ee server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0dc007d10 con 0x7fc0d0004e70 2022-08-01T22:59:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc007030 con 0x7fc0d0004e70 2022-08-01T22:59:16.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0e959a700 1 -- 172.21.15.32:0/1602073199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0dc007390 con 0x7fc0d0004e70 2022-08-01T22:59:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 -- 172.21.15.32:0/1602073199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0d0004e70 msgr2=0x7fc0d000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0d0004e70 0x7fc0d000b750 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fc0ec061c10 tx=0x7fc0dc007a00).stop 2022-08-01T22:59:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 -- 172.21.15.32:0/1602073199 shutdown_connections 2022-08-01T22:59:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/1602073199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0d0004e70 0x7fc0d000b750 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 -- 172.21.15.32:0/1602073199 >> 172.21.15.32:0/1602073199 conn(0x7fc0ec060310 msgr2=0x7fc0ec0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 -- 172.21.15.32:0/1602073199 shutdown_connections 2022-08-01T22:59:16.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.391+0000 7fc0eb59e700 1 -- 172.21.15.32:0/1602073199 wait complete. 2022-08-01T22:59:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 Processor -- start 2022-08-01T22:59:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 -- start start 2022-08-01T22:59:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0004e70 0x7fc0ec05c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 0x7fc0ec0abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0ec159ba0 0x7fc0ec15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec063f30 con 0x7fc0ec0ab960 2022-08-01T22:59:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec05cd40 con 0x7fc0d0004e70 2022-08-01T22:59:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0ab410 con 0x7fc0ec159ba0 2022-08-01T22:59:16.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 0x7fc0ec0abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 0x7fc0ec0abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43254/0 (socket says 172.21.15.32:43254) 2022-08-01T22:59:16.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 -- 172.21.15.32:0/808189918 learned_addr learned my addr 172.21.15.32:0/808189918 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:16.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0ea59c700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0004e70 0x7fc0ec05c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0ead9d700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0ec159ba0 0x7fc0ec15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 -- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0ec159ba0 msgr2=0x7fc0ec15a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0ec159ba0 0x7fc0ec15a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 -- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0004e70 msgr2=0x7fc0ec05c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0004e70 0x7fc0ec05c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 -- 172.21.15.32:0/808189918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec1571f0 con 0x7fc0ec0ab960 2022-08-01T22:59:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0e9d9b700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 0x7fc0ec0abde0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fc0e000b000 tx=0x7fc0ec0a7b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0e0015630 con 0x7fc0ec0ab960 2022-08-01T22:59:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e0010040 con 0x7fc0ec0ab960 2022-08-01T22:59:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec0a92d0 con 0x7fc0ec0ab960 2022-08-01T22:59:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0e0015980 con 0x7fc0ec0ab960 2022-08-01T22:59:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.395+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec14b040 con 0x7fc0ec0ab960 2022-08-01T22:59:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.399+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec0ab5d0 con 0x7fc0ec0ab960 2022-08-01T22:59:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.399+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc0e0015b40 con 0x7fc0ec0ab960 2022-08-01T22:59:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.399+0000 7fc0db7fe700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0c8053aa0 0x7fc0c8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.399+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc0e006bd30 con 0x7fc0ec0ab960 2022-08-01T22:59:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.399+0000 7fc0ea59c700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0c8053aa0 0x7fc0c8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.399+0000 7fc0ea59c700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0c8053aa0 0x7fc0c8055f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc0e400b000 tx=0x7fc0e40091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.403+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc0e0042e30 con 0x7fc0ec0ab960 2022-08-01T22:59:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.531+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0ec157020 con 0x7fc0ec0ab960 2022-08-01T22:59:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0db7fe700 1 -- 172.21.15.32:0/808189918 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13414 (secure 0 0 0) 0x7fc0e0046450 con 0x7fc0ec0ab960 2022-08-01T22:59:16.537 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:16.538 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:59:15.926532+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:59:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0c8053aa0 msgr2=0x7fc0c8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0c8053aa0 0x7fc0c8055f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc0e400b000 tx=0x7fc0e40091d0).stop 2022-08-01T22:59:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 msgr2=0x7fc0ec0abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 0x7fc0ec0abde0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fc0e000b000 tx=0x7fc0ec0a7b50).stop 2022-08-01T22:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 shutdown_connections 2022-08-01T22:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0c8053aa0 0x7fc0c8055f60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0ec159ba0 0x7fc0ec15a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0ec0ab960 0x7fc0ec0abde0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 --2- 172.21.15.32:0/808189918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0d0004e70 0x7fc0ec05c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 >> 172.21.15.32:0/808189918 conn(0x7fc0ec060310 msgr2=0x7fc0ec148b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 shutdown_connections 2022-08-01T22:59:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.535+0000 7fc0eb59e700 1 -- 172.21.15.32:0/808189918 wait complete. 2022-08-01T22:59:16.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T22:59:16.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:59:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 Processor -- start 2022-08-01T22:59:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 -- start start 2022-08-01T22:59:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb6cc0b7840 0x7fb6cc060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc0610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc0671e0 con 0x7fb6cc146720 2022-08-01T22:59:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6cc065540 con 0x7fb6cc0b7840 2022-08-01T22:59:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d202e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc0b7c20 con 0x7fb6cc0b2930 2022-08-01T22:59:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d182d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb6cc146720 0x7fb6cc146b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59038/0 (socket says 172.21.15.32:59038) 2022-08-01T22:59:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d182d700 1 -- 172.21.15.32:0/3630291020 learned_addr learned my addr 172.21.15.32:0/3630291020 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d082b700 1 --2- 172.21.15.32:0/3630291020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc0610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.659+0000 7fb6d102c700 1 --2- 172.21.15.32:0/3630291020 >> v2:172.21.15.55:3300/0 conn(0x7fb6cc0b7840 0x7fb6cc060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6c3fff700 1 -- 172.21.15.32:0/3630291020 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745772899 0 0) 0x7fb6cc0671e0 con 0x7fb6cc146720 2022-08-01T22:59:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6c3fff700 1 -- 172.21.15.32:0/3630291020 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b0003260 con 0x7fb6cc146720 2022-08-01T22:59:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6c3fff700 1 -- 172.21.15.32:0/3630291020 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047965704 0 0) 0x7fb6b0003260 con 0x7fb6cc146720 2022-08-01T22:59:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d082b700 1 -- 172.21.15.32:0/3630291020 >> v2:172.21.15.55:3300/0 conn(0x7fb6cc0b7840 msgr2=0x7fb6cc060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:16.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d082b700 1 --2- 172.21.15.32:0/3630291020 >> v2:172.21.15.55:3300/0 conn(0x7fb6cc0b7840 0x7fb6cc060be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d082b700 1 -- 172.21.15.32:0/3630291020 >> v1:172.21.15.32:6789/0 conn(0x7fb6cc146720 legacy=0x7fb6cc146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d082b700 1 -- 172.21.15.32:0/3630291020 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc066ce0 con 0x7fb6cc0b2930 2022-08-01T22:59:16.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d082b700 1 --2- 172.21.15.32:0/3630291020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc0610d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb6c4001b80 tx=0x7fb6bc006df0).ready entity=mon.2 client_cookie=2e6ebd0f16c20643 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6c3fff700 1 -- 172.21.15.32:0/3630291020 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c40019e0 con 0x7fb6cc0b2930 2022-08-01T22:59:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6c3fff700 1 -- 172.21.15.32:0/3630291020 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc000e90 con 0x7fb6cc0b2930 2022-08-01T22:59:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6c3fff700 1 -- 172.21.15.32:0/3630291020 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6bc012460 con 0x7fb6cc0b2930 2022-08-01T22:59:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- 172.21.15.32:0/3630291020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 msgr2=0x7fb6cc0610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 --2- 172.21.15.32:0/3630291020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc0610d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb6c4001b80 tx=0x7fb6bc006df0).stop 2022-08-01T22:59:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- 172.21.15.32:0/3630291020 shutdown_connections 2022-08-01T22:59:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 --2- 172.21.15.32:0/3630291020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc0610d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 --2- 172.21.15.32:0/3630291020 >> v2:172.21.15.55:3300/0 conn(0x7fb6cc0b7840 0x7fb6cc060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- 172.21.15.32:0/3630291020 >> 172.21.15.32:0/3630291020 conn(0x7fb6cc05f840 msgr2=0x7fb6cc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- 172.21.15.32:0/3630291020 shutdown_connections 2022-08-01T22:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- 172.21.15.32:0/3630291020 wait complete. 2022-08-01T22:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 Processor -- start 2022-08-01T22:59:16.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- start start 2022-08-01T22:59:16.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7840 0x7fb6cc152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 0x7fb6cc14a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc13faf0 con 0x7fb6cc146720 2022-08-01T22:59:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc140ba0 con 0x7fb6cc0b7840 2022-08-01T22:59:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d202e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc140dd0 con 0x7fb6cc0b2930 2022-08-01T22:59:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d182d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 0x7fb6cc14a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d182d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 0x7fb6cc14a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43268/0 (socket says 172.21.15.32:43268) 2022-08-01T22:59:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.663+0000 7fb6d182d700 1 -- 172.21.15.32:0/1635463305 learned_addr learned my addr 172.21.15.32:0/1635463305 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d102c700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d082b700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7840 0x7fb6cc152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d182d700 1 -- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 msgr2=0x7fb6cc148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d182d700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc148de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d182d700 1 -- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7840 msgr2=0x7fb6cc152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d182d700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7840 0x7fb6cc152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d182d700 1 -- 172.21.15.32:0/1635463305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc00b180 con 0x7fb6cc146720 2022-08-01T22:59:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d102c700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d182d700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 0x7fb6cc14a1f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fb6c4001c40 tx=0x7fb6c4002100).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c4004ab0 con 0x7fb6cc146720 2022-08-01T22:59:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c4004c10 con 0x7fb6cc146720 2022-08-01T22:59:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc141070 con 0x7fb6cc146720 2022-08-01T22:59:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c4007170 con 0x7fb6cc146720 2022-08-01T22:59:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc14a6e0 con 0x7fb6cc146720 2022-08-01T22:59:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb6c4010070 con 0x7fb6cc146720 2022-08-01T22:59:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc0574e0 con 0x7fb6cc146720 2022-08-01T22:59:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6c27fc700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb6b4053aa0 0x7fb6b4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb6c406b860 con 0x7fb6cc146720 2022-08-01T22:59:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.667+0000 7fb6d102c700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb6b4053aa0 0x7fb6b4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.671+0000 7fb6d102c700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb6b4053aa0 0x7fb6b4055f60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb6c8006000 tx=0x7fb6c8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.671+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6c4042960 con 0x7fb6cc146720 2022-08-01T22:59:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb6cc060330 con 0x7fb6cc146720 2022-08-01T22:59:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6c27fc700 1 -- 172.21.15.32:0/1635463305 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v51) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb6c4045f80 con 0x7fb6cc146720 2022-08-01T22:59:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:59:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb6b4053aa0 msgr2=0x7fb6b4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb6b4053aa0 0x7fb6b4055f60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb6c8006000 tx=0x7fb6c8008040).stop 2022-08-01T22:59:16.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 msgr2=0x7fb6cc14a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 0x7fb6cc14a1f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fb6c4001c40 tx=0x7fb6c4002100).stop 2022-08-01T22:59:16.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 shutdown_connections 2022-08-01T22:59:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb6b4053aa0 0x7fb6b4055f60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc146720 0x7fb6cc14a1f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7840 0x7fb6cc152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 --2- 172.21.15.32:0/1635463305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6cc0b2930 0x7fb6cc148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 >> 172.21.15.32:0/1635463305 conn(0x7fb6cc05f840 msgr2=0x7fb6cc0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 shutdown_connections 2022-08-01T22:59:16.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.799+0000 7fb6d202e700 1 -- 172.21.15.32:0/1635463305 wait complete. 2022-08-01T22:59:16.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:59:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.919+0000 7ff4ae9b7700 1 Processor -- start 2022-08-01T22:59:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.919+0000 7ff4ae9b7700 1 -- start start 2022-08-01T22:59:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4a8064b10 0x7ff4a8144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a80b36e0 0x7ff4a8141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a8064ef0 con 0x7ff4a8141860 2022-08-01T22:59:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a813ebf0 con 0x7ff4a8064b10 2022-08-01T22:59:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8066690 con 0x7ff4a80b36e0 2022-08-01T22:59:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae1b6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4a8141860 0x7ff4a8066ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59052/0 (socket says 172.21.15.32:59052) 2022-08-01T22:59:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae1b6700 1 -- 172.21.15.32:0/777065597 learned_addr learned my addr 172.21.15.32:0/777065597 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad1b4700 1 --2- 172.21.15.32:0/777065597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a80b36e0 0x7ff4a8141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad9b5700 1 --2- 172.21.15.32:0/777065597 >> v2:172.21.15.55:3300/0 conn(0x7ff4a8064b10 0x7ff4a8144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ac9b3700 1 -- 172.21.15.32:0/777065597 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811118833 0 0) 0x7ff4a8064ef0 con 0x7ff4a8141860 2022-08-01T22:59:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ac9b3700 1 -- 172.21.15.32:0/777065597 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff490003260 con 0x7ff4a8141860 2022-08-01T22:59:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/777065597 >> v2:172.21.15.55:3300/0 conn(0x7ff4a8064b10 msgr2=0x7ff4a8144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad1b4700 1 --2- 172.21.15.32:0/777065597 >> v2:172.21.15.55:3300/0 conn(0x7ff4a8064b10 0x7ff4a8144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/777065597 >> v1:172.21.15.32:6789/0 conn(0x7ff4a8141860 legacy=0x7ff4a8066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/777065597 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80b4850 con 0x7ff4a80b36e0 2022-08-01T22:59:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ad1b4700 1 --2- 172.21.15.32:0/777065597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a80b36e0 0x7ff4a8141370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff4a40032f0 tx=0x7ff4a40091d0).ready entity=mon.2 client_cookie=3c8cebf39d7ad1f1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ac9b3700 1 -- 172.21.15.32:0/777065597 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4a400b600 con 0x7ff4a80b36e0 2022-08-01T22:59:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ac9b3700 1 -- 172.21.15.32:0/777065597 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400b760 con 0x7ff4a80b36e0 2022-08-01T22:59:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ac9b3700 1 -- 172.21.15.32:0/777065597 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4a400b8d0 con 0x7ff4a80b36e0 2022-08-01T22:59:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/777065597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a80b36e0 msgr2=0x7ff4a8141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/777065597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a80b36e0 0x7ff4a8141370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff4a40032f0 tx=0x7ff4a40091d0).stop 2022-08-01T22:59:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/777065597 shutdown_connections 2022-08-01T22:59:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/777065597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a80b36e0 0x7ff4a8141370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/777065597 >> v2:172.21.15.55:3300/0 conn(0x7ff4a8064b10 0x7ff4a8144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/777065597 >> 172.21.15.32:0/777065597 conn(0x7ff4a805f840 msgr2=0x7ff4a805fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/777065597 shutdown_connections 2022-08-01T22:59:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.923+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/777065597 wait complete. 2022-08-01T22:59:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.927+0000 7ff4ae9b7700 1 Processor -- start 2022-08-01T22:59:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.927+0000 7ff4ae9b7700 1 -- start start 2022-08-01T22:59:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a8064b10 0x7ff4a805cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 0x7ff4a8150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8141860 0x7ff4a8150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80ae880 con 0x7ff4a80b36e0 2022-08-01T22:59:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8150420 con 0x7ff4a8141860 2022-08-01T22:59:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80b1320 con 0x7ff4a8064b10 2022-08-01T22:59:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 0x7ff4a8150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 0x7ff4a8150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43282/0 (socket says 172.21.15.32:43282) 2022-08-01T22:59:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/449653740 learned_addr learned my addr 172.21.15.32:0/449653740 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad9b5700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a8064b10 0x7ff4a805cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae1b6700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8141860 0x7ff4a8150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a8064b10 msgr2=0x7ff4a805cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a8064b10 0x7ff4a805cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8141860 msgr2=0x7ff4a8150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8141860 0x7ff4a8150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 -- 172.21.15.32:0/449653740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009470 con 0x7ff4a80b36e0 2022-08-01T22:59:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ad1b4700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 0x7ff4a8150810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7ff4a40032f0 tx=0x7ff4a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4a400b8d0 con 0x7ff4a80b36e0 2022-08-01T22:59:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4021630 con 0x7ff4a80b36e0 2022-08-01T22:59:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a80b15c0 con 0x7ff4a80b36e0 2022-08-01T22:59:16.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4a400b600 con 0x7ff4a80b36e0 2022-08-01T22:59:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a813bfd0 con 0x7ff4a80b36e0 2022-08-01T22:59:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.931+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a80b4af0 con 0x7ff4a80b36e0 2022-08-01T22:59:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.935+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff4a400f070 con 0x7ff4a80b36e0 2022-08-01T22:59:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.935+0000 7ff496ffd700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff488053950 0x7ff488055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.935+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff4a406ce90 con 0x7ff4a80b36e0 2022-08-01T22:59:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.935+0000 7ff4ad9b5700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff488053950 0x7ff488055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.935+0000 7ff4ad9b5700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff488053950 0x7ff488055e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff49800b000 tx=0x7ff498006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:16.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:16.939+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4a4043f90 con 0x7ff4a80b36e0 2022-08-01T22:59:17.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.067+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4a80574e0 con 0x7ff4a80b36e0 2022-08-01T22:59:17.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.067+0000 7ff496ffd700 1 -- 172.21.15.32:0/449653740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4a4015020 con 0x7ff4a80b36e0 2022-08-01T22:59:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff488053950 msgr2=0x7ff488055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff488053950 0x7ff488055e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff49800b000 tx=0x7ff498006df0).stop 2022-08-01T22:59:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 msgr2=0x7ff4a8150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 0x7ff4a8150810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7ff4a40032f0 tx=0x7ff4a4016000).stop 2022-08-01T22:59:17.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 shutdown_connections 2022-08-01T22:59:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff488053950 0x7ff488055e10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8141860 0x7ff4a8150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a80b36e0 0x7ff4a8150810 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 --2- 172.21.15.32:0/449653740 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4a8064b10 0x7ff4a805cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 >> 172.21.15.32:0/449653740 conn(0x7ff4a805f840 msgr2=0x7ff4a80b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:17.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 shutdown_connections 2022-08-01T22:59:17.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.071+0000 7ff4ae9b7700 1 -- 172.21.15.32:0/449653740 wait complete. 2022-08-01T22:59:17.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T22:59:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 Processor -- start 2022-08-01T22:59:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 -- start start 2022-08-01T22:59:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8067150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5df80a9930 0x7f5df80a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df80ac030 con 0x7f5df8060410 2022-08-01T22:59:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df80ab4a0 con 0x7f5df80a9930 2022-08-01T22:59:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5e00043700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8140490 con 0x7f5df80b1b30 2022-08-01T22:59:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfdddf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5df8060410 0x7f5df8066a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59062/0 (socket says 172.21.15.32:59062) 2022-08-01T22:59:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfe5e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5df80a9930 0x7f5df80a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfdddf700 1 -- 172.21.15.32:0/2208660617 learned_addr learned my addr 172.21.15.32:0/2208660617 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfd5de700 1 --2- 172.21.15.32:0/2208660617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfcddd700 1 -- 172.21.15.32:0/2208660617 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476804159 0 0) 0x7f5df80ac030 con 0x7f5df8060410 2022-08-01T22:59:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfcddd700 1 -- 172.21.15.32:0/2208660617 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003260 con 0x7f5df8060410 2022-08-01T22:59:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfd5de700 1 -- 172.21.15.32:0/2208660617 >> v2:172.21.15.32:3300/0 conn(0x7f5df80a9930 msgr2=0x7f5df80a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfd5de700 1 --2- 172.21.15.32:0/2208660617 >> v2:172.21.15.32:3300/0 conn(0x7f5df80a9930 0x7f5df80a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfd5de700 1 -- 172.21.15.32:0/2208660617 >> v1:172.21.15.32:6789/0 conn(0x7f5df8060410 legacy=0x7f5df8066a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.187+0000 7f5dfd5de700 1 -- 172.21.15.32:0/2208660617 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df8140730 con 0x7f5df80b1b30 2022-08-01T22:59:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5dfd5de700 1 --2- 172.21.15.32:0/2208660617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8067150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5de8002f70 tx=0x7f5de8006df0).ready entity=mon.2 client_cookie=d3e87f853a7996f2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5dfcddd700 1 -- 172.21.15.32:0/2208660617 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5de800d4a0 con 0x7f5df80b1b30 2022-08-01T22:59:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5dfcddd700 1 -- 172.21.15.32:0/2208660617 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8000e90 con 0x7f5df80b1b30 2022-08-01T22:59:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5dfcddd700 1 -- 172.21.15.32:0/2208660617 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5de800d7f0 con 0x7f5df80b1b30 2022-08-01T22:59:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 -- 172.21.15.32:0/2208660617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 msgr2=0x7f5df8067150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 --2- 172.21.15.32:0/2208660617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8067150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5de8002f70 tx=0x7f5de8006df0).stop 2022-08-01T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 -- 172.21.15.32:0/2208660617 shutdown_connections 2022-08-01T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 --2- 172.21.15.32:0/2208660617 >> v2:172.21.15.32:3300/0 conn(0x7f5df80a9930 0x7f5df80a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 --2- 172.21.15.32:0/2208660617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8067150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 -- 172.21.15.32:0/2208660617 >> 172.21.15.32:0/2208660617 conn(0x7f5df80b5570 msgr2=0x7f5df80b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 -- 172.21.15.32:0/2208660617 shutdown_connections 2022-08-01T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.191+0000 7f5e00043700 1 -- 172.21.15.32:0/2208660617 wait complete. 2022-08-01T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 Processor -- start 2022-08-01T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 -- start start 2022-08-01T22:59:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df8060410 0x7f5df80b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 0x7f5df8142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80ae1e0 con 0x7f5df80a9930 2022-08-01T22:59:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8143610 con 0x7f5df8060410 2022-08-01T22:59:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81438b0 con 0x7f5df80b1b30 2022-08-01T22:59:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 0x7f5df8142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 0x7f5df8142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43296/0 (socket says 172.21.15.32:43296) 2022-08-01T22:59:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 -- 172.21.15.32:0/1275892944 learned_addr learned my addr 172.21.15.32:0/1275892944 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfe5e0700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfdddf700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df8060410 0x7f5df80b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 -- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 msgr2=0x7f5df8142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 -- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df8060410 msgr2=0x7f5df80b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df8060410 0x7f5df80b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 -- 172.21.15.32:0/1275892944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de800b180 con 0x7f5df80a9930 2022-08-01T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5dfd5de700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 0x7f5df8142350 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f5de8002f70 tx=0x7f5de8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5de8013ac0 con 0x7f5df80a9930 2022-08-01T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8013c20 con 0x7f5df80a9930 2022-08-01T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df8143b50 con 0x7f5df80a9930 2022-08-01T22:59:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5de801e400 con 0x7f5df80a9930 2022-08-01T22:59:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.195+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df8141c10 con 0x7f5df80a9930 2022-08-01T22:59:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.199+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df80a9490 con 0x7f5df80a9930 2022-08-01T22:59:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.199+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5de8025030 con 0x7f5df80a9930 2022-08-01T22:59:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.199+0000 7f5de6ffd700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5dd8053b00 0x7f5dd8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.199+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5de806cb30 con 0x7f5df80a9930 2022-08-01T22:59:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.199+0000 7f5dfdddf700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5dd8053b00 0x7f5dd8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.199+0000 7f5dfdddf700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5dd8053b00 0x7f5dd8055fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f5df400b000 tx=0x7f5df4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:17.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.203+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5de8043c30 con 0x7f5df80a9930 2022-08-01T22:59:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5df80574e0 con 0x7f5df80a9930 2022-08-01T22:59:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5de6ffd700 1 -- 172.21.15.32:0/1275892944 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5de8023070 con 0x7f5df80a9930 2022-08-01T22:59:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:17.338 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T22:59:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T22:59:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T22:59:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T22:59:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T22:59:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T22:59:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:17.340 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T22:59:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5dd8053b00 msgr2=0x7f5dd8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5dd8053b00 0x7f5dd8055fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f5df400b000 tx=0x7f5df4009040).stop 2022-08-01T22:59:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 msgr2=0x7f5df8142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 0x7f5df8142350 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f5de8002f70 tx=0x7f5de8014000).stop 2022-08-01T22:59:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 shutdown_connections 2022-08-01T22:59:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5dd8053b00 0x7f5dd8055fc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5df80b1b30 0x7f5df8142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df80a9930 0x7f5df8142350 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 --2- 172.21.15.32:0/1275892944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df8060410 0x7f5df80b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 >> 172.21.15.32:0/1275892944 conn(0x7f5df80b5570 msgr2=0x7f5df80afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 shutdown_connections 2022-08-01T22:59:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.335+0000 7f5e00043700 1 -- 172.21.15.32:0/1275892944 wait complete. 2022-08-01T22:59:17.351 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T22:59:17.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T22:59:17.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.459+0000 7f8c84fae700 1 Processor -- start 2022-08-01T22:59:17.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.459+0000 7f8c84fae700 1 -- start start 2022-08-01T22:59:17.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.459+0000 7f8c84fae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c800b4310 0x7f8c80140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8c800b4820 0x7f8c800b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c80143cf0 con 0x7f8c80146ae0 2022-08-01T22:59:17.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c801476a0 con 0x7f8c800b4820 2022-08-01T22:59:17.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80143f90 con 0x7f8c800b4310 2022-08-01T22:59:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c80146ae0 0x7f8c801403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59076/0 (socket says 172.21.15.32:59076) 2022-08-01T22:59:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7e59c700 1 -- 172.21.15.32:0/2497176994 learned_addr learned my addr 172.21.15.32:0/2497176994 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7dd9b700 1 --2- 172.21.15.32:0/2497176994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c800b4310 0x7f8c80140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7ed9d700 1 --2- 172.21.15.32:0/2497176994 >> v2:172.21.15.55:3300/0 conn(0x7f8c800b4820 0x7f8c800b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7d59a700 1 -- 172.21.15.32:0/2497176994 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472800708 0 0) 0x7f8c80143cf0 con 0x7f8c80146ae0 2022-08-01T22:59:17.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7d59a700 1 -- 172.21.15.32:0/2497176994 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c60003260 con 0x7f8c80146ae0 2022-08-01T22:59:17.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7d59a700 1 -- 172.21.15.32:0/2497176994 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517316407 0 0) 0x7f8c60003260 con 0x7f8c80146ae0 2022-08-01T22:59:17.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/2497176994 >> v2:172.21.15.55:3300/0 conn(0x7f8c800b4820 msgr2=0x7f8c800b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:17.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7dd9b700 1 --2- 172.21.15.32:0/2497176994 >> v2:172.21.15.55:3300/0 conn(0x7f8c800b4820 0x7f8c800b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/2497176994 >> v1:172.21.15.32:6789/0 conn(0x7f8c80146ae0 legacy=0x7f8c801403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/2497176994 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800b0e20 con 0x7f8c800b4310 2022-08-01T22:59:17.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7dd9b700 1 --2- 172.21.15.32:0/2497176994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c800b4310 0x7f8c80140ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f8c70001800 tx=0x7f8c68006df0).ready entity=mon.2 client_cookie=a6332d4ebafcfbdc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:17.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7d59a700 1 -- 172.21.15.32:0/2497176994 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c70001660 con 0x7f8c800b4310 2022-08-01T22:59:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7d59a700 1 -- 172.21.15.32:0/2497176994 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68000e90 con 0x7f8c800b4310 2022-08-01T22:59:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c7d59a700 1 -- 172.21.15.32:0/2497176994 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c68012460 con 0x7f8c800b4310 2022-08-01T22:59:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- 172.21.15.32:0/2497176994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c800b4310 msgr2=0x7f8c80140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 --2- 172.21.15.32:0/2497176994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c800b4310 0x7f8c80140ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f8c70001800 tx=0x7f8c68006df0).stop 2022-08-01T22:59:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- 172.21.15.32:0/2497176994 shutdown_connections 2022-08-01T22:59:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 --2- 172.21.15.32:0/2497176994 >> v2:172.21.15.55:3300/0 conn(0x7f8c800b4820 0x7f8c800b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 --2- 172.21.15.32:0/2497176994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c800b4310 0x7f8c80140ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- 172.21.15.32:0/2497176994 >> 172.21.15.32:0/2497176994 conn(0x7f8c8005c620 msgr2=0x7f8c8005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- 172.21.15.32:0/2497176994 shutdown_connections 2022-08-01T22:59:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.463+0000 7f8c84fae700 1 -- 172.21.15.32:0/2497176994 wait complete. 2022-08-01T22:59:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 Processor -- start 2022-08-01T22:59:17.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 -- start start 2022-08-01T22:59:17.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800b4310 0x7f8c80152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 0x7f8c80148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c80146ae0 0x7f8c801561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801558a0 con 0x7f8c800b4820 2022-08-01T22:59:17.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80142fa0 con 0x7f8c800b4310 2022-08-01T22:59:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80155a90 con 0x7f8c80146ae0 2022-08-01T22:59:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 0x7f8c80148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 0x7f8c80148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43308/0 (socket says 172.21.15.32:43308) 2022-08-01T22:59:17.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/3855409025 learned_addr learned my addr 172.21.15.32:0/3855409025 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7ed9d700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c80146ae0 0x7f8c801561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7e59c700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800b4310 0x7f8c80152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c80146ae0 msgr2=0x7f8c801561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c80146ae0 0x7f8c801561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800b4310 msgr2=0x7f8c80152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800b4310 0x7f8c80152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 -- 172.21.15.32:0/3855409025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6800b180 con 0x7f8c800b4820 2022-08-01T22:59:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7e59c700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800b4310 0x7f8c80152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:59:17.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c7dd9b700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 0x7f8c80148b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f8c70001800 tx=0x7f8c68014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c68016030 con 0x7f8c800b4820 2022-08-01T22:59:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68013e40 con 0x7f8c800b4820 2022-08-01T22:59:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c80155d30 con 0x7f8c800b4820 2022-08-01T22:59:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6800de60 con 0x7f8c800b4820 2022-08-01T22:59:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.467+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c80149060 con 0x7f8c800b4820 2022-08-01T22:59:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.471+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80155ec0 con 0x7f8c800b4820 2022-08-01T22:59:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.471+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c68024020 con 0x7f8c800b4820 2022-08-01T22:59:17.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.471+0000 7f8c777fe700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c640539e0 0x7f8c64055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:17.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.471+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8c68017070 con 0x7f8c800b4820 2022-08-01T22:59:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.471+0000 7f8c7e59c700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c640539e0 0x7f8c64055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.471+0000 7f8c7e59c700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c640539e0 0x7f8c64055ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f8c70008000 tx=0x7f8c70009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.479+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c68043e70 con 0x7f8c800b4820 2022-08-01T22:59:17.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:17.603+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f8c80143160 con 0x7f8c800b4820 2022-08-01T22:59:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.303+0000 7f8c777fe700 1 -- 172.21.15.32:0/3855409025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8c68047490 con 0x7f8c800b4820 2022-08-01T22:59:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c640539e0 msgr2=0x7f8c64055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c640539e0 0x7f8c64055ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f8c70008000 tx=0x7f8c70009040).stop 2022-08-01T22:59:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 msgr2=0x7f8c80148b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 0x7f8c80148b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f8c70001800 tx=0x7f8c68014000).stop 2022-08-01T22:59:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 shutdown_connections 2022-08-01T22:59:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c640539e0 0x7f8c64055ea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c80146ae0 0x7f8c801561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c800b4820 0x7f8c80148b70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 --2- 172.21.15.32:0/3855409025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800b4310 0x7f8c80152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 >> 172.21.15.32:0/3855409025 conn(0x7f8c8005c620 msgr2=0x7f8c80147a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 shutdown_connections 2022-08-01T22:59:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.307+0000 7f8c84fae700 1 -- 172.21.15.32:0/3855409025 wait complete. 2022-08-01T22:59:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T22:59:18.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T22:59:18.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 Processor -- start 2022-08-01T22:59:18.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 -- start start 2022-08-01T22:59:18.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380617f0 0x7f46380b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:18.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4638144d20 0x7f463814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:18.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4638061bd0 con 0x7f46380b3f90 2022-08-01T22:59:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380a9fd0 con 0x7f4638144d20 2022-08-01T22:59:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463ef93700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380a88e0 con 0x7f46380617f0 2022-08-01T22:59:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463e792700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4638144d20 0x7f463814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463d790700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46380b3f90 0x7f4638144610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59092/0 (socket says 172.21.15.32:59092) 2022-08-01T22:59:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463d790700 1 -- 172.21.15.32:0/2756160774 learned_addr learned my addr 172.21.15.32:0/2756160774 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463df91700 1 --2- 172.21.15.32:0/2756160774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380617f0 0x7f46380b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463cf8f700 1 -- 172.21.15.32:0/2756160774 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656422456 0 0) 0x7f4638061bd0 con 0x7f46380b3f90 2022-08-01T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463cf8f700 1 -- 172.21.15.32:0/2756160774 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003260 con 0x7f46380b3f90 2022-08-01T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463e792700 1 -- 172.21.15.32:0/2756160774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380617f0 msgr2=0x7f46380b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463e792700 1 --2- 172.21.15.32:0/2756160774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380617f0 0x7f46380b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463e792700 1 -- 172.21.15.32:0/2756160774 >> v1:172.21.15.32:6789/0 conn(0x7f46380b3f90 legacy=0x7f4638144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463e792700 1 -- 172.21.15.32:0/2756160774 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380a8b80 con 0x7f4638144d20 2022-08-01T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463e792700 1 --2- 172.21.15.32:0/2756160774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638144d20 0x7f463814aaa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f4630002f70 tx=0x7f4630006df0).ready entity=mon.0 client_cookie=2cd37f9d2f70eba9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:18.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463cf8f700 1 -- 172.21.15.32:0/2756160774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f463000d4a0 con 0x7f4638144d20 2022-08-01T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463cf8f700 1 -- 172.21.15.32:0/2756160774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4630000e90 con 0x7f4638144d20 2022-08-01T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.435+0000 7f463cf8f700 1 -- 172.21.15.32:0/2756160774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f463000d7f0 con 0x7f4638144d20 2022-08-01T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- 172.21.15.32:0/2756160774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638144d20 msgr2=0x7f463814aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 --2- 172.21.15.32:0/2756160774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638144d20 0x7f463814aaa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f4630002f70 tx=0x7f4630006df0).stop 2022-08-01T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- 172.21.15.32:0/2756160774 shutdown_connections 2022-08-01T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 --2- 172.21.15.32:0/2756160774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4638144d20 0x7f463814aaa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 --2- 172.21.15.32:0/2756160774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380617f0 0x7f46380b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- 172.21.15.32:0/2756160774 >> 172.21.15.32:0/2756160774 conn(0x7f4638066f70 msgr2=0x7f4638067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- 172.21.15.32:0/2756160774 shutdown_connections 2022-08-01T22:59:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- 172.21.15.32:0/2756160774 wait complete. 2022-08-01T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 Processor -- start 2022-08-01T22:59:18.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- start start 2022-08-01T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 0x7f463814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380b3f90 0x7f4638062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4638144d20 0x7f4638147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:18.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638064560 con 0x7f46380617f0 2022-08-01T22:59:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380658e0 con 0x7f4638144d20 2022-08-01T22:59:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463ef93700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4638063010 con 0x7f46380b3f90 2022-08-01T22:59:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463df91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 0x7f463814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463df91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 0x7f463814c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43324/0 (socket says 172.21.15.32:43324) 2022-08-01T22:59:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463df91700 1 -- 172.21.15.32:0/637152228 learned_addr learned my addr 172.21.15.32:0/637152228 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:18.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463d790700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380b3f90 0x7f4638062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:18.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.439+0000 7f463e792700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4638144d20 0x7f4638147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:18.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463df91700 1 -- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380b3f90 msgr2=0x7f4638062b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463df91700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380b3f90 0x7f4638062b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463df91700 1 -- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4638144d20 msgr2=0x7f4638147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463df91700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4638144d20 0x7f4638147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463df91700 1 -- 172.21.15.32:0/637152228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463000b180 con 0x7f46380617f0 2022-08-01T22:59:18.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463df91700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 0x7f463814c660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f4638065df0 tx=0x7f462c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:18.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f462c010780 con 0x7f46380617f0 2022-08-01T22:59:18.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c0108e0 con 0x7f46380617f0 2022-08-01T22:59:18.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4638148610 con 0x7f46380617f0 2022-08-01T22:59:18.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f462c010c40 con 0x7f46380617f0 2022-08-01T22:59:18.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4638148350 con 0x7f46380617f0 2022-08-01T22:59:18.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46380574e0 con 0x7f46380617f0 2022-08-01T22:59:18.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f462c01c460 con 0x7f46380617f0 2022-08-01T22:59:18.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f462affd700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4618053a60 0x7f4618055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:18.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f462c06c190 con 0x7f46380617f0 2022-08-01T22:59:18.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.443+0000 7f463d790700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4618053a60 0x7f4618055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:18.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.447+0000 7f463d790700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4618053a60 0x7f4618055f20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f463400d000 tx=0x7f4634009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:18.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.447+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f462c0430c0 con 0x7f46380617f0 2022-08-01T22:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.575+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f46380b8c30 con 0x7f46380617f0 2022-08-01T22:59:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.923+0000 7f462affd700 1 -- 172.21.15.32:0/637152228 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7f462c0466e0 con 0x7f46380617f0 2022-08-01T22:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.927+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4618053a60 msgr2=0x7f4618055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4618053a60 0x7f4618055f20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f463400d000 tx=0x7f4634009040).stop 2022-08-01T22:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 msgr2=0x7f463814c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 0x7f463814c660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f4638065df0 tx=0x7f462c006df0).stop 2022-08-01T22:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 shutdown_connections 2022-08-01T22:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4618053a60 0x7f4618055f20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4638144d20 0x7f4638147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46380b3f90 0x7f4638062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 --2- 172.21.15.32:0/637152228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46380617f0 0x7f463814c660 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 >> 172.21.15.32:0/637152228 conn(0x7f4638066f70 msgr2=0x7f463814b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 shutdown_connections 2022-08-01T22:59:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:18.931+0000 7f463ef93700 1 -- 172.21.15.32:0/637152228 wait complete. 2022-08-01T22:59:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T22:59:18.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T22:59:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935b59e700 1 Processor -- start 2022-08-01T22:59:19.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935b59e700 1 -- start start 2022-08-01T22:59:19.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0654c0 con 0x7f935c066ba0 2022-08-01T22:59:19.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0b3570 con 0x7f935c0b3190 2022-08-01T22:59:19.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c062190 con 0x7f935c0b2730 2022-08-01T22:59:19.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f935c066ba0 0x7f935c14dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59108/0 (socket says 172.21.15.32:59108) 2022-08-01T22:59:19.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f935ad9d700 1 -- 172.21.15.32:0/1256451815 learned_addr learned my addr 172.21.15.32:0/1256451815 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:19.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.063+0000 7f9359d9b700 1 --2- 172.21.15.32:0/1256451815 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017607793 0 0) 0x7f935c0654c0 con 0x7f935c066ba0 2022-08-01T22:59:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f933c003260 con 0x7f935c066ba0 2022-08-01T22:59:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827634709 0 0) 0x7f935c0b3570 con 0x7f935c0b3190 2022-08-01T22:59:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f935c0654c0 con 0x7f935c0b3190 2022-08-01T22:59:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f9359d9b700 1 -- 172.21.15.32:0/1256451815 >> v1:172.21.15.55:6789/0 conn(0x7f935c0b3190 legacy=0x7f935c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f9359d9b700 1 -- 172.21.15.32:0/1256451815 >> v1:172.21.15.32:6789/0 conn(0x7f935c066ba0 legacy=0x7f935c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f9359d9b700 1 -- 172.21.15.32:0/1256451815 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c05fbf0 con 0x7f935c0b2730 2022-08-01T22:59:19.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f9359d9b700 1 --2- 172.21.15.32:0/1256451815 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0666b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f93440032f0 tx=0x7f93440091d0).ready entity=mon.2 client_cookie=64c7822a3948cdc2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934400b600 con 0x7f935c0b2730 2022-08-01T22:59:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934400b760 con 0x7f935c0b2730 2022-08-01T22:59:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935959a700 1 -- 172.21.15.32:0/1256451815 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934400b8d0 con 0x7f935c0b2730 2022-08-01T22:59:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 -- 172.21.15.32:0/1256451815 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 msgr2=0x7f935c0666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 --2- 172.21.15.32:0/1256451815 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0666b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f93440032f0 tx=0x7f93440091d0).stop 2022-08-01T22:59:19.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 -- 172.21.15.32:0/1256451815 shutdown_connections 2022-08-01T22:59:19.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 --2- 172.21.15.32:0/1256451815 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0666b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 -- 172.21.15.32:0/1256451815 >> 172.21.15.32:0/1256451815 conn(0x7f935c063870 msgr2=0x7f935c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 -- 172.21.15.32:0/1256451815 shutdown_connections 2022-08-01T22:59:19.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.067+0000 7f935b59e700 1 -- 172.21.15.32:0/1256451815 wait complete. 2022-08-01T22:59:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 Processor -- start 2022-08-01T22:59:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 -- start start 2022-08-01T22:59:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 0x7f935c0b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f935c0b3190 0x7f935c0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1558a0 con 0x7f935c066ba0 2022-08-01T22:59:19.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0ad080 con 0x7f935c0b3190 2022-08-01T22:59:19.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0ac8a0 con 0x7f935c0b2730 2022-08-01T22:59:19.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 0x7f935c0b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 0x7f935c0b6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43336/0 (socket says 172.21.15.32:43336) 2022-08-01T22:59:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 -- 172.21.15.32:0/3592182004 learned_addr learned my addr 172.21.15.32:0/3592182004 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935ad9d700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f935c0b3190 0x7f935c0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f9359d9b700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 -- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 msgr2=0x7f935c0aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 -- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f935c0b3190 msgr2=0x7f935c0aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f935c0b3190 0x7f935c0aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 -- 172.21.15.32:0/3592182004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009470 con 0x7f935c066ba0 2022-08-01T22:59:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935ad9d700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f935c0b3190 0x7f935c0aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.071+0000 7f935a59c700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 0x7f935c0b6b60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f9354001110 tx=0x7f9354007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9354012400 con 0x7f935c066ba0 2022-08-01T22:59:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354012560 con 0x7f935c066ba0 2022-08-01T22:59:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c0acb40 con 0x7f935c066ba0 2022-08-01T22:59:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93540128c0 con 0x7f935c066ba0 2022-08-01T22:59:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c0ab960 con 0x7f935c066ba0 2022-08-01T22:59:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c05fe90 con 0x7f935c066ba0 2022-08-01T22:59:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9354019030 con 0x7f935c066ba0 2022-08-01T22:59:19.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f93537fe700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9340053a70 0x7f9340055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f935406b330 con 0x7f935c066ba0 2022-08-01T22:59:19.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.075+0000 7f9359d9b700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9340053a70 0x7f9340055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.079+0000 7f9359d9b700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9340053a70 0x7f9340055f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9344019000 tx=0x7f9344016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.083+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93540420d0 con 0x7f935c066ba0 2022-08-01T22:59:19.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.247+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f935c0574e0 con 0x7f935c066ba0 2022-08-01T22:59:19.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.255+0000 7f93537fe700 1 -- 172.21.15.32:0/3592182004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f93540456f0 con 0x7f935c066ba0 2022-08-01T22:59:19.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9340053a70 msgr2=0x7f9340055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9340053a70 0x7f9340055f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9344019000 tx=0x7f9344016000).stop 2022-08-01T22:59:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 msgr2=0x7f935c0b6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 0x7f935c0b6b60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f9354001110 tx=0x7f9354007ae0).stop 2022-08-01T22:59:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 shutdown_connections 2022-08-01T22:59:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9340053a70 0x7f9340055f30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f935c0b3190 0x7f935c0aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f935c0b2730 0x7f935c0aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 --2- 172.21.15.32:0/3592182004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f935c066ba0 0x7f935c0b6b60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 >> 172.21.15.32:0/3592182004 conn(0x7f935c063870 msgr2=0x7f935c0b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 shutdown_connections 2022-08-01T22:59:19.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.259+0000 7f935b59e700 1 -- 172.21.15.32:0/3592182004 wait complete. 2022-08-01T22:59:19.265 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 9 and data pool 10 2022-08-01T22:59:19.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 Processor -- start 2022-08-01T22:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 -- start start 2022-08-01T22:59:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f99780b6710 0x7f99780b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 0x7f99780b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9978144240 con 0x7f9978061ec0 2022-08-01T22:59:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99780b6fe0 con 0x7f99780b6710 2022-08-01T22:59:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.367+0000 7f997dd93700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9978143d00 con 0x7f99780b4550 2022-08-01T22:59:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 0x7f99780b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f99777fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f99780b6710 0x7f99780b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 0x7f99780b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55084/0 (socket says 172.21.15.32:55084) 2022-08-01T22:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 -- 172.21.15.32:0/936069924 learned_addr learned my addr 172.21.15.32:0/936069924 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 -- 172.21.15.32:0/936069924 >> v2:172.21.15.55:3300/0 conn(0x7f99780b6710 msgr2=0x7f99780b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 --2- 172.21.15.32:0/936069924 >> v2:172.21.15.55:3300/0 conn(0x7f99780b6710 0x7f99780b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 -- 172.21.15.32:0/936069924 >> v1:172.21.15.55:6789/0 conn(0x7f9978061ec0 legacy=0x7f9978146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 -- 172.21.15.32:0/936069924 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9978143fa0 con 0x7f99780b4550 2022-08-01T22:59:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f9976ffd700 1 --2- 172.21.15.32:0/936069924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 0x7f99780b6af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f996c002f70 tx=0x7f996c006eb0).ready entity=mon.2 client_cookie=b649e672024bbb20 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f99767fc700 1 -- 172.21.15.32:0/936069924 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996c00d570 con 0x7f99780b4550 2022-08-01T22:59:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f99767fc700 1 -- 172.21.15.32:0/936069924 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c000e90 con 0x7f99780b4550 2022-08-01T22:59:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f99767fc700 1 -- 172.21.15.32:0/936069924 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996c00d900 con 0x7f99780b4550 2022-08-01T22:59:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 -- 172.21.15.32:0/936069924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 msgr2=0x7f99780b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 --2- 172.21.15.32:0/936069924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 0x7f99780b6af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f996c002f70 tx=0x7f996c006eb0).stop 2022-08-01T22:59:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 -- 172.21.15.32:0/936069924 shutdown_connections 2022-08-01T22:59:19.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 --2- 172.21.15.32:0/936069924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b4550 0x7f99780b6af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 --2- 172.21.15.32:0/936069924 >> v2:172.21.15.55:3300/0 conn(0x7f99780b6710 0x7f99780b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 -- 172.21.15.32:0/936069924 >> 172.21.15.32:0/936069924 conn(0x7f997805c620 msgr2=0x7f997805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 -- 172.21.15.32:0/936069924 shutdown_connections 2022-08-01T22:59:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.371+0000 7f997dd93700 1 -- 172.21.15.32:0/936069924 wait complete. 2022-08-01T22:59:19.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 Processor -- start 2022-08-01T22:59:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 -- start start 2022-08-01T22:59:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9978061ec0 0x7f997812af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 0x7f99780b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b6710 0x7f99781270b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780b1e50 con 0x7f99780b4550 2022-08-01T22:59:19.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997813e8b0 con 0x7f9978061ec0 2022-08-01T22:59:19.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f997dd93700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997813f350 con 0x7f99780b6710 2022-08-01T22:59:19.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 0x7f99780b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f99777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9978061ec0 0x7f997812af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 0x7f99780b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43354/0 (socket says 172.21.15.32:43354) 2022-08-01T22:59:19.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f99777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9978061ec0 0x7f997812af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53136/0 (socket says 172.21.15.32:53136) 2022-08-01T22:59:19.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f9976ffd700 1 -- 172.21.15.32:0/2168721455 learned_addr learned my addr 172.21.15.32:0/2168721455 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:19.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.375+0000 7f9977fff700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b6710 0x7f99781270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9976ffd700 1 -- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b6710 msgr2=0x7f99781270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9976ffd700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b6710 0x7f99781270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9976ffd700 1 -- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9978061ec0 msgr2=0x7f997812af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9976ffd700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9978061ec0 0x7f997812af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9976ffd700 1 -- 172.21.15.32:0/2168721455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c00b1e0 con 0x7f99780b4550 2022-08-01T22:59:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9976ffd700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 0x7f99780b3710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f996c002f70 tx=0x7f996c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996c0117e0 con 0x7f99780b4550 2022-08-01T22:59:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c011940 con 0x7f99780b4550 2022-08-01T22:59:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997813efc0 con 0x7f99780b4550 2022-08-01T22:59:19.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996c011ca0 con 0x7f99780b4550 2022-08-01T22:59:19.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9978141120 con 0x7f99780b4550 2022-08-01T22:59:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9978061780 con 0x7f99780b4550 2022-08-01T22:59:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.379+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f996c025030 con 0x7f99780b4550 2022-08-01T22:59:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.383+0000 7f9974ff9700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9958053b00 0x7f9958055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.383+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f996c06d240 con 0x7f99780b4550 2022-08-01T22:59:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.383+0000 7f99777fe700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9958053b00 0x7f9958055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.383+0000 7f99777fe700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9958053b00 0x7f9958055fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f996800b000 tx=0x7f9968006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.387+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f996c043ef0 con 0x7f99780b4550 2022-08-01T22:59:19.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.511+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99780574e0 con 0x7f99780b4550 2022-08-01T22:59:19.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f9974ff9700 1 -- 172.21.15.32:0/2168721455 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f996c023090 con 0x7f99780b4550 2022-08-01T22:59:19.517 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:19.517 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:59:19.252922+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:59:17.662803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T22:59:18.633694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:59:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9958053b00 msgr2=0x7f9958055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9958053b00 0x7f9958055fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f996800b000 tx=0x7f9968006df0).stop 2022-08-01T22:59:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 msgr2=0x7f99780b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 0x7f99780b3710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f996c002f70 tx=0x7f996c015000).stop 2022-08-01T22:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 shutdown_connections 2022-08-01T22:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9958053b00 0x7f9958055fc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99780b6710 0x7f99781270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780b4550 0x7f99780b3710 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 --2- 172.21.15.32:0/2168721455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9978061ec0 0x7f997812af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 >> 172.21.15.32:0/2168721455 conn(0x7f997805c620 msgr2=0x7f9978145790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 shutdown_connections 2022-08-01T22:59:19.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.515+0000 7f997dd93700 1 -- 172.21.15.32:0/2168721455 wait complete. 2022-08-01T22:59:19.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:59:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f361a24c700 1 Processor -- start 2022-08-01T22:59:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f361a24c700 1 -- start start 2022-08-01T22:59:19.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f361a24c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f360c0422b0 0x7f360c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f361a24c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c045ff0 con 0x7f360c045640 2022-08-01T22:59:19.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f361a24c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c047740 con 0x7f360c047360 2022-08-01T22:59:19.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f361a24c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c044ac0 con 0x7f360c0422b0 2022-08-01T22:59:19.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f3618a49700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f360c045640 0x7f360c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59134/0 (socket says 172.21.15.32:59134) 2022-08-01T22:59:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f3618a49700 1 -- 172.21.15.32:0/2309056753 learned_addr learned my addr 172.21.15.32:0/2309056753 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.627+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2309056753 >> v2:172.21.15.55:3300/0 conn(0x7f360c0422b0 0x7f360c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460977812 0 0) 0x7f360c045ff0 con 0x7f360c045640 2022-08-01T22:59:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f8003260 con 0x7f360c045640 2022-08-01T22:59:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197392241 0 0) 0x7f360c047740 con 0x7f360c047360 2022-08-01T22:59:19.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f360c045ff0 con 0x7f360c047360 2022-08-01T22:59:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.1 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1465793494 0 0) 0x7f360c045ff0 con 0x7f360c047360 2022-08-01T22:59:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 >> v2:172.21.15.55:3300/0 conn(0x7f360c0422b0 msgr2=0x7f360c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 --2- 172.21.15.32:0/2309056753 >> v2:172.21.15.55:3300/0 conn(0x7f360c0422b0 0x7f360c0d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 >> v1:172.21.15.32:6789/0 conn(0x7f360c045640 legacy=0x7f360c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c046220 con 0x7f360c047360 2022-08-01T22:59:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.1 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f3610001660 con 0x7f360c047360 2022-08-01T22:59:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2309056753 >> v2:172.21.15.55:3300/0 conn(0x7f360c0422b0 0x7f360c0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 >> v1:172.21.15.55:6789/0 conn(0x7f360c047360 legacy=0x7f360c041ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8004940 0x7f35f8004d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35f80034a0 0x7f35f8003920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f35f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3608001230 con 0x7f35f8003e10 2022-08-01T22:59:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c044ac0 con 0x7f35f8004940 2022-08-01T22:59:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f800be20 con 0x7f35f80034a0 2022-08-01T22:59:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f35f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361924a700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8004940 0x7f35f8004d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3618a49700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35f80034a0 0x7f35f8003920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 -- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35f80034a0 msgr2=0x7f35f8003920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35f80034a0 0x7f35f8003920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 -- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8004940 msgr2=0x7f35f8004d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8004940 0x7f35f8004d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 -- 172.21.15.32:0/2309056753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c045ff0 con 0x7f35f8003e10 2022-08-01T22:59:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 -- 172.21.15.32:0/2309056753 reap_dead start 2022-08-01T22:59:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f35f800b750 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f3614062000 tx=0x7f361405f750).ready entity=mon.0 client_cookie=558c70d51856faeb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3614086070 con 0x7f35f8003e10 2022-08-01T22:59:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140447a0 con 0x7f35f8003e10 2022-08-01T22:59:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f3607fff700 1 -- 172.21.15.32:0/2309056753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361405d280 con 0x7f35f8003e10 2022-08-01T22:59:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 -- 172.21.15.32:0/2309056753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 msgr2=0x7f35f800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f35f800b750 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f3614062000 tx=0x7f361405f750).stop 2022-08-01T22:59:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 -- 172.21.15.32:0/2309056753 shutdown_connections 2022-08-01T22:59:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 --2- 172.21.15.32:0/2309056753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f35f800b750 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 -- 172.21.15.32:0/2309056753 >> 172.21.15.32:0/2309056753 conn(0x7f360c010d90 msgr2=0x7f360c0111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 -- 172.21.15.32:0/2309056753 shutdown_connections 2022-08-01T22:59:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.631+0000 7f361a24c700 1 -- 172.21.15.32:0/2309056753 wait complete. 2022-08-01T22:59:19.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 Processor -- start 2022-08-01T22:59:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 -- start start 2022-08-01T22:59:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f360c0e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f360c048710 0x7f360c0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f360c0e3220 0x7f360c0e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0d3650 con 0x7f35f8003e10 2022-08-01T22:59:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0d3110 con 0x7f360c048710 2022-08-01T22:59:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.635+0000 7f361a24c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0d33b0 con 0x7f360c0e3220 2022-08-01T22:59:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f360c0e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f360c0e6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43370/0 (socket says 172.21.15.32:43370) 2022-08-01T22:59:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 -- 172.21.15.32:0/2990787386 learned_addr learned my addr 172.21.15.32:0/2990787386 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f3619a4b700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f360c0e3220 0x7f360c0e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f3618a49700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f360c048710 0x7f360c0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 -- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f360c0e3220 msgr2=0x7f360c0e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f360c0e3220 0x7f360c0e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 -- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f360c048710 msgr2=0x7f360c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f360c048710 0x7f360c0cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 -- 172.21.15.32:0/2990787386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0d3ba0 con 0x7f35f8003e10 2022-08-01T22:59:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361924a700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f360c0e6e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f361000d000 tx=0x7f360c0463b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3610012750 con 0x7f35f8003e10 2022-08-01T22:59:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36100128b0 con 0x7f35f8003e10 2022-08-01T22:59:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c0e3b90 con 0x7f35f8003e10 2022-08-01T22:59:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3610012c10 con 0x7f35f8003e10 2022-08-01T22:59:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c0d1f20 con 0x7f35f8003e10 2022-08-01T22:59:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c0446c0 con 0x7f35f8003e10 2022-08-01T22:59:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3610012d70 con 0x7f35f8003e10 2022-08-01T22:59:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f36067fc700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35fc053b00 0x7f35fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.639+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f361006c430 con 0x7f35f8003e10 2022-08-01T22:59:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.643+0000 7f3618a49700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35fc053b00 0x7f35fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.643+0000 7f3618a49700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35fc053b00 0x7f35fc055fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3608008000 tx=0x7f3608009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.647+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3610043190 con 0x7f35f8003e10 2022-08-01T22:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.771+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360c004d30 con 0x7f35f8003e10 2022-08-01T22:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.771+0000 7f36067fc700 1 -- 172.21.15.32:0/2990787386 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f36100467b0 con 0x7f35f8003e10 2022-08-01T22:59:19.776 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:19.776 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:59:19.252922+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:59:17.662803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T22:59:18.633694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:59:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35fc053b00 msgr2=0x7f35fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35fc053b00 0x7f35fc055fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3608008000 tx=0x7f3608009040).stop 2022-08-01T22:59:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 msgr2=0x7f360c0e6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f360c0e6e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f361000d000 tx=0x7f360c0463b0).stop 2022-08-01T22:59:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 shutdown_connections 2022-08-01T22:59:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35fc053b00 0x7f35fc055fc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f360c0e3220 0x7f360c0e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f360c048710 0x7f360c0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 --2- 172.21.15.32:0/2990787386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f8003e10 0x7f360c0e6e20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 >> 172.21.15.32:0/2990787386 conn(0x7f360c010d90 msgr2=0x7f360c0d8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 shutdown_connections 2022-08-01T22:59:19.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.775+0000 7f361a24c700 1 -- 172.21.15.32:0/2990787386 wait complete. 2022-08-01T22:59:19.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T22:59:19.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.899+0000 7fc655152700 1 Processor -- start 2022-08-01T22:59:19.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.899+0000 7fc655152700 1 -- start start 2022-08-01T22:59:19.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6500b38a0 0x7fc6500b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500a9da0 0x7fc6500b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc650143ab0 con 0x7fc6500b0740 2022-08-01T22:59:19.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc650143d50 con 0x7fc6500b38a0 2022-08-01T22:59:19.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc650143ff0 con 0x7fc6500a9da0 2022-08-01T22:59:19.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500a9da0 0x7fc6500b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6500b38a0 0x7fc6500b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64ed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc6500b0740 0x7fc65013d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:51826/0 (socket says 172.21.15.32:51826) 2022-08-01T22:59:19.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64ed9d700 1 -- 172.21.15.32:0/3559033517 learned_addr learned my addr 172.21.15.32:0/3559033517 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:19.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64e59c700 1 -- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500a9da0 msgr2=0x7fc6500b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64e59c700 1 --2- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500a9da0 0x7fc6500b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64dd9b700 1 -- 172.21.15.32:0/3559033517 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613583994 0 0) 0x7fc650143ab0 con 0x7fc6500b0740 2022-08-01T22:59:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64e59c700 1 -- 172.21.15.32:0/3559033517 >> v1:172.21.15.55:6789/0 conn(0x7fc6500b0740 legacy=0x7fc65013d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64e59c700 1 -- 172.21.15.32:0/3559033517 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc650144290 con 0x7fc6500b38a0 2022-08-01T22:59:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64f59e700 1 --2- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500a9da0 0x7fc6500b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:19.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64e59c700 1 --2- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500b38a0 0x7fc6500b0b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fc6400032f0 tx=0x7fc6400091d0).ready entity=mon.1 client_cookie=50162211f9d667f3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64dd9b700 1 -- 172.21.15.32:0/3559033517 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc64000b6c0 con 0x7fc6500b38a0 2022-08-01T22:59:19.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64dd9b700 1 -- 172.21.15.32:0/3559033517 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000b820 con 0x7fc6500b38a0 2022-08-01T22:59:19.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc64dd9b700 1 -- 172.21.15.32:0/3559033517 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc64000b990 con 0x7fc6500b38a0 2022-08-01T22:59:19.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500b38a0 msgr2=0x7fc6500b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 --2- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500b38a0 0x7fc6500b0b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fc6400032f0 tx=0x7fc6400091d0).stop 2022-08-01T22:59:19.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- 172.21.15.32:0/3559033517 shutdown_connections 2022-08-01T22:59:19.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 --2- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500a9da0 0x7fc6500b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 --2- 172.21.15.32:0/3559033517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500b38a0 0x7fc6500b0b20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- 172.21.15.32:0/3559033517 >> 172.21.15.32:0/3559033517 conn(0x7fc650060950 msgr2=0x7fc650060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- 172.21.15.32:0/3559033517 shutdown_connections 2022-08-01T22:59:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.903+0000 7fc655152700 1 -- 172.21.15.32:0/3559033517 wait complete. 2022-08-01T22:59:19.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 Processor -- start 2022-08-01T22:59:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 -- start start 2022-08-01T22:59:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500a9da0 0x7fc6500ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500b0740 0x7fc6500abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6500b38a0 0x7fc65013efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500acb90 con 0x7fc6500b38a0 2022-08-01T22:59:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500ac340 con 0x7fc6500a9da0 2022-08-01T22:59:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc655152700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500ab620 con 0x7fc6500b0740 2022-08-01T22:59:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6500b38a0 0x7fc65013efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500b0740 0x7fc6500abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500a9da0 0x7fc6500ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6500b38a0 0x7fc65013efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43388/0 (socket says 172.21.15.32:43388) 2022-08-01T22:59:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500b0740 0x7fc6500abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55124/0 (socket says 172.21.15.32:55124) 2022-08-01T22:59:19.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64f59e700 1 -- 172.21.15.32:0/2156178219 learned_addr learned my addr 172.21.15.32:0/2156178219 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 -- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500b0740 msgr2=0x7fc6500abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500b0740 0x7fc6500abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 -- 172.21.15.32:0/2156178219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6500b38a0 msgr2=0x7fc65013efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6500b38a0 0x7fc65013efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:19.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 -- 172.21.15.32:0/2156178219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64800b180 con 0x7fc6500a9da0 2022-08-01T22:59:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc64ed9d700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500a9da0 0x7fc6500ad060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc6500618c0 tx=0x7fc6380079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.907+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc63800cdf0 con 0x7fc6500a9da0 2022-08-01T22:59:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6380070d0 con 0x7fc6500a9da0 2022-08-01T22:59:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc640009470 con 0x7fc6500a9da0 2022-08-01T22:59:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc638007430 con 0x7fc6500a9da0 2022-08-01T22:59:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65014ffc0 con 0x7fc6500a9da0 2022-08-01T22:59:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc650150150 con 0x7fc6500a9da0 2022-08-01T22:59:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc638010070 con 0x7fc6500a9da0 2022-08-01T22:59:19.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc647fff700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc634053aa0 0x7fc634055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:19.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc63806b550 con 0x7fc6500a9da0 2022-08-01T22:59:19.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.911+0000 7fc64e59c700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc634053aa0 0x7fc634055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:19.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.915+0000 7fc64e59c700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc634053aa0 0x7fc634055f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc64001c000 tx=0x7fc640016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:19.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:19.919+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc638042200 con 0x7fc6500a9da0 2022-08-01T22:59:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.087+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc65013ddf0 con 0x7fc6500a9da0 2022-08-01T22:59:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.307+0000 7fc647fff700 1 -- 172.21.15.32:0/2156178219 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc638045820 con 0x7fc6500a9da0 2022-08-01T22:59:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc634053aa0 msgr2=0x7fc634055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc634053aa0 0x7fc634055f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc64001c000 tx=0x7fc640016000).stop 2022-08-01T22:59:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500a9da0 msgr2=0x7fc6500ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500a9da0 0x7fc6500ad060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc6500618c0 tx=0x7fc6380079e0).stop 2022-08-01T22:59:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 shutdown_connections 2022-08-01T22:59:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc634053aa0 0x7fc634055f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6500b38a0 0x7fc65013efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc6500b0740 0x7fc6500abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 --2- 172.21.15.32:0/2156178219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6500a9da0 0x7fc6500ad060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 >> 172.21.15.32:0/2156178219 conn(0x7fc650060950 msgr2=0x7fc6500b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 shutdown_connections 2022-08-01T22:59:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.311+0000 7fc655152700 1 -- 172.21.15.32:0/2156178219 wait complete. 2022-08-01T22:59:20.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 Processor -- start 2022-08-01T22:59:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- start start 2022-08-01T22:59:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbcc148530 0x7fcbcc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcbcc0a7250 0x7fcbcc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc0b04e0 con 0x7fcbcc0b1f40 2022-08-01T22:59:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0a9250 con 0x7fcbcc148530 2022-08-01T22:59:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0b0780 con 0x7fcbcc0a7250 2022-08-01T22:59:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbcc148530 0x7fcbcc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbcc148530 0x7fcbcc0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43394/0 (socket says 172.21.15.32:43394) 2022-08-01T22:59:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 -- 172.21.15.32:0/1565662570 learned_addr learned my addr 172.21.15.32:0/1565662570 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcbfff700 1 --2- 172.21.15.32:0/1565662570 >> v2:172.21.15.55:3300/0 conn(0x7fcbcc0a7250 0x7fcbcc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 -- 172.21.15.32:0/1565662570 >> v2:172.21.15.55:3300/0 conn(0x7fcbcc0a7250 msgr2=0x7fcbcc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 --2- 172.21.15.32:0/1565662570 >> v2:172.21.15.55:3300/0 conn(0x7fcbcc0a7250 0x7fcbcc1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 -- 172.21.15.32:0/1565662570 >> v1:172.21.15.55:6789/0 conn(0x7fcbcc0b1f40 legacy=0x7fcbcc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 -- 172.21.15.32:0/1565662570 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0b0a20 con 0x7fcbcc148530 2022-08-01T22:59:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcbfff700 1 --2- 172.21.15.32:0/1565662570 >> v2:172.21.15.55:3300/0 conn(0x7fcbcc0a7250 0x7fcbcc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:59:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbcaffd700 1 --2- 172.21.15.32:0/1565662570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0b1160 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fcbcc1478c0 tx=0x7fcbbc009290).ready entity=mon.0 client_cookie=97d95c041eed0c75 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbca7fc700 1 -- 172.21.15.32:0/1565662570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbbc00b860 con 0x7fcbcc148530 2022-08-01T22:59:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbca7fc700 1 -- 172.21.15.32:0/1565662570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc00b9c0 con 0x7fcbcc148530 2022-08-01T22:59:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbca7fc700 1 -- 172.21.15.32:0/1565662570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbbc00bb30 con 0x7fcbcc148530 2022-08-01T22:59:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- 172.21.15.32:0/1565662570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 msgr2=0x7fcbcc0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 --2- 172.21.15.32:0/1565662570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0b1160 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fcbcc1478c0 tx=0x7fcbbc009290).stop 2022-08-01T22:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- 172.21.15.32:0/1565662570 shutdown_connections 2022-08-01T22:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 --2- 172.21.15.32:0/1565662570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0b1160 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 --2- 172.21.15.32:0/1565662570 >> v2:172.21.15.55:3300/0 conn(0x7fcbcc0a7250 0x7fcbcc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- 172.21.15.32:0/1565662570 >> 172.21.15.32:0/1565662570 conn(0x7fcbcc05f840 msgr2=0x7fcbcc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- 172.21.15.32:0/1565662570 shutdown_connections 2022-08-01T22:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.439+0000 7fcbd2228700 1 -- 172.21.15.32:0/1565662570 wait complete. 2022-08-01T22:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 Processor -- start 2022-08-01T22:59:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 -- start start 2022-08-01T22:59:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbcc0a7250 0x7fcbcc0aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbcc0b1f40 0x7fcbcc147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0ab7e0 con 0x7fcbcc148530 2022-08-01T22:59:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0aba10 con 0x7fcbcc0a7250 2022-08-01T22:59:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbd2228700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0abcb0 con 0x7fcbcc0b1f40 2022-08-01T22:59:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.443+0000 7fcbcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43402/0 (socket says 172.21.15.32:43402) 2022-08-01T22:59:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 -- 172.21.15.32:0/2356864481 learned_addr learned my addr 172.21.15.32:0/2356864481 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcb7fe700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbcc0a7250 0x7fcbcc0aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcaffd700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbcc0b1f40 0x7fcbcc147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 -- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbcc0b1f40 msgr2=0x7fcbcc147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbcc0b1f40 0x7fcbcc147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 -- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbcc0a7250 msgr2=0x7fcbcc0aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbcc0a7250 0x7fcbcc0aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 -- 172.21.15.32:0/2356864481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc009530 con 0x7fcbcc148530 2022-08-01T22:59:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbcbfff700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0adfc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fcbc4007000 tx=0x7fcbc4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbc4015070 con 0x7fcbcc148530 2022-08-01T22:59:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc400ce80 con 0x7fcbcc148530 2022-08-01T22:59:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc0aa9d0 con 0x7fcbcc148530 2022-08-01T22:59:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbc4010770 con 0x7fcbcc148530 2022-08-01T22:59:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc14acb0 con 0x7fcbcc148530 2022-08-01T22:59:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.447+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc0b0cc0 con 0x7fcbcc148530 2022-08-01T22:59:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.451+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcbc40108d0 con 0x7fcbcc148530 2022-08-01T22:59:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.451+0000 7fcbc8ff9700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbb0053aa0 0x7fcbb0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.451+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcbc406bba0 con 0x7fcbcc148530 2022-08-01T22:59:20.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.451+0000 7fcbcb7fe700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbb0053aa0 0x7fcbb0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.451+0000 7fcbcb7fe700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbb0053aa0 0x7fcbb0055f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fcbb400d000 tx=0x7fcbb4000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.459+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcbc40427f0 con 0x7fcbcc148530 2022-08-01T22:59:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.623+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbcc0574e0 con 0x7fcbcc148530 2022-08-01T22:59:20.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.623+0000 7fcbc8ff9700 1 -- 172.21.15.32:0/2356864481 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fcbc406b030 con 0x7fcbcc148530 2022-08-01T22:59:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:20.627 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:20.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbb0053aa0 msgr2=0x7fcbb0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbb0053aa0 0x7fcbb0055f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fcbb400d000 tx=0x7fcbb4000bc0).stop 2022-08-01T22:59:20.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 msgr2=0x7fcbcc0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0adfc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fcbc4007000 tx=0x7fcbc4008040).stop 2022-08-01T22:59:20.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 shutdown_connections 2022-08-01T22:59:20.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbb0053aa0 0x7fcbb0055f60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbcc148530 0x7fcbcc0adfc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbcc0b1f40 0x7fcbcc147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 --2- 172.21.15.32:0/2356864481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbcc0a7250 0x7fcbcc0aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 >> 172.21.15.32:0/2356864481 conn(0x7fcbcc05f840 msgr2=0x7fcbcc0af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 shutdown_connections 2022-08-01T22:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.627+0000 7fcbd2228700 1 -- 172.21.15.32:0/2356864481 wait complete. 2022-08-01T22:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:20.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:59:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 Processor -- start 2022-08-01T22:59:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 -- start start 2022-08-01T22:59:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b2710 0x7f491814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49180a8c50 0x7f491814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180a7250 con 0x7f49180b01a0 2022-08-01T22:59:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180a7480 con 0x7f49180a8c50 2022-08-01T22:59:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180a7720 con 0x7f49180b2710 2022-08-01T22:59:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4916d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49180a8c50 0x7f491814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4915d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49180b01a0 0x7f49180a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59180/0 (socket says 172.21.15.32:59180) 2022-08-01T22:59:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4915d9b700 1 -- 172.21.15.32:0/2762093540 learned_addr learned my addr 172.21.15.32:0/2762093540 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491659c700 1 --2- 172.21.15.32:0/2762093540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b2710 0x7f491814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491559a700 1 -- 172.21.15.32:0/2762093540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431139843 0 0) 0x7f49180a7250 con 0x7f49180b01a0 2022-08-01T22:59:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f491559a700 1 -- 172.21.15.32:0/2762093540 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48fc003260 con 0x7f49180b01a0 2022-08-01T22:59:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4916d9d700 1 -- 172.21.15.32:0/2762093540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b2710 msgr2=0x7f491814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4916d9d700 1 --2- 172.21.15.32:0/2762093540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b2710 0x7f491814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4916d9d700 1 -- 172.21.15.32:0/2762093540 >> v1:172.21.15.32:6789/0 conn(0x7f49180b01a0 legacy=0x7f49180a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.743+0000 7f4916d9d700 1 -- 172.21.15.32:0/2762093540 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180a79c0 con 0x7f49180a8c50 2022-08-01T22:59:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f4916d9d700 1 --2- 172.21.15.32:0/2762093540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814b4c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f4910002f70 tx=0x7f4910006df0).ready entity=mon.0 client_cookie=33a0066ecf8d6157 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491559a700 1 -- 172.21.15.32:0/2762093540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491000d4a0 con 0x7f49180a8c50 2022-08-01T22:59:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491559a700 1 -- 172.21.15.32:0/2762093540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910000e90 con 0x7f49180a8c50 2022-08-01T22:59:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491559a700 1 -- 172.21.15.32:0/2762093540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f491000d7f0 con 0x7f49180a8c50 2022-08-01T22:59:20.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 -- 172.21.15.32:0/2762093540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 msgr2=0x7f491814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 --2- 172.21.15.32:0/2762093540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814b4c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f4910002f70 tx=0x7f4910006df0).stop 2022-08-01T22:59:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 -- 172.21.15.32:0/2762093540 shutdown_connections 2022-08-01T22:59:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 --2- 172.21.15.32:0/2762093540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814b4c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 --2- 172.21.15.32:0/2762093540 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b2710 0x7f491814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 -- 172.21.15.32:0/2762093540 >> 172.21.15.32:0/2762093540 conn(0x7f4918060310 msgr2=0x7f49180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 -- 172.21.15.32:0/2762093540 shutdown_connections 2022-08-01T22:59:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 -- 172.21.15.32:0/2762093540 wait complete. 2022-08-01T22:59:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 Processor -- start 2022-08-01T22:59:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.747+0000 7f491759e700 1 -- start start 2022-08-01T22:59:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b01a0 0x7f491815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49180b2710 0x7f49180b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918067e60 con 0x7f49180a8c50 2022-08-01T22:59:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180b57f0 con 0x7f49180b2710 2022-08-01T22:59:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180b5a90 con 0x7f49180b01a0 2022-08-01T22:59:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43410/0 (socket says 172.21.15.32:43410) 2022-08-01T22:59:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 -- 172.21.15.32:0/2056653532 learned_addr learned my addr 172.21.15.32:0/2056653532 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f4916d9d700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49180b2710 0x7f49180b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f4915d9b700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b01a0 0x7f491815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 -- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b01a0 msgr2=0x7f491815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b01a0 0x7f491815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 -- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49180b2710 msgr2=0x7f49180b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49180b2710 0x7f49180b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 -- 172.21.15.32:0/2056653532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491000b180 con 0x7f49180a8c50 2022-08-01T22:59:20.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f4916d9d700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49180b2710 0x7f49180b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491659c700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814cd50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f490800c000 tx=0x7f49080091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4908012600 con 0x7f49180a8c50 2022-08-01T22:59:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908012760 con 0x7f49180a8c50 2022-08-01T22:59:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4918159c40 con 0x7f49180a8c50 2022-08-01T22:59:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4908012ac0 con 0x7f49180a8c50 2022-08-01T22:59:20.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.751+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49180b4bc0 con 0x7f49180a8c50 2022-08-01T22:59:20.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.755+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4908022070 con 0x7f49180a8c50 2022-08-01T22:59:20.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.755+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49180574e0 con 0x7f49180a8c50 2022-08-01T22:59:20.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.755+0000 7f490f7fe700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48f0053b00 0x7f48f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:20.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.755+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f490806c690 con 0x7f49180a8c50 2022-08-01T22:59:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.755+0000 7f4915d9b700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48f0053b00 0x7f48f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.755+0000 7f4915d9b700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48f0053b00 0x7f48f0055fc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f490000d000 tx=0x7f4900009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.759+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49080432c0 con 0x7f49180a8c50 2022-08-01T22:59:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.887+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4918069a70 con 0x7f49180a8c50 2022-08-01T22:59:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.887+0000 7f490f7fe700 1 -- 172.21.15.32:0/2056653532 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16286 (secure 0 0 0) 0x7f49080468e0 con 0x7f49180a8c50 2022-08-01T22:59:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:20.891 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":55,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:59:20.256580+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:59:17.662803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T22:59:18.633694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:59:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48f0053b00 msgr2=0x7f48f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48f0053b00 0x7f48f0055fc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f490000d000 tx=0x7f4900009040).stop 2022-08-01T22:59:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 msgr2=0x7f491814cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814cd50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f490800c000 tx=0x7f49080091d0).stop 2022-08-01T22:59:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 shutdown_connections 2022-08-01T22:59:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48f0053b00 0x7f48f0055fc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49180b2710 0x7f49180b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49180b01a0 0x7f491815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 --2- 172.21.15.32:0/2056653532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49180a8c50 0x7f491814cd50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 >> 172.21.15.32:0/2056653532 conn(0x7f4918060310 msgr2=0x7f4918148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 shutdown_connections 2022-08-01T22:59:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:20.891+0000 7f491759e700 1 -- 172.21.15.32:0/2056653532 wait complete. 2022-08-01T22:59:20.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T22:59:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 Processor -- start 2022-08-01T22:59:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 -- start start 2022-08-01T22:59:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300ad4b0 0x7fd0300af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0300ad9e0 0x7fd0300a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0300aff30 con 0x7fd0300af450 2022-08-01T22:59:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03005c7a0 con 0x7fd0300ad9e0 2022-08-01T22:59:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd036f62700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd030140990 con 0x7fd0300ad4b0 2022-08-01T22:59:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd035f60700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0300af450 0x7fd0301432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59192/0 (socket says 172.21.15.32:59192) 2022-08-01T22:59:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.019+0000 7fd035f60700 1 -- 172.21.15.32:0/1590022386 learned_addr learned my addr 172.21.15.32:0/1590022386 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036761700 1 --2- 172.21.15.32:0/1590022386 >> v2:172.21.15.55:3300/0 conn(0x7fd0300ad9e0 0x7fd0300a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd03575f700 1 --2- 172.21.15.32:0/1590022386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300ad4b0 0x7fd0300af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd034f5e700 1 -- 172.21.15.32:0/1590022386 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041725281 0 0) 0x7fd0300aff30 con 0x7fd0300af450 2022-08-01T22:59:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd034f5e700 1 -- 172.21.15.32:0/1590022386 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd018003260 con 0x7fd0300af450 2022-08-01T22:59:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd03575f700 1 -- 172.21.15.32:0/1590022386 >> v2:172.21.15.55:3300/0 conn(0x7fd0300ad9e0 msgr2=0x7fd0300a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd03575f700 1 --2- 172.21.15.32:0/1590022386 >> v2:172.21.15.55:3300/0 conn(0x7fd0300ad9e0 0x7fd0300a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd03575f700 1 -- 172.21.15.32:0/1590022386 >> v1:172.21.15.32:6789/0 conn(0x7fd0300af450 legacy=0x7fd0301432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd03575f700 1 -- 172.21.15.32:0/1590022386 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0301439f0 con 0x7fd0300ad4b0 2022-08-01T22:59:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd03575f700 1 --2- 172.21.15.32:0/1590022386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300ad4b0 0x7fd0300af830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd020002f70 tx=0x7fd020006df0).ready entity=mon.2 client_cookie=60a9ce5b55afe2b3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd034f5e700 1 -- 172.21.15.32:0/1590022386 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd020014070 con 0x7fd0300ad4b0 2022-08-01T22:59:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd034f5e700 1 -- 172.21.15.32:0/1590022386 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020000e90 con 0x7fd0300ad4b0 2022-08-01T22:59:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd034f5e700 1 -- 172.21.15.32:0/1590022386 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd02000fa90 con 0x7fd0300ad4b0 2022-08-01T22:59:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 -- 172.21.15.32:0/1590022386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300ad4b0 msgr2=0x7fd0300af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 --2- 172.21.15.32:0/1590022386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300ad4b0 0x7fd0300af830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd020002f70 tx=0x7fd020006df0).stop 2022-08-01T22:59:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 -- 172.21.15.32:0/1590022386 shutdown_connections 2022-08-01T22:59:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 --2- 172.21.15.32:0/1590022386 >> v2:172.21.15.55:3300/0 conn(0x7fd0300ad9e0 0x7fd0300a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 --2- 172.21.15.32:0/1590022386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300ad4b0 0x7fd0300af830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 -- 172.21.15.32:0/1590022386 >> 172.21.15.32:0/1590022386 conn(0x7fd0300612d0 msgr2=0x7fd030062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 -- 172.21.15.32:0/1590022386 shutdown_connections 2022-08-01T22:59:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.023+0000 7fd036f62700 1 -- 172.21.15.32:0/1590022386 wait complete. 2022-08-01T22:59:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 Processor -- start 2022-08-01T22:59:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 -- start start 2022-08-01T22:59:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 0x7fd030139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0300ad9e0 0x7fd0300a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300af450 0x7fd0300a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300abb70 con 0x7fd0300ad4b0 2022-08-01T22:59:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030067dc0 con 0x7fd0300ad9e0 2022-08-01T22:59:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036f62700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300aaea0 con 0x7fd0300af450 2022-08-01T22:59:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd035f60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 0x7fd030139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd035f60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 0x7fd030139440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43424/0 (socket says 172.21.15.32:43424) 2022-08-01T22:59:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd035f60700 1 -- 172.21.15.32:0/1583154067 learned_addr learned my addr 172.21.15.32:0/1583154067 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd03575f700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0300ad9e0 0x7fd0300a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.027+0000 7fd036761700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300af450 0x7fd0300a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd035f60700 1 -- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300af450 msgr2=0x7fd0300a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd035f60700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300af450 0x7fd0300a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd035f60700 1 -- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0300ad9e0 msgr2=0x7fd0300a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd035f60700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0300ad9e0 0x7fd0300a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd035f60700 1 -- 172.21.15.32:0/1583154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02000b180 con 0x7fd0300ad4b0 2022-08-01T22:59:21.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd035f60700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 0x7fd030139440 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fd02c001110 tx=0x7fd02c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd02c0070d0 con 0x7fd0300ad4b0 2022-08-01T22:59:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c007230 con 0x7fd0300ad4b0 2022-08-01T22:59:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0300ab140 con 0x7fd0300ad4b0 2022-08-01T22:59:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd02c005610 con 0x7fd0300ad4b0 2022-08-01T22:59:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0300ab880 con 0x7fd0300ad4b0 2022-08-01T22:59:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0300ab5e0 con 0x7fd0300ad4b0 2022-08-01T22:59:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd02c010070 con 0x7fd0300ad4b0 2022-08-01T22:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd01effd700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd010053950 0x7fd010055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd02c06b720 con 0x7fd0300ad4b0 2022-08-01T22:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.031+0000 7fd03575f700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd010053950 0x7fd010055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.035+0000 7fd03575f700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd010053950 0x7fd010055e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd02001d000 tx=0x7fd020016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.039+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0300ab5e0 con 0x7fd0300ad4b0 2022-08-01T22:59:21.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.207+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd0300ab5e0 con 0x7fd0300ad4b0 2022-08-01T22:59:21.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.219+0000 7fd01effd700 1 -- 172.21.15.32:0/1583154067 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd0300ab5e0 con 0x7fd0300ad4b0 2022-08-01T22:59:21.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd010053950 msgr2=0x7fd010055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd010053950 0x7fd010055e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd02001d000 tx=0x7fd020016000).stop 2022-08-01T22:59:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 msgr2=0x7fd030139440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 0x7fd030139440 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fd02c001110 tx=0x7fd02c007ad0).stop 2022-08-01T22:59:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 shutdown_connections 2022-08-01T22:59:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd010053950 0x7fd010055e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0300af450 0x7fd0300a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0300ad9e0 0x7fd0300a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 --2- 172.21.15.32:0/1583154067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0300ad4b0 0x7fd030139440 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 >> 172.21.15.32:0/1583154067 conn(0x7fd0300612d0 msgr2=0x7fd03013a270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 shutdown_connections 2022-08-01T22:59:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.223+0000 7fd036f62700 1 -- 172.21.15.32:0/1583154067 wait complete. 2022-08-01T22:59:21.228 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T22:59:21.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T22:59:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 Processor -- start 2022-08-01T22:59:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 -- start start 2022-08-01T22:59:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc0b31d0 0x7fb3fc066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3fc0a9da0 0x7fb3fc060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc067030 con 0x7fb3fc1434b0 2022-08-01T22:59:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc064a70 con 0x7fb3fc0a9da0 2022-08-01T22:59:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401d96700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc061f60 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3fc0a9da0 0x7fb3fc060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3fc0a9da0 0x7fb3fc060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43434/0 (socket says 172.21.15.32:43434) 2022-08-01T22:59:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb401595700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3fc1434b0 0x7fb3fc143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59210/0 (socket says 172.21.15.32:59210) 2022-08-01T22:59:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 -- 172.21.15.32:0/1435031017 learned_addr learned my addr 172.21.15.32:0/1435031017 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb400d94700 1 --2- 172.21.15.32:0/1435031017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc0b31d0 0x7fb3fc066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 -- 172.21.15.32:0/1435031017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc0b31d0 msgr2=0x7fb3fc066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 --2- 172.21.15.32:0/1435031017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc0b31d0 0x7fb3fc066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 -- 172.21.15.32:0/1435031017 >> v1:172.21.15.32:6789/0 conn(0x7fb3fc1434b0 legacy=0x7fb3fc143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 -- 172.21.15.32:0/1435031017 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0b35b0 con 0x7fb3fc0a9da0 2022-08-01T22:59:21.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.355+0000 7fb3fbfff700 1 --2- 172.21.15.32:0/1435031017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc060950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fb3ec002f70 tx=0x7fb3ec006f40).ready entity=mon.0 client_cookie=3c3967f5c8cbb02f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb3fb7fe700 1 -- 172.21.15.32:0/1435031017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3ec00da80 con 0x7fb3fc0a9da0 2022-08-01T22:59:21.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb3fb7fe700 1 -- 172.21.15.32:0/1435031017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec000e90 con 0x7fb3fc0a9da0 2022-08-01T22:59:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb3fb7fe700 1 -- 172.21.15.32:0/1435031017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3ec00de10 con 0x7fb3fc0a9da0 2022-08-01T22:59:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 -- 172.21.15.32:0/1435031017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0a9da0 msgr2=0x7fb3fc060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 --2- 172.21.15.32:0/1435031017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc060950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fb3ec002f70 tx=0x7fb3ec006f40).stop 2022-08-01T22:59:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 -- 172.21.15.32:0/1435031017 shutdown_connections 2022-08-01T22:59:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 --2- 172.21.15.32:0/1435031017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc060950 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 --2- 172.21.15.32:0/1435031017 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc0b31d0 0x7fb3fc066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 -- 172.21.15.32:0/1435031017 >> 172.21.15.32:0/1435031017 conn(0x7fb3fc063870 msgr2=0x7fb3fc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 -- 172.21.15.32:0/1435031017 shutdown_connections 2022-08-01T22:59:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 -- 172.21.15.32:0/1435031017 wait complete. 2022-08-01T22:59:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 Processor -- start 2022-08-01T22:59:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.359+0000 7fb401d96700 1 -- start start 2022-08-01T22:59:21.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 0x7fb3fc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc1434b0 0x7fb3fc1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b5c60 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc14e190 con 0x7fb3fc0a9da0 2022-08-01T22:59:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc14e430 con 0x7fb3fc1434b0 2022-08-01T22:59:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 0x7fb3fc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 0x7fb3fc14e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43440/0 (socket says 172.21.15.32:43440) 2022-08-01T22:59:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 -- 172.21.15.32:0/2586742587 learned_addr learned my addr 172.21.15.32:0/2586742587 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:21.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401595700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc1434b0 0x7fb3fc1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb400d94700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 -- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc1434b0 msgr2=0x7fb3fc1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc1434b0 0x7fb3fc1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 -- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc0a9da0 msgr2=0x7fb3fc144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 -- 172.21.15.32:0/2586742587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec00b460 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb400d94700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3fbfff700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 0x7fb3fc14e8d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb3ec002f70 tx=0x7fb3fc067360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3ec00dde0 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec010e10 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc0b5f00 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3ec018800 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.363+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc14edc0 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.367+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc066af0 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.367+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb3ec021020 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.367+0000 7fb3f9ffb700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3dc0539e0 0x7fb3dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.367+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb3ec06fd80 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.367+0000 7fb400d94700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3dc0539e0 0x7fb3dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.367+0000 7fb400d94700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3dc0539e0 0x7fb3dc055ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fb3f000b000 tx=0x7fb3f0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.375+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb3ec046a30 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.539+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb3fc14e600 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.547+0000 7fb3f9ffb700 1 -- 172.21.15.32:0/2586742587 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb3ec04a050 con 0x7fb3fc0b31d0 2022-08-01T22:59:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3dc0539e0 msgr2=0x7fb3dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3dc0539e0 0x7fb3dc055ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fb3f000b000 tx=0x7fb3f0008040).stop 2022-08-01T22:59:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 msgr2=0x7fb3fc14e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 0x7fb3fc14e8d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb3ec002f70 tx=0x7fb3fc067360).stop 2022-08-01T22:59:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 shutdown_connections 2022-08-01T22:59:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3dc0539e0 0x7fb3dc055ea0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3fc1434b0 0x7fb3fc1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3fc0b31d0 0x7fb3fc14e8d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 --2- 172.21.15.32:0/2586742587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc0a9da0 0x7fb3fc144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 >> 172.21.15.32:0/2586742587 conn(0x7fb3fc063870 msgr2=0x7fb3fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 shutdown_connections 2022-08-01T22:59:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.551+0000 7fb401d96700 1 -- 172.21.15.32:0/2586742587 wait complete. 2022-08-01T22:59:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T22:59:21.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.667+0000 7ffb8759e700 1 Processor -- start 2022-08-01T22:59:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- start start 2022-08-01T22:59:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffb88064b10 0x7ffb88144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb880b36e0 0x7ffb88141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88141860 0x7ffb88066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8813ebf0 con 0x7ffb880b36e0 2022-08-01T22:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb88064ef0 con 0x7ffb88064b10 2022-08-01T22:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88066690 con 0x7ffb88141860 2022-08-01T22:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb880b36e0 0x7ffb88141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb880b36e0 0x7ffb88141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43448/0 (socket says 172.21.15.32:43448) 2022-08-01T22:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffb88064b10 0x7ffb88144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 -- 172.21.15.32:0/571598820 learned_addr learned my addr 172.21.15.32:0/571598820 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb86d9d700 1 --2- 172.21.15.32:0/571598820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88141860 0x7ffb88066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 -- 172.21.15.32:0/571598820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88141860 msgr2=0x7ffb88066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 --2- 172.21.15.32:0/571598820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88141860 0x7ffb88066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 -- 172.21.15.32:0/571598820 >> v2:172.21.15.55:3300/0 conn(0x7ffb88064b10 msgr2=0x7ffb88144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 --2- 172.21.15.32:0/571598820 >> v2:172.21.15.55:3300/0 conn(0x7ffb88064b10 0x7ffb88144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 -- 172.21.15.32:0/571598820 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880b4850 con 0x7ffb880b36e0 2022-08-01T22:59:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb85d9b700 1 --2- 172.21.15.32:0/571598820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb880b36e0 0x7ffb88141370 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ffb70002f70 tx=0x7ffb70006eb0).ready entity=mon.0 client_cookie=2ee2bb2c1352521c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8559a700 1 -- 172.21.15.32:0/571598820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7000d570 con 0x7ffb880b36e0 2022-08-01T22:59:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8559a700 1 -- 172.21.15.32:0/571598820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb70000e90 con 0x7ffb880b36e0 2022-08-01T22:59:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8559a700 1 -- 172.21.15.32:0/571598820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7000d900 con 0x7ffb880b36e0 2022-08-01T22:59:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- 172.21.15.32:0/571598820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb880b36e0 msgr2=0x7ffb88141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- 172.21.15.32:0/571598820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb880b36e0 0x7ffb88141370 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ffb70002f70 tx=0x7ffb70006eb0).stop 2022-08-01T22:59:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- 172.21.15.32:0/571598820 shutdown_connections 2022-08-01T22:59:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- 172.21.15.32:0/571598820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb880b36e0 0x7ffb88141370 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- 172.21.15.32:0/571598820 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88141860 0x7ffb88066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 --2- 172.21.15.32:0/571598820 >> v2:172.21.15.55:3300/0 conn(0x7ffb88064b10 0x7ffb88144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- 172.21.15.32:0/571598820 >> 172.21.15.32:0/571598820 conn(0x7ffb8805f840 msgr2=0x7ffb8805fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- 172.21.15.32:0/571598820 shutdown_connections 2022-08-01T22:59:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.671+0000 7ffb8759e700 1 -- 172.21.15.32:0/571598820 wait complete. 2022-08-01T22:59:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.675+0000 7ffb8759e700 1 Processor -- start 2022-08-01T22:59:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.675+0000 7ffb8759e700 1 -- start start 2022-08-01T22:59:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88064b10 0x7ffb8813ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb880b36e0 0x7ffb88155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 0x7ffb88155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb880adf50 con 0x7ffb88141860 2022-08-01T22:59:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88061d30 con 0x7ffb880b36e0 2022-08-01T22:59:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88061fd0 con 0x7ffb88064b10 2022-08-01T22:59:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 0x7ffb88155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 0x7ffb88155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43456/0 (socket says 172.21.15.32:43456) 2022-08-01T22:59:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 -- 172.21.15.32:0/1517602095 learned_addr learned my addr 172.21.15.32:0/1517602095 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8659c700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88064b10 0x7ffb8813ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb85d9b700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb880b36e0 0x7ffb88155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 -- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88064b10 msgr2=0x7ffb8813ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88064b10 0x7ffb8813ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 -- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb880b36e0 msgr2=0x7ffb88155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb880b36e0 0x7ffb88155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 -- 172.21.15.32:0/1517602095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7000b1e0 con 0x7ffb88141860 2022-08-01T22:59:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb86d9d700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 0x7ffb88155ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7ffb7800c000 tx=0x7ffb78006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb78011750 con 0x7ffb88141860 2022-08-01T22:59:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb780118b0 con 0x7ffb88141860 2022-08-01T22:59:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb880b0460 con 0x7ffb88141860 2022-08-01T22:59:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb78011c10 con 0x7ffb88141860 2022-08-01T22:59:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8813ff10 con 0x7ffb88141860 2022-08-01T22:59:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb880ae150 con 0x7ffb88141860 2022-08-01T22:59:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.679+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb78011d70 con 0x7ffb88141860 2022-08-01T22:59:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.683+0000 7ffb7f7fe700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb6c0539e0 0x7ffb6c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.683+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb7806c480 con 0x7ffb88141860 2022-08-01T22:59:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.683+0000 7ffb8659c700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb6c0539e0 0x7ffb6c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.683+0000 7ffb8659c700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb6c0539e0 0x7ffb6c055ea0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7ffb8000b000 tx=0x7ffb800091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.683+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb78043130 con 0x7ffb88141860 2022-08-01T22:59:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb880b4af0 con 0x7ffb88141860 2022-08-01T22:59:21.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb7f7fe700 1 -- 172.21.15.32:0/1517602095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ffb78046750 con 0x7ffb88141860 2022-08-01T22:59:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb6c0539e0 msgr2=0x7ffb6c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb6c0539e0 0x7ffb6c055ea0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7ffb8000b000 tx=0x7ffb800091d0).stop 2022-08-01T22:59:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 msgr2=0x7ffb88155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 0x7ffb88155ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7ffb7800c000 tx=0x7ffb78006df0).stop 2022-08-01T22:59:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 shutdown_connections 2022-08-01T22:59:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb6c0539e0 0x7ffb6c055ea0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb88141860 0x7ffb88155ef0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb880b36e0 0x7ffb88155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 --2- 172.21.15.32:0/1517602095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffb88064b10 0x7ffb8813ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 >> 172.21.15.32:0/1517602095 conn(0x7ffb8805f840 msgr2=0x7ffb88140420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 shutdown_connections 2022-08-01T22:59:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:21.847+0000 7ffb8759e700 1 -- 172.21.15.32:0/1517602095 wait complete. 2022-08-01T22:59:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:22.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:59:19.253309+0000', 'modified': '2022-08-01T22:59:20.304429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5604': {'gid': 5604, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/2848708845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2848708845}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2848708845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:59:22.166 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:59:22.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T22:59:22.283 INFO:teuthology.orchestra.run.smithi055.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5604,"want_state":"up:active","state":"up:active","rank_uptime":3.0185632120000001,"mdsmap_epoch":23,"osdmap_epoch":56,"osdmap_epoch_barrier":55,"uptime":7.3625543440000003} 2022-08-01T22:59:22.283 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5604, "mdsmap_epoch": 23, "osdmap_epoch": 56, "osdmap_epoch_barrier": 55, "rank_uptime": 3.018563212, "state": "up:active", "uptime": 7.362554344, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:59:22.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:59:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 Processor -- start 2022-08-01T22:59:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 -- start start 2022-08-01T22:59:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c0b31d0 0x7f1c7c066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c7c0a9da0 0x7f1c7c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c7c1434b0 0x7f1c7c143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c064a70 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c061f60 con 0x7f1c7c1434b0 2022-08-01T22:59:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c067030 con 0x7f1c7c0b31d0 2022-08-01T22:59:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c7c0a9da0 0x7f1c7c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c7c0a9da0 0x7f1c7c060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43462/0 (socket says 172.21.15.32:43462) 2022-08-01T22:59:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 -- 172.21.15.32:0/627670662 learned_addr learned my addr 172.21.15.32:0/627670662 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7a59c700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c0b31d0 0x7f1c7c066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7ad9d700 1 --2- 172.21.15.32:0/627670662 >> v2:172.21.15.55:3300/0 conn(0x7f1c7c1434b0 0x7f1c7c143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 -- 172.21.15.32:0/627670662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c0b31d0 msgr2=0x7f1c7c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c0b31d0 0x7f1c7c066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 -- 172.21.15.32:0/627670662 >> v2:172.21.15.55:3300/0 conn(0x7f1c7c1434b0 msgr2=0x7f1c7c143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 --2- 172.21.15.32:0/627670662 >> v2:172.21.15.55:3300/0 conn(0x7f1c7c1434b0 0x7f1c7c143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 -- 172.21.15.32:0/627670662 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0b35b0 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7a59c700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c0b31d0 0x7f1c7c066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7ad9d700 1 --2- 172.21.15.32:0/627670662 >> v2:172.21.15.55:3300/0 conn(0x7f1c7c1434b0 0x7f1c7c143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c79d9b700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c060950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f1c70002f70 tx=0x7f1c70006f40).ready entity=mon.0 client_cookie=531a5d66836b8d6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7959a700 1 -- 172.21.15.32:0/627670662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c7000d9a0 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7959a700 1 -- 172.21.15.32:0/627670662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c70000e90 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.395+0000 7f1c7959a700 1 -- 172.21.15.32:0/627670662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c7000dd30 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- 172.21.15.32:0/627670662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 msgr2=0x7f1c7c060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c060950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f1c70002f70 tx=0x7f1c70006f40).stop 2022-08-01T22:59:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- 172.21.15.32:0/627670662 shutdown_connections 2022-08-01T22:59:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c060950 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/627670662 >> v2:172.21.15.55:3300/0 conn(0x7f1c7c1434b0 0x7f1c7c143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/627670662 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c0b31d0 0x7f1c7c066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- 172.21.15.32:0/627670662 >> 172.21.15.32:0/627670662 conn(0x7f1c7c063870 msgr2=0x7f1c7c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- 172.21.15.32:0/627670662 shutdown_connections 2022-08-01T22:59:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- 172.21.15.32:0/627670662 wait complete. 2022-08-01T22:59:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 Processor -- start 2022-08-01T22:59:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- start start 2022-08-01T22:59:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c7c0b31d0 0x7f1c7c0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c1434b0 0x7f1c7c0aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0af390 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c155860 con 0x7f1c7c0b31d0 2022-08-01T22:59:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c155b00 con 0x7f1c7c1434b0 2022-08-01T22:59:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.399+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43466/0 (socket says 172.21.15.32:43466) 2022-08-01T22:59:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 -- 172.21.15.32:0/4230565585 learned_addr learned my addr 172.21.15.32:0/4230565585 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7ad9d700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c1434b0 0x7f1c7c0aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c79d9b700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c7c0b31d0 0x7f1c7c0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 -- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c1434b0 msgr2=0x7f1c7c0aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c1434b0 0x7f1c7c0aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 -- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c7c0b31d0 msgr2=0x7f1c7c0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c7c0b31d0 0x7f1c7c0ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 -- 172.21.15.32:0/4230565585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009470 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7ad9d700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c1434b0 0x7f1c7c0aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7a59c700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c141ad0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1c7400c000 tx=0x7f1c740091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c7400c850 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7400c9b0 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c00b180 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c7400cd10 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7000b460 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c066af0 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.403+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c7401b400 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.407+0000 7f1c6b7fe700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c58053950 0x7f1c58055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.407+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1c7406c9b0 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.407+0000 7f1c79d9b700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c58053950 0x7f1c58055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.407+0000 7f1c79d9b700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c58053950 0x7f1c58055e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f1c7001b000 tx=0x7f1c7c067360).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.411+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c74043710 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.539+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c7c1561e0 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.539+0000 7f1c6b7fe700 1 -- 172.21.15.32:0/4230565585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f1c74046d30 con 0x7f1c7c0a9da0 2022-08-01T22:59:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:22.544 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:59:21.260196+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:59:17.662803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T22:59:18.633694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:59:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c58053950 msgr2=0x7f1c58055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c58053950 0x7f1c58055e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f1c7001b000 tx=0x7f1c7c067360).stop 2022-08-01T22:59:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 msgr2=0x7f1c7c141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c141ad0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1c7400c000 tx=0x7f1c740091d0).stop 2022-08-01T22:59:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 shutdown_connections 2022-08-01T22:59:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c58053950 0x7f1c58055e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c7c1434b0 0x7f1c7c0aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c7c0b31d0 0x7f1c7c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 --2- 172.21.15.32:0/4230565585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c7c0a9da0 0x7f1c7c141ad0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 >> 172.21.15.32:0/4230565585 conn(0x7f1c7c063870 msgr2=0x7f1c7c0b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 shutdown_connections 2022-08-01T22:59:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.543+0000 7f1c7b59e700 1 -- 172.21.15.32:0/4230565585 wait complete. 2022-08-01T22:59:22.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:59:17.662803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:59:18.633694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:59:22.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:59:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.663+0000 7f9ecffff700 1 Processor -- start 2022-08-01T22:59:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.663+0000 7f9ecffff700 1 -- start start 2022-08-01T22:59:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ed00b01a0 0x7f9ed00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ed00afd60 0x7f9ed00a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00a7250 con 0x7f9ed00b2710 2022-08-01T22:59:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00a7480 con 0x7f9ed00b01a0 2022-08-01T22:59:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00a7720 con 0x7f9ed00afd60 2022-08-01T22:59:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ed00b01a0 0x7f9ed00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ed00b01a0 0x7f9ed00a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43476/0 (socket says 172.21.15.32:43476) 2022-08-01T22:59:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 -- 172.21.15.32:0/984184495 learned_addr learned my addr 172.21.15.32:0/984184495 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecf7fe700 1 --2- 172.21.15.32:0/984184495 >> v2:172.21.15.55:3300/0 conn(0x7f9ed00afd60 0x7f9ed00a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 -- 172.21.15.32:0/984184495 >> v2:172.21.15.55:3300/0 conn(0x7f9ed00afd60 msgr2=0x7f9ed00a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 --2- 172.21.15.32:0/984184495 >> v2:172.21.15.55:3300/0 conn(0x7f9ed00afd60 0x7f9ed00a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 -- 172.21.15.32:0/984184495 >> v1:172.21.15.55:6789/0 conn(0x7f9ed00b2710 legacy=0x7f9ed014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 -- 172.21.15.32:0/984184495 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00a79c0 con 0x7f9ed00b01a0 2022-08-01T22:59:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ece7fc700 1 --2- 172.21.15.32:0/984184495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b01a0 0x7f9ed00a8540 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f9eb8002f70 tx=0x7f9eb8006eb0).ready entity=mon.0 client_cookie=844de3f01a2a06a1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecdffb700 1 -- 172.21.15.32:0/984184495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb800d5c0 con 0x7f9ed00b01a0 2022-08-01T22:59:22.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecdffb700 1 -- 172.21.15.32:0/984184495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8000e90 con 0x7f9ed00b01a0 2022-08-01T22:59:22.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecdffb700 1 -- 172.21.15.32:0/984184495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb800d950 con 0x7f9ed00b01a0 2022-08-01T22:59:22.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- 172.21.15.32:0/984184495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b01a0 msgr2=0x7f9ed00a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 --2- 172.21.15.32:0/984184495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b01a0 0x7f9ed00a8540 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f9eb8002f70 tx=0x7f9eb8006eb0).stop 2022-08-01T22:59:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- 172.21.15.32:0/984184495 shutdown_connections 2022-08-01T22:59:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 --2- 172.21.15.32:0/984184495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b01a0 0x7f9ed00a8540 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 --2- 172.21.15.32:0/984184495 >> v2:172.21.15.55:3300/0 conn(0x7f9ed00afd60 0x7f9ed00a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- 172.21.15.32:0/984184495 >> 172.21.15.32:0/984184495 conn(0x7f9ed0060310 msgr2=0x7f9ed00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- 172.21.15.32:0/984184495 shutdown_connections 2022-08-01T22:59:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.667+0000 7f9ecffff700 1 -- 172.21.15.32:0/984184495 wait complete. 2022-08-01T22:59:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 Processor -- start 2022-08-01T22:59:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 -- start start 2022-08-01T22:59:22.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ed00afd60 0x7f9ed00646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 0x7f9ed0156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ed0066380 0x7f9ed0066810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00636f0 con 0x7f9ed00b2710 2022-08-01T22:59:22.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00abe90 con 0x7f9ed00afd60 2022-08-01T22:59:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0064ba0 con 0x7f9ed0066380 2022-08-01T22:59:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 0x7f9ed0156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 0x7f9ed0156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43482/0 (socket says 172.21.15.32:43482) 2022-08-01T22:59:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 -- 172.21.15.32:0/1323493664 learned_addr learned my addr 172.21.15.32:0/1323493664 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecf7fe700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ed0066380 0x7f9ed0066810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9eceffd700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ed00afd60 0x7f9ed00646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 -- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ed0066380 msgr2=0x7f9ed0066810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ed0066380 0x7f9ed0066810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 -- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ed00afd60 msgr2=0x7f9ed00646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ed00afd60 0x7f9ed00646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 -- 172.21.15.32:0/1323493664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb800b1e0 con 0x7f9ed00b2710 2022-08-01T22:59:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ece7fc700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 0x7f9ed0156c00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f9eb8000b60 tx=0x7f9eb8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb8011840 con 0x7f9ed00b2710 2022-08-01T22:59:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb80119a0 con 0x7f9ed00b2710 2022-08-01T22:59:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed0066ec0 con 0x7f9ed00b2710 2022-08-01T22:59:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eb8011d00 con 0x7f9ed00b2710 2022-08-01T22:59:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.671+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00abb00 con 0x7f9ed00b2710 2022-08-01T22:59:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.675+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00ac000 con 0x7f9ed00b2710 2022-08-01T22:59:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.675+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9eb801a040 con 0x7f9ed00b2710 2022-08-01T22:59:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.675+0000 7f9ec7fff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9eb4053aa0 0x7f9eb4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.675+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9eb806cfb0 con 0x7f9ed00b2710 2022-08-01T22:59:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.675+0000 7f9eceffd700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9eb4053aa0 0x7f9eb4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.679+0000 7f9eceffd700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9eb4053aa0 0x7f9eb4055f60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f9ec000d000 tx=0x7f9ec0007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.683+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ed00ac000 con 0x7f9ed00b2710 2022-08-01T22:59:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.807+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9ed00ac000 con 0x7f9ed00b2710 2022-08-01T22:59:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.807+0000 7f9ec7fff700 1 -- 172.21.15.32:0/1323493664 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9ed00ac000 con 0x7f9ed00b2710 2022-08-01T22:59:22.812 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:59:22.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9eb4053aa0 msgr2=0x7f9eb4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9eb4053aa0 0x7f9eb4055f60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f9ec000d000 tx=0x7f9ec0007f10).stop 2022-08-01T22:59:22.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 msgr2=0x7f9ed0156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 0x7f9ed0156c00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f9eb8000b60 tx=0x7f9eb8015000).stop 2022-08-01T22:59:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 shutdown_connections 2022-08-01T22:59:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9eb4053aa0 0x7f9eb4055f60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ed0066380 0x7f9ed0066810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ed00b2710 0x7f9ed0156c00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 --2- 172.21.15.32:0/1323493664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ed00afd60 0x7f9ed00646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 >> 172.21.15.32:0/1323493664 conn(0x7f9ed0060310 msgr2=0x7f9ed0148ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 shutdown_connections 2022-08-01T22:59:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.811+0000 7f9ecffff700 1 -- 172.21.15.32:0/1323493664 wait complete. 2022-08-01T22:59:22.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:59:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 Processor -- start 2022-08-01T22:59:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- start start 2022-08-01T22:59:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba40b6510 0x7f1ba413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ba40b0e60 0x7f1ba413d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba413dc80 con 0x7f1ba4064500 2022-08-01T22:59:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba4066af0 con 0x7f1ba40b0e60 2022-08-01T22:59:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4066d90 con 0x7f1ba40b6510 2022-08-01T22:59:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ba40b0e60 0x7f1ba413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ba40b0e60 0x7f1ba413d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53274/0 (socket says 172.21.15.32:53274) 2022-08-01T22:59:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 -- 172.21.15.32:0/3549024747 learned_addr learned my addr 172.21.15.32:0/3549024747 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1baa0bb700 1 --2- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba40b6510 0x7f1ba413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 -- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba40b6510 msgr2=0x7f1ba413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 --2- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba40b6510 0x7f1ba413d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 -- 172.21.15.32:0/3549024747 >> v1:172.21.15.55:6789/0 conn(0x7f1ba4064500 legacy=0x7f1ba40609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba90b9700 1 -- 172.21.15.32:0/3549024747 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007513636 0 0) 0x7f1ba413dc80 con 0x7f1ba4064500 2022-08-01T22:59:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 -- 172.21.15.32:0/3549024747 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba4067030 con 0x7f1ba40b0e60 2022-08-01T22:59:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba98ba700 1 --2- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413d790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f1b98002f70 tx=0x7f1b98006eb0).ready entity=mon.1 client_cookie=f5bfae2d1f8d2e38 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba90b9700 1 -- 172.21.15.32:0/3549024747 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b9800d570 con 0x7f1ba40b0e60 2022-08-01T22:59:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba90b9700 1 -- 172.21.15.32:0/3549024747 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b98000e90 con 0x7f1ba40b0e60 2022-08-01T22:59:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1ba90b9700 1 -- 172.21.15.32:0/3549024747 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b9800d8c0 con 0x7f1ba40b0e60 2022-08-01T22:59:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 msgr2=0x7f1ba413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413d790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f1b98002f70 tx=0x7f1b98006eb0).stop 2022-08-01T22:59:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- 172.21.15.32:0/3549024747 shutdown_connections 2022-08-01T22:59:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413d790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/3549024747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba40b6510 0x7f1ba413d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- 172.21.15.32:0/3549024747 >> 172.21.15.32:0/3549024747 conn(0x7f1ba4063800 msgr2=0x7f1ba4063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- 172.21.15.32:0/3549024747 shutdown_connections 2022-08-01T22:59:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.931+0000 7f1bab0bd700 1 -- 172.21.15.32:0/3549024747 wait complete. 2022-08-01T22:59:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 Processor -- start 2022-08-01T22:59:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 -- start start 2022-08-01T22:59:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba4064500 0x7f1ba40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 0x7f1ba413cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4140c60 con 0x7f1ba40b6510 2022-08-01T22:59:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4140f00 con 0x7f1ba40b0e60 2022-08-01T22:59:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba41451c0 con 0x7f1ba4064500 2022-08-01T22:59:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 0x7f1ba413cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 0x7f1ba413cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43496/0 (socket says 172.21.15.32:43496) 2022-08-01T22:59:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 -- 172.21.15.32:0/2364382621 learned_addr learned my addr 172.21.15.32:0/2364382621 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1ba98ba700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa0bb700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba4064500 0x7f1ba40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 -- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba4064500 msgr2=0x7f1ba40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba4064500 0x7f1ba40afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 -- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 msgr2=0x7f1ba413c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 -- 172.21.15.32:0/2364382621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9800b1e0 con 0x7f1ba40b6510 2022-08-01T22:59:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1baa8bc700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 0x7f1ba413cae0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f1b9c000f40 tx=0x7f1b9c007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b9c00e4a0 con 0x7f1ba40b6510 2022-08-01T22:59:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c00e600 con 0x7f1ba40b6510 2022-08-01T22:59:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba4145460 con 0x7f1ba40b6510 2022-08-01T22:59:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b9c00e960 con 0x7f1ba40b6510 2022-08-01T22:59:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.935+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba413cfd0 con 0x7f1ba40b6510 2022-08-01T22:59:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.939+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba40574e0 con 0x7f1ba40b6510 2022-08-01T22:59:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.939+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b9c010070 con 0x7f1ba40b6510 2022-08-01T22:59:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.939+0000 7f1b977fe700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b840539e0 0x7f1b84055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.939+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1b9c06b450 con 0x7f1ba40b6510 2022-08-01T22:59:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.939+0000 7f1baa0bb700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b840539e0 0x7f1b84055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.939+0000 7f1baa0bb700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b840539e0 0x7f1b84055ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1ba000b000 tx=0x7f1ba0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:22.947+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b9c042210 con 0x7f1ba40b6510 2022-08-01T22:59:23.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.071+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ba406ac80 con 0x7f1ba40b6510 2022-08-01T22:59:23.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.071+0000 7f1b977fe700 1 -- 172.21.15.32:0/2364382621 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1b9c045830 con 0x7f1ba40b6510 2022-08-01T22:59:23.075 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b840539e0 msgr2=0x7f1b84055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b840539e0 0x7f1b84055ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1ba000b000 tx=0x7f1ba0008040).stop 2022-08-01T22:59:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 msgr2=0x7f1ba413cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 0x7f1ba413cae0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f1b9c000f40 tx=0x7f1b9c007d60).stop 2022-08-01T22:59:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 shutdown_connections 2022-08-01T22:59:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1b840539e0 0x7f1b84055ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ba40b6510 0x7f1ba413cae0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ba40b0e60 0x7f1ba413c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 --2- 172.21.15.32:0/2364382621 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ba4064500 0x7f1ba40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 >> 172.21.15.32:0/2364382621 conn(0x7f1ba4063800 msgr2=0x7f1ba413d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 shutdown_connections 2022-08-01T22:59:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.075+0000 7f1bab0bd700 1 -- 172.21.15.32:0/2364382621 wait complete. 2022-08-01T22:59:23.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:59:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 Processor -- start 2022-08-01T22:59:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 -- start start 2022-08-01T22:59:23.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9d900614b0 0x7f9d90061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d901405d0 0x7f9d90140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900aee80 con 0x7f9d900ad750 2022-08-01T22:59:23.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900b70b0 con 0x7f9d900614b0 2022-08-01T22:59:23.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900af0b0 con 0x7f9d901405d0 2022-08-01T22:59:23.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9d900614b0 0x7f9d90061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d901405d0 0x7f9d90140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9d900614b0 0x7f9d90061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53288/0 (socket says 172.21.15.32:53288) 2022-08-01T22:59:23.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 -- 172.21.15.32:0/596391966 learned_addr learned my addr 172.21.15.32:0/596391966 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:23.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8e7fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9d900ad750 0x7f9d900b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:51952/0 (socket says 172.21.15.32:51952) 2022-08-01T22:59:23.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 -- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d901405d0 msgr2=0x7f9d90140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:23.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 --2- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d901405d0 0x7f9d90140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 -- 172.21.15.32:0/596391966 >> v1:172.21.15.55:6789/0 conn(0x7f9d900ad750 legacy=0x7f9d900b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8effd700 1 -- 172.21.15.32:0/596391966 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900af350 con 0x7f9d900614b0 2022-08-01T22:59:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8dffb700 1 -- 172.21.15.32:0/596391966 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420143548 0 0) 0x7f9d900aee80 con 0x7f9d900ad750 2022-08-01T22:59:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.195+0000 7f9d8f7fe700 1 --2- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d901405d0 0x7f9d90140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8effd700 1 --2- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900614b0 0x7f9d90061890 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f9d900aed30 tx=0x7f9d80006eb0).ready entity=mon.1 client_cookie=e8ef14b5acb89e08 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8dffb700 1 -- 172.21.15.32:0/596391966 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d8000d5c0 con 0x7f9d900614b0 2022-08-01T22:59:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8dffb700 1 -- 172.21.15.32:0/596391966 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d80000e90 con 0x7f9d900614b0 2022-08-01T22:59:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8dffb700 1 -- 172.21.15.32:0/596391966 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d8000d910 con 0x7f9d900614b0 2022-08-01T22:59:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 -- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900614b0 msgr2=0x7f9d90061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900614b0 0x7f9d90061890 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f9d900aed30 tx=0x7f9d80006eb0).stop 2022-08-01T22:59:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 -- 172.21.15.32:0/596391966 shutdown_connections 2022-08-01T22:59:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d901405d0 0x7f9d90140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/596391966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900614b0 0x7f9d90061890 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 -- 172.21.15.32:0/596391966 >> 172.21.15.32:0/596391966 conn(0x7f9d90066f70 msgr2=0x7f9d90067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 -- 172.21.15.32:0/596391966 shutdown_connections 2022-08-01T22:59:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 -- 172.21.15.32:0/596391966 wait complete. 2022-08-01T22:59:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.199+0000 7f9d8ffff700 1 Processor -- start 2022-08-01T22:59:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 -- start start 2022-08-01T22:59:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 0x7f9d900b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900ad750 0x7f9d900b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 0x7f9d90141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900aaba0 con 0x7f9d901405d0 2022-08-01T22:59:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900ab520 con 0x7f9d900ad750 2022-08-01T22:59:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900aadd0 con 0x7f9d900614b0 2022-08-01T22:59:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 0x7f9d90141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900ad750 0x7f9d900b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 0x7f9d900b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900ad750 0x7f9d900b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53296/0 (socket says 172.21.15.32:53296) 2022-08-01T22:59:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 0x7f9d90141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43512/0 (socket says 172.21.15.32:43512) 2022-08-01T22:59:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8e7fc700 1 -- 172.21.15.32:0/3915960924 learned_addr learned my addr 172.21.15.32:0/3915960924 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 0x7f9d900b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55246/0 (socket says 172.21.15.32:55246) 2022-08-01T22:59:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 -- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900ad750 msgr2=0x7f9d900b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900ad750 0x7f9d900b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 -- 172.21.15.32:0/3915960924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 msgr2=0x7f9d90141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 0x7f9d90141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 -- 172.21.15.32:0/3915960924 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009470 con 0x7f9d900614b0 2022-08-01T22:59:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8f7fe700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 0x7f9d90141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8effd700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 0x7f9d900b0a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9d90152e60 tx=0x7f9d80015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d80002f50 con 0x7f9d900614b0 2022-08-01T22:59:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8000d940 con 0x7f9d900614b0 2022-08-01T22:59:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d80011de0 con 0x7f9d900614b0 2022-08-01T22:59:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8000b1e0 con 0x7f9d900614b0 2022-08-01T22:59:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.203+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d90143bf0 con 0x7f9d900614b0 2022-08-01T22:59:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.207+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d900b3cb0 con 0x7f9d900614b0 2022-08-01T22:59:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.207+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d8001a040 con 0x7f9d900614b0 2022-08-01T22:59:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.207+0000 7f9d7ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d6c053980 0x7f9d6c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.207+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9d8006cf90 con 0x7f9d900614b0 2022-08-01T22:59:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.207+0000 7f9d8e7fc700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d6c053980 0x7f9d6c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.207+0000 7f9d8e7fc700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d6c053980 0x7f9d6c055e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f9d8400d000 tx=0x7f9d84007a60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.219+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d80043cc0 con 0x7f9d900614b0 2022-08-01T22:59:23.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.343+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d90152ea0 con 0x7f9d900614b0 2022-08-01T22:59:23.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.343+0000 7f9d7ffff700 1 -- 172.21.15.32:0/3915960924 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9d80047260 con 0x7f9d900614b0 2022-08-01T22:59:23.347 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:23.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d6c053980 msgr2=0x7f9d6c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d6c053980 0x7f9d6c055e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f9d8400d000 tx=0x7f9d84007a60).stop 2022-08-01T22:59:23.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 msgr2=0x7f9d900b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 0x7f9d900b0a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9d90152e60 tx=0x7f9d80015000).stop 2022-08-01T22:59:23.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 shutdown_connections 2022-08-01T22:59:23.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d6c053980 0x7f9d6c055e40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d901405d0 0x7f9d90141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d900ad750 0x7f9d900b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 --2- 172.21.15.32:0/3915960924 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9d900614b0 0x7f9d900b0a60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 >> 172.21.15.32:0/3915960924 conn(0x7f9d90066f70 msgr2=0x7f9d9005c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 shutdown_connections 2022-08-01T22:59:23.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.347+0000 7f9d8ffff700 1 -- 172.21.15.32:0/3915960924 wait complete. 2022-08-01T22:59:23.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:59:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2359e700 1 Processor -- start 2022-08-01T22:59:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2359e700 1 -- start start 2022-08-01T22:59:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3e240ad750 0x7f3e24061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e240aee80 con 0x7f3e240614b0 2022-08-01T22:59:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e240b70b0 con 0x7f3e241405d0 2022-08-01T22:59:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240af0b0 con 0x7f3e240ad750 2022-08-01T22:59:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3e240614b0 0x7f3e240b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59290/0 (socket says 172.21.15.32:59290) 2022-08-01T22:59:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2259c700 1 -- 172.21.15.32:0/922695245 learned_addr learned my addr 172.21.15.32:0/922695245 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/922695245 >> v2:172.21.15.55:3300/0 conn(0x7f3e240ad750 0x7f3e24061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800210830 0 0) 0x7f3e240aee80 con 0x7f3e240614b0 2022-08-01T22:59:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e04003260 con 0x7f3e240614b0 2022-08-01T22:59:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3345659867 0 0) 0x7f3e04003260 con 0x7f3e240614b0 2022-08-01T22:59:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 >> v2:172.21.15.55:3300/0 conn(0x7f3e240ad750 msgr2=0x7f3e24061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 --2- 172.21.15.32:0/922695245 >> v2:172.21.15.55:3300/0 conn(0x7f3e240ad750 0x7f3e24061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 >> v1:172.21.15.55:6789/0 conn(0x7f3e241405d0 legacy=0x7f3e241466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240af350 con 0x7f3e240614b0 2022-08-01T22:59:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f3e14001660 con 0x7f3e240614b0 2022-08-01T22:59:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 >> v1:172.21.15.32:6789/0 conn(0x7f3e240614b0 legacy=0x7f3e240b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e04003750 0x7f3e04003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e04004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e04004e70 0x7f3e0400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240af0b0 con 0x7f3e04004500 2022-08-01T22:59:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0400bd20 con 0x7f3e04003750 2022-08-01T22:59:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0400bee0 con 0x7f3e04004e70 2022-08-01T22:59:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e04004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e2259c700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e04003750 0x7f3e04003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e22d9d700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e04004e70 0x7f3e0400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 -- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e04004e70 msgr2=0x7f3e0400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e04004e70 0x7f3e0400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 -- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e04003750 msgr2=0x7f3e04003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e04003750 0x7f3e04003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e22d9d700 1 -- 172.21.15.32:0/922695245 reap_dead start 2022-08-01T22:59:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 -- 172.21.15.32:0/922695245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04003260 con 0x7f3e04004500 2022-08-01T22:59:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.471+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e04004980 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f3e0c00c000 tx=0x7f3e0c006df0).ready entity=mon.0 client_cookie=9606fa51701e757e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0c00e800 con 0x7f3e04004500 2022-08-01T22:59:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c00e960 con 0x7f3e04004500 2022-08-01T22:59:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e21d9b700 1 -- 172.21.15.32:0/922695245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0c00ecc0 con 0x7f3e04004500 2022-08-01T22:59:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 -- 172.21.15.32:0/922695245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 msgr2=0x7f3e04004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e04004980 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f3e0c00c000 tx=0x7f3e0c006df0).stop 2022-08-01T22:59:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 -- 172.21.15.32:0/922695245 shutdown_connections 2022-08-01T22:59:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 --2- 172.21.15.32:0/922695245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e04004980 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 -- 172.21.15.32:0/922695245 >> 172.21.15.32:0/922695245 conn(0x7f3e24066f70 msgr2=0x7f3e24067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 -- 172.21.15.32:0/922695245 shutdown_connections 2022-08-01T22:59:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 -- 172.21.15.32:0/922695245 wait complete. 2022-08-01T22:59:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 Processor -- start 2022-08-01T22:59:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.475+0000 7f3e2359e700 1 -- start start 2022-08-01T22:59:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e240b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e240afac0 0x7f3e240b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e240b3530 0x7f3e240b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24153370 con 0x7f3e04004500 2022-08-01T22:59:23.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240b21b0 con 0x7f3e240afac0 2022-08-01T22:59:23.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e241535a0 con 0x7f3e240b3530 2022-08-01T22:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e240b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e240b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43528/0 (socket says 172.21.15.32:43528) 2022-08-01T22:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 -- 172.21.15.32:0/3565866718 learned_addr learned my addr 172.21.15.32:0/3565866718 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e22d9d700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e240b3530 0x7f3e240b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e240afac0 0x7f3e240b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 -- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e240b3530 msgr2=0x7f3e240b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e240b3530 0x7f3e240b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 -- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e240afac0 msgr2=0x7f3e240b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e240afac0 0x7f3e240b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 -- 172.21.15.32:0/3565866718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24153840 con 0x7f3e04004500 2022-08-01T22:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2259c700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e240b17d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f3e1400d000 tx=0x7f3e240af4e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e14011780 con 0x7f3e04004500 2022-08-01T22:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e140118e0 con 0x7f3e04004500 2022-08-01T22:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e14011c40 con 0x7f3e04004500 2022-08-01T22:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e240ab1d0 con 0x7f3e04004500 2022-08-01T22:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e240acfb0 con 0x7f3e04004500 2022-08-01T22:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.479+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e240af5f0 con 0x7f3e04004500 2022-08-01T22:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.483+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3e1401c360 con 0x7f3e04004500 2022-08-01T22:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.483+0000 7f3e1bfff700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3e08053b00 0x7f3e08055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.483+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3e1406bc20 con 0x7f3e04004500 2022-08-01T22:59:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.483+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3e08053b00 0x7f3e08055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.483+0000 7f3e19d9b700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3e08053b00 0x7f3e08055fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f3e0c01b000 tx=0x7f3e0c019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.491+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3e14042980 con 0x7f3e04004500 2022-08-01T22:59:23.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.615+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e240574e0 con 0x7f3e04004500 2022-08-01T22:59:23.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.615+0000 7f3e1bfff700 1 -- 172.21.15.32:0/3565866718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3e14019020 con 0x7f3e04004500 2022-08-01T22:59:23.618 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.615+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3e08053b00 msgr2=0x7f3e08055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.615+0000 7f3e2359e700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3e08053b00 0x7f3e08055fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f3e0c01b000 tx=0x7f3e0c019000).stop 2022-08-01T22:59:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.615+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 msgr2=0x7f3e240b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.615+0000 7f3e2359e700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e240b17d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f3e1400d000 tx=0x7f3e240af4e0).stop 2022-08-01T22:59:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 shutdown_connections 2022-08-01T22:59:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3e08053b00 0x7f3e08055fc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3e240b3530 0x7f3e240b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e240afac0 0x7f3e240b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 --2- 172.21.15.32:0/3565866718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e04004500 0x7f3e240b17d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 >> 172.21.15.32:0/3565866718 conn(0x7f3e24066f70 msgr2=0x7f3e2413fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 shutdown_connections 2022-08-01T22:59:23.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.619+0000 7f3e2359e700 1 -- 172.21.15.32:0/3565866718 wait complete. 2022-08-01T22:59:23.632 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:59:23.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.731+0000 7fb4656f9700 1 Processor -- start 2022-08-01T22:59:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.731+0000 7fb4656f9700 1 -- start start 2022-08-01T22:59:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4600ad750 0x7fb460061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4601405d0 0x7fb460140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4600aee80 con 0x7fb4600614b0 2022-08-01T22:59:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600b70b0 con 0x7fb4601405d0 2022-08-01T22:59:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600af0b0 con 0x7fb4600ad750 2022-08-01T22:59:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb45ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb4600614b0 0x7fb4600b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59310/0 (socket says 172.21.15.32:59310) 2022-08-01T22:59:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb464ef8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4601405d0 0x7fb460140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb45ffff700 1 -- 172.21.15.32:0/1478927099 learned_addr learned my addr 172.21.15.32:0/1478927099 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb457fff700 1 --2- 172.21.15.32:0/1478927099 >> v2:172.21.15.55:3300/0 conn(0x7fb4600ad750 0x7fb460061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb464ef8700 1 -- 172.21.15.32:0/1478927099 >> v2:172.21.15.55:3300/0 conn(0x7fb4600ad750 msgr2=0x7fb460061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb464ef8700 1 --2- 172.21.15.32:0/1478927099 >> v2:172.21.15.55:3300/0 conn(0x7fb4600ad750 0x7fb460061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb464ef8700 1 -- 172.21.15.32:0/1478927099 >> v1:172.21.15.32:6789/0 conn(0x7fb4600614b0 legacy=0x7fb4600b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb464ef8700 1 -- 172.21.15.32:0/1478927099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600af350 con 0x7fb4601405d0 2022-08-01T22:59:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb464ef8700 1 --2- 172.21.15.32:0/1478927099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4601405d0 0x7fb460140a80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fb4580032f0 tx=0x7fb4580091d0).ready entity=mon.0 client_cookie=be38577b51104c82 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb45f7fe700 1 -- 172.21.15.32:0/1478927099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45800b680 con 0x7fb4601405d0 2022-08-01T22:59:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb45f7fe700 1 -- 172.21.15.32:0/1478927099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800b7e0 con 0x7fb4601405d0 2022-08-01T22:59:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb45f7fe700 1 -- 172.21.15.32:0/1478927099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45800b950 con 0x7fb4601405d0 2022-08-01T22:59:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- 172.21.15.32:0/1478927099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4601405d0 msgr2=0x7fb460140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 --2- 172.21.15.32:0/1478927099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4601405d0 0x7fb460140a80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fb4580032f0 tx=0x7fb4580091d0).stop 2022-08-01T22:59:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- 172.21.15.32:0/1478927099 shutdown_connections 2022-08-01T22:59:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 --2- 172.21.15.32:0/1478927099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4601405d0 0x7fb460140a80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 --2- 172.21.15.32:0/1478927099 >> v2:172.21.15.55:3300/0 conn(0x7fb4600ad750 0x7fb460061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- 172.21.15.32:0/1478927099 >> 172.21.15.32:0/1478927099 conn(0x7fb460066f70 msgr2=0x7fb460067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- 172.21.15.32:0/1478927099 shutdown_connections 2022-08-01T22:59:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.735+0000 7fb4656f9700 1 -- 172.21.15.32:0/1478927099 wait complete. 2022-08-01T22:59:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 Processor -- start 2022-08-01T22:59:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 -- start start 2022-08-01T22:59:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 0x7fb46005c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4600ad750 0x7fb46013f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4601405d0 0x7fb46013fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600b0c00 con 0x7fb4600614b0 2022-08-01T22:59:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600ad580 con 0x7fb4600ad750 2022-08-01T22:59:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb4656f9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600a9c60 con 0x7fb4601405d0 2022-08-01T22:59:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 0x7fb46005c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb464ef8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4601405d0 0x7fb46013fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb457fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4600ad750 0x7fb46013f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 0x7fb46005c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43542/0 (socket says 172.21.15.32:43542) 2022-08-01T22:59:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 -- 172.21.15.32:0/579653049 learned_addr learned my addr 172.21.15.32:0/579653049 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 -- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4601405d0 msgr2=0x7fb46013fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4601405d0 0x7fb46013fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 -- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4600ad750 msgr2=0x7fb46013f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4600ad750 0x7fb46013f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.739+0000 7fb45ffff700 1 -- 172.21.15.32:0/579653049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009470 con 0x7fb4600614b0 2022-08-01T22:59:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45ffff700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 0x7fb46005c170 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fb4600ae2b0 tx=0x7fb450007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4500070d0 con 0x7fb4600614b0 2022-08-01T22:59:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb450007230 con 0x7fb4600614b0 2022-08-01T22:59:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4600a9f00 con 0x7fb4600614b0 2022-08-01T22:59:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb450005610 con 0x7fb4600614b0 2022-08-01T22:59:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46009f5a0 con 0x7fb4600614b0 2022-08-01T22:59:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4600af5f0 con 0x7fb4600614b0 2022-08-01T22:59:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb450010070 con 0x7fb4600614b0 2022-08-01T22:59:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45dffb700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb444053980 0x7fb444055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb45006b5b0 con 0x7fb4600614b0 2022-08-01T22:59:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.743+0000 7fb457fff700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb444053980 0x7fb444055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.747+0000 7fb457fff700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb444053980 0x7fb444055e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fb44800b000 tx=0x7fb448006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.751+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb450042370 con 0x7fb4600614b0 2022-08-01T22:59:23.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:23.768 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:23.771 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:23.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:23.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.915+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4600574e0 con 0x7fb4600614b0 2022-08-01T22:59:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.915+0000 7fb45dffb700 1 -- 172.21.15.32:0/579653049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fb450018070 con 0x7fb4600614b0 2022-08-01T22:59:23.919 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:23.919 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb444053980 msgr2=0x7fb444055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb444053980 0x7fb444055e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fb44800b000 tx=0x7fb448006df0).stop 2022-08-01T22:59:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 msgr2=0x7fb46005c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 0x7fb46005c170 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fb4600ae2b0 tx=0x7fb450007c00).stop 2022-08-01T22:59:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 shutdown_connections 2022-08-01T22:59:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb444053980 0x7fb444055e40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4601405d0 0x7fb46013fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4600ad750 0x7fb46013f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 --2- 172.21.15.32:0/579653049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4600614b0 0x7fb46005c170 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 >> 172.21.15.32:0/579653049 conn(0x7fb460066f70 msgr2=0x7fb46005cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 shutdown_connections 2022-08-01T22:59:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:23.919+0000 7fb4656f9700 1 -- 172.21.15.32:0/579653049 wait complete. 2022-08-01T22:59:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:24.240 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:59:19.253309+0000', 'modified': '2022-08-01T22:59:20.304429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5604': {'gid': 5604, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/2848708845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2848708845}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2848708845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:59:24.241 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:59:24.242 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T22:59:24.366 INFO:teuthology.orchestra.run.smithi055.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5604,"want_state":"up:active","state":"up:active","rank_uptime":5.1021395949999997,"mdsmap_epoch":23,"osdmap_epoch":56,"osdmap_epoch_barrier":55,"uptime":9.4461195930000006} 2022-08-01T22:59:24.367 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5604, "mdsmap_epoch": 23, "osdmap_epoch": 56, "osdmap_epoch_barrier": 55, "rank_uptime": 5.102139595, "state": "up:active", "uptime": 9.446119593, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:59:24.368 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:59:24.368 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T22:59:24.368 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T22:59:24.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.369 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:24.378 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:59:24.440 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:59:24.441 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:59:24.453 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:59:24.453 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:59:24.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.455 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:59:24.460 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:59:24.461 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:24.464 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:59:24.483 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:59:24.484 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:59:24.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T22:59:24.537+0000 7ff4d7de7080 -1 init, newargv = 0x562a7e052870 newargc=9 2022-08-01T22:59:24.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22689]: starting ceph client 2022-08-01T22:59:24.546 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:59:24.547 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:59:24.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.548 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:59:24.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22689]: starting fuse 2022-08-01T22:59:24.556 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T22:59:24.557 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:59:24.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.557 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T22:59:24.558 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T22:59:24.558 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T22:59:24.559 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:59:24.559 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T22:59:24.560 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.560 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T22:59:24.561 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T22:59:24.561 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T22:59:24.562 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.562 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T22:59:24.563 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T22:59:24.563 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T22:59:24.564 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T22:59:24.564 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.564 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T22:59:24.565 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:59:24.565 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:59:24.566 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T22:59:24.566 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:59:24.566 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.567 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T22:59:24.567 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:59:24.602 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22689.asok 2022-08-01T22:59:24.607 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22689.asok 2022-08-01T22:59:24.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.608 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22689.asok status 2022-08-01T22:59:24.734 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T22:59:24.735 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T22:59:24.735 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:59:24.735 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:59:24.735 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T22:59:24.736 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T22:59:24.736 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T22:59:24.736 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22689", 2022-08-01T22:59:24.736 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T22:59:24.737 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:24.737 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T22:59:24.737 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T22:59:24.737 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 5348, 2022-08-01T22:59:24.738 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T22:59:24.738 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T22:59:24.738 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T22:59:24.739 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 5348 2022-08-01T22:59:24.739 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:24.739 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:59:24.739 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:59:24.740 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:59:24.740 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1884393188 2022-08-01T22:59:24.740 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T22:59:24.740 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:24.740 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:59:24.741 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:59:24.741 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:59:24.741 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1884393188 2022-08-01T22:59:24.742 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:24.742 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.5348 172.21.15.130:0/1884393188", 2022-08-01T22:59:24.742 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/1884393188", 2022-08-01T22:59:24.742 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T22:59:24.743 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 23, 2022-08-01T22:59:24.743 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 56, 2022-08-01T22:59:24.743 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:59:24.743 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T22:59:24.743 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T22:59:24.744 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T22:59:24.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.746 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:24.747 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:24.754 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:59:24.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:59:24.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.757 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:24.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:24.821 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:24.822 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:24.830 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:59:24.831 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:59:24.832 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T22:59:24.833 DEBUG:teuthology.orchestra.run.smithi130:> import sys 2022-08-01T22:59:24.833 DEBUG:teuthology.orchestra.run.smithi130:> import time 2022-08-01T22:59:24.833 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:59:24.834 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.834 DEBUG:teuthology.orchestra.run.smithi130:> n = 1000 2022-08-01T22:59:24.835 DEBUG:teuthology.orchestra.run.smithi130:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T22:59:24.835 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.835 DEBUG:teuthology.orchestra.run.smithi130:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T22:59:24.836 DEBUG:teuthology.orchestra.run.smithi130:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T22:59:24.836 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:24.837 DEBUG:teuthology.orchestra.run.smithi130:> for i in range(0, n): 2022-08-01T22:59:24.837 DEBUG:teuthology.orchestra.run.smithi130:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T22:59:24.837 DEBUG:teuthology.orchestra.run.smithi130:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T22:59:24.838 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content'"'"') 2022-08-01T22:59:24.838 DEBUG:teuthology.orchestra.run.smithi130:> if True: 2022-08-01T22:59:24.839 DEBUG:teuthology.orchestra.run.smithi130:> f.flush() 2022-08-01T22:59:24.839 DEBUG:teuthology.orchestra.run.smithi130:> os.fsync(f.fileno()) 2022-08-01T22:59:24.840 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:59:36.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.231+0000 7fcbe2ead700 1 Processor -- start 2022-08-01T22:59:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.231+0000 7fcbe2ead700 1 -- start start 2022-08-01T22:59:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe2ead700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b08a0 0x7fcbdc142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe2ead700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcbdc0ad000 0x7fcbdc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe2ead700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbdc143ab0 con 0x7fcbdc0b3260 2022-08-01T22:59:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe2ead700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b0c80 con 0x7fcbdc0ad000 2022-08-01T22:59:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe2ead700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0ae810 con 0x7fcbdc0b08a0 2022-08-01T22:59:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe16aa700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcbdc0b3260 0x7fcbdc1433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59326/0 (socket says 172.21.15.32:59326) 2022-08-01T22:59:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe16aa700 1 -- 172.21.15.32:0/1944284923 learned_addr learned my addr 172.21.15.32:0/1944284923 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe26ac700 1 --2- 172.21.15.32:0/1944284923 >> v2:172.21.15.55:3300/0 conn(0x7fcbdc0ad000 0x7fcbdc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe1eab700 1 --2- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b08a0 0x7fcbdc142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe0ea9700 1 -- 172.21.15.32:0/1944284923 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268577477 0 0) 0x7fcbdc143ab0 con 0x7fcbdc0b3260 2022-08-01T22:59:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe0ea9700 1 -- 172.21.15.32:0/1944284923 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc4003260 con 0x7fcbdc0b3260 2022-08-01T22:59:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe26ac700 1 -- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b08a0 msgr2=0x7fcbdc142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe26ac700 1 --2- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b08a0 0x7fcbdc142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe26ac700 1 -- 172.21.15.32:0/1944284923 >> v1:172.21.15.32:6789/0 conn(0x7fcbdc0b3260 legacy=0x7fcbdc1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe26ac700 1 -- 172.21.15.32:0/1944284923 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0aeab0 con 0x7fcbdc0ad000 2022-08-01T22:59:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe26ac700 1 --2- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc13fa30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fcbd40032f0 tx=0x7fcbd40091d0).ready entity=mon.1 client_cookie=7a3a497528722216 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe0ea9700 1 -- 172.21.15.32:0/1944284923 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbd400b600 con 0x7fcbdc0ad000 2022-08-01T22:59:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe0ea9700 1 -- 172.21.15.32:0/1944284923 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd400b760 con 0x7fcbdc0ad000 2022-08-01T22:59:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.235+0000 7fcbe0ea9700 1 -- 172.21.15.32:0/1944284923 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbd400b8d0 con 0x7fcbdc0ad000 2022-08-01T22:59:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 -- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0ad000 msgr2=0x7fcbdc13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc13fa30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fcbd40032f0 tx=0x7fcbd40091d0).stop 2022-08-01T22:59:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 -- 172.21.15.32:0/1944284923 shutdown_connections 2022-08-01T22:59:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc13fa30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/1944284923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b08a0 0x7fcbdc142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 -- 172.21.15.32:0/1944284923 >> 172.21.15.32:0/1944284923 conn(0x7fcbdc0612d0 msgr2=0x7fcbdc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 -- 172.21.15.32:0/1944284923 shutdown_connections 2022-08-01T22:59:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 -- 172.21.15.32:0/1944284923 wait complete. 2022-08-01T22:59:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 Processor -- start 2022-08-01T22:59:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.239+0000 7fcbe2ead700 1 -- start start 2022-08-01T22:59:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0b08a0 0x7fcbdc13ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b3260 0x7fcbdc13d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b39f0 con 0x7fcbdc0ad000 2022-08-01T22:59:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc067240 con 0x7fcbdc0b08a0 2022-08-01T22:59:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0b5690 con 0x7fcbdc0b3260 2022-08-01T22:59:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43556/0 (socket says 172.21.15.32:43556) 2022-08-01T22:59:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 -- 172.21.15.32:0/3039858471 learned_addr learned my addr 172.21.15.32:0/3039858471 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe16aa700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0b08a0 0x7fcbdc13ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe26ac700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b3260 0x7fcbdc13d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 -- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b3260 msgr2=0x7fcbdc13d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b3260 0x7fcbdc13d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 -- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0b08a0 msgr2=0x7fcbdc13ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0b08a0 0x7fcbdc13ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 -- 172.21.15.32:0/3039858471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4009470 con 0x7fcbdc0ad000 2022-08-01T22:59:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe16aa700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0b08a0 0x7fcbdc13ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe1eab700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc141090 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fcbd800d000 tx=0x7fcbd8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbd8010820 con 0x7fcbdc0ad000 2022-08-01T22:59:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd8010980 con 0x7fcbdc0ad000 2022-08-01T22:59:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc0b3c20 con 0x7fcbdc0ad000 2022-08-01T22:59:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbd8010ce0 con 0x7fcbdc0ad000 2022-08-01T22:59:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.243+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc13d840 con 0x7fcbdc0ad000 2022-08-01T22:59:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.247+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc0b40c0 con 0x7fcbdc0ad000 2022-08-01T22:59:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.247+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcbd8017030 con 0x7fcbdc0ad000 2022-08-01T22:59:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.247+0000 7fcbcaffd700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbbc053950 0x7fcbbc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.247+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcbd806c150 con 0x7fcbdc0ad000 2022-08-01T22:59:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.247+0000 7fcbe16aa700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbbc053950 0x7fcbbc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.247+0000 7fcbe16aa700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbbc053950 0x7fcbbc055e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fcbcc00d000 tx=0x7fcbcc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.255+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcbd8042e00 con 0x7fcbdc0ad000 2022-08-01T22:59:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.415+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbdc0b44a0 con 0x7fcbdc0ad000 2022-08-01T22:59:36.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.419+0000 7fcbcaffd700 1 -- 172.21.15.32:0/3039858471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fcbd801f020 con 0x7fcbdc0ad000 2022-08-01T22:59:36.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:36.421 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbbc053950 msgr2=0x7fcbbc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbbc053950 0x7fcbbc055e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fcbcc00d000 tx=0x7fcbcc009040).stop 2022-08-01T22:59:36.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 msgr2=0x7fcbdc141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc141090 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fcbd800d000 tx=0x7fcbd8008040).stop 2022-08-01T22:59:36.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 shutdown_connections 2022-08-01T22:59:36.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbbc053950 0x7fcbbc055e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbdc0b3260 0x7fcbdc13d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbdc0b08a0 0x7fcbdc13ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 --2- 172.21.15.32:0/3039858471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc0ad000 0x7fcbdc141090 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 >> 172.21.15.32:0/3039858471 conn(0x7fcbdc0612d0 msgr2=0x7fcbdc142f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:36.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 shutdown_connections 2022-08-01T22:59:36.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.423+0000 7fcbe2ead700 1 -- 172.21.15.32:0/3039858471 wait complete. 2022-08-01T22:59:36.429 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:36.438 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-08-01T22:59:36.552 INFO:teuthology.orchestra.run.smithi055.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T22:59:36.552 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T22:59:36.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 Processor -- start 2022-08-01T22:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 -- start start 2022-08-01T22:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feed40b2710 0x7feed414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 0x7feed414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feed40afd60 0x7feed40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed40a7250 con 0x7feed40b2710 2022-08-01T22:59:36.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed40a7480 con 0x7feed40afd60 2022-08-01T22:59:36.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed3fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed40a7720 con 0x7feed40b01a0 2022-08-01T22:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feed40b2710 0x7feed414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed27fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 0x7feed414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed27fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 0x7feed414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55302/0 (socket says 172.21.15.32:55302) 2022-08-01T22:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed27fc700 1 -- 172.21.15.32:0/2961935452 learned_addr learned my addr 172.21.15.32:0/2961935452 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feed40b2710 0x7feed414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43566/0 (socket says 172.21.15.32:43566) 2022-08-01T22:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.663+0000 7feed37fe700 1 --2- 172.21.15.32:0/2961935452 >> v2:172.21.15.55:3300/0 conn(0x7feed40afd60 0x7feed40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed27fc700 1 -- 172.21.15.32:0/2961935452 >> v2:172.21.15.55:3300/0 conn(0x7feed40afd60 msgr2=0x7feed40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed27fc700 1 --2- 172.21.15.32:0/2961935452 >> v2:172.21.15.55:3300/0 conn(0x7feed40afd60 0x7feed40a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed27fc700 1 -- 172.21.15.32:0/2961935452 >> v2:172.21.15.32:3300/0 conn(0x7feed40b2710 msgr2=0x7feed414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed27fc700 1 --2- 172.21.15.32:0/2961935452 >> v2:172.21.15.32:3300/0 conn(0x7feed40b2710 0x7feed414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed27fc700 1 -- 172.21.15.32:0/2961935452 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed40a79c0 con 0x7feed40b01a0 2022-08-01T22:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed27fc700 1 --2- 172.21.15.32:0/2961935452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 0x7feed414b4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7feec4002f70 tx=0x7feec4006eb0).ready entity=mon.2 client_cookie=4880e295693550af server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed1ffb700 1 -- 172.21.15.32:0/2961935452 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feec400d570 con 0x7feed40b01a0 2022-08-01T22:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed1ffb700 1 -- 172.21.15.32:0/2961935452 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4000e90 con 0x7feed40b01a0 2022-08-01T22:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed1ffb700 1 -- 172.21.15.32:0/2961935452 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feec400d900 con 0x7feed40b01a0 2022-08-01T22:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 -- 172.21.15.32:0/2961935452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 msgr2=0x7feed414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 --2- 172.21.15.32:0/2961935452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 0x7feed414b4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7feec4002f70 tx=0x7feec4006eb0).stop 2022-08-01T22:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 -- 172.21.15.32:0/2961935452 shutdown_connections 2022-08-01T22:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 --2- 172.21.15.32:0/2961935452 >> v2:172.21.15.55:3300/0 conn(0x7feed40afd60 0x7feed40a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 --2- 172.21.15.32:0/2961935452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b01a0 0x7feed414b4c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 --2- 172.21.15.32:0/2961935452 >> v2:172.21.15.32:3300/0 conn(0x7feed40b2710 0x7feed414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 -- 172.21.15.32:0/2961935452 >> 172.21.15.32:0/2961935452 conn(0x7feed4060310 msgr2=0x7feed40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 -- 172.21.15.32:0/2961935452 shutdown_connections 2022-08-01T22:59:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 -- 172.21.15.32:0/2961935452 wait complete. 2022-08-01T22:59:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 Processor -- start 2022-08-01T22:59:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.667+0000 7feed3fff700 1 -- start start 2022-08-01T22:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed40afd60 0x7feed40aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 0x7feed414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b2710 0x7feed414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed40b37a0 con 0x7feed40b01a0 2022-08-01T22:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed415a650 con 0x7feed40afd60 2022-08-01T22:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed414cef0 con 0x7feed40b2710 2022-08-01T22:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 0x7feed414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 0x7feed414b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43570/0 (socket says 172.21.15.32:43570) 2022-08-01T22:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 -- 172.21.15.32:0/1040474728 learned_addr learned my addr 172.21.15.32:0/1040474728 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed37fe700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b2710 0x7feed414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed2ffd700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed40afd60 0x7feed40aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 -- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b2710 msgr2=0x7feed414beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b2710 0x7feed414beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 -- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed40afd60 msgr2=0x7feed40aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed40afd60 0x7feed40aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 -- 172.21.15.32:0/1040474728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec400b1e0 con 0x7feed40b01a0 2022-08-01T22:59:36.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed37fe700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b2710 0x7feed414beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed27fc700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 0x7feed414b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7feec4002f70 tx=0x7feec4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feec40118c0 con 0x7feed40b01a0 2022-08-01T22:59:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4011a20 con 0x7feed40b01a0 2022-08-01T22:59:36.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed40b6000 con 0x7feed40b01a0 2022-08-01T22:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feec4011d80 con 0x7feed40b01a0 2022-08-01T22:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed414c3a0 con 0x7feed40b01a0 2022-08-01T22:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feec4025030 con 0x7feed40b01a0 2022-08-01T22:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed414c590 con 0x7feed40b01a0 2022-08-01T22:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feec3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feeb0053a70 0x7feeb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.671+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feec406d140 con 0x7feed40b01a0 2022-08-01T22:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.675+0000 7feed2ffd700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feeb0053a70 0x7feeb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.675+0000 7feed2ffd700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feeb0053a70 0x7feeb0055f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7feec800b000 tx=0x7feec8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.683+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feec4043f00 con 0x7feed40b01a0 2022-08-01T22:59:36.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.847+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feed415a480 con 0x7feed40b01a0 2022-08-01T22:59:36.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.847+0000 7feec3fff700 1 -- 172.21.15.32:0/1040474728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7feec4023020 con 0x7feed40b01a0 2022-08-01T22:59:36.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:36.851 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:36.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feeb0053a70 msgr2=0x7feeb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feeb0053a70 0x7feeb0055f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7feec800b000 tx=0x7feec8008040).stop 2022-08-01T22:59:36.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 msgr2=0x7feed414b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:36.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 0x7feed414b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7feec4002f70 tx=0x7feec4015000).stop 2022-08-01T22:59:36.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 shutdown_connections 2022-08-01T22:59:36.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feeb0053a70 0x7feeb0055f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7feed40b2710 0x7feed414beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feed40b01a0 0x7feed414b9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 --2- 172.21.15.32:0/1040474728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed40afd60 0x7feed40aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:36.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 >> 172.21.15.32:0/1040474728 conn(0x7feed4060310 msgr2=0x7feed41460a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:36.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 shutdown_connections 2022-08-01T22:59:36.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:36.851+0000 7feed3fff700 1 -- 172.21.15.32:0/1040474728 wait complete. 2022-08-01T22:59:36.856 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:36.864 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T22:59:36.980 INFO:teuthology.orchestra.run.smithi055.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T22:59:36.981 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T22:59:36.981 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:59:36.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:37.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff956764700 1 Processor -- start 2022-08-01T22:59:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff956764700 1 -- start start 2022-08-01T22:59:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff956764700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9500b1160 0x7ff9501448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff956764700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500b04e0 con 0x7ff9500b1f40 2022-08-01T22:59:37.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff956764700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500b0780 con 0x7ff950148530 2022-08-01T22:59:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff956764700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500a9250 con 0x7ff9500b1160 2022-08-01T22:59:37.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9500b1f40 0x7ff9500b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59352/0 (socket says 172.21.15.32:59352) 2022-08-01T22:59:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 -- 172.21.15.32:0/4152008062 learned_addr learned my addr 172.21.15.32:0/4152008062 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff954d01700 1 --2- 172.21.15.32:0/4152008062 >> v2:172.21.15.55:3300/0 conn(0x7ff9500b1160 0x7ff9501448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299564315 0 0) 0x7ff9500b04e0 con 0x7ff9500b1f40 2022-08-01T22:59:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff930003260 con 0x7ff9500b1f40 2022-08-01T22:59:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389939506 0 0) 0x7ff9500b0780 con 0x7ff950148530 2022-08-01T22:59:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9500b04e0 con 0x7ff950148530 2022-08-01T22:59:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 620126230 0 0) 0x7ff930003260 con 0x7ff9500b1f40 2022-08-01T22:59:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 >> v2:172.21.15.55:3300/0 conn(0x7ff9500b1160 msgr2=0x7ff9501448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 --2- 172.21.15.32:0/4152008062 >> v2:172.21.15.55:3300/0 conn(0x7ff9500b1160 0x7ff9501448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 >> v1:172.21.15.55:6789/0 conn(0x7ff950148530 legacy=0x7ff9500a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500b0a20 con 0x7ff9500b1f40 2022-08-01T22:59:37.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7ff938001660 con 0x7ff9500b1f40 2022-08-01T22:59:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 >> v1:172.21.15.32:6789/0 conn(0x7ff9500b1f40 legacy=0x7ff9500b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff930003930 0x7ff930003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930004500 0x7ff930004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff930004e70 0x7ff93000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff930003930 0x7ff930003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93000bcf0 con 0x7ff930003930 2022-08-01T22:59:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93000be60 con 0x7ff930004500 2022-08-01T22:59:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93000c020 con 0x7ff930004e70 2022-08-01T22:59:37.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94f7fe700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930004500 0x7ff930004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff954d01700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff930004e70 0x7ff93000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 -- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff930004e70 msgr2=0x7ff93000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff930004e70 0x7ff93000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 -- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930004500 msgr2=0x7ff930004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930004500 0x7ff930004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 -- 172.21.15.32:0/4152008062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930003260 con 0x7ff930003930 2022-08-01T22:59:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff954d01700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff930004e70 0x7ff93000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:59:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff954d01700 1 -- 172.21.15.32:0/4152008062 reap_dead start 2022-08-01T22:59:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94ffff700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff930003930 0x7ff930003d30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7ff938001430 tx=0x7ff9500b0910).ready entity=mon.0 client_cookie=45c043cc8c468cc2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9380070d0 con 0x7ff930003930 2022-08-01T22:59:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938007230 con 0x7ff930003930 2022-08-01T22:59:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.107+0000 7ff94effd700 1 -- 172.21.15.32:0/4152008062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff93800c8f0 con 0x7ff930003930 2022-08-01T22:59:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- 172.21.15.32:0/4152008062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff930003930 msgr2=0x7ff930003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff930003930 0x7ff930003d30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7ff938001430 tx=0x7ff9500b0910).stop 2022-08-01T22:59:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- 172.21.15.32:0/4152008062 shutdown_connections 2022-08-01T22:59:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 --2- 172.21.15.32:0/4152008062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff930003930 0x7ff930003d30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- 172.21.15.32:0/4152008062 >> 172.21.15.32:0/4152008062 conn(0x7ff95005f840 msgr2=0x7ff95005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- 172.21.15.32:0/4152008062 shutdown_connections 2022-08-01T22:59:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- 172.21.15.32:0/4152008062 wait complete. 2022-08-01T22:59:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 Processor -- start 2022-08-01T22:59:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- start start 2022-08-01T22:59:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930003930 0x7ff9500ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9501439d0 0x7ff950143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 0x7ff95015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500a9640 con 0x7ff95015a150 2022-08-01T22:59:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500a98e0 con 0x7ff930003930 2022-08-01T22:59:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.111+0000 7ff956764700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500a9b80 con 0x7ff9501439d0 2022-08-01T22:59:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 0x7ff95015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 0x7ff95015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43594/0 (socket says 172.21.15.32:43594) 2022-08-01T22:59:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 -- 172.21.15.32:0/2431438874 learned_addr learned my addr 172.21.15.32:0/2431438874 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94f7fe700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9501439d0 0x7ff950143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930003930 0x7ff9500ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 -- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9501439d0 msgr2=0x7ff950143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9501439d0 0x7ff950143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 -- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930003930 msgr2=0x7ff9500ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930003930 0x7ff9500ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 -- 172.21.15.32:0/2431438874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95014a070 con 0x7ff95015a150 2022-08-01T22:59:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930003930 0x7ff9500ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff954d01700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 0x7ff95015a5d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff94800b000 tx=0x7ff948008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff948019070 con 0x7ff95015a150 2022-08-01T22:59:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff948014810 con 0x7ff95015a150 2022-08-01T22:59:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9500aee10 con 0x7ff95015a150 2022-08-01T22:59:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff948014b70 con 0x7ff95015a150 2022-08-01T22:59:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff950144340 con 0x7ff95015a150 2022-08-01T22:59:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9500b0cc0 con 0x7ff95015a150 2022-08-01T22:59:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff948012040 con 0x7ff95015a150 2022-08-01T22:59:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94d7fa700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff934053aa0 0x7ff934055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.115+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff94806c300 con 0x7ff95015a150 2022-08-01T22:59:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.119+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff934053aa0 0x7ff934055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.119+0000 7ff94ffff700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff934053aa0 0x7ff934055f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7ff938017000 tx=0x7ff938013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.123+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff948042f30 con 0x7ff95015a150 2022-08-01T22:59:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.287+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9500574e0 con 0x7ff95015a150 2022-08-01T22:59:37.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff94d7fa700 1 -- 172.21.15.32:0/2431438874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ff948046550 con 0x7ff95015a150 2022-08-01T22:59:37.293 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:37.293 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff934053aa0 msgr2=0x7ff934055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff934053aa0 0x7ff934055f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7ff938017000 tx=0x7ff938013000).stop 2022-08-01T22:59:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 msgr2=0x7ff95015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 0x7ff95015a5d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff94800b000 tx=0x7ff948008040).stop 2022-08-01T22:59:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 shutdown_connections 2022-08-01T22:59:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff934053aa0 0x7ff934055f60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95015a150 0x7ff95015a5d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9501439d0 0x7ff950143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 --2- 172.21.15.32:0/2431438874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff930003930 0x7ff9500ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 >> 172.21.15.32:0/2431438874 conn(0x7ff95005f840 msgr2=0x7ff9500b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 shutdown_connections 2022-08-01T22:59:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.291+0000 7ff956764700 1 -- 172.21.15.32:0/2431438874 wait complete. 2022-08-01T22:59:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:37.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 5 2022-08-01T22:59:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 Processor -- start 2022-08-01T22:59:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 -- start start 2022-08-01T22:59:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f089c0b7840 0x7f089c060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c0610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f089c146720 0x7f089c146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0671e0 con 0x7f089c0b7840 2022-08-01T22:59:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c065540 con 0x7f089c146720 2022-08-01T22:59:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.419+0000 7f08a0f23700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0b7c20 con 0x7f089c0b2930 2022-08-01T22:59:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f089c0b7840 0x7f089c060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f089c0b7840 0x7f089c060be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43598/0 (socket says 172.21.15.32:43598) 2022-08-01T22:59:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 -- 172.21.15.32:0/2839846045 learned_addr learned my addr 172.21.15.32:0/2839846045 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089bfff700 1 --2- 172.21.15.32:0/2839846045 >> v2:172.21.15.55:3300/0 conn(0x7f089c146720 0x7f089c146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f0892ffd700 1 --2- 172.21.15.32:0/2839846045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c0610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 -- 172.21.15.32:0/2839846045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 msgr2=0x7f089c0610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2839846045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c0610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 -- 172.21.15.32:0/2839846045 >> v2:172.21.15.55:3300/0 conn(0x7f089c146720 msgr2=0x7f089c146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2839846045 >> v2:172.21.15.55:3300/0 conn(0x7f089c146720 0x7f089c146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 -- 172.21.15.32:0/2839846045 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c066ce0 con 0x7f089c0b7840 2022-08-01T22:59:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089bfff700 1 --2- 172.21.15.32:0/2839846045 >> v2:172.21.15.55:3300/0 conn(0x7f089c146720 0x7f089c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2839846045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c0b7840 0x7f089c060be0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f088c002f70 tx=0x7f088c006eb0).ready entity=mon.0 client_cookie=a25da228590e0b4c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089affd700 1 -- 172.21.15.32:0/2839846045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f088c00d570 con 0x7f089c0b7840 2022-08-01T22:59:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089affd700 1 -- 172.21.15.32:0/2839846045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c000e90 con 0x7f089c0b7840 2022-08-01T22:59:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f089affd700 1 -- 172.21.15.32:0/2839846045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f088c00d900 con 0x7f089c0b7840 2022-08-01T22:59:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 -- 172.21.15.32:0/2839846045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c0b7840 msgr2=0x7f089c060be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 --2- 172.21.15.32:0/2839846045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c0b7840 0x7f089c060be0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f088c002f70 tx=0x7f088c006eb0).stop 2022-08-01T22:59:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 -- 172.21.15.32:0/2839846045 shutdown_connections 2022-08-01T22:59:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 --2- 172.21.15.32:0/2839846045 >> v2:172.21.15.55:3300/0 conn(0x7f089c146720 0x7f089c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 --2- 172.21.15.32:0/2839846045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c0610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 --2- 172.21.15.32:0/2839846045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c0b7840 0x7f089c060be0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 -- 172.21.15.32:0/2839846045 >> 172.21.15.32:0/2839846045 conn(0x7f089c05f840 msgr2=0x7f089c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 -- 172.21.15.32:0/2839846045 shutdown_connections 2022-08-01T22:59:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.423+0000 7f08a0f23700 1 -- 172.21.15.32:0/2839846045 wait complete. 2022-08-01T22:59:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 Processor -- start 2022-08-01T22:59:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 -- start start 2022-08-01T22:59:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c05c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f089c0b7840 0x7f089c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 0x7f089c0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c156490 con 0x7f089c146720 2022-08-01T22:59:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0aaa90 con 0x7f089c0b7840 2022-08-01T22:59:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0aa7f0 con 0x7f089c0b2930 2022-08-01T22:59:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 0x7f089c0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 0x7f089c0ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43608/0 (socket says 172.21.15.32:43608) 2022-08-01T22:59:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 -- 172.21.15.32:0/1132322119 learned_addr learned my addr 172.21.15.32:0/1132322119 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089b7fe700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c05c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f0892ffd700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f089c0b7840 0x7f089c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 -- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 msgr2=0x7f089c05c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c05c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 -- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f089c0b7840 msgr2=0x7f089c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f089c0b7840 0x7f089c0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 -- 172.21.15.32:0/1132322119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894009470 con 0x7f089c146720 2022-08-01T22:59:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089b7fe700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c05c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f089bfff700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 0x7f089c0ad750 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f089400c000 tx=0x7f08940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08997fa700 1 -- 172.21.15.32:0/1132322119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f089400c9d0 con 0x7f089c146720 2022-08-01T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08997fa700 1 -- 172.21.15.32:0/1132322119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089400cb30 con 0x7f089c146720 2022-08-01T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c00b1e0 con 0x7f089c146720 2022-08-01T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.427+0000 7f08997fa700 1 -- 172.21.15.32:0/1132322119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f089400ce90 con 0x7f089c146720 2022-08-01T22:59:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.431+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c05d260 con 0x7f089c146720 2022-08-01T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.431+0000 7f08997fa700 1 -- 172.21.15.32:0/1132322119 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0894022070 con 0x7f089c146720 2022-08-01T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.431+0000 7f08997fa700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08800539e0 0x7f0880055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.431+0000 7f08997fa700 1 -- 172.21.15.32:0/1132322119 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f089406ca50 con 0x7f089c146720 2022-08-01T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.431+0000 7f089b7fe700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08800539e0 0x7f0880055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.431+0000 7f089b7fe700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08800539e0 0x7f0880055ea0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f088c018000 tx=0x7f088c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 Processor -- start 2022-08-01T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- start start 2022-08-01T22:59:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 0x25d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x25df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x253b3c0 con 0x25d9b40 2022-08-01T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x254f050 con 0x25dc2b0 2022-08-01T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x252f290 con 0x25db940 2022-08-01T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 0x25d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 0x25d9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43612/0 (socket says 172.21.15.32:43612) 2022-08-01T22:59:37.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 -- 172.21.15.32:0/2383738747 learned_addr learned my addr 172.21.15.32:0/2383738747 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f0892ffd700 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089bfff700 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x25df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 -- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 msgr2=0x25dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 -- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 msgr2=0x25df940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x25df940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 -- 172.21.15.32:0/2383738747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2586e90 con 0x25d9b40 2022-08-01T22:59:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 0x25d9f20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f088c006a00 tx=0x7f088c01e000).ready entity=mon.0 client_cookie=eb2fe4c7fdee3d31 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08a0f23700 1 -- 172.21.15.32:0/2383738747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f088c013570 con 0x25d9b40 2022-08-01T22:59:37.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08a0f23700 1 -- 172.21.15.32:0/2383738747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c0136d0 con 0x25d9b40 2022-08-01T22:59:37.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08a0f23700 1 -- 172.21.15.32:0/2383738747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f088c013a30 con 0x25d9b40 2022-08-01T22:59:37.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2383738747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 msgr2=0x25d9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 0x25d9f20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f088c006a00 tx=0x7f088c01e000).stop 2022-08-01T22:59:37.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2383738747 shutdown_connections 2022-08-01T22:59:37.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x25df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2383738747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25d9b40 0x25d9f20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.435+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2383738747 >> 172.21.15.32:0/2383738747 conn(0x2617930 msgr2=0x2617d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:37.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2383738747 shutdown_connections 2022-08-01T22:59:37.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2383738747 wait complete. 2022-08-01T22:59:37.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- start start 2022-08-01T22:59:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 Processor -- start 2022-08-01T22:59:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 0x26169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x2616ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e1cd0 con 0x25db940 2022-08-01T22:59:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x26173b0 con 0x25dc2b0 2022-08-01T22:59:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x25e1f00 con 0x25d9b40 2022-08-01T22:59:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 10 client.0 ll_register_callbacks cb 0x2596470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T22:59:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 0x26169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 0x26169d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43620/0 (socket says 172.21.15.32:43620) 2022-08-01T22:59:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 -- 172.21.15.32:0/906426785 learned_addr learned my addr 172.21.15.32:0/906426785 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f089b7fe700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f089bfff700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x2616ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 -- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 msgr2=0x26234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 -- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 msgr2=0x2616ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x2616ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 -- 172.21.15.32:0/906426785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c066ce0 con 0x25db940 2022-08-01T22:59:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f089b7fe700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f0892ffd700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 0x26169d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f088400b000 tx=0x7f0884008040).ready entity=mon.0 client_cookie=bb77eb848e31af49 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f087a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T22:59:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f087a7fc700 1 -- 172.21.15.32:0/906426785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0884015630 con 0x25db940 2022-08-01T22:59:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f087a7fc700 1 -- 172.21.15.32:0/906426785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0884015790 con 0x25db940 2022-08-01T22:59:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2545000 con 0x25db940 2022-08-01T22:59:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.439+0000 7f087a7fc700 1 -- 172.21.15.32:0/906426785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0884015950 con 0x25db940 2022-08-01T22:59:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 10 client.5784 fetch_fsmap learned FSMap version 23 2022-08-01T22:59:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x25219f0 con 0x25db940 2022-08-01T22:59:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f087a7fc700 1 -- 172.21.15.32:0/906426785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2545000 con 0x25db940 2022-08-01T22:59:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f087a7fc700 1 -- 172.21.15.32:0/906426785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f0884015ab0 con 0x25db940 2022-08-01T22:59:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 10 client.5784 fetch_fsmap finished waiting for FSMap version 23 2022-08-01T22:59:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 10 client.5784 resolve_mds: resolved ID 'b' to GID 5604 2022-08-01T22:59:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 20 client.5784 populate_metadata read hostname 'smithi032' 2022-08-01T22:59:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2610230 0x2610610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 4 client.5784 mds_command: new command op to 5604 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T22:59:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 --> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x253d080 con 0x2610230 2022-08-01T22:59:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f089b7fe700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2610230 0x2610610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f089b7fe700 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2610230 0x2610610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=498361e97b161c00 server_cookie=be849e336f4ce600 in_seq=0 out_seq=0 2022-08-01T22:59:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f087a7fc700 10 client.5784 ms_handle_connect on v2:172.21.15.55:6834/2848708845 2022-08-01T22:59:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f087a7fc700 1 -- 172.21.15.32:0/906426785 <== mds.0 v2:172.21.15.55:6834/2848708845 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x253d080 con 0x2610230 2022-08-01T22:59:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f087a7fc700 10 client.5784 handle_command_reply: tid=0 2022-08-01T22:59:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 client.5784 shutdown 2022-08-01T22:59:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 msgr2=0x26169d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 0x26169d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f088400b000 tx=0x7f0884008040).stop 2022-08-01T22:59:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 shutdown_connections 2022-08-01T22:59:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25dc2b0 0x2616ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25db940 0x26169d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2610230 0x2610610 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/906426785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f087a7fc700 0 client.5784 ms_handle_reset on v2:172.21.15.55:6834/2848708845 2022-08-01T22:59:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 >> 172.21.15.32:0/906426785 conn(0x2617930 msgr2=0x2617d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:37.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 shutdown_connections 2022-08-01T22:59:37.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 1 -- 172.21.15.32:0/906426785 wait complete. 2022-08-01T22:59:37.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.443+0000 7f08ac3e9740 20 client.5784 trim_cache size 0 max 16384 2022-08-01T22:59:37.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 Processor -- start 2022-08-01T22:59:37.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- start start 2022-08-01T22:59:37.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25d9b40 0x26144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25e0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x25e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e4c70 con 0x25dc2b0 2022-08-01T22:59:37.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d9a00 con 0x25d9b40 2022-08-01T22:59:37.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x25e25e0 con 0x25db940 2022-08-01T22:59:37.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x25e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x25e0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43630/0 (socket says 172.21.15.32:43630) 2022-08-01T22:59:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 -- 172.21.15.32:0/318056297 learned_addr learned my addr 172.21.15.32:0/318056297 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f0892ffd700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089b7fe700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25d9b40 0x26144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 -- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 msgr2=0x25e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25e0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 -- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25d9b40 msgr2=0x26144f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25d9b40 0x26144f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 -- 172.21.15.32:0/318056297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x253e5f0 con 0x25dc2b0 2022-08-01T22:59:37.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f0892ffd700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089b7fe700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25d9b40 0x26144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f089bfff700 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x25e0c10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x2627080 tx=0x7f089401d040).ready entity=mon.0 client_cookie=1dbfb44032a77c50 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f0890ff9700 1 -- 172.21.15.32:0/318056297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f089402a070 con 0x25dc2b0 2022-08-01T22:59:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f0890ff9700 1 -- 172.21.15.32:0/318056297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0894046ae0 con 0x25dc2b0 2022-08-01T22:59:37.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f0890ff9700 1 -- 172.21.15.32:0/318056297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0894022d80 con 0x25dc2b0 2022-08-01T22:59:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- 172.21.15.32:0/318056297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 msgr2=0x25e0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x25e0c10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x2627080 tx=0x7f089401d040).stop 2022-08-01T22:59:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- 172.21.15.32:0/318056297 shutdown_connections 2022-08-01T22:59:37.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x25e0c10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25db940 0x25e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/318056297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25d9b40 0x26144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- 172.21.15.32:0/318056297 >> 172.21.15.32:0/318056297 conn(0x2617930 msgr2=0x25dc710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- 172.21.15.32:0/318056297 shutdown_connections 2022-08-01T22:59:37.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- 172.21.15.32:0/318056297 wait complete. 2022-08-01T22:59:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 -- start start 2022-08-01T22:59:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.463+0000 7f08ac3e9740 1 Processor -- start 2022-08-01T22:59:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25db940 0x26707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x2622930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x2670cb0 con 0x25dc2b0 2022-08-01T22:59:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e3f30 con 0x25db940 2022-08-01T22:59:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x2676fe0 con 0x25d9b40 2022-08-01T22:59:37.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 10 client.0 ll_register_callbacks cb 0x25e5830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T22:59:37.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x2622930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x2622930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43636/0 (socket says 172.21.15.32:43636) 2022-08-01T22:59:37.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 -- 172.21.15.32:0/2054109886 learned_addr learned my addr 172.21.15.32:0/2054109886 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:37.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f0892ffd700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25db940 0x26707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 -- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 msgr2=0x26702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 -- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25db940 msgr2=0x26707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:37.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25db940 0x26707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:37.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 -- 172.21.15.32:0/2054109886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c05d260 con 0x25dc2b0 2022-08-01T22:59:37.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089bfff700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x2622930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x2627080 tx=0x7f0894025000).ready entity=mon.0 client_cookie=96b619d63fdea1a4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:37.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f087b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T22:59:37.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f087b7fe700 1 -- 172.21.15.32:0/2054109886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08940455a0 con 0x25dc2b0 2022-08-01T22:59:37.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f087b7fe700 1 -- 172.21.15.32:0/2054109886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0894046570 con 0x25dc2b0 2022-08-01T22:59:37.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f088c0136d0 con 0x25dc2b0 2022-08-01T22:59:37.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f087b7fe700 1 -- 172.21.15.32:0/2054109886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0894046730 con 0x25dc2b0 2022-08-01T22:59:37.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 10 client.5790 fetch_fsmap learned FSMap version 23 2022-08-01T22:59:37.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x25471e0 con 0x25dc2b0 2022-08-01T22:59:37.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f087b7fe700 1 -- 172.21.15.32:0/2054109886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c0136d0 con 0x25dc2b0 2022-08-01T22:59:37.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f087b7fe700 1 -- 172.21.15.32:0/2054109886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f0894005a70 con 0x25dc2b0 2022-08-01T22:59:37.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 10 client.5790 fetch_fsmap finished waiting for FSMap version 23 2022-08-01T22:59:37.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 10 client.5790 resolve_mds: resolved ID 'b' to GID 5604 2022-08-01T22:59:37.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 20 client.5790 populate_metadata read hostname 'smithi032' 2022-08-01T22:59:37.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2615f30 0x2622e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:37.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 4 client.5790 mds_command: new command op to 5604 tid=0[{"prefix": "cache drop", "timeout": 5}] 2022-08-01T22:59:37.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 --> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x24325e0 con 0x2615f30 2022-08-01T22:59:37.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.467+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2615f30 0x2622e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:37.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.471+0000 7f087b7fe700 10 client.5790 ms_handle_connect on v2:172.21.15.55:6834/2848708845 2022-08-01T22:59:37.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:37.471+0000 7f089b7fe700 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2615f30 0x2622e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=eda4b82f7be15221 server_cookie=3891f8a38c6e4716 in_seq=0 out_seq=0 2022-08-01T22:59:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f087b7fe700 1 -- 172.21.15.32:0/2054109886 <== mds.0 v2:172.21.15.55:6834/2848708845 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x24325e0 con 0x2615f30 2022-08-01T22:59:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f087b7fe700 10 client.5790 handle_command_reply: tid=0 2022-08-01T22:59:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 client.5790 shutdown 2022-08-01T22:59:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 msgr2=0x2622930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x2622930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x2627080 tx=0x7f0894025000).stop 2022-08-01T22:59:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 shutdown_connections 2022-08-01T22:59:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x25dc2b0 0x2622930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x25db940 0x26707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:6834/2848708845,v1:172.21.15.55:6835/2848708845] conn(0x2615f30 0x2622e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 --2- 172.21.15.32:0/2054109886 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x25d9b40 0x26702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 >> 172.21.15.32:0/2054109886 conn(0x2617930 msgr2=0x26142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f087b7fe700 0 client.5790 ms_handle_reset on v2:172.21.15.55:6834/2848708845 2022-08-01T22:59:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 shutdown_connections 2022-08-01T22:59:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 1 -- 172.21.15.32:0/2054109886 wait complete. 2022-08-01T22:59:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08ac3e9740 20 client.5790 trim_cache size 0 max 16384 2022-08-01T22:59:42.669 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T22:59:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2022-08-01T22:59:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": -110, 2022-08-01T22:59:42.669 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(110) Connection timed out", 2022-08-01T22:59:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 120 2022-08-01T22:59:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2022-08-01T22:59:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2022-08-01T22:59:42.670 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2022-08-01T22:59:42.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:42.671 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2022-08-01T22:59:42.671 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 0 2022-08-01T22:59:42.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:42.672 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2022-08-01T22:59:42.672 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2022-08-01T22:59:42.672 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 19972, 2022-08-01T22:59:42.672 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 3098528 2022-08-01T22:59:42.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T22:59:42.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T22:59:42.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1916052840000004 2022-08-01T22:59:42.673 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T22:59:42.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08800539e0 msgr2=0x7f0880055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08800539e0 0x7f0880055ea0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f088c018000 tx=0x7f088c014000).stop 2022-08-01T22:59:42.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 msgr2=0x7f089c0ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 0x7f089c0ad750 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f089400c000 tx=0x7f08940091d0).stop 2022-08-01T22:59:42.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 shutdown_connections 2022-08-01T22:59:42.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f08800539e0 0x7f0880055ea0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c146720 0x7f089c0ad750 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f089c0b7840 0x7f089c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 --2- 172.21.15.32:0/1132322119 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f089c0b2930 0x7f089c05c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 >> 172.21.15.32:0/1132322119 conn(0x7f089c05f840 msgr2=0x7f089c0b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 shutdown_connections 2022-08-01T22:59:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.663+0000 7f08a0f23700 1 -- 172.21.15.32:0/1132322119 wait complete. 2022-08-01T22:59:42.680 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-08-01T22:59:42.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:42.681 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:42.681 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:42.689 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T22:59:42.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T22:59:42.690 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T22:59:42.691 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T22:59:42.691 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T22:59:42.692 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:42.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:42.712 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:42.712 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:42.761 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T22:59:42.762 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T22:59:42.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T22:59:42.763 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:42.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T22:59:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.883+0000 7ff98692a700 1 Processor -- start 2022-08-01T22:59:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff98692a700 1 -- start start 2022-08-01T22:59:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff98692a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff980060330 0x7ff9800b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff98692a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 0x7ff9800a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff98692a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9800aa630 con 0x7ff9800af550 2022-08-01T22:59:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff98692a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9800b15e0 con 0x7ff980060330 2022-08-01T22:59:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff98692a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800af930 con 0x7ff98005c6f0 2022-08-01T22:59:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff985928700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9800af550 0x7ff9801445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59414/0 (socket says 172.21.15.32:59414) 2022-08-01T22:59:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff985928700 1 -- 172.21.15.32:0/3178811555 learned_addr learned my addr 172.21.15.32:0/3178811555 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff985127700 1 --2- 172.21.15.32:0/3178811555 >> v2:172.21.15.55:3300/0 conn(0x7ff980060330 0x7ff9800b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 0x7ff9800a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888305721 0 0) 0x7ff9800aa630 con 0x7ff9800af550 2022-08-01T22:59:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff968003260 con 0x7ff9800af550 2022-08-01T22:59:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3908593412 0 0) 0x7ff968003260 con 0x7ff9800af550 2022-08-01T22:59:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 msgr2=0x7ff9800a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 0x7ff9800a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 >> v2:172.21.15.55:3300/0 conn(0x7ff980060330 msgr2=0x7ff9800b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 --2- 172.21.15.32:0/3178811555 >> v2:172.21.15.55:3300/0 conn(0x7ff980060330 0x7ff9800b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98005fd10 con 0x7ff9800af550 2022-08-01T22:59:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7ff97c001680 con 0x7ff9800af550 2022-08-01T22:59:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 >> v1:172.21.15.32:6789/0 conn(0x7ff9800af550 legacy=0x7ff9801445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968003750 0x7ff968003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 0x7ff968004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680049f0 0x7ff968004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800b15e0 con 0x7ff9680049f0 2022-08-01T22:59:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800af930 con 0x7ff968003750 2022-08-01T22:59:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96800b750 con 0x7ff98005c6f0 2022-08-01T22:59:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff985127700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 0x7ff968004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680049f0 0x7ff968004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff985928700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968003750 0x7ff968003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 -- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 msgr2=0x7ff968004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff98005c6f0 0x7ff968004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 -- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968003750 msgr2=0x7ff968003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968003750 0x7ff968003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 -- 172.21.15.32:0/3178811555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968003260 con 0x7ff9680049f0 2022-08-01T22:59:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff985928700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff968003750 0x7ff968003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff986129700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680049f0 0x7ff968004e70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7ff97800c000 tx=0x7ff9780091d0).ready entity=mon.0 client_cookie=f2f0e0d7fcb351c5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff97800e7e0 con 0x7ff9680049f0 2022-08-01T22:59:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800e940 con 0x7ff9680049f0 2022-08-01T22:59:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.887+0000 7ff984926700 1 -- 172.21.15.32:0/3178811555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff97800eca0 con 0x7ff9680049f0 2022-08-01T22:59:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- 172.21.15.32:0/3178811555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680049f0 msgr2=0x7ff968004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 --2- 172.21.15.32:0/3178811555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680049f0 0x7ff968004e70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7ff97800c000 tx=0x7ff9780091d0).stop 2022-08-01T22:59:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff986129700 1 -- 172.21.15.32:0/3178811555 reap_dead start 2022-08-01T22:59:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- 172.21.15.32:0/3178811555 shutdown_connections 2022-08-01T22:59:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- 172.21.15.32:0/3178811555 >> 172.21.15.32:0/3178811555 conn(0x7ff9800612d0 msgr2=0x7ff980062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- 172.21.15.32:0/3178811555 shutdown_connections 2022-08-01T22:59:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- 172.21.15.32:0/3178811555 wait complete. 2022-08-01T22:59:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 Processor -- start 2022-08-01T22:59:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- start start 2022-08-01T22:59:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9800ae040 0x7ff980151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9800ac120 0x7ff980063ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 0x7ff980064440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff980064930 con 0x7ff980063fc0 2022-08-01T22:59:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800ac6a0 con 0x7ff9800ae040 2022-08-01T22:59:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff98692a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801518a0 con 0x7ff9800ac120 2022-08-01T22:59:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff986129700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 0x7ff980064440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff986129700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 0x7ff980064440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43656/0 (socket says 172.21.15.32:43656) 2022-08-01T22:59:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff986129700 1 -- 172.21.15.32:0/320176762 learned_addr learned my addr 172.21.15.32:0/320176762 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff985127700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9800ac120 0x7ff980063ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.891+0000 7ff985928700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9800ae040 0x7ff980151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff986129700 1 -- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9800ac120 msgr2=0x7ff980063ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff986129700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9800ac120 0x7ff980063ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff986129700 1 -- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9800ae040 msgr2=0x7ff980151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff986129700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9800ae040 0x7ff980151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff986129700 1 -- 172.21.15.32:0/320176762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98005f840 con 0x7ff980063fc0 2022-08-01T22:59:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff985928700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9800ae040 0x7ff980151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T22:59:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff986129700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 0x7ff980064440 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff9800b0160 tx=0x7ff97801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff97800e7e0 con 0x7ff980063fc0 2022-08-01T22:59:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800eed0 con 0x7ff980063fc0 2022-08-01T22:59:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff980151b40 con 0x7ff980063fc0 2022-08-01T22:59:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff978009600 con 0x7ff980063fc0 2022-08-01T22:59:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff980066780 con 0x7ff980063fc0 2022-08-01T22:59:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff97801ba40 con 0x7ff980063fc0 2022-08-01T22:59:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980151cd0 con 0x7ff980063fc0 2022-08-01T22:59:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff96effd700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff960053a10 0x7ff960055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.895+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff97806e280 con 0x7ff980063fc0 2022-08-01T22:59:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.899+0000 7ff985928700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff960053a10 0x7ff960055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.899+0000 7ff985928700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff960053a10 0x7ff960055ed0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff97c011000 tx=0x7ff97c00d040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:42.903+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff978044fa0 con 0x7ff980063fc0 2022-08-01T22:59:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.031+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff98005fa70 con 0x7ff980063fc0 2022-08-01T22:59:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.031+0000 7ff96effd700 1 -- 172.21.15.32:0/320176762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16273 (secure 0 0 0) 0x7ff978016020 con 0x7ff980063fc0 2022-08-01T22:59:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:43.035 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T22:59:21.260196+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T22:59:17.662803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T22:59:18.633694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T22:59:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff960053a10 msgr2=0x7ff960055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff960053a10 0x7ff960055ed0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff97c011000 tx=0x7ff97c00d040).stop 2022-08-01T22:59:43.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 msgr2=0x7ff980064440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 0x7ff980064440 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff9800b0160 tx=0x7ff97801a000).stop 2022-08-01T22:59:43.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 shutdown_connections 2022-08-01T22:59:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff960053a10 0x7ff960055ed0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff980063fc0 0x7ff980064440 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9800ac120 0x7ff980063ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 --2- 172.21.15.32:0/320176762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9800ae040 0x7ff980151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 >> 172.21.15.32:0/320176762 conn(0x7ff9800612d0 msgr2=0x7ff9800b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 shutdown_connections 2022-08-01T22:59:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.035+0000 7ff98692a700 1 -- 172.21.15.32:0/320176762 wait complete. 2022-08-01T22:59:43.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T22:59:17.662803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T22:59:18.633694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T22:59:43.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T22:59:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 Processor -- start 2022-08-01T22:59:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 -- start start 2022-08-01T22:59:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5274146ae0 0x7f52741403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52740b4310 0x7f52741408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5274143cf0 con 0x7f52740b4820 2022-08-01T22:59:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52741476a0 con 0x7f52740b4310 2022-08-01T22:59:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527c7b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5274143f90 con 0x7f5274146ae0 2022-08-01T22:59:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52740b4310 0x7f52741408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52740b4310 0x7f52741408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43662/0 (socket says 172.21.15.32:43662) 2022-08-01T22:59:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 -- 172.21.15.32:0/869975266 learned_addr learned my addr 172.21.15.32:0/869975266 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527a54f700 1 --2- 172.21.15.32:0/869975266 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5274146ae0 0x7f52741403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 -- 172.21.15.32:0/869975266 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5274146ae0 msgr2=0x7f52741403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 --2- 172.21.15.32:0/869975266 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5274146ae0 0x7f52741403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 -- 172.21.15.32:0/869975266 >> v1:172.21.15.55:6789/0 conn(0x7f52740b4820 legacy=0x7f52740b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 -- 172.21.15.32:0/869975266 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740b0e20 con 0x7f52740b4310 2022-08-01T22:59:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f5279d4e700 1 --2- 172.21.15.32:0/869975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52740b4310 0x7f52741408b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f52680032f0 tx=0x7f5268009290).ready entity=mon.0 client_cookie=f7f8fb9f16bee974 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527954d700 1 -- 172.21.15.32:0/869975266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526800b8a0 con 0x7f52740b4310 2022-08-01T22:59:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527954d700 1 -- 172.21.15.32:0/869975266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526800ba00 con 0x7f52740b4310 2022-08-01T22:59:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.159+0000 7f527954d700 1 -- 172.21.15.32:0/869975266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526800bb70 con 0x7f52740b4310 2022-08-01T22:59:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- 172.21.15.32:0/869975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52740b4310 msgr2=0x7f52741408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 --2- 172.21.15.32:0/869975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52740b4310 0x7f52741408b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f52680032f0 tx=0x7f5268009290).stop 2022-08-01T22:59:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- 172.21.15.32:0/869975266 shutdown_connections 2022-08-01T22:59:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 --2- 172.21.15.32:0/869975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52740b4310 0x7f52741408b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 --2- 172.21.15.32:0/869975266 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5274146ae0 0x7f52741403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- 172.21.15.32:0/869975266 >> 172.21.15.32:0/869975266 conn(0x7f527405c620 msgr2=0x7f527405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- 172.21.15.32:0/869975266 shutdown_connections 2022-08-01T22:59:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- 172.21.15.32:0/869975266 wait complete. 2022-08-01T22:59:43.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 Processor -- start 2022-08-01T22:59:43.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- start start 2022-08-01T22:59:43.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52740b4310 0x7f5274143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52740b4820 0x7f52740b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 0x7f52741526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740b0800 con 0x7f5274146ae0 2022-08-01T22:59:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740b0020 con 0x7f52740b4310 2022-08-01T22:59:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527c7b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740b02c0 con 0x7f52740b4820 2022-08-01T22:59:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.163+0000 7f527ad50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 0x7f52741526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527a54f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52740b4310 0x7f5274143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 0x7f52741526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43668/0 (socket says 172.21.15.32:43668) 2022-08-01T22:59:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 -- 172.21.15.32:0/1361521398 learned_addr learned my addr 172.21.15.32:0/1361521398 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f5279d4e700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52740b4820 0x7f52740b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 -- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52740b4820 msgr2=0x7f52740b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52740b4820 0x7f52740b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 -- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52740b4310 msgr2=0x7f5274143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52740b4310 0x7f5274143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 -- 172.21.15.32:0/1361521398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268009530 con 0x7f5274146ae0 2022-08-01T22:59:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527ad50700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 0x7f52741526a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f5274060de0 tx=0x7f526c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c012400 con 0x7f5274146ae0 2022-08-01T22:59:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c012560 con 0x7f5274146ae0 2022-08-01T22:59:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c0128c0 con 0x7f5274146ae0 2022-08-01T22:59:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52740b0560 con 0x7f5274146ae0 2022-08-01T22:59:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52740b8ad0 con 0x7f5274146ae0 2022-08-01T22:59:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52740574e0 con 0x7f5274146ae0 2022-08-01T22:59:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f526c019030 con 0x7f5274146ae0 2022-08-01T22:59:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f52677fe700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5254053950 0x7f5254055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.167+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f526c06b490 con 0x7f5274146ae0 2022-08-01T22:59:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.171+0000 7f527a54f700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5254053950 0x7f5254055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.171+0000 7f527a54f700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5254053950 0x7f5254055e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f527000c000 tx=0x7f5270008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.175+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f526c042140 con 0x7f5274146ae0 2022-08-01T22:59:43.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.299+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f52740b8340 con 0x7f5274146ae0 2022-08-01T22:59:43.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.299+0000 7f52677fe700 1 -- 172.21.15.32:0/1361521398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f526c045760 con 0x7f5274146ae0 2022-08-01T22:59:43.303 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T22:59:43.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5254053950 msgr2=0x7f5254055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5254053950 0x7f5254055e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f527000c000 tx=0x7f5270008040).stop 2022-08-01T22:59:43.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 msgr2=0x7f52741526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 0x7f52741526a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f5274060de0 tx=0x7f526c007bc0).stop 2022-08-01T22:59:43.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 shutdown_connections 2022-08-01T22:59:43.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5254053950 0x7f5254055e10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274146ae0 0x7f52741526a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52740b4820 0x7f52740b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 --2- 172.21.15.32:0/1361521398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52740b4310 0x7f5274143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 >> 172.21.15.32:0/1361521398 conn(0x7f527405c620 msgr2=0x7f52740b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 shutdown_connections 2022-08-01T22:59:43.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.303+0000 7f527c7b3700 1 -- 172.21.15.32:0/1361521398 wait complete. 2022-08-01T22:59:43.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T22:59:43.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 Processor -- start 2022-08-01T22:59:43.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 -- start start 2022-08-01T22:59:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 0x7f962c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 0x7f962c13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0b2510 con 0x7f962c13e8a0 2022-08-01T22:59:43.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c1443e0 con 0x7f962c0b7500 2022-08-01T22:59:43.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.427+0000 7f9631cbb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b3700 con 0x7f962c0b31f0 2022-08-01T22:59:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9630cb9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 0x7f962c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 0x7f962c13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9630cb9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 0x7f962c0b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53460/0 (socket says 172.21.15.32:53460) 2022-08-01T22:59:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 0x7f962c13e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55416/0 (socket says 172.21.15.32:55416) 2022-08-01T22:59:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f96314ba700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f962c13e8a0 0x7f962c064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52124/0 (socket says 172.21.15.32:52124) 2022-08-01T22:59:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9630cb9700 1 -- 172.21.15.32:0/3779023673 learned_addr learned my addr 172.21.15.32:0/3779023673 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962b7fe700 1 -- 172.21.15.32:0/3779023673 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228434228 0 0) 0x7f962c0b2510 con 0x7f962c13e8a0 2022-08-01T22:59:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962b7fe700 1 -- 172.21.15.32:0/3779023673 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9618003260 con 0x7f962c13e8a0 2022-08-01T22:59:43.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 -- 172.21.15.32:0/3779023673 >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 msgr2=0x7f962c0b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:43.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 --2- 172.21.15.32:0/3779023673 >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 0x7f962c0b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 -- 172.21.15.32:0/3779023673 >> v1:172.21.15.55:6789/0 conn(0x7f962c13e8a0 legacy=0x7f962c064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 -- 172.21.15.32:0/3779023673 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0b2740 con 0x7f962c0b31f0 2022-08-01T22:59:43.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9630cb9700 1 --2- 172.21.15.32:0/3779023673 >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 0x7f962c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962bfff700 1 --2- 172.21.15.32:0/3779023673 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 0x7f962c13e3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9620002f70 tx=0x7f9620006df0).ready entity=mon.2 client_cookie=cc287cb4bb87cb94 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962b7fe700 1 -- 172.21.15.32:0/3779023673 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f962000d4a0 con 0x7f962c0b31f0 2022-08-01T22:59:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962b7fe700 1 -- 172.21.15.32:0/3779023673 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620000e90 con 0x7f962c0b31f0 2022-08-01T22:59:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f962b7fe700 1 -- 172.21.15.32:0/3779023673 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f962000d7f0 con 0x7f962c0b31f0 2022-08-01T22:59:43.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 -- 172.21.15.32:0/3779023673 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 msgr2=0x7f962c13e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 --2- 172.21.15.32:0/3779023673 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 0x7f962c13e3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9620002f70 tx=0x7f9620006df0).stop 2022-08-01T22:59:43.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 -- 172.21.15.32:0/3779023673 shutdown_connections 2022-08-01T22:59:43.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 --2- 172.21.15.32:0/3779023673 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c0b31f0 0x7f962c13e3b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 --2- 172.21.15.32:0/3779023673 >> v2:172.21.15.55:3300/0 conn(0x7f962c0b7500 0x7f962c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 -- 172.21.15.32:0/3779023673 >> 172.21.15.32:0/3779023673 conn(0x7f962c05f840 msgr2=0x7f962c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 -- 172.21.15.32:0/3779023673 shutdown_connections 2022-08-01T22:59:43.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 -- 172.21.15.32:0/3779023673 wait complete. 2022-08-01T22:59:43.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 Processor -- start 2022-08-01T22:59:43.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.431+0000 7f9631cbb700 1 -- start start 2022-08-01T22:59:43.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 0x7f962c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c0b7500 0x7f962c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c13e8a0 0x7f962c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0af580 con 0x7f962c0b31f0 2022-08-01T22:59:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0ab610 con 0x7f962c0b7500 2022-08-01T22:59:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b0910 con 0x7f962c13e8a0 2022-08-01T22:59:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 0x7f962c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 0x7f962c152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43684/0 (socket says 172.21.15.32:43684) 2022-08-01T22:59:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 -- 172.21.15.32:0/4289387280 learned_addr learned my addr 172.21.15.32:0/4289387280 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f962bfff700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c0b7500 0x7f962c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f96314ba700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c13e8a0 0x7f962c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 -- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c13e8a0 msgr2=0x7f962c0b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c13e8a0 0x7f962c0b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 -- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c0b7500 msgr2=0x7f962c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c0b7500 0x7f962c0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 -- 172.21.15.32:0/4289387280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c00b1e0 con 0x7f962c0b31f0 2022-08-01T22:59:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9630cb9700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 0x7f962c152cf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f961c00c000 tx=0x7f961c006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c010780 con 0x7f962c0b31f0 2022-08-01T22:59:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c0108e0 con 0x7f962c0b31f0 2022-08-01T22:59:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962000b180 con 0x7f962c0b31f0 2022-08-01T22:59:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f961c010c40 con 0x7f962c0b31f0 2022-08-01T22:59:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0ac9c0 con 0x7f962c0b31f0 2022-08-01T22:59:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c0574e0 con 0x7f962c0b31f0 2022-08-01T22:59:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.435+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f961c01c460 con 0x7f962c0b31f0 2022-08-01T22:59:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.439+0000 7f9629ffb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f960c053b30 0x7f960c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.439+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f961c06c570 con 0x7f962c0b31f0 2022-08-01T22:59:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.439+0000 7f962bfff700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f960c053b30 0x7f960c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.439+0000 7f962bfff700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f960c053b30 0x7f960c055ff0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f962001a000 tx=0x7f9620014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.443+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f961c043230 con 0x7f962c0b31f0 2022-08-01T22:59:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.567+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f962c060330 con 0x7f962c0b31f0 2022-08-01T22:59:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.567+0000 7f9629ffb700 1 -- 172.21.15.32:0/4289387280 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f961c046850 con 0x7f962c0b31f0 2022-08-01T22:59:43.571 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f960c053b30 msgr2=0x7f960c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f960c053b30 0x7f960c055ff0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f962001a000 tx=0x7f9620014000).stop 2022-08-01T22:59:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 msgr2=0x7f962c152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 0x7f962c152cf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f961c00c000 tx=0x7f961c006e50).stop 2022-08-01T22:59:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 shutdown_connections 2022-08-01T22:59:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f960c053b30 0x7f960c055ff0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f962c13e8a0 0x7f962c0b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c0b7500 0x7f962c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 --2- 172.21.15.32:0/4289387280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c0b31f0 0x7f962c152cf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 >> 172.21.15.32:0/4289387280 conn(0x7f962c05f840 msgr2=0x7f962c0aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 shutdown_connections 2022-08-01T22:59:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.571+0000 7f9631cbb700 1 -- 172.21.15.32:0/4289387280 wait complete. 2022-08-01T22:59:43.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T22:59:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 Processor -- start 2022-08-01T22:59:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- start start 2022-08-01T22:59:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c3c0b31f0 0x7f0c3c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c3c0aadb0 0x7f0c3c064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3c0b2510 con 0x7f0c3c0b7500 2022-08-01T22:59:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c1443e0 con 0x7f0c3c0b31f0 2022-08-01T22:59:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c0b3700 con 0x7f0c3c0aadb0 2022-08-01T22:59:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c3c0b31f0 0x7f0c3c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c41f50700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c3c0b7500 0x7f0c3c13e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59462/0 (socket says 172.21.15.32:59462) 2022-08-01T22:59:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c3c0b31f0 0x7f0c3c0b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43690/0 (socket says 172.21.15.32:43690) 2022-08-01T22:59:43.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c41f50700 1 -- 172.21.15.32:0/1150117771 learned_addr learned my addr 172.21.15.32:0/1150117771 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42751700 1 --2- 172.21.15.32:0/1150117771 >> v2:172.21.15.55:3300/0 conn(0x7f0c3c0aadb0 0x7f0c3c064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 -- 172.21.15.32:0/1150117771 >> v2:172.21.15.55:3300/0 conn(0x7f0c3c0aadb0 msgr2=0x7f0c3c064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c40f4e700 1 -- 172.21.15.32:0/1150117771 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76875877 0 0) 0x7f0c3c0b2510 con 0x7f0c3c0b7500 2022-08-01T22:59:43.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 --2- 172.21.15.32:0/1150117771 >> v2:172.21.15.55:3300/0 conn(0x7f0c3c0aadb0 0x7f0c3c064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 -- 172.21.15.32:0/1150117771 >> v1:172.21.15.32:6789/0 conn(0x7f0c3c0b7500 legacy=0x7f0c3c13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 -- 172.21.15.32:0/1150117771 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0b2740 con 0x7f0c3c0b31f0 2022-08-01T22:59:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42751700 1 --2- 172.21.15.32:0/1150117771 >> v2:172.21.15.55:3300/0 conn(0x7f0c3c0aadb0 0x7f0c3c064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:59:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c4174f700 1 --2- 172.21.15.32:0/1150117771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b31f0 0x7f0c3c0b78e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0c38002f70 tx=0x7f0c38006df0).ready entity=mon.0 client_cookie=6b32d6b0809016b9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c40f4e700 1 -- 172.21.15.32:0/1150117771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3800d5c0 con 0x7f0c3c0b31f0 2022-08-01T22:59:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c40f4e700 1 -- 172.21.15.32:0/1150117771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c38000e90 con 0x7f0c3c0b31f0 2022-08-01T22:59:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c40f4e700 1 -- 172.21.15.32:0/1150117771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3800d910 con 0x7f0c3c0b31f0 2022-08-01T22:59:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- 172.21.15.32:0/1150117771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b31f0 msgr2=0x7f0c3c0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 --2- 172.21.15.32:0/1150117771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b31f0 0x7f0c3c0b78e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0c38002f70 tx=0x7f0c38006df0).stop 2022-08-01T22:59:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- 172.21.15.32:0/1150117771 shutdown_connections 2022-08-01T22:59:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 --2- 172.21.15.32:0/1150117771 >> v2:172.21.15.55:3300/0 conn(0x7f0c3c0aadb0 0x7f0c3c064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 --2- 172.21.15.32:0/1150117771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b31f0 0x7f0c3c0b78e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- 172.21.15.32:0/1150117771 >> 172.21.15.32:0/1150117771 conn(0x7f0c3c05f840 msgr2=0x7f0c3c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- 172.21.15.32:0/1150117771 shutdown_connections 2022-08-01T22:59:43.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.691+0000 7f0c42f52700 1 -- 172.21.15.32:0/1150117771 wait complete. 2022-08-01T22:59:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 Processor -- start 2022-08-01T22:59:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 -- start start 2022-08-01T22:59:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c3c0aadb0 0x7f0c3c146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0c3c0b31f0 0x7f0c3c143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 0x7f0c3c0ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c147f80 con 0x7f0c3c0b7500 2022-08-01T22:59:43.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c145960 con 0x7f0c3c0aadb0 2022-08-01T22:59:43.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c145c00 con 0x7f0c3c0b31f0 2022-08-01T22:59:43.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 0x7f0c3c0ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 0x7f0c3c0ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43698/0 (socket says 172.21.15.32:43698) 2022-08-01T22:59:43.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 -- 172.21.15.32:0/826902566 learned_addr learned my addr 172.21.15.32:0/826902566 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c4174f700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0c3c0b31f0 0x7f0c3c143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c41f50700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c3c0aadb0 0x7f0c3c146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 -- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0c3c0b31f0 msgr2=0x7f0c3c143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0c3c0b31f0 0x7f0c3c143cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 -- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c3c0aadb0 msgr2=0x7f0c3c146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c3c0aadb0 0x7f0c3c146f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 -- 172.21.15.32:0/826902566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3800b180 con 0x7f0c3c0b7500 2022-08-01T22:59:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c41f50700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c3c0aadb0 0x7f0c3c146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:59:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c4174f700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0c3c0b31f0 0x7f0c3c143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T22:59:43.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42751700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 0x7f0c3c0ac1a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f0c34007000 tx=0x7f0c34008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c34015070 con 0x7f0c3c0b7500 2022-08-01T22:59:43.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400a5f0 con 0x7f0c3c0b7500 2022-08-01T22:59:43.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c146c80 con 0x7f0c3c0b7500 2022-08-01T22:59:43.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c3400a950 con 0x7f0c3c0b7500 2022-08-01T22:59:43.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.695+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c147c70 con 0x7f0c3c0b7500 2022-08-01T22:59:43.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.699+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3c0abf40 con 0x7f0c3c0b7500 2022-08-01T22:59:43.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.699+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0c3400aab0 con 0x7f0c3c0b7500 2022-08-01T22:59:43.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.699+0000 7f0c32ffd700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c24053a70 0x7f0c24055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.699+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0c3406c500 con 0x7f0c3c0b7500 2022-08-01T22:59:43.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.699+0000 7f0c41f50700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c24053a70 0x7f0c24055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.699+0000 7f0c41f50700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c24053a70 0x7f0c24055f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0c2c008000 tx=0x7f0c2c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.707+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0c34043150 con 0x7f0c3c0b7500 2022-08-01T22:59:43.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.831+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c3c0574e0 con 0x7f0c3c0b7500 2022-08-01T22:59:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.831+0000 7f0c32ffd700 1 -- 172.21.15.32:0/826902566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0c34046770 con 0x7f0c3c0b7500 2022-08-01T22:59:43.834 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c24053a70 msgr2=0x7f0c24055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c24053a70 0x7f0c24055f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0c2c008000 tx=0x7f0c2c009040).stop 2022-08-01T22:59:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 msgr2=0x7f0c3c0ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 0x7f0c3c0ac1a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f0c34007000 tx=0x7f0c34008040).stop 2022-08-01T22:59:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 shutdown_connections 2022-08-01T22:59:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c24053a70 0x7f0c24055f30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c3c0b7500 0x7f0c3c0ac1a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0c3c0b31f0 0x7f0c3c143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 --2- 172.21.15.32:0/826902566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c3c0aadb0 0x7f0c3c146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 >> 172.21.15.32:0/826902566 conn(0x7f0c3c05f840 msgr2=0x7f0c3c141a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 shutdown_connections 2022-08-01T22:59:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.835+0000 7f0c42f52700 1 -- 172.21.15.32:0/826902566 wait complete. 2022-08-01T22:59:43.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T22:59:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 Processor -- start 2022-08-01T22:59:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 -- start start 2022-08-01T22:59:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 0x7f45a40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45a40afd60 0x7f45a40a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40a7250 con 0x7f45a40b2710 2022-08-01T22:59:43.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a40a7480 con 0x7f45a40afd60 2022-08-01T22:59:43.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a889a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40a7720 con 0x7f45a40b01a0 2022-08-01T22:59:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 0x7f45a40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45a40afd60 0x7f45a40a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 0x7f45a40a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55442/0 (socket says 172.21.15.32:55442) 2022-08-01T22:59:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 -- 172.21.15.32:0/416262 learned_addr learned my addr 172.21.15.32:0/416262 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45a40afd60 0x7f45a40a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53492/0 (socket says 172.21.15.32:53492) 2022-08-01T22:59:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a2d9d700 1 --1- 172.21.15.32:0/416262 >> v1:172.21.15.55:6789/0 conn(0x7f45a40b2710 0x7f45a414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52150/0 (socket says 172.21.15.32:52150) 2022-08-01T22:59:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 -- 172.21.15.32:0/416262 >> v2:172.21.15.55:3300/0 conn(0x7f45a40afd60 msgr2=0x7f45a40a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T22:59:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 --2- 172.21.15.32:0/416262 >> v2:172.21.15.55:3300/0 conn(0x7f45a40afd60 0x7f45a40a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 -- 172.21.15.32:0/416262 >> v1:172.21.15.55:6789/0 conn(0x7f45a40b2710 legacy=0x7f45a414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.955+0000 7f45a259c700 1 -- 172.21.15.32:0/416262 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40a79c0 con 0x7f45a40b01a0 2022-08-01T22:59:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a259c700 1 --2- 172.21.15.32:0/416262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 0x7f45a40a8540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f4594002f70 tx=0x7f4594006eb0).ready entity=mon.2 client_cookie=8f6e04a64230a8a6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a1d9b700 1 -- 172.21.15.32:0/416262 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459400d570 con 0x7f45a40b01a0 2022-08-01T22:59:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a1d9b700 1 -- 172.21.15.32:0/416262 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594000e90 con 0x7f45a40b01a0 2022-08-01T22:59:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a1d9b700 1 -- 172.21.15.32:0/416262 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459400d900 con 0x7f45a40b01a0 2022-08-01T22:59:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 -- 172.21.15.32:0/416262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 msgr2=0x7f45a40a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 --2- 172.21.15.32:0/416262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 0x7f45a40a8540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f4594002f70 tx=0x7f4594006eb0).stop 2022-08-01T22:59:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 -- 172.21.15.32:0/416262 shutdown_connections 2022-08-01T22:59:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 --2- 172.21.15.32:0/416262 >> v2:172.21.15.55:3300/0 conn(0x7f45a40afd60 0x7f45a40a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 --2- 172.21.15.32:0/416262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40b01a0 0x7f45a40a8540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 -- 172.21.15.32:0/416262 >> 172.21.15.32:0/416262 conn(0x7f45a4060310 msgr2=0x7f45a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 -- 172.21.15.32:0/416262 shutdown_connections 2022-08-01T22:59:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 -- 172.21.15.32:0/416262 wait complete. 2022-08-01T22:59:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 Processor -- start 2022-08-01T22:59:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 -- start start 2022-08-01T22:59:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40afd60 0x7f45a40b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 0x7f45a40a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.959+0000 7f45a889a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a40b2710 0x7f45a40ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a889a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40b43e0 con 0x7f45a40b01a0 2022-08-01T22:59:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a889a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a415a850 con 0x7f45a40b2710 2022-08-01T22:59:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a889a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40aa150 con 0x7f45a40afd60 2022-08-01T22:59:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 0x7f45a40a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a2d9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40afd60 0x7f45a40b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a40b2710 0x7f45a40ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 0x7f45a40a9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43712/0 (socket says 172.21.15.32:43712) 2022-08-01T22:59:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 -- 172.21.15.32:0/9880441 learned_addr learned my addr 172.21.15.32:0/9880441 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 -- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40afd60 msgr2=0x7f45a40b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40afd60 0x7f45a40b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 -- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a40b2710 msgr2=0x7f45a40ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:43.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a40b2710 0x7f45a40ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:43.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 -- 172.21.15.32:0/9880441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459400b1e0 con 0x7f45a40b01a0 2022-08-01T22:59:43.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a259c700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 0x7f45a40a9590 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f4594002f70 tx=0x7f4594015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4594011760 con 0x7f45a40b01a0 2022-08-01T22:59:43.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45940118c0 con 0x7f45a40b01a0 2022-08-01T22:59:43.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4594011c20 con 0x7f45a40b01a0 2022-08-01T22:59:43.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40aa3f0 con 0x7f45a40b01a0 2022-08-01T22:59:43.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a415a4e0 con 0x7f45a40b01a0 2022-08-01T22:59:43.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.963+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a40aa580 con 0x7f45a40b01a0 2022-08-01T22:59:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.967+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f459401a040 con 0x7f45a40b01a0 2022-08-01T22:59:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.967+0000 7f4593fff700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4580053b00 0x7f4580055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.967+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f459406d3b0 con 0x7f45a40b01a0 2022-08-01T22:59:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.967+0000 7f45a2d9d700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4580053b00 0x7f4580055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:43.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.967+0000 7f45a2d9d700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4580053b00 0x7f4580055fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f459800d000 tx=0x7f4598007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:43.971+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4594044060 con 0x7f45a40b01a0 2022-08-01T22:59:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.099+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45a40574e0 con 0x7f45a40b01a0 2022-08-01T22:59:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.099+0000 7f4593fff700 1 -- 172.21.15.32:0/9880441 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4594023020 con 0x7f45a40b01a0 2022-08-01T22:59:44.101 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T22:59:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4580053b00 msgr2=0x7f4580055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4580053b00 0x7f4580055fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f459800d000 tx=0x7f4598007b80).stop 2022-08-01T22:59:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 msgr2=0x7f45a40a9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 0x7f45a40a9590 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f4594002f70 tx=0x7f4594015000).stop 2022-08-01T22:59:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 shutdown_connections 2022-08-01T22:59:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4580053b00 0x7f4580055fc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a40b2710 0x7f45a40ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a40b01a0 0x7f45a40a9590 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 --2- 172.21.15.32:0/9880441 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45a40afd60 0x7f45a40b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 >> 172.21.15.32:0/9880441 conn(0x7f45a4060310 msgr2=0x7f45a40af310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 shutdown_connections 2022-08-01T22:59:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.103+0000 7f45a889a700 1 -- 172.21.15.32:0/9880441 wait complete. 2022-08-01T22:59:44.118 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T22:59:44.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T22:59:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07eeaaf700 1 Processor -- start 2022-08-01T22:59:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07eeaaf700 1 -- start start 2022-08-01T22:59:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07eeaaf700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07e80b2710 0x7f07e814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07eeaaf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80a7250 con 0x7f07e80a8c50 2022-08-01T22:59:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07eeaaf700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80a7480 con 0x7f07e80b01a0 2022-08-01T22:59:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07eeaaf700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80a7720 con 0x7f07e80b2710 2022-08-01T22:59:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ee2ae700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07e80a8c50 0x7f07e8151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59494/0 (socket says 172.21.15.32:59494) 2022-08-01T22:59:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ee2ae700 1 -- 172.21.15.32:0/1260746171 learned_addr learned my addr 172.21.15.32:0/1260746171 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T22:59:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07edaad700 1 --2- 172.21.15.32:0/1260746171 >> v2:172.21.15.55:3300/0 conn(0x7f07e80b2710 0x7f07e814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ecaab700 1 -- 172.21.15.32:0/1260746171 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483325024 0 0) 0x7f07e80a7250 con 0x7f07e80a8c50 2022-08-01T22:59:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ecaab700 1 -- 172.21.15.32:0/1260746171 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d0003260 con 0x7f07e80a8c50 2022-08-01T22:59:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07edaad700 1 -- 172.21.15.32:0/1260746171 >> v1:172.21.15.55:6789/0 conn(0x7f07e80b01a0 legacy=0x7f07e80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07edaad700 1 -- 172.21.15.32:0/1260746171 >> v1:172.21.15.32:6789/0 conn(0x7f07e80a8c50 legacy=0x7f07e8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07edaad700 1 -- 172.21.15.32:0/1260746171 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80a79c0 con 0x7f07e80b2710 2022-08-01T22:59:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07edaad700 1 --2- 172.21.15.32:0/1260746171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b2710 0x7f07e814afd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f07d8002f70 tx=0x7f07d8006df0).ready entity=mon.1 client_cookie=dea1e7c1b468b8ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ecaab700 1 -- 172.21.15.32:0/1260746171 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d800d610 con 0x7f07e80b2710 2022-08-01T22:59:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ecaab700 1 -- 172.21.15.32:0/1260746171 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8000e90 con 0x7f07e80b2710 2022-08-01T22:59:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.223+0000 7f07ecaab700 1 -- 172.21.15.32:0/1260746171 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d800d960 con 0x7f07e80b2710 2022-08-01T22:59:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- 172.21.15.32:0/1260746171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b2710 msgr2=0x7f07e814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/1260746171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b2710 0x7f07e814afd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f07d8002f70 tx=0x7f07d8006df0).stop 2022-08-01T22:59:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- 172.21.15.32:0/1260746171 shutdown_connections 2022-08-01T22:59:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/1260746171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b2710 0x7f07e814afd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- 172.21.15.32:0/1260746171 >> 172.21.15.32:0/1260746171 conn(0x7f07e8060310 msgr2=0x7f07e80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- 172.21.15.32:0/1260746171 shutdown_connections 2022-08-01T22:59:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- 172.21.15.32:0/1260746171 wait complete. 2022-08-01T22:59:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 Processor -- start 2022-08-01T22:59:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- start start 2022-08-01T22:59:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07e80a8c50 0x7f07e8065340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 0x7f07e8149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 0x7f07e81565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80ab920 con 0x7f07e80b2710 2022-08-01T22:59:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8065970 con 0x7f07e80b01a0 2022-08-01T22:59:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07eeaaf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8065c10 con 0x7f07e80a8c50 2022-08-01T22:59:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ee2ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 0x7f07e81565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07edaad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07e80a8c50 0x7f07e8065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 0x7f07e8149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 0x7f07e8149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53510/0 (socket says 172.21.15.32:53510) 2022-08-01T22:59:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ee2ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 0x7f07e81565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43726/0 (socket says 172.21.15.32:43726) 2022-08-01T22:59:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 -- 172.21.15.32:0/2648703782 learned_addr learned my addr 172.21.15.32:0/2648703782 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T22:59:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07edaad700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07e80a8c50 0x7f07e8065340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55460/0 (socket says 172.21.15.32:55460) 2022-08-01T22:59:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 -- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07e80a8c50 msgr2=0x7f07e8065340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07e80a8c50 0x7f07e8065340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 -- 172.21.15.32:0/2648703782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 msgr2=0x7f07e81565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 0x7f07e81565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 -- 172.21.15.32:0/2648703782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d800b180 con 0x7f07e80b01a0 2022-08-01T22:59:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ee2ae700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 0x7f07e81565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T22:59:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.227+0000 7f07ed2ac700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 0x7f07e8149b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f07e8061c10 tx=0x7f07e4007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07e400f470 con 0x7f07e80b01a0 2022-08-01T22:59:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e400f5d0 con 0x7f07e80b01a0 2022-08-01T22:59:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e8156ae0 con 0x7f07e80b01a0 2022-08-01T22:59:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07e400f930 con 0x7f07e80b01a0 2022-08-01T22:59:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e80ac420 con 0x7f07e80b01a0 2022-08-01T22:59:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8065dd0 con 0x7f07e80b01a0 2022-08-01T22:59:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07e4019030 con 0x7f07e80b01a0 2022-08-01T22:59:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07d6ffd700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07c8053b30 0x7f07c8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T22:59:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f07e406b830 con 0x7f07e80b01a0 2022-08-01T22:59:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.231+0000 7f07edaad700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07c8053b30 0x7f07c8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T22:59:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.235+0000 7f07edaad700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07c8053b30 0x7f07c8055ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f07d801a000 tx=0x7f07d8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T22:59:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.243+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07e4042590 con 0x7f07e80b01a0 2022-08-01T22:59:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.403+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07e80574e0 con 0x7f07e80b01a0 2022-08-01T22:59:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07d6ffd700 1 -- 172.21.15.32:0/2648703782 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f07e4045bb0 con 0x7f07e80b01a0 2022-08-01T22:59:44.408 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T22:59:44.409 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T22:59:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07c8053b30 msgr2=0x7f07c8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07c8053b30 0x7f07c8055ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f07d801a000 tx=0x7f07d8014000).stop 2022-08-01T22:59:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 msgr2=0x7f07e8149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T22:59:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 0x7f07e8149b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f07e8061c10 tx=0x7f07e4007f70).stop 2022-08-01T22:59:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 shutdown_connections 2022-08-01T22:59:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07c8053b30 0x7f07c8055ff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80b2710 0x7f07e81565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07e80b01a0 0x7f07e8149b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 --2- 172.21.15.32:0/2648703782 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07e80a8c50 0x7f07e8065340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T22:59:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 >> 172.21.15.32:0/2648703782 conn(0x7f07e8060310 msgr2=0x7f07e814b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T22:59:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 shutdown_connections 2022-08-01T22:59:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T22:59:44.407+0000 7f07eeaaf700 1 -- 172.21.15.32:0/2648703782 wait complete. 2022-08-01T22:59:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T22:59:44.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T22:59:19.253309+0000', 'modified': '2022-08-01T22:59:20.304429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5604': {'gid': 5604, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/2848708845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2848708845}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2848708845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T22:59:44.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T22:59:44.725 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T22:59:44.844 INFO:teuthology.orchestra.run.smithi055.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5604,"want_state":"up:active","state":"up:active","rank_uptime":25.580436533,"mdsmap_epoch":23,"osdmap_epoch":56,"osdmap_epoch_barrier":55,"uptime":29.924411521} 2022-08-01T22:59:44.845 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5604, "mdsmap_epoch": 23, "osdmap_epoch": 56, "osdmap_epoch_barrier": 55, "rank_uptime": 25.580436533, "state": "up:active", "uptime": 29.924411521, "want_state": "up:active", "whoami": 0 } 2022-08-01T22:59:44.845 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T22:59:44.845 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T22:59:44.846 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T22:59:44.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:44.847 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:44.855 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:59:44.915 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:59:44.916 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:59:44.930 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:59:44.931 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:59:44.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:44.933 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:59:44.939 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T22:59:44.940 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:44.984 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:59:44.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:59:45.000 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:59:45.010 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:59:45.011 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:59:45.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:45.013 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:59:45.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T22:59:45.061+0000 7f04319c7080 -1 init, newargv = 0x5626946112c0 newargc=9ceph-fuse[22777]: starting ceph client 2022-08-01T22:59:45.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr: 2022-08-01T22:59:45.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22777]: starting fuse 2022-08-01T22:59:46.119 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T22:59:46.139 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T22:59:46.140 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T22:59:46.198 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T22:59:46.199 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T22:59:46.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:46.200 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T22:59:46.205 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T22:59:46.205 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T22:59:46.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:46.206 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T22:59:46.207 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T22:59:46.207 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T22:59:46.207 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T22:59:46.208 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T22:59:46.208 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:46.209 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T22:59:46.209 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T22:59:46.210 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T22:59:46.210 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:46.210 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T22:59:46.211 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T22:59:46.211 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T22:59:46.212 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T22:59:46.212 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:46.213 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T22:59:46.213 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T22:59:46.214 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T22:59:46.214 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T22:59:46.214 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T22:59:46.215 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T22:59:46.215 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T22:59:46.216 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T22:59:46.261 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22777.asok 2022-08-01T22:59:46.266 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22777.asok 2022-08-01T22:59:46.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:46.268 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22777.asok status 2022-08-01T22:59:46.357 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T22:59:46.357 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T22:59:46.357 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T22:59:46.358 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T22:59:46.358 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T22:59:46.359 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T22:59:46.359 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T22:59:46.359 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22777", 2022-08-01T22:59:46.360 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T22:59:46.360 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:46.361 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T22:59:46.361 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T22:59:46.362 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 5770, 2022-08-01T22:59:46.362 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T22:59:46.363 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T22:59:46.363 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T22:59:46.364 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 5770 2022-08-01T22:59:46.364 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:46.364 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:59:46.365 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:59:46.365 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:59:46.366 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1310665561 2022-08-01T22:59:46.366 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T22:59:46.367 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:46.367 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T22:59:46.367 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T22:59:46.368 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T22:59:46.368 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1310665561 2022-08-01T22:59:46.368 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T22:59:46.369 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.5770 172.21.15.130:0/1310665561", 2022-08-01T22:59:46.369 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/1310665561", 2022-08-01T22:59:46.369 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T22:59:46.370 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 23, 2022-08-01T22:59:46.370 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 56, 2022-08-01T22:59:46.371 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T22:59:46.371 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T22:59:46.372 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T22:59:46.372 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T22:59:46.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:46.374 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T22:59:46.374 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:46.381 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T22:59:46.383 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T22:59:46.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T22:59:46.384 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T22:59:53.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:53.842 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:53.845 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T22:59:53.847 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:23.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:23.904 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:23.906 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:23.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:39.919 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:00:39.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:39.941 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:39.963 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:39.985 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:40.007 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:00:40.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:40.008 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:00:40.009 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:40.033 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:00:40.035 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:00:40.036 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:00:40.036 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:00:40.037 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:40.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22777]: fuse finished with error 0 and tester_r 0 2022-08-01T23:00:40.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:40.056 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:00:40.057 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:40.064 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:00:40.065 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:00:40.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:00:40.132 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:40.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:40.140 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:00:40.141 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:00:40.148 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:00:40.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:00:40.150 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:00:40.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:00:40.151 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:00:40.158 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:00:40.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:00:40.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:00:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 Processor -- start 2022-08-01T23:00:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- start start 2022-08-01T23:00:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01480b1f40 0x7f01480b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0148148530 0x7f01480b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01480b04e0 con 0x7f01480a7250 2022-08-01T23:00:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01480a9250 con 0x7f01480b1f40 2022-08-01T23:00:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01480b0780 con 0x7f0148148530 2022-08-01T23:00:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01480b1f40 0x7f01480b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014e5ce700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01480a7250 0x7f01481448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59508/0 (socket says 172.21.15.32:59508) 2022-08-01T23:00:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01480b1f40 0x7f01480b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43730/0 (socket says 172.21.15.32:43730) 2022-08-01T23:00:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014e5ce700 1 -- 172.21.15.32:0/118812944 learned_addr learned my addr 172.21.15.32:0/118812944 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:40.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014d5cc700 1 --2- 172.21.15.32:0/118812944 >> v2:172.21.15.55:3300/0 conn(0x7f0148148530 0x7f01480b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 -- 172.21.15.32:0/118812944 >> v2:172.21.15.55:3300/0 conn(0x7f0148148530 msgr2=0x7f01480b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 --2- 172.21.15.32:0/118812944 >> v2:172.21.15.55:3300/0 conn(0x7f0148148530 0x7f01480b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 -- 172.21.15.32:0/118812944 >> v1:172.21.15.32:6789/0 conn(0x7f01480a7250 legacy=0x7f01481448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014cdcb700 1 -- 172.21.15.32:0/118812944 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524291070 0 0) 0x7f01480b04e0 con 0x7f01480a7250 2022-08-01T23:00:40.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 -- 172.21.15.32:0/118812944 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01480b0a20 con 0x7f01480b1f40 2022-08-01T23:00:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014ddcd700 1 --2- 172.21.15.32:0/118812944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01480b1f40 0x7f01480b1160 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f01481478c0 tx=0x7f0144006df0).ready entity=mon.0 client_cookie=af340316429fe743 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014cdcb700 1 -- 172.21.15.32:0/118812944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014400d5c0 con 0x7f01480b1f40 2022-08-01T23:00:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014cdcb700 1 -- 172.21.15.32:0/118812944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0144000e90 con 0x7f01480b1f40 2022-08-01T23:00:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f014cdcb700 1 -- 172.21.15.32:0/118812944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014400d910 con 0x7f01480b1f40 2022-08-01T23:00:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- 172.21.15.32:0/118812944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01480b1f40 msgr2=0x7f01480b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 --2- 172.21.15.32:0/118812944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01480b1f40 0x7f01480b1160 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f01481478c0 tx=0x7f0144006df0).stop 2022-08-01T23:00:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- 172.21.15.32:0/118812944 shutdown_connections 2022-08-01T23:00:40.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 --2- 172.21.15.32:0/118812944 >> v2:172.21.15.55:3300/0 conn(0x7f0148148530 0x7f01480b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 --2- 172.21.15.32:0/118812944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01480b1f40 0x7f01480b1160 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- 172.21.15.32:0/118812944 >> 172.21.15.32:0/118812944 conn(0x7f014805f840 msgr2=0x7f014805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.271+0000 7f0150031700 1 -- 172.21.15.32:0/118812944 shutdown_connections 2022-08-01T23:00:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 -- 172.21.15.32:0/118812944 wait complete. 2022-08-01T23:00:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 Processor -- start 2022-08-01T23:00:40.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 -- start start 2022-08-01T23:00:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01480a7250 0x7f014814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01480b1f40 0x7f01481439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 0x7f0148143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014814a250 con 0x7f0148148530 2022-08-01T23:00:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014814b6d0 con 0x7f01480a7250 2022-08-01T23:00:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f0150031700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014814a480 con 0x7f01480b1f40 2022-08-01T23:00:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 0x7f0148143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 0x7f0148143ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43740/0 (socket says 172.21.15.32:43740) 2022-08-01T23:00:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 -- 172.21.15.32:0/116036773 learned_addr learned my addr 172.21.15.32:0/116036773 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014d5cc700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01480b1f40 0x7f01481439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014ddcd700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01480a7250 0x7f014814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 -- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01480b1f40 msgr2=0x7f01481439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01480b1f40 0x7f01481439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 -- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01480a7250 msgr2=0x7f014814e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01480a7250 0x7f014814e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 -- 172.21.15.32:0/116036773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014400b180 con 0x7f0148148530 2022-08-01T23:00:40.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f014e5ce700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 0x7f0148143ec0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f0138000ed0 tx=0x7f0138007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.275+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0138007110 con 0x7f0148148530 2022-08-01T23:00:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138007270 con 0x7f0148148530 2022-08-01T23:00:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014814b950 con 0x7f0148148530 2022-08-01T23:00:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0138005030 con 0x7f0148148530 2022-08-01T23:00:40.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014814b230 con 0x7f0148148530 2022-08-01T23:00:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01480b0cc0 con 0x7f0148148530 2022-08-01T23:00:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0138005190 con 0x7f0148148530 2022-08-01T23:00:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f013effd700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0128053a70 0x7f0128055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f013806b8d0 con 0x7f0148148530 2022-08-01T23:00:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f014ddcd700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0128053a70 0x7f0128055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.279+0000 7f014ddcd700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0128053a70 0x7f0128055f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f014401a000 tx=0x7f0144014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:40.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.287+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0138042690 con 0x7f0148148530 2022-08-01T23:00:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.411+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f01480574e0 con 0x7f0148148530 2022-08-01T23:00:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.415+0000 7f013effd700 1 -- 172.21.15.32:0/116036773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+34039 (secure 0 0 0) 0x7f0138045cb0 con 0x7f0148148530 2022-08-01T23:00:40.418 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:40.418 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:00:40.418 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 56, 2022-08-01T23:00:40.418 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:00:40.419 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:00:40.419 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T22:59:21.260196+0000", 2022-08-01T23:00:40.419 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:00:40.419 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:00:40.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:00:40.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:00:40.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:00:40.420 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:00:40.421 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:00:40.421 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:00:40.421 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:00:40.421 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:40.422 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:00:40.422 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:00:40.422 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:00:40.422 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:00:40.422 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:00:40.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 10, 2022-08-01T23:00:40.423 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:00:40.423 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:00:40.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:00:40.424 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:00:40.424 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:00:40.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.424 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:00:40.425 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:00:40.425 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:00:40.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:00:40.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:00:40.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:00:40.426 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:00:40.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:00:40.426 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:00:40.426 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:00:40.427 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:00:40.427 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:00:40.427 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:00:40.427 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:00:40.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:00:40.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:00:40.428 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:00:40.428 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:00:40.428 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:00:40.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:00:40.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:00:40.429 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:00:40.429 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:00:40.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.430 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:00:40.430 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:00:40.430 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:00:40.431 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:00:40.431 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:00:40.431 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:00:40.431 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:00:40.431 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:00:40.432 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:00:40.432 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:00:40.432 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:00:40.432 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:00:40.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:00:40.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:00:40.433 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:00:40.433 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:00:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:00:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:00:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:00:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:00:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:00:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:00:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:00:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:00:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:00:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:00:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:00:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:00:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:00:40.437 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:00:40.437 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:00:40.437 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:00:40.437 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:00:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:00:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:00:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:00:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:00:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:00:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:00:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:00:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:00:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:00:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:00:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:00:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:00:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:00:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:00:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:00:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:00:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:00:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:00:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:00:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:00:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:00:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:00:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:00:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:00:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:00:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:00:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:00:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:00:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:00:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:00:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:00:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:00:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:00:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:00:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:00:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:00:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:00:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:00:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:00:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:00:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:00:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:00:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:00:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:00:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:00:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:00:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:00:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:00:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:00:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:00:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:00:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:00:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:00:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:00:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:00:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:00:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:00:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:00:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:00:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:00:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:00:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:00:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:00:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:00:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:00:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:00:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:00:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:00:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:00:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:00:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:00:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:00:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:00:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 9, 2022-08-01T23:00:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:00:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:59:17.662803+0000", 2022-08-01T23:00:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:00:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:00:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:00:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:00:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:00:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:00:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:00:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:00:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:00:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:00:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:00:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:00:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:00:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:00:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:00:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:00:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:00:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:00:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:00:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:00:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "55", 2022-08-01T23:00:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:00:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:00:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:00:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:00:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:00:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:00:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:00:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:00:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:00:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:00:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:00:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:00:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:00:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:00:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:00:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:00:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:00:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:00:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:00:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:00:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:00:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:00:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:00:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:00:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:00:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:00:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:00:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:00:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:00:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:00:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:00:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:00:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:00:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:00:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:00:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:00:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:00:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:00:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:00:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:00:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:00:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:00:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:00:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:00:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 10, 2022-08-01T23:00:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:00:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:59:18.633694+0000", 2022-08-01T23:00:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:00:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:00:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:00:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:00:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:00:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:00:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:00:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:00:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:00:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:00:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:00:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:00:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:00:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:00:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:00:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:00:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:00:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:00:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:00:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:00:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2022-08-01T23:00:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:00:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:00:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:00:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:00:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:00:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:00:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:00:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:00:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:00:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:00:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:00:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:00:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:00:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:00:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:00:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:00:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:00:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:00:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:00:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:00:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:00:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:00:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:00:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:00:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:00:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:00:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:00:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:00:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:00:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:00:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:00:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:00:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:00:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:00:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:00:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:00:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:00:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:00:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:00:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:00:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:00:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:00:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:00:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:00:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:00:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:00:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:00:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:00:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:00:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:00:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:00:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:00:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:00:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:00:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:00:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:00:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:00:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:00:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:00:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:00:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:00:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:00:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:00:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:00:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.520 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.520 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:00:40.521 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.522 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:00:40.522 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:40.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.523 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:00:40.523 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:00:40.523 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:00:40.524 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:00:40.524 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.524 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.524 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.525 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.525 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:00:40.526 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:00:40.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.526 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.526 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.526 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.527 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.527 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:40.528 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.528 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.528 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:00:40.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:00:40.530 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.531 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:00:40.532 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:00:40.533 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.534 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:00:40.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:00:40.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.537 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:00:40.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:00:40.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:40.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.540 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:00:40.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:00:40.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:00:40.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:00:40.541 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.542 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.542 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:00:40.543 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:00:40.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.543 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.544 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.544 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.545 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.545 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2022-08-01T23:00:40.545 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.545 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.545 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:00:40.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:00:40.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.549 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:00:40.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:00:40.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:00:40.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:00:40.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.555 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:00:40.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:00:40.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:40.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.558 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:00:40.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:00:40.558 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:00:40.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:00:40.559 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.559 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.560 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:00:40.560 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:00:40.561 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.561 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.561 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.561 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.562 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.562 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:40.563 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.563 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.563 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:00:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:00:40.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.566 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:00:40.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:00:40.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:00:40.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:00:40.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:00:40.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:00:40.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:40.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.576 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:00:40.576 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:00:40.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:00:40.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:00:40.576 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.577 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.578 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:00:40.578 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:00:40.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.579 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.579 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.579 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.579 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.579 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.580 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.580 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:40.580 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.580 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.581 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:00:40.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:00:40.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.584 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:00:40.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:00:40.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:00:40.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:00:40.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:40.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:00:40.593 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:00:40.594 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:00:40.594 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.594 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.595 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.596 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.597 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 54, 2022-08-01T23:00:40.598 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.598 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.598 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:00:40.599 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:00:40.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.601 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:00:40.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:00:40.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.604 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:00:40.605 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:00:40.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.607 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:00:40.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.609 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:00:40.610 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:40.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.610 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:00:40.611 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:00:40.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:00:40.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:00:40.611 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.612 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.612 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:00:40.613 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:00:40.613 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:40.613 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:40.614 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:40.614 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:40.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:40.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:40.614 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:40.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2022-08-01T23:00:40.615 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:40.615 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:40.615 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:40.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:00:40.616 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:00:40.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.618 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:40.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:00:40.620 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.621 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:00:40.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.622 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:40.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:00:40.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:00:40.624 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.625 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:40.625 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:40.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:40.626 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:00:40.626 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:40.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:00:40.627 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:40.627 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.627 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.628 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:00:40.628 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:00:40.628 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:00:40.628 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:40.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.630 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:40.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:00:40.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:00:40.631 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.631 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.631 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.631 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.632 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.632 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:00:40.632 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.633 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:00:40.633 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.633 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.633 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.634 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.634 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.634 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:00:40.634 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.635 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:00:40.635 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.635 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.636 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.636 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.636 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.636 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:00:40.636 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.637 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:00:40.637 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.638 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.638 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.638 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.638 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.638 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:00:40.639 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.639 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:00:40.640 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.640 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.640 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.640 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.640 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.641 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:00:40.641 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.642 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:00:40.642 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.642 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.642 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:00:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:00:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:00:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:00:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:00:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:00:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:00:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:00:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:00:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:00:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:00:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:00:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:00:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:00:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:00:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:00:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:00:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:00:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:00:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:00:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:00:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:00:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:00:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:00:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:00:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:00:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:00:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:00:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:00:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:40.656 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:00:40.656 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:00:40.656 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:00:40.656 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:00:40.656 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:00:40.657 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:00:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0128053a70 msgr2=0x7f0128055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0128053a70 0x7f0128055f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f014401a000 tx=0x7f0144014000).stop 2022-08-01T23:00:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 msgr2=0x7f0148143ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 0x7f0148143ec0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f0138000ed0 tx=0x7f0138007a40).stop 2022-08-01T23:00:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 shutdown_connections 2022-08-01T23:00:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0128053a70 0x7f0128055f30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0148148530 0x7f0148143ec0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01480b1f40 0x7f01481439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 --2- 172.21.15.32:0/116036773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01480a7250 0x7f014814e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 >> 172.21.15.32:0/116036773 conn(0x7f014805f840 msgr2=0x7f01480aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:40.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 shutdown_connections 2022-08-01T23:00:40.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.419+0000 7f0150031700 1 -- 172.21.15.32:0/116036773 wait complete. 2022-08-01T23:00:40.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:40.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 Processor -- start 2022-08-01T23:00:40.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- start start 2022-08-01T23:00:40.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb40a9da0 0x7fecb4060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb41402d0 0x7fecb4140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb4067030 con 0x7fecb40b31d0 2022-08-01T23:00:40.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb4064a70 con 0x7fecb40a9da0 2022-08-01T23:00:40.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4061f60 con 0x7fecb41402d0 2022-08-01T23:00:40.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb40a9da0 0x7fecb4060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb40a9da0 0x7fecb4060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43746/0 (socket says 172.21.15.32:43746) 2022-08-01T23:00:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 -- 172.21.15.32:0/821072851 learned_addr learned my addr 172.21.15.32:0/821072851 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecba4a4700 1 --2- 172.21.15.32:0/821072851 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb41402d0 0x7fecb4140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 -- 172.21.15.32:0/821072851 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb41402d0 msgr2=0x7fecb4140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 --2- 172.21.15.32:0/821072851 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb41402d0 0x7fecb4140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 -- 172.21.15.32:0/821072851 >> v1:172.21.15.55:6789/0 conn(0x7fecb40b31d0 legacy=0x7fecb4066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 -- 172.21.15.32:0/821072851 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40b35b0 con 0x7fecb40a9da0 2022-08-01T23:00:40.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb94a2700 1 --2- 172.21.15.32:0/821072851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb40a9da0 0x7fecb4060950 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fecb4063230 tx=0x7fecb0006f40).ready entity=mon.0 client_cookie=5a533793317fee63 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:40.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb8ca1700 1 -- 172.21.15.32:0/821072851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecb000da40 con 0x7fecb40a9da0 2022-08-01T23:00:40.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb8ca1700 1 -- 172.21.15.32:0/821072851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb0000e90 con 0x7fecb40a9da0 2022-08-01T23:00:40.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecb8ca1700 1 -- 172.21.15.32:0/821072851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecb000ddd0 con 0x7fecb40a9da0 2022-08-01T23:00:40.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- 172.21.15.32:0/821072851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb40a9da0 msgr2=0x7fecb4060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 --2- 172.21.15.32:0/821072851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb40a9da0 0x7fecb4060950 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fecb4063230 tx=0x7fecb0006f40).stop 2022-08-01T23:00:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- 172.21.15.32:0/821072851 shutdown_connections 2022-08-01T23:00:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 --2- 172.21.15.32:0/821072851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb40a9da0 0x7fecb4060950 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 --2- 172.21.15.32:0/821072851 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb41402d0 0x7fecb4140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- 172.21.15.32:0/821072851 >> 172.21.15.32:0/821072851 conn(0x7fecb4063870 msgr2=0x7fecb4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- 172.21.15.32:0/821072851 shutdown_connections 2022-08-01T23:00:40.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.775+0000 7fecbaca5700 1 -- 172.21.15.32:0/821072851 wait complete. 2022-08-01T23:00:40.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 Processor -- start 2022-08-01T23:00:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 -- start start 2022-08-01T23:00:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecb40a9da0 0x7fecb4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb40b31d0 0x7fecb40ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 0x7fecb40acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40ab3d0 con 0x7fecb41402d0 2022-08-01T23:00:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40ad210 con 0x7fecb40a9da0 2022-08-01T23:00:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40aaf10 con 0x7fecb40b31d0 2022-08-01T23:00:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 0x7fecb40acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 0x7fecb40acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43754/0 (socket says 172.21.15.32:43754) 2022-08-01T23:00:40.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 -- 172.21.15.32:0/1319067929 learned_addr learned my addr 172.21.15.32:0/1319067929 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:40.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecb94a2700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb40b31d0 0x7fecb40ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecb9ca3700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecb40a9da0 0x7fecb4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 -- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb40b31d0 msgr2=0x7fecb40ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb40b31d0 0x7fecb40ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 -- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecb40a9da0 msgr2=0x7fecb4155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecb40a9da0 0x7fecb4155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 -- 172.21.15.32:0/1319067929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb000b460 con 0x7fecb41402d0 2022-08-01T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecb9ca3700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecb40a9da0 0x7fecb4155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecba4a4700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 0x7fecb40acd20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fecac006e00 tx=0x7fecac0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecac00eeb0 con 0x7fecb41402d0 2022-08-01T23:00:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb41563f0 con 0x7fecb41402d0 2022-08-01T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac00c850 con 0x7fecb41402d0 2022-08-01T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecac00cbb0 con 0x7fecb41402d0 2022-08-01T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.779+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb40ab830 con 0x7fecb41402d0 2022-08-01T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.783+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fecac00cd10 con 0x7fecb41402d0 2022-08-01T23:00:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.783+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90003410 con 0x7fecb41402d0 2022-08-01T23:00:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.783+0000 7fecaaffd700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec9c053a70 0x7fec9c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.783+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fecac06c3e0 con 0x7fecb41402d0 2022-08-01T23:00:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.783+0000 7fecb9ca3700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec9c053a70 0x7fec9c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.783+0000 7fecb9ca3700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec9c053a70 0x7fec9c055f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7feca400b000 tx=0x7feca4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:40.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.791+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fecac043100 con 0x7fecb41402d0 2022-08-01T23:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.951+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec90003c40 con 0x7fecb41402d0 2022-08-01T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecaaffd700 1 -- 172.21.15.32:0/1319067929 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fecac046720 con 0x7fecb41402d0 2022-08-01T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:40.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T22:59:20.304429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5604":{"gid":5604,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2848708845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2848708845},{"type":"v1","addr":"172.21.15.55:6835","nonce":2848708845}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T23:00:40.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec9c053a70 msgr2=0x7fec9c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec9c053a70 0x7fec9c055f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7feca400b000 tx=0x7feca4009040).stop 2022-08-01T23:00:40.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 msgr2=0x7fecb40acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 0x7fecb40acd20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fecac006e00 tx=0x7fecac0091d0).stop 2022-08-01T23:00:40.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 shutdown_connections 2022-08-01T23:00:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec9c053a70 0x7fec9c055f30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb41402d0 0x7fecb40acd20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecb40b31d0 0x7fecb40ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 --2- 172.21.15.32:0/1319067929 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecb40a9da0 0x7fecb4155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 >> 172.21.15.32:0/1319067929 conn(0x7fecb4063870 msgr2=0x7fecb405fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 shutdown_connections 2022-08-01T23:00:40.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:40.955+0000 7fecbaca5700 1 -- 172.21.15.32:0/1319067929 wait complete. 2022-08-01T23:00:40.964 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2022-08-01T23:00:40.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:00:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 Processor -- start 2022-08-01T23:00:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 -- start start 2022-08-01T23:00:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0aaf10 0x7f841c067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f841c0ab2f0 0x7f841c064990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c065af0 con 0x7f841c064e80 2022-08-01T23:00:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c0b3e70 con 0x7f841c0ab2f0 2022-08-01T23:00:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8422043700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c066ff0 con 0x7f841c0aaf10 2022-08-01T23:00:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f841c0ab2f0 0x7f841c064990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8421041700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0aaf10 0x7f841c067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f841c0ab2f0 0x7f841c064990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53544/0 (socket says 172.21.15.32:53544) 2022-08-01T23:00:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 -- 172.21.15.32:0/2023728875 learned_addr learned my addr 172.21.15.32:0/2023728875 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8421842700 1 --1- 172.21.15.32:0/2023728875 >> v1:172.21.15.55:6789/0 conn(0x7f841c064e80 0x7f841c0b76b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52210/0 (socket says 172.21.15.32:52210) 2022-08-01T23:00:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 -- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0aaf10 msgr2=0x7f841c067490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 --2- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0aaf10 0x7f841c067490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 -- 172.21.15.32:0/2023728875 >> v1:172.21.15.55:6789/0 conn(0x7f841c064e80 legacy=0x7f841c0b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 -- 172.21.15.32:0/2023728875 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c0b7dc0 con 0x7f841c0ab2f0 2022-08-01T23:00:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8420840700 1 --2- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0ab2f0 0x7f841c064990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f8418002f70 tx=0x7f8418006e50).ready entity=mon.1 client_cookie=6fac1b13d99cca93 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.071+0000 7f8413fff700 1 -- 172.21.15.32:0/2023728875 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418006020 con 0x7f841c0ab2f0 2022-08-01T23:00:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8413fff700 1 -- 172.21.15.32:0/2023728875 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418006180 con 0x7f841c0ab2f0 2022-08-01T23:00:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8413fff700 1 -- 172.21.15.32:0/2023728875 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418006520 con 0x7f841c0ab2f0 2022-08-01T23:00:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0ab2f0 msgr2=0x7f841c064990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 --2- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0ab2f0 0x7f841c064990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f8418002f70 tx=0x7f8418006e50).stop 2022-08-01T23:00:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- 172.21.15.32:0/2023728875 shutdown_connections 2022-08-01T23:00:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 --2- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0ab2f0 0x7f841c064990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 --2- 172.21.15.32:0/2023728875 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0aaf10 0x7f841c067490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- 172.21.15.32:0/2023728875 >> 172.21.15.32:0/2023728875 conn(0x7f841c062d70 msgr2=0x7f841c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- 172.21.15.32:0/2023728875 shutdown_connections 2022-08-01T23:00:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- 172.21.15.32:0/2023728875 wait complete. 2022-08-01T23:00:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 Processor -- start 2022-08-01T23:00:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- start start 2022-08-01T23:00:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0aaf10 0x7f841c0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 0x7f841c0aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0afb40 0x7f841c060a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0ac040 con 0x7f841c0ab2f0 2022-08-01T23:00:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c061b10 con 0x7f841c0aaf10 2022-08-01T23:00:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8422043700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0ad3e0 con 0x7f841c0afb40 2022-08-01T23:00:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 0x7f841c0aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 0x7f841c0aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43764/0 (socket says 172.21.15.32:43764) 2022-08-01T23:00:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 -- 172.21.15.32:0/1173053946 learned_addr learned my addr 172.21.15.32:0/1173053946 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8421842700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0afb40 0x7f841c060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8421041700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0aaf10 0x7f841c0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 -- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0afb40 msgr2=0x7f841c060a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0afb40 0x7f841c060a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 -- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0aaf10 msgr2=0x7f841c0adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0aaf10 0x7f841c0adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 -- 172.21.15.32:0/1173053946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841800b1e0 con 0x7f841c0ab2f0 2022-08-01T23:00:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.075+0000 7f8420840700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 0x7f841c0aba10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8418002f70 tx=0x7f8418014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418016070 con 0x7f841c0ab2f0 2022-08-01T23:00:41.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418005900 con 0x7f841c0ab2f0 2022-08-01T23:00:41.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c13e6e0 con 0x7f841c0ab2f0 2022-08-01T23:00:41.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8418005c60 con 0x7f841c0ab2f0 2022-08-01T23:00:41.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c0b1980 con 0x7f841c0ab2f0 2022-08-01T23:00:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c0574e0 con 0x7f841c0ab2f0 2022-08-01T23:00:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8418024030 con 0x7f841c0ab2f0 2022-08-01T23:00:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f84127fc700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f84040539e0 0x7f8404055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f841806cd90 con 0x7f841c0ab2f0 2022-08-01T23:00:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.079+0000 7f8421041700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f84040539e0 0x7f8404055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.083+0000 7f8421041700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f84040539e0 0x7f8404055ea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f840c00b000 tx=0x7f840c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.087+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8418043a40 con 0x7f841c0ab2f0 2022-08-01T23:00:41.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:41.251+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f841c05db60 con 0x7f841c0ab2f0 2022-08-01T23:00:42.155 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:00:42.155+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:00:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.167+0000 7f84127fc700 1 -- 172.21.15.32:0/1173053946 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8418022020 con 0x7f841c0ab2f0 2022-08-01T23:00:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f84040539e0 msgr2=0x7f8404055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f84040539e0 0x7f8404055ea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f840c00b000 tx=0x7f840c006df0).stop 2022-08-01T23:00:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 msgr2=0x7f841c0aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 0x7f841c0aba10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8418002f70 tx=0x7f8418014000).stop 2022-08-01T23:00:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 shutdown_connections 2022-08-01T23:00:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f84040539e0 0x7f8404055ea0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f841c0afb40 0x7f841c060a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f841c0ab2f0 0x7f841c0aba10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 --2- 172.21.15.32:0/1173053946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f841c0aaf10 0x7f841c0adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 >> 172.21.15.32:0/1173053946 conn(0x7f841c062d70 msgr2=0x7f841c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 shutdown_connections 2022-08-01T23:00:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.171+0000 7f8422043700 1 -- 172.21.15.32:0/1173053946 wait complete. 2022-08-01T23:00:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:00:42.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:42.208 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:00:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.291+0000 7f0e1d93e700 1 Processor -- start 2022-08-01T23:00:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.291+0000 7f0e1d93e700 1 -- start start 2022-08-01T23:00:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e180b08a0 0x7f0e18142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e1813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e18143ab0 con 0x7f0e180b3260 2022-08-01T23:00:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e180b0c80 con 0x7f0e180b08a0 2022-08-01T23:00:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180ae810 con 0x7f0e180ad000 2022-08-01T23:00:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d13d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e1813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1c93c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e180b08a0 0x7f0e18142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e17fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0e180b3260 0x7f0e181433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52220/0 (socket says 172.21.15.32:52220) 2022-08-01T23:00:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e17fff700 1 -- 172.21.15.32:0/3198902253 learned_addr learned my addr 172.21.15.32:0/3198902253 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e177fe700 1 -- 172.21.15.32:0/3198902253 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623154645 0 0) 0x7f0e18143ab0 con 0x7f0e180b3260 2022-08-01T23:00:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e177fe700 1 -- 172.21.15.32:0/3198902253 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e04003260 con 0x7f0e180b3260 2022-08-01T23:00:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3198902253 >> v2:172.21.15.55:3300/0 conn(0x7f0e180b08a0 msgr2=0x7f0e18142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d13d700 1 --2- 172.21.15.32:0/3198902253 >> v2:172.21.15.55:3300/0 conn(0x7f0e180b08a0 0x7f0e18142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3198902253 >> v1:172.21.15.55:6789/0 conn(0x7f0e180b3260 legacy=0x7f0e181433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3198902253 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180aeab0 con 0x7f0e180ad000 2022-08-01T23:00:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1c93c700 1 --2- 172.21.15.32:0/3198902253 >> v2:172.21.15.55:3300/0 conn(0x7f0e180b08a0 0x7f0e18142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d13d700 1 --2- 172.21.15.32:0/3198902253 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e1813fa30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f0e100032f0 tx=0x7f0e100091d0).ready entity=mon.2 client_cookie=889aec45c3b23441 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e177fe700 1 -- 172.21.15.32:0/3198902253 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1000b600 con 0x7f0e180ad000 2022-08-01T23:00:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e177fe700 1 -- 172.21.15.32:0/3198902253 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000b760 con 0x7f0e180ad000 2022-08-01T23:00:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e177fe700 1 -- 172.21.15.32:0/3198902253 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1000b8d0 con 0x7f0e180ad000 2022-08-01T23:00:42.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3198902253 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 msgr2=0x7f0e1813fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3198902253 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e1813fa30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f0e100032f0 tx=0x7f0e100091d0).stop 2022-08-01T23:00:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3198902253 shutdown_connections 2022-08-01T23:00:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3198902253 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e1813fa30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3198902253 >> v2:172.21.15.55:3300/0 conn(0x7f0e180b08a0 0x7f0e18142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3198902253 >> 172.21.15.32:0/3198902253 conn(0x7f0e180612d0 msgr2=0x7f0e18062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3198902253 shutdown_connections 2022-08-01T23:00:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.295+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3198902253 wait complete. 2022-08-01T23:00:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 Processor -- start 2022-08-01T23:00:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 -- start start 2022-08-01T23:00:42.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e18143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 0x7f0e18135990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 0x7f0e18135e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e18140240 con 0x7f0e180b3260 2022-08-01T23:00:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1813f5c0 con 0x7f0e180b08a0 2022-08-01T23:00:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d93e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1813f860 con 0x7f0e180ad000 2022-08-01T23:00:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 0x7f0e18135e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e17fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 0x7f0e18135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1c93c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e18143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 0x7f0e18135e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43782/0 (socket says 172.21.15.32:43782) 2022-08-01T23:00:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e17fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 0x7f0e18135990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53564/0 (socket says 172.21.15.32:53564) 2022-08-01T23:00:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3768193082 learned_addr learned my addr 172.21.15.32:0/3768193082 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 msgr2=0x7f0e18143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e18143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 msgr2=0x7f0e18135990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 0x7f0e18135990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e1d13d700 1 -- 172.21.15.32:0/3768193082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c00b180 con 0x7f0e180b3260 2022-08-01T23:00:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.299+0000 7f0e17fff700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 0x7f0e18135990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e1d13d700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 0x7f0e18135e80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f0e100032f0 tx=0x7f0e10016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1000b600 con 0x7f0e180b3260 2022-08-01T23:00:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e10011b60 con 0x7f0e180b3260 2022-08-01T23:00:42.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1001f3f0 con 0x7f0e180b3260 2022-08-01T23:00:42.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e10009470 con 0x7f0e180b3260 2022-08-01T23:00:42.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1813fda0 con 0x7f0e180b3260 2022-08-01T23:00:42.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1813ff60 con 0x7f0e180b3260 2022-08-01T23:00:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e10026030 con 0x7f0e180b3260 2022-08-01T23:00:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e15ffb700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0df8053b30 0x7f0df8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0e1006cba0 con 0x7f0e180b3260 2022-08-01T23:00:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.303+0000 7f0e1c93c700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0df8053b30 0x7f0df8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.307+0000 7f0e1c93c700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0df8053b30 0x7f0df8055ff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f0e0c00b000 tx=0x7f0e0c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:42.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.311+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e10043910 con 0x7f0e180b3260 2022-08-01T23:00:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e180574e0 con 0x7f0e180b3260 2022-08-01T23:00:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e15ffb700 1 -- 172.21.15.32:0/3768193082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2131 (secure 0 0 0) 0x7f0e10024090 con 0x7f0e180b3260 2022-08-01T23:00:42.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:42.481 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T22:59:19.253309+0000","modified":"2022-08-01T23:00:42.159105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T23:00:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0df8053b30 msgr2=0x7f0df8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0df8053b30 0x7f0df8055ff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f0e0c00b000 tx=0x7f0e0c006df0).stop 2022-08-01T23:00:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 msgr2=0x7f0e18135e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 0x7f0e18135e80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f0e100032f0 tx=0x7f0e10016000).stop 2022-08-01T23:00:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 shutdown_connections 2022-08-01T23:00:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0df8053b30 0x7f0df8055ff0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e180b3260 0x7f0e18135e80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e180b08a0 0x7f0e18135990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 --2- 172.21.15.32:0/3768193082 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0e180ad000 0x7f0e18143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.479+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 >> 172.21.15.32:0/3768193082 conn(0x7f0e180612d0 msgr2=0x7f0e181406e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.483+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 shutdown_connections 2022-08-01T23:00:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.483+0000 7f0e1d93e700 1 -- 172.21.15.32:0/3768193082 wait complete. 2022-08-01T23:00:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2022-08-01T23:00:42.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:00:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.599+0000 7f9395517700 1 Processor -- start 2022-08-01T23:00:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.599+0000 7f9395517700 1 -- start start 2022-08-01T23:00:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.599+0000 7f9395517700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93900ad670 0x7f93900b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f939013fa40 0x7f939013fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900b0c40 con 0x7f93900af120 2022-08-01T23:00:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900b0ee0 con 0x7f939013fa40 2022-08-01T23:00:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93900ae9e0 con 0x7f93900ad670 2022-08-01T23:00:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f939013fa40 0x7f939013fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f939013fa40 0x7f939013fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43790/0 (socket says 172.21.15.32:43790) 2022-08-01T23:00:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 -- 172.21.15.32:0/602984985 learned_addr learned my addr 172.21.15.32:0/602984985 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f938ffff700 1 --2- 172.21.15.32:0/602984985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93900ad670 0x7f93900b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 -- 172.21.15.32:0/602984985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93900ad670 msgr2=0x7f93900b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 --2- 172.21.15.32:0/602984985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93900ad670 0x7f93900b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 -- 172.21.15.32:0/602984985 >> v1:172.21.15.55:6789/0 conn(0x7f93900af120 legacy=0x7f93900b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 -- 172.21.15.32:0/602984985 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900aec80 con 0x7f939013fa40 2022-08-01T23:00:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9394d16700 1 --2- 172.21.15.32:0/602984985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939013fa40 0x7f939013fed0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f93880032f0 tx=0x7f938800b530).ready entity=mon.0 client_cookie=3a467f38eb7bb08d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f938effd700 1 -- 172.21.15.32:0/602984985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93880153f0 con 0x7f939013fa40 2022-08-01T23:00:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f938effd700 1 -- 172.21.15.32:0/602984985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9388015550 con 0x7f939013fa40 2022-08-01T23:00:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f938effd700 1 -- 172.21.15.32:0/602984985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93880158b0 con 0x7f939013fa40 2022-08-01T23:00:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- 172.21.15.32:0/602984985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939013fa40 msgr2=0x7f939013fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 --2- 172.21.15.32:0/602984985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939013fa40 0x7f939013fed0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f93880032f0 tx=0x7f938800b530).stop 2022-08-01T23:00:42.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- 172.21.15.32:0/602984985 shutdown_connections 2022-08-01T23:00:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 --2- 172.21.15.32:0/602984985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939013fa40 0x7f939013fed0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 --2- 172.21.15.32:0/602984985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93900ad670 0x7f93900b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- 172.21.15.32:0/602984985 >> 172.21.15.32:0/602984985 conn(0x7f9390061360 msgr2=0x7f9390062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:42.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- 172.21.15.32:0/602984985 shutdown_connections 2022-08-01T23:00:42.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.603+0000 7f9395517700 1 -- 172.21.15.32:0/602984985 wait complete. 2022-08-01T23:00:42.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 Processor -- start 2022-08-01T23:00:42.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 -- start start 2022-08-01T23:00:42.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93900ad670 0x7f9390066590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 0x7f9390066a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f939013fa40 0x7f9390065070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900aa4d0 con 0x7f93900af120 2022-08-01T23:00:42.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900b4560 con 0x7f93900ad670 2022-08-01T23:00:42.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93900aac10 con 0x7f939013fa40 2022-08-01T23:00:42.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 0x7f9390066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 0x7f9390066a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43794/0 (socket says 172.21.15.32:43794) 2022-08-01T23:00:42.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938ffff700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93900ad670 0x7f9390066590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 -- 172.21.15.32:0/1150218980 learned_addr learned my addr 172.21.15.32:0/1150218980 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:42.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9394d16700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f939013fa40 0x7f9390065070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 -- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f939013fa40 msgr2=0x7f9390065070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f939013fa40 0x7f9390065070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 -- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93900ad670 msgr2=0x7f9390066590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:42.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93900ad670 0x7f9390066590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:42.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 -- 172.21.15.32:0/1150218980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938800b7d0 con 0x7f93900af120 2022-08-01T23:00:42.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938f7fe700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 0x7f9390066a80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f93900b3a70 tx=0x7f9384007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:42.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9384012400 con 0x7f93900af120 2022-08-01T23:00:42.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384012560 con 0x7f93900af120 2022-08-01T23:00:42.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93900aaeb0 con 0x7f93900af120 2022-08-01T23:00:42.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93840128c0 con 0x7f93900af120 2022-08-01T23:00:42.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.607+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9390067a00 con 0x7f93900af120 2022-08-01T23:00:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.611+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93900ab040 con 0x7f93900af120 2022-08-01T23:00:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.611+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9384019030 con 0x7f93900af120 2022-08-01T23:00:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.611+0000 7f938d7fa700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9370053980 0x7f9370055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.611+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f938406b410 con 0x7f93900af120 2022-08-01T23:00:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.611+0000 7f938ffff700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9370053980 0x7f9370055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:42.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.611+0000 7f938ffff700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9370053980 0x7f9370055e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f938000b000 tx=0x7f9380006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:42.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.615+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9384042120 con 0x7f93900af120 2022-08-01T23:00:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:42.775+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f93900b34e0 con 0x7f93900af120 2022-08-01T23:00:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.167+0000 7f938d7fa700 1 -- 172.21.15.32:0/1150218980 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9384045740 con 0x7f93900af120 2022-08-01T23:00:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9370053980 msgr2=0x7f9370055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9370053980 0x7f9370055e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f938000b000 tx=0x7f9380006df0).stop 2022-08-01T23:00:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 msgr2=0x7f9390066a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 0x7f9390066a80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f93900b3a70 tx=0x7f9384007c50).stop 2022-08-01T23:00:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 shutdown_connections 2022-08-01T23:00:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9370053980 0x7f9370055e40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f939013fa40 0x7f9390065070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93900af120 0x7f9390066a80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 --2- 172.21.15.32:0/1150218980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93900ad670 0x7f9390066590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 >> 172.21.15.32:0/1150218980 conn(0x7f9390061360 msgr2=0x7f93900601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 shutdown_connections 2022-08-01T23:00:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.171+0000 7f9395517700 1 -- 172.21.15.32:0/1150218980 wait complete. 2022-08-01T23:00:43.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:00:43.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 Processor -- start 2022-08-01T23:00:43.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 -- start start 2022-08-01T23:00:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1a80b3630 0x7ff1a8143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80aa0d0 0x7ff1a80af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1a80af6f0 0x7ff1a805f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a81443f0 con 0x7ff1a80b3630 2022-08-01T23:00:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80b3a10 con 0x7ff1a80af6f0 2022-08-01T23:00:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1ad90d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80afb80 con 0x7ff1a80aa0d0 2022-08-01T23:00:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1a6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1a80b3630 0x7ff1a8143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1a6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1a80b3630 0x7ff1a8143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43800/0 (socket says 172.21.15.32:43800) 2022-08-01T23:00:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1a6ffd700 1 -- 172.21.15.32:0/53358038 learned_addr learned my addr 172.21.15.32:0/53358038 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1a67fc700 1 --2- 172.21.15.32:0/53358038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80aa0d0 0x7ff1a80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.311+0000 7ff1a77fe700 1 --2- 172.21.15.32:0/53358038 >> v2:172.21.15.55:3300/0 conn(0x7ff1a80af6f0 0x7ff1a805f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a6ffd700 1 -- 172.21.15.32:0/53358038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80aa0d0 msgr2=0x7ff1a80af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a6ffd700 1 --2- 172.21.15.32:0/53358038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80aa0d0 0x7ff1a80af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a6ffd700 1 -- 172.21.15.32:0/53358038 >> v2:172.21.15.55:3300/0 conn(0x7ff1a80af6f0 msgr2=0x7ff1a805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a6ffd700 1 --2- 172.21.15.32:0/53358038 >> v2:172.21.15.55:3300/0 conn(0x7ff1a80af6f0 0x7ff1a805f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a6ffd700 1 -- 172.21.15.32:0/53358038 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8146b40 con 0x7ff1a80b3630 2022-08-01T23:00:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a77fe700 1 --2- 172.21.15.32:0/53358038 >> v2:172.21.15.55:3300/0 conn(0x7ff1a80af6f0 0x7ff1a805f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a6ffd700 1 --2- 172.21.15.32:0/53358038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8143f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7ff198002f70 tx=0x7ff198006eb0).ready entity=mon.0 client_cookie=c2a59cb983159245 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a5ffb700 1 -- 172.21.15.32:0/53358038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff19800d570 con 0x7ff1a80b3630 2022-08-01T23:00:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a5ffb700 1 -- 172.21.15.32:0/53358038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198000e90 con 0x7ff1a80b3630 2022-08-01T23:00:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1a5ffb700 1 -- 172.21.15.32:0/53358038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff19800d900 con 0x7ff1a80b3630 2022-08-01T23:00:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 -- 172.21.15.32:0/53358038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 msgr2=0x7ff1a8143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/53358038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8143f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7ff198002f70 tx=0x7ff198006eb0).stop 2022-08-01T23:00:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 -- 172.21.15.32:0/53358038 shutdown_connections 2022-08-01T23:00:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/53358038 >> v2:172.21.15.55:3300/0 conn(0x7ff1a80af6f0 0x7ff1a805f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/53358038 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80aa0d0 0x7ff1a80af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/53358038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8143f00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 -- 172.21.15.32:0/53358038 >> 172.21.15.32:0/53358038 conn(0x7ff1a8060950 msgr2=0x7ff1a8060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 -- 172.21.15.32:0/53358038 shutdown_connections 2022-08-01T23:00:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.315+0000 7ff1ad90d700 1 -- 172.21.15.32:0/53358038 wait complete. 2022-08-01T23:00:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 Processor -- start 2022-08-01T23:00:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 -- start start 2022-08-01T23:00:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1a80aa0d0 0x7ff1a81425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 0x7ff1a8152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80b58b0 con 0x7ff1a80b3630 2022-08-01T23:00:43.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80b65b0 con 0x7ff1a80aa0d0 2022-08-01T23:00:43.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1ad90d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80b50d0 con 0x7ff1a80af6f0 2022-08-01T23:00:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1a67fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 0x7ff1a8152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1a6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1a80aa0d0 0x7ff1a81425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43810/0 (socket says 172.21.15.32:43810) 2022-08-01T23:00:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1a67fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 0x7ff1a8152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55546/0 (socket says 172.21.15.32:55546) 2022-08-01T23:00:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.319+0000 7ff1a77fe700 1 -- 172.21.15.32:0/1967546685 learned_addr learned my addr 172.21.15.32:0/1967546685 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a77fe700 1 -- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 msgr2=0x7ff1a8152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a77fe700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 0x7ff1a8152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a77fe700 1 -- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1a80aa0d0 msgr2=0x7ff1a81425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a77fe700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1a80aa0d0 0x7ff1a81425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a77fe700 1 -- 172.21.15.32:0/1967546685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a0009470 con 0x7ff1a80b3630 2022-08-01T23:00:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a67fc700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 0x7ff1a8152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a6ffd700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1a80aa0d0 0x7ff1a81425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1a77fe700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8152f40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7ff1a000c000 tx=0x7ff1a00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a000ee60 con 0x7ff1a80b3630 2022-08-01T23:00:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000c850 con 0x7ff1a80b3630 2022-08-01T23:00:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a000cbb0 con 0x7ff1a80b3630 2022-08-01T23:00:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19800b1e0 con 0x7ff1a80b3630 2022-08-01T23:00:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a80b5610 con 0x7ff1a80b3630 2022-08-01T23:00:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a81535d0 con 0x7ff1a80b3630 2022-08-01T23:00:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff1a0022070 con 0x7ff1a80b3630 2022-08-01T23:00:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.323+0000 7ff197fff700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff184053b00 0x7ff184055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.327+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff1a006c760 con 0x7ff1a80b3630 2022-08-01T23:00:43.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.327+0000 7ff1a6ffd700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff184053b00 0x7ff184055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:43.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.327+0000 7ff1a6ffd700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff184053b00 0x7ff184055fc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7ff19801c000 tx=0x7ff198015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:43.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.331+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1a00434d0 con 0x7ff1a80b3630 2022-08-01T23:00:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:43.463+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1a8146de0 con 0x7ff1a80b3630 2022-08-01T23:00:44.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.395+0000 7ff197fff700 1 -- 172.21.15.32:0/1967546685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 185+0+0 (secure 0 0 0) 0x7ff1a0046af0 con 0x7ff1a80b3630 2022-08-01T23:00:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff184053b00 msgr2=0x7ff184055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff184053b00 0x7ff184055fc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7ff19801c000 tx=0x7ff198015000).stop 2022-08-01T23:00:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 msgr2=0x7ff1a8152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8152f40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7ff1a000c000 tx=0x7ff1a00091d0).stop 2022-08-01T23:00:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 shutdown_connections 2022-08-01T23:00:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff184053b00 0x7ff184055fc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1a80b3630 0x7ff1a8152f40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1a80af6f0 0x7ff1a8152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 --2- 172.21.15.32:0/1967546685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1a80aa0d0 0x7ff1a81425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 >> 172.21.15.32:0/1967546685 conn(0x7ff1a8060950 msgr2=0x7ff1a8143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 shutdown_connections 2022-08-01T23:00:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.399+0000 7ff1ad90d700 1 -- 172.21.15.32:0/1967546685 wait complete. 2022-08-01T23:00:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:00:44.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:00:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 Processor -- start 2022-08-01T23:00:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 -- start start 2022-08-01T23:00:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b6710 0x7fbf7c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf7c0b6af0 0x7fbf7c12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c144240 con 0x7fbf7c0b4550 2022-08-01T23:00:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c143d00 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf821f5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c143fa0 con 0x7fbf7c0b6710 2022-08-01T23:00:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf7c0b6af0 0x7fbf7c12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf7c0b6af0 0x7fbf7c12bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43818/0 (socket says 172.21.15.32:43818) 2022-08-01T23:00:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7bfff700 1 -- 172.21.15.32:0/3766535229 learned_addr learned my addr 172.21.15.32:0/3766535229 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7b7fe700 1 --2- 172.21.15.32:0/3766535229 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b6710 0x7fbf7c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7bfff700 1 -- 172.21.15.32:0/3766535229 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b6710 msgr2=0x7fbf7c0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7bfff700 1 --2- 172.21.15.32:0/3766535229 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b6710 0x7fbf7c0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.523+0000 7fbf7bfff700 1 -- 172.21.15.32:0/3766535229 >> v1:172.21.15.32:6789/0 conn(0x7fbf7c0b4550 legacy=0x7fbf7c061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf7bfff700 1 -- 172.21.15.32:0/3766535229 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c061780 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf7bfff700 1 --2- 172.21.15.32:0/3766535229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c12bb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fbf740032f0 tx=0x7fbf74009290).ready entity=mon.0 client_cookie=df997b442c74604 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf7a7fc700 1 -- 172.21.15.32:0/3766535229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7400b8a0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf7a7fc700 1 -- 172.21.15.32:0/3766535229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400ba00 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf7a7fc700 1 -- 172.21.15.32:0/3766535229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7400bb70 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 -- 172.21.15.32:0/3766535229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 msgr2=0x7fbf7c12bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 --2- 172.21.15.32:0/3766535229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c12bb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fbf740032f0 tx=0x7fbf74009290).stop 2022-08-01T23:00:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 -- 172.21.15.32:0/3766535229 shutdown_connections 2022-08-01T23:00:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 --2- 172.21.15.32:0/3766535229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c12bb90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 --2- 172.21.15.32:0/3766535229 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b6710 0x7fbf7c0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 -- 172.21.15.32:0/3766535229 >> 172.21.15.32:0/3766535229 conn(0x7fbf7c05c620 msgr2=0x7fbf7c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 -- 172.21.15.32:0/3766535229 shutdown_connections 2022-08-01T23:00:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 -- 172.21.15.32:0/3766535229 wait complete. 2022-08-01T23:00:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 Processor -- start 2022-08-01T23:00:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.527+0000 7fbf821f5700 1 -- start start 2022-08-01T23:00:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b4550 0x7fbf7c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf7c0b6710 0x7fbf7c153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c0b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c1413e0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b07d0 con 0x7fbf7c0b6710 2022-08-01T23:00:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c152740 con 0x7fbf7c0b4550 2022-08-01T23:00:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c0b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c0b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43822/0 (socket says 172.21.15.32:43822) 2022-08-01T23:00:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 -- 172.21.15.32:0/429978624 learned_addr learned my addr 172.21.15.32:0/429978624 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7b7fe700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b4550 0x7fbf7c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7affd700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf7c0b6710 0x7fbf7c153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 -- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b4550 msgr2=0x7fbf7c0b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b4550 0x7fbf7c0b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 -- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf7c0b6710 msgr2=0x7fbf7c153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf7c0b6710 0x7fbf7c153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 -- 172.21.15.32:0/429978624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf74009530 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7b7fe700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b4550 0x7fbf7c0b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf7bfff700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c0b7af0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fbf740032f0 tx=0x7fbf74016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7400b8a0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf74021440 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c1529e0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf740217a0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c0afde0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c061a20 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf74027020 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf78ff9700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf60053b30 0x7fbf60055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.531+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fbf7406d480 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.535+0000 7fbf7b7fe700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf60053b30 0x7fbf60055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.535+0000 7fbf7b7fe700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf60053b30 0x7fbf60055ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fbf6400b000 tx=0x7fbf64008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.539+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf740443a0 con 0x7fbf7c0b6af0 2022-08-01T23:00:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:44.663+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbf7c0574e0 con 0x7fbf7c0b6af0 2022-08-01T23:00:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.411+0000 7fbf78ff9700 1 -- 172.21.15.32:0/429978624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbf74015020 con 0x7fbf7c0b6af0 2022-08-01T23:00:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf60053b30 msgr2=0x7fbf60055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf60053b30 0x7fbf60055ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fbf6400b000 tx=0x7fbf64008040).stop 2022-08-01T23:00:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 msgr2=0x7fbf7c0b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c0b7af0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fbf740032f0 tx=0x7fbf74016000).stop 2022-08-01T23:00:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 shutdown_connections 2022-08-01T23:00:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf60053b30 0x7fbf60055ff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf7c0b6af0 0x7fbf7c0b7af0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf7c0b6710 0x7fbf7c153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 --2- 172.21.15.32:0/429978624 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf7c0b4550 0x7fbf7c0b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 >> 172.21.15.32:0/429978624 conn(0x7fbf7c05c620 msgr2=0x7fbf7c142340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 shutdown_connections 2022-08-01T23:00:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.419+0000 7fbf821f5700 1 -- 172.21.15.32:0/429978624 wait complete. 2022-08-01T23:00:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:00:45.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:00:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 Processor -- start 2022-08-01T23:00:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- start start 2022-08-01T23:00:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faeb40b3260 0x7faeb41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faeb40ad000 0x7faeb413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb4143ab0 con 0x7faeb40b08a0 2022-08-01T23:00:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40b0c80 con 0x7faeb40b3260 2022-08-01T23:00:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40ae810 con 0x7faeb40ad000 2022-08-01T23:00:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faeb40b3260 0x7faeb41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faeb40b3260 0x7faeb41435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43830/0 (socket says 172.21.15.32:43830) 2022-08-01T23:00:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 -- 172.21.15.32:0/282221192 learned_addr learned my addr 172.21.15.32:0/282221192 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb9c2a700 1 --2- 172.21.15.32:0/282221192 >> v2:172.21.15.55:3300/0 conn(0x7faeb40ad000 0x7faeb413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 -- 172.21.15.32:0/282221192 >> v2:172.21.15.55:3300/0 conn(0x7faeb40ad000 msgr2=0x7faeb413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 --2- 172.21.15.32:0/282221192 >> v2:172.21.15.55:3300/0 conn(0x7faeb40ad000 0x7faeb413fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 -- 172.21.15.32:0/282221192 >> v1:172.21.15.32:6789/0 conn(0x7faeb40b08a0 legacy=0x7faeb4142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 -- 172.21.15.32:0/282221192 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb40aeab0 con 0x7faeb40b3260 2022-08-01T23:00:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb9c2a700 1 --2- 172.21.15.32:0/282221192 >> v2:172.21.15.55:3300/0 conn(0x7faeb40ad000 0x7faeb413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeabfff700 1 -- 172.21.15.32:0/282221192 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102857529 0 0) 0x7faeb4143ab0 con 0x7faeb40b08a0 2022-08-01T23:00:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeb8c28700 1 --2- 172.21.15.32:0/282221192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40b3260 0x7faeb41435c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7faeac0032f0 tx=0x7faeac00b530).ready entity=mon.0 client_cookie=692efcdd72fc24aa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeabfff700 1 -- 172.21.15.32:0/282221192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeac0155e0 con 0x7faeb40b3260 2022-08-01T23:00:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeabfff700 1 -- 172.21.15.32:0/282221192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac015740 con 0x7faeb40b3260 2022-08-01T23:00:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeabfff700 1 -- 172.21.15.32:0/282221192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeac015aa0 con 0x7faeb40b3260 2022-08-01T23:00:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- 172.21.15.32:0/282221192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40b3260 msgr2=0x7faeb41435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 --2- 172.21.15.32:0/282221192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40b3260 0x7faeb41435c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7faeac0032f0 tx=0x7faeac00b530).stop 2022-08-01T23:00:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- 172.21.15.32:0/282221192 shutdown_connections 2022-08-01T23:00:45.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 --2- 172.21.15.32:0/282221192 >> v2:172.21.15.55:3300/0 conn(0x7faeb40ad000 0x7faeb413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 --2- 172.21.15.32:0/282221192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40b3260 0x7faeb41435c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- 172.21.15.32:0/282221192 >> 172.21.15.32:0/282221192 conn(0x7faeb40612d0 msgr2=0x7faeb4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:45.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- 172.21.15.32:0/282221192 shutdown_connections 2022-08-01T23:00:45.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.547+0000 7faeba42b700 1 -- 172.21.15.32:0/282221192 wait complete. 2022-08-01T23:00:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 Processor -- start 2022-08-01T23:00:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 -- start start 2022-08-01T23:00:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 0x7faeb409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb40b08a0 0x7faeb4151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faeb40b3260 0x7faeb4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb41516e0 con 0x7faeb40ad000 2022-08-01T23:00:45.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40645e0 con 0x7faeb40b08a0 2022-08-01T23:00:45.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40640a0 con 0x7faeb40b3260 2022-08-01T23:00:45.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 0x7faeb409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 0x7faeb409ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43834/0 (socket says 172.21.15.32:43834) 2022-08-01T23:00:45.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 -- 172.21.15.32:0/1956311300 learned_addr learned my addr 172.21.15.32:0/1956311300 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:45.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9c2a700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faeb40b3260 0x7faeb4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb8c28700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb40b08a0 0x7faeb4151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 -- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faeb40b3260 msgr2=0x7faeb4152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faeb40b3260 0x7faeb4152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 -- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb40b08a0 msgr2=0x7faeb4151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb40b08a0 0x7faeb4151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 -- 172.21.15.32:0/1956311300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac00b7d0 con 0x7faeb40ad000 2022-08-01T23:00:45.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9c2a700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faeb40b3260 0x7faeb4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:45.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeb9429700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 0x7faeb409ed80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7faea4000ed0 tx=0x7faea4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:45.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faea400ce20 con 0x7faeb40ad000 2022-08-01T23:00:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400f470 con 0x7faeb40ad000 2022-08-01T23:00:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb4064340 con 0x7faeb40ad000 2022-08-01T23:00:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faea400f7d0 con 0x7faeb40ad000 2022-08-01T23:00:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.551+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb4067c00 con 0x7faeb40ad000 2022-08-01T23:00:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.555+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb41518a0 con 0x7faeb40ad000 2022-08-01T23:00:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.555+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faea4019030 con 0x7faeb40ad000 2022-08-01T23:00:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.555+0000 7faeaa7fc700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae9c053a70 0x7fae9c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.555+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7faea406b190 con 0x7faeb40ad000 2022-08-01T23:00:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.555+0000 7faeb8c28700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae9c053a70 0x7fae9c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.555+0000 7faeb8c28700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae9c053a70 0x7fae9c055f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7faeac01a000 tx=0x7faeac019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.563+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faeb41518a0 con 0x7faeb40ad000 2022-08-01T23:00:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.699+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faeb41518a0 con 0x7faeb40ad000 2022-08-01T23:00:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.703+0000 7faeaa7fc700 1 -- 172.21.15.32:0/1956311300 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7faeb41518a0 con 0x7faeb40ad000 2022-08-01T23:00:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae9c053a70 msgr2=0x7fae9c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae9c053a70 0x7fae9c055f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7faeac01a000 tx=0x7faeac019000).stop 2022-08-01T23:00:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 msgr2=0x7faeb409ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 0x7faeb409ed80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7faea4000ed0 tx=0x7faea4007ac0).stop 2022-08-01T23:00:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 shutdown_connections 2022-08-01T23:00:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae9c053a70 0x7fae9c055f30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faeb40b3260 0x7faeb4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb40b08a0 0x7faeb4151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 --2- 172.21.15.32:0/1956311300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb40ad000 0x7faeb409ed80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 >> 172.21.15.32:0/1956311300 conn(0x7faeb40612d0 msgr2=0x7faeb4142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 shutdown_connections 2022-08-01T23:00:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.707+0000 7faeba42b700 1 -- 172.21.15.32:0/1956311300 wait complete. 2022-08-01T23:00:45.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-08-01T23:00:45.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.815+0000 7f5c52111700 1 Processor -- start 2022-08-01T23:00:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c52111700 1 -- start start 2022-08-01T23:00:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c52111700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c0b3190 0x7f5c4c060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c52111700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c0654c0 con 0x7f5c4c0b2730 2022-08-01T23:00:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c52111700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c0b3570 con 0x7f5c4c061190 2022-08-01T23:00:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c52111700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c062190 con 0x7f5c4c0b3190 2022-08-01T23:00:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5090e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c4c0b2730 0x7f5c4c0666b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59618/0 (socket says 172.21.15.32:59618) 2022-08-01T23:00:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5090e700 1 -- 172.21.15.32:0/627713756 learned_addr learned my addr 172.21.15.32:0/627713756 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5110f700 1 --2- 172.21.15.32:0/627713756 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c0b3190 0x7f5c4c060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c3bfff700 1 -- 172.21.15.32:0/627713756 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928621914 0 0) 0x7f5c4c0654c0 con 0x7f5c4c0b2730 2022-08-01T23:00:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c3bfff700 1 -- 172.21.15.32:0/627713756 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c34003260 con 0x7f5c4c0b2730 2022-08-01T23:00:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5110f700 1 -- 172.21.15.32:0/627713756 >> v1:172.21.15.55:6789/0 conn(0x7f5c4c061190 legacy=0x7f5c4c066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5110f700 1 -- 172.21.15.32:0/627713756 >> v1:172.21.15.32:6789/0 conn(0x7f5c4c0b2730 legacy=0x7f5c4c0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5110f700 1 -- 172.21.15.32:0/627713756 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c05fbf0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c5110f700 1 --2- 172.21.15.32:0/627713756 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c0b3190 0x7f5c4c060ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5c48002f70 tx=0x7f5c48006df0).ready entity=mon.2 client_cookie=cd9ca9a31a1ad583 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.819+0000 7f5c3bfff700 1 -- 172.21.15.32:0/627713756 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c4800d570 con 0x7f5c4c0b3190 2022-08-01T23:00:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c3bfff700 1 -- 172.21.15.32:0/627713756 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48000e90 con 0x7f5c4c0b3190 2022-08-01T23:00:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 -- 172.21.15.32:0/627713756 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c0b3190 msgr2=0x7f5c4c060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 --2- 172.21.15.32:0/627713756 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c0b3190 0x7f5c4c060ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5c48002f70 tx=0x7f5c48006df0).stop 2022-08-01T23:00:45.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 -- 172.21.15.32:0/627713756 shutdown_connections 2022-08-01T23:00:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 --2- 172.21.15.32:0/627713756 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c0b3190 0x7f5c4c060ca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 -- 172.21.15.32:0/627713756 >> 172.21.15.32:0/627713756 conn(0x7f5c4c063870 msgr2=0x7f5c4c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 -- 172.21.15.32:0/627713756 shutdown_connections 2022-08-01T23:00:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.823+0000 7f5c52111700 1 -- 172.21.15.32:0/627713756 wait complete. 2022-08-01T23:00:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 Processor -- start 2022-08-01T23:00:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 -- start start 2022-08-01T23:00:45.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c061190 0x7f5c4c0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c0b2730 0x7f5c4c0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 0x7f5c4c0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c152ea0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0ad1f0 con 0x7f5c4c0b2730 2022-08-01T23:00:45.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c52111700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0ad490 con 0x7f5c4c061190 2022-08-01T23:00:45.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 0x7f5c4c0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 0x7f5c4c0b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43852/0 (socket says 172.21.15.32:43852) 2022-08-01T23:00:45.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 -- 172.21.15.32:0/580164108 learned_addr learned my addr 172.21.15.32:0/580164108 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:45.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c5110f700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c061190 0x7f5c4c0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c5090e700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c0b2730 0x7f5c4c0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 -- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c061190 msgr2=0x7f5c4c0b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c061190 0x7f5c4c0b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 -- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c0b2730 msgr2=0x7f5c4c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c0b2730 0x7f5c4c0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 -- 172.21.15.32:0/580164108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4800b180 con 0x7f5c4c0b3190 2022-08-01T23:00:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c51910700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 0x7f5c4c0b0360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f5c4c068010 tx=0x7f5c44007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:45.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.827+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c4400f470 con 0x7f5c4c0b3190 2022-08-01T23:00:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4400f5d0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c4400f930 con 0x7f5c4c0b3190 2022-08-01T23:00:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c152bc0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c0a2ff0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c1530a0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c44019030 con 0x7f5c4c0b3190 2022-08-01T23:00:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c3a7fc700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c2c053b00 0x7f5c2c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5c4406b3d0 con 0x7f5c4c0b3190 2022-08-01T23:00:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c5110f700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c2c053b00 0x7f5c2c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.831+0000 7f5c5110f700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c2c053b00 0x7f5c2c055fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f5c4801a000 tx=0x7f5c48014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.835+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c44042370 con 0x7f5c4c0b3190 2022-08-01T23:00:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:45.999+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f5c4c0ad650 con 0x7f5c4c0b3190 2022-08-01T23:00:46.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.415+0000 7f5c3a7fc700 1 -- 172.21.15.32:0/580164108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v215) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5c44045990 con 0x7f5c4c0b3190 2022-08-01T23:00:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c2c053b00 msgr2=0x7f5c2c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c2c053b00 0x7f5c2c055fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f5c4801a000 tx=0x7f5c48014000).stop 2022-08-01T23:00:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 msgr2=0x7f5c4c0b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 0x7f5c4c0b0360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f5c4c068010 tx=0x7f5c44007f40).stop 2022-08-01T23:00:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 shutdown_connections 2022-08-01T23:00:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c2c053b00 0x7f5c2c055fc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c0b3190 0x7f5c4c0b0360 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c0b2730 0x7f5c4c0aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 --2- 172.21.15.32:0/580164108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c4c061190 0x7f5c4c0b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 >> 172.21.15.32:0/580164108 conn(0x7f5c4c063870 msgr2=0x7f5c4c0b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 shutdown_connections 2022-08-01T23:00:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.419+0000 7f5c52111700 1 -- 172.21.15.32:0/580164108 wait complete. 2022-08-01T23:00:46.437 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T23:00:46.438 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T23:00:46.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-08-01T23:00:46.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 Processor -- start 2022-08-01T23:00:46.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- start start 2022-08-01T23:00:46.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 0x7f42200b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:46.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f42200a7250 0x7f42201448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:46.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200b04e0 con 0x7f4220148530 2022-08-01T23:00:46.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200a9250 con 0x7f42200a7250 2022-08-01T23:00:46.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42200b0780 con 0x7f42200b1f40 2022-08-01T23:00:46.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 0x7f42200b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:46.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f422541d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f42200a7250 0x7f42201448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:46.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 0x7f42200b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55594/0 (socket says 172.21.15.32:55594) 2022-08-01T23:00:46.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f422541d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f42200a7250 0x7f42201448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53644/0 (socket says 172.21.15.32:53644) 2022-08-01T23:00:46.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f421ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4220148530 0x7f42200b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52304/0 (socket says 172.21.15.32:52304) 2022-08-01T23:00:46.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f422541d700 1 -- 172.21.15.32:0/3266589902 learned_addr learned my addr 172.21.15.32:0/3266589902 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:46.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 -- 172.21.15.32:0/3266589902 >> v2:172.21.15.55:3300/0 conn(0x7f42200a7250 msgr2=0x7f42201448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:46.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 --2- 172.21.15.32:0/3266589902 >> v2:172.21.15.55:3300/0 conn(0x7f42200a7250 0x7f42201448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 -- 172.21.15.32:0/3266589902 >> v1:172.21.15.55:6789/0 conn(0x7f4220148530 legacy=0x7f42200b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:46.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 -- 172.21.15.32:0/3266589902 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200b0a20 con 0x7f42200b1f40 2022-08-01T23:00:46.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4224c1c700 1 --2- 172.21.15.32:0/3266589902 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 0x7f42200b1160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f4214002f70 tx=0x7f4214006df0).ready entity=mon.2 client_cookie=aec335e269033b6d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:46.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f421f7fe700 1 -- 172.21.15.32:0/3266589902 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421400d570 con 0x7f42200b1f40 2022-08-01T23:00:46.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f421f7fe700 1 -- 172.21.15.32:0/3266589902 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4214000e90 con 0x7f42200b1f40 2022-08-01T23:00:46.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f421f7fe700 1 -- 172.21.15.32:0/3266589902 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421400d8c0 con 0x7f42200b1f40 2022-08-01T23:00:46.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- 172.21.15.32:0/3266589902 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 msgr2=0x7f42200b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:46.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 --2- 172.21.15.32:0/3266589902 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 0x7f42200b1160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f4214002f70 tx=0x7f4214006df0).stop 2022-08-01T23:00:46.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- 172.21.15.32:0/3266589902 shutdown_connections 2022-08-01T23:00:46.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 --2- 172.21.15.32:0/3266589902 >> v2:172.21.15.55:3300/0 conn(0x7f42200a7250 0x7f42201448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 --2- 172.21.15.32:0/3266589902 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42200b1f40 0x7f42200b1160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.547+0000 7f4226e80700 1 -- 172.21.15.32:0/3266589902 >> 172.21.15.32:0/3266589902 conn(0x7f422005f840 msgr2=0x7f422005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 -- 172.21.15.32:0/3266589902 shutdown_connections 2022-08-01T23:00:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 -- 172.21.15.32:0/3266589902 wait complete. 2022-08-01T23:00:46.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 Processor -- start 2022-08-01T23:00:46.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 -- start start 2022-08-01T23:00:46.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 0x7f42200aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:46.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42200b1f40 0x7f4220156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4220148530 0x7f4220157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422014ad60 con 0x7f42200a7250 2022-08-01T23:00:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422014b260 con 0x7f42200b1f40 2022-08-01T23:00:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4226e80700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4220156a00 con 0x7f4220148530 2022-08-01T23:00:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 0x7f42200aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 0x7f42200aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43860/0 (socket says 172.21.15.32:43860) 2022-08-01T23:00:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 -- 172.21.15.32:0/1720730350 learned_addr learned my addr 172.21.15.32:0/1720730350 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f421ffff700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42200b1f40 0x7f4220156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f422541d700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4220148530 0x7f4220157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 -- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4220148530 msgr2=0x7f4220157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4220148530 0x7f4220157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 -- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42200b1f40 msgr2=0x7f4220156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42200b1f40 0x7f4220156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 -- 172.21.15.32:0/1720730350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421400b180 con 0x7f42200a7250 2022-08-01T23:00:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.551+0000 7f4224c1c700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 0x7f42200aecd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f4214002f70 tx=0x7f4214014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4214016070 con 0x7f42200a7250 2022-08-01T23:00:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421400d8f0 con 0x7f42200a7250 2022-08-01T23:00:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422014b490 con 0x7f42200a7250 2022-08-01T23:00:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4214013de0 con 0x7f42200a7250 2022-08-01T23:00:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422014b9d0 con 0x7f42200a7250 2022-08-01T23:00:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422015a020 con 0x7f42200a7250 2022-08-01T23:00:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4214025030 con 0x7f42200a7250 2022-08-01T23:00:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421dffb700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4200053a70 0x7f4200055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:46.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f421406cce0 con 0x7f42200a7250 2022-08-01T23:00:46.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.555+0000 7f421ffff700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4200053a70 0x7f4200055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:46.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.559+0000 7f421ffff700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4200053a70 0x7f4200055f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f421000d000 tx=0x7f4210000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:46.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.563+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4214043c70 con 0x7f42200a7250 2022-08-01T23:00:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:46.727+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f422014a930 con 0x7f42200a7250 2022-08-01T23:00:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.419+0000 7f421dffb700 1 -- 172.21.15.32:0/1720730350 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v216) v1 ==== 150+0+0 (secure 0 0 0) 0x7f42140230d0 con 0x7f42200a7250 2022-08-01T23:00:47.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.419+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4200053a70 msgr2=0x7f4200055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.419+0000 7f4226e80700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4200053a70 0x7f4200055f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f421000d000 tx=0x7f4210000bc0).stop 2022-08-01T23:00:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.419+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 msgr2=0x7f42200aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.419+0000 7f4226e80700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 0x7f42200aecd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f4214002f70 tx=0x7f4214014000).stop 2022-08-01T23:00:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 shutdown_connections 2022-08-01T23:00:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4200053a70 0x7f4200055f30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4220148530 0x7f4220157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42200b1f40 0x7f4220156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 --2- 172.21.15.32:0/1720730350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42200a7250 0x7f42200aecd0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 >> 172.21.15.32:0/1720730350 conn(0x7f422005f840 msgr2=0x7f4220144dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 shutdown_connections 2022-08-01T23:00:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.423+0000 7f4226e80700 1 -- 172.21.15.32:0/1720730350 wait complete. 2022-08-01T23:00:47.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:00:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 Processor -- start 2022-08-01T23:00:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 -- start start 2022-08-01T23:00:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa27c0b1460 0x7fa27c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 0x7fa27c1413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c0a7960 con 0x7fa27c0b2050 2022-08-01T23:00:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c0b18d0 con 0x7fa27c0b1460 2022-08-01T23:00:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa281637700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b07e0 con 0x7fa27c0b2430 2022-08-01T23:00:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa27c0b2050 0x7fa27c0a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59642/0 (socket says 172.21.15.32:59642) 2022-08-01T23:00:47.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 -- 172.21.15.32:0/1123758101 learned_addr learned my addr 172.21.15.32:0/1123758101 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:47.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa273fff700 1 --2- 172.21.15.32:0/1123758101 >> v2:172.21.15.55:3300/0 conn(0x7fa27c0b1460 0x7fa27c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27b7fe700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 0x7fa27c1413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385530763 0 0) 0x7fa27c0a7960 con 0x7fa27c0b2050 2022-08-01T23:00:47.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25c003260 con 0x7fa27c0b2050 2022-08-01T23:00:47.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030288495 0 0) 0x7fa25c003260 con 0x7fa27c0b2050 2022-08-01T23:00:47.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 msgr2=0x7fa27c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:47.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 0x7fa27c1413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 >> v2:172.21.15.55:3300/0 conn(0x7fa27c0b1460 msgr2=0x7fa27c0b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:47.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 --2- 172.21.15.32:0/1123758101 >> v2:172.21.15.55:3300/0 conn(0x7fa27c0b1460 0x7fa27c0b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c0b0a80 con 0x7fa27c0b2050 2022-08-01T23:00:47.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fa26c001660 con 0x7fa27c0b2050 2022-08-01T23:00:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 >> v1:172.21.15.32:6789/0 conn(0x7fa27c0b2050 legacy=0x7fa27c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa25c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa25c004500 0x7fa25c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 0x7fa25c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b18d0 con 0x7fa25c003750 2022-08-01T23:00:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b07e0 con 0x7fa25c004500 2022-08-01T23:00:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c00b750 con 0x7fa27c0b2430 2022-08-01T23:00:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa25c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa273fff700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa25c004500 0x7fa25c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 -- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 msgr2=0x7fa25c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b2430 0x7fa25c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T23:00:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 -- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa25c004500 msgr2=0x7fa25c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa25c004500 0x7fa25c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 -- 172.21.15.32:0/1123758101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c003260 con 0x7fa25c003750 2022-08-01T23:00:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27affd700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa25c003b70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fa26c001370 tx=0x7fa27c0b0c10).ready entity=mon.0 client_cookie=67db4cd53db6e758 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa26c0071a0 con 0x7fa25c003750 2022-08-01T23:00:47.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c007300 con 0x7fa25c003750 2022-08-01T23:00:47.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.551+0000 7fa27a7fc700 1 -- 172.21.15.32:0/1123758101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa26c00cbb0 con 0x7fa25c003750 2022-08-01T23:00:47.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 -- 172.21.15.32:0/1123758101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 msgr2=0x7fa25c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 --2- 172.21.15.32:0/1123758101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa25c003b70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fa26c001370 tx=0x7fa27c0b0c10).stop 2022-08-01T23:00:47.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa27b7fe700 1 -- 172.21.15.32:0/1123758101 reap_dead start 2022-08-01T23:00:47.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 -- 172.21.15.32:0/1123758101 shutdown_connections 2022-08-01T23:00:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 -- 172.21.15.32:0/1123758101 >> 172.21.15.32:0/1123758101 conn(0x7fa27c05f840 msgr2=0x7fa27c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 -- 172.21.15.32:0/1123758101 shutdown_connections 2022-08-01T23:00:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 -- 172.21.15.32:0/1123758101 wait complete. 2022-08-01T23:00:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 Processor -- start 2022-08-01T23:00:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.555+0000 7fa281637700 1 -- start start 2022-08-01T23:00:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa27c13e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa27c15a800 0x7fa27c15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b4d20 0x7fa27c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c15a3d0 con 0x7fa25c003750 2022-08-01T23:00:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b4620 con 0x7fa27c15a800 2022-08-01T23:00:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b0340 con 0x7fa27c0b4d20 2022-08-01T23:00:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa27c13e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa27c13e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43880/0 (socket says 172.21.15.32:43880) 2022-08-01T23:00:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 -- 172.21.15.32:0/1425338909 learned_addr learned my addr 172.21.15.32:0/1425338909 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa273fff700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa27c15a800 0x7fa27c15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27b7fe700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b4d20 0x7fa27c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 -- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b4d20 msgr2=0x7fa27c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b4d20 0x7fa27c0b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 -- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa27c15a800 msgr2=0x7fa27c15ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa27c15a800 0x7fa27c15ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 -- 172.21.15.32:0/1425338909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c0b33a0 con 0x7fa25c003750 2022-08-01T23:00:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa27affd700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa27c13e0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa26c001370 tx=0x7fa26c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa26c007470 con 0x7fa25c003750 2022-08-01T23:00:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00ce40 con 0x7fa25c003750 2022-08-01T23:00:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c15b150 con 0x7fa25c003750 2022-08-01T23:00:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa26c004920 con 0x7fa25c003750 2022-08-01T23:00:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.559+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c0af600 con 0x7fa25c003750 2022-08-01T23:00:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.563+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c0574e0 con 0x7fa25c003750 2022-08-01T23:00:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.563+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa26c026070 con 0x7fa25c003750 2022-08-01T23:00:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.563+0000 7fa278ff9700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa260053a70 0x7fa260055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.563+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa26c06c490 con 0x7fa25c003750 2022-08-01T23:00:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.563+0000 7fa273fff700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa260053a70 0x7fa260055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.563+0000 7fa273fff700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa260053a70 0x7fa260055f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fa26400b000 tx=0x7fa264006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:47.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.571+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa26c043420 con 0x7fa25c003750 2022-08-01T23:00:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.699+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa27c15a560 con 0x7fa25c003750 2022-08-01T23:00:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.707+0000 7fa278ff9700 1 -- 172.21.15.32:0/1425338909 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa26c046a40 con 0x7fa25c003750 2022-08-01T23:00:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa260053a70 msgr2=0x7fa260055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa260053a70 0x7fa260055f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fa26400b000 tx=0x7fa264006df0).stop 2022-08-01T23:00:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 msgr2=0x7fa27c13e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa27c13e0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa26c001370 tx=0x7fa26c013000).stop 2022-08-01T23:00:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 shutdown_connections 2022-08-01T23:00:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa260053a70 0x7fa260055f30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa27c0b4d20 0x7fa27c0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa27c15a800 0x7fa27c15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 --2- 172.21.15.32:0/1425338909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa25c003750 0x7fa27c13e0a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 >> 172.21.15.32:0/1425338909 conn(0x7fa27c05f840 msgr2=0x7fa27c140390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 shutdown_connections 2022-08-01T23:00:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.711+0000 7fa281637700 1 -- 172.21.15.32:0/1425338909 wait complete. 2022-08-01T23:00:47.727 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:00:47.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:47.747 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:47.769 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:47.791 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:00:47.811 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:00:47.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:47.813 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:00:47.813 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:47.819 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:00:47.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:00:47.821 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:00:47.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:47.822 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:00:47.822 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:00:47.829 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:00:47.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:00:47.831 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:00:47.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:00:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6711343700 1 Processor -- start 2022-08-01T23:00:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6711343700 1 -- start start 2022-08-01T23:00:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6711343700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f670c0ac6a0 0x7f670c14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6711343700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670c1453e0 con 0x7f670c0b2390 2022-08-01T23:00:47.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6711343700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670c145680 con 0x7f670c0b0160 2022-08-01T23:00:47.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6711343700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c141390 con 0x7f670c0ac6a0 2022-08-01T23:00:47.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f670affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f670c0b2390 0x7f670c0abf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59662/0 (socket says 172.21.15.32:59662) 2022-08-01T23:00:47.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f670affd700 1 -- 172.21.15.32:0/113458557 learned_addr learned my addr 172.21.15.32:0/113458557 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f670b7fe700 1 --2- 172.21.15.32:0/113458557 >> v2:172.21.15.55:3300/0 conn(0x7f670c0ac6a0 0x7f670c14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6709ffb700 1 -- 172.21.15.32:0/113458557 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391131665 0 0) 0x7f670c1453e0 con 0x7f670c0b2390 2022-08-01T23:00:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.939+0000 7f6709ffb700 1 -- 172.21.15.32:0/113458557 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f0003260 con 0x7f670c0b2390 2022-08-01T23:00:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f670b7fe700 1 -- 172.21.15.32:0/113458557 >> v1:172.21.15.55:6789/0 conn(0x7f670c0b0160 legacy=0x7f670c141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f670b7fe700 1 -- 172.21.15.32:0/113458557 >> v1:172.21.15.32:6789/0 conn(0x7f670c0b2390 legacy=0x7f670c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f670b7fe700 1 -- 172.21.15.32:0/113458557 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0b0670 con 0x7f670c0ac6a0 2022-08-01T23:00:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f670b7fe700 1 --2- 172.21.15.32:0/113458557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0ac6a0 0x7f670c14cc00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f67040032f0 tx=0x7f67040091d0).ready entity=mon.1 client_cookie=d634aecde0b15dc2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:47.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6709ffb700 1 -- 172.21.15.32:0/113458557 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670400b6c0 con 0x7f670c0ac6a0 2022-08-01T23:00:47.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6709ffb700 1 -- 172.21.15.32:0/113458557 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670400b820 con 0x7f670c0ac6a0 2022-08-01T23:00:47.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6709ffb700 1 -- 172.21.15.32:0/113458557 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670400b990 con 0x7f670c0ac6a0 2022-08-01T23:00:47.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 -- 172.21.15.32:0/113458557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0ac6a0 msgr2=0x7f670c14cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 --2- 172.21.15.32:0/113458557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0ac6a0 0x7f670c14cc00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f67040032f0 tx=0x7f67040091d0).stop 2022-08-01T23:00:47.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 -- 172.21.15.32:0/113458557 shutdown_connections 2022-08-01T23:00:47.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 --2- 172.21.15.32:0/113458557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0ac6a0 0x7f670c14cc00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 -- 172.21.15.32:0/113458557 >> 172.21.15.32:0/113458557 conn(0x7f670c05f840 msgr2=0x7f670c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 -- 172.21.15.32:0/113458557 shutdown_connections 2022-08-01T23:00:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 -- 172.21.15.32:0/113458557 wait complete. 2022-08-01T23:00:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 Processor -- start 2022-08-01T23:00:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.943+0000 7f6711343700 1 -- start start 2022-08-01T23:00:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f670c0ac6a0 0x7f670c141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0b0160 0x7f670c154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 0x7f670c138b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c144840 con 0x7f670c0b2390 2022-08-01T23:00:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c150680 con 0x7f670c0b0160 2022-08-01T23:00:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c13bd60 con 0x7f670c0ac6a0 2022-08-01T23:00:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 0x7f670c138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 0x7f670c138b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43898/0 (socket says 172.21.15.32:43898) 2022-08-01T23:00:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 -- 172.21.15.32:0/3736067351 learned_addr learned my addr 172.21.15.32:0/3736067351 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670a7fc700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0b0160 0x7f670c154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670affd700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f670c0ac6a0 0x7f670c141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 -- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f670c0ac6a0 msgr2=0x7f670c141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f670c0ac6a0 0x7f670c141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 -- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0b0160 msgr2=0x7f670c154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0b0160 0x7f670c154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 -- 172.21.15.32:0/3736067351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009470 con 0x7f670c0b2390 2022-08-01T23:00:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f670b7fe700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 0x7f670c138b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f67040032f0 tx=0x7f6704016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f670400b6c0 con 0x7f670c0b2390 2022-08-01T23:00:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040107a0 con 0x7f670c0b2390 2022-08-01T23:00:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c13bf90 con 0x7f670c0b2390 2022-08-01T23:00:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6704010b00 con 0x7f670c0b2390 2022-08-01T23:00:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c13c670 con 0x7f670c0b2390 2022-08-01T23:00:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.947+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c13c430 con 0x7f670c0b2390 2022-08-01T23:00:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.951+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f670400f070 con 0x7f670c0b2390 2022-08-01T23:00:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.951+0000 7f66fbfff700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66e8053b00 0x7f66e8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.951+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f670406d270 con 0x7f670c0b2390 2022-08-01T23:00:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.951+0000 7f670affd700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66e8053b00 0x7f66e8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.951+0000 7f670affd700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66e8053b00 0x7f66e8055fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f670000b000 tx=0x7f6700009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:47.959+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6704044310 con 0x7f670c0b2390 2022-08-01T23:00:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.083+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f670c0574e0 con 0x7f670c0b2390 2022-08-01T23:00:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f66fbfff700 1 -- 172.21.15.32:0/3736067351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v59) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f6704015090 con 0x7f670c0b2390 2022-08-01T23:00:48.088 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:48.089 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:00:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 59, 2022-08-01T23:00:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:00:48.089 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:00:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:00:45.394724+0000", 2022-08-01T23:00:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:00:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:00:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:00:48.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:00:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:00:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:00:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:00:48.091 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:00:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:00:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:00:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:00:48.092 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:00:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:00:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:00:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 10, 2022-08-01T23:00:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:00:48.093 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:00:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:00:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:00:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:00:48.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:00:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:00:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:00:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:00:48.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:00:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:00:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:00:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:00:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:00:48.096 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:00:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:00:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:00:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:00:48.097 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:00:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:00:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:00:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:00:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:00:48.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:00:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:00:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:00:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:00:48.100 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:00:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:00:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:00:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:00:48.101 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:00:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:00:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:00:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:00:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:00:48.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:00:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:00:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:00:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:00:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:00:48.103 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:00:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:00:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:00:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:00:48.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:00:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:00:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:00:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:00:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:00:48.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:00:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:00:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:00:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:00:48.106 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:00:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:00:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:00:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:00:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:00:48.107 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:00:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:00:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:00:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:00:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.108 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:00:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:00:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:00:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:00:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:00:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:00:48.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:00:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:00:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:00:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:00:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:00:48.111 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:00:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:00:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:00:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:00:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:00:48.112 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:00:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:00:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:00:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:00:48.113 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:00:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:00:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:00:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:00:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:00:48.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:00:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:00:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:00:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:00:48.115 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:00:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:00:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:00:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:00:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:00:48.116 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:00:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:00:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:00:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:00:48.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:00:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:00:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:00:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:00:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:00:48.118 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:00:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:00:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:00:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:00:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:00:48.119 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:00:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:00:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:00:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:00:48.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:00:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:00:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:00:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:00:48.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:00:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:00:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:00:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:00:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:00:48.122 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:00:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:00:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:00:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:00:48.123 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:00:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:00:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:00:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:00:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.125 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.126 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:00:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:00:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:00:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:00:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:00:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:00:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:00:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:00:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:00:48.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:00:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:00:48.139 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:00:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:00:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:00:48.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:00:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.142 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:48.143 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:00:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:00:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:00:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:00:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:00:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:00:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:00:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:00:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:00:48.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:00:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:00:48.156 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:00:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:00:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.157 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:00:48.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:00:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:48.160 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:00:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:00:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.163 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:00:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:00:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:00:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:00:48.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.170 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:00:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:00:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:00:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:00:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:00:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:00:48.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:00:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:00:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:00:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.176 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.177 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2022-08-01T23:00:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.178 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:00:48.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:00:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.181 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:00:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:00:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.184 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:00:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:00:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.187 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:00:48.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:00:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:00:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.190 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:00:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:00:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:00:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:00:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.191 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:00:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:00:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.193 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.194 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:00:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:00:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:00:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:00:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:00:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:00:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:00:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:00:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:00:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:00:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:00:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:00:48.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:00:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:00:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:00:48.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.210 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 53, 2022-08-01T23:00:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.211 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:00:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:00:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:00:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:00:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.217 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:00:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.219 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:00:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.220 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:00:48.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:00:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:00:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:00:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:00:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:00:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:00:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.224 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:00:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:00:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.226 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 54, 2022-08-01T23:00:48.227 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:00:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:00:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.230 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:00:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:00:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:00:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:00:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:00:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:00:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:00:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:00:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:00:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:00:48.240 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:00:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:00:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:00:48.242 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:00:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:00:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:00:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:00:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:00:48.243 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:00:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:00:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2022-08-01T23:00:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:00:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:00:48.244 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:00:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:00:48.245 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.246 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:00:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.247 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:00:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:00:48.248 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:00:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.250 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:00:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.251 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:00:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.252 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:00:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:00:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:00:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:00:48.254 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:00:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:00:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:00:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:00:48.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:00:48.257 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:00:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:00:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:00:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:00:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:00:48.258 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:00:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:00:48.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:00:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.260 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:00:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:00:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.262 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:00:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:00:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.264 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:00:48.265 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.55:6835/2848708845": "2022-08-02T23:00:41.255474+0000", 2022-08-01T23:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.55:6834/2848708845": "2022-08-02T23:00:41.255474+0000" 2022-08-01T23:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:00:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66e8053b00 msgr2=0x7f66e8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66e8053b00 0x7f66e8055fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f670000b000 tx=0x7f6700009040).stop 2022-08-01T23:00:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 msgr2=0x7f670c138b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 0x7f670c138b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f67040032f0 tx=0x7f6704016000).stop 2022-08-01T23:00:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 shutdown_connections 2022-08-01T23:00:48.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66e8053b00 0x7f66e8055fc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f670c0b2390 0x7f670c138b50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f670c0b0160 0x7f670c154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 --2- 172.21.15.32:0/3736067351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f670c0ac6a0 0x7f670c141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 >> 172.21.15.32:0/3736067351 conn(0x7f670c05f840 msgr2=0x7f670c139e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 shutdown_connections 2022-08-01T23:00:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.087+0000 7f6711343700 1 -- 172.21.15.32:0/3736067351 wait complete. 2022-08-01T23:00:48.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 Processor -- start 2022-08-01T23:00:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 -- start start 2022-08-01T23:00:48.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b7840 0x7f8468065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84680b3070 0x7f8468061120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84680b2c20 con 0x7f846814b040 2022-08-01T23:00:48.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8468065c10 con 0x7f84680b3070 2022-08-01T23:00:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846e5d6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680b7c20 con 0x7f84680b7840 2022-08-01T23:00:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84680b3070 0x7f8468061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84680b3070 0x7f8468061120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43902/0 (socket says 172.21.15.32:43902) 2022-08-01T23:00:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 -- 172.21.15.32:0/2229409432 learned_addr learned my addr 172.21.15.32:0/2229409432 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846d5d4700 1 --2- 172.21.15.32:0/2229409432 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b7840 0x7f8468065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 -- 172.21.15.32:0/2229409432 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b7840 msgr2=0x7f8468065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 --2- 172.21.15.32:0/2229409432 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b7840 0x7f8468065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 -- 172.21.15.32:0/2229409432 >> v1:172.21.15.55:6789/0 conn(0x7f846814b040 legacy=0x7f846814b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.399+0000 7f846cdd3700 1 -- 172.21.15.32:0/2229409432 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680b2980 con 0x7f84680b3070 2022-08-01T23:00:48.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846cdd3700 1 --2- 172.21.15.32:0/2229409432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b3070 0x7f8468061120 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f84580032f0 tx=0x7f8458009290).ready entity=mon.0 client_cookie=47a8cb2d16c4121c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:48.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f8467fff700 1 -- 172.21.15.32:0/2229409432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f845800b8a0 con 0x7f84680b3070 2022-08-01T23:00:48.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f8467fff700 1 -- 172.21.15.32:0/2229409432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845800ba00 con 0x7f84680b3070 2022-08-01T23:00:48.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f8467fff700 1 -- 172.21.15.32:0/2229409432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f845800bb70 con 0x7f84680b3070 2022-08-01T23:00:48.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 -- 172.21.15.32:0/2229409432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b3070 msgr2=0x7f8468061120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 --2- 172.21.15.32:0/2229409432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b3070 0x7f8468061120 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f84580032f0 tx=0x7f8458009290).stop 2022-08-01T23:00:48.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 -- 172.21.15.32:0/2229409432 shutdown_connections 2022-08-01T23:00:48.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 --2- 172.21.15.32:0/2229409432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b3070 0x7f8468061120 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 --2- 172.21.15.32:0/2229409432 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b7840 0x7f8468065720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 -- 172.21.15.32:0/2229409432 >> 172.21.15.32:0/2229409432 conn(0x7f846805f840 msgr2=0x7f846805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:48.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 -- 172.21.15.32:0/2229409432 shutdown_connections 2022-08-01T23:00:48.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.403+0000 7f846e5d6700 1 -- 172.21.15.32:0/2229409432 wait complete. 2022-08-01T23:00:48.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 Processor -- start 2022-08-01T23:00:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 -- start start 2022-08-01T23:00:48.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b3070 0x7f846814a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 0x7f846809ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846814b040 0x7f846809f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680aed30 con 0x7f84680b7840 2022-08-01T23:00:48.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846809f730 con 0x7f846814b040 2022-08-01T23:00:48.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680aff50 con 0x7f84680b3070 2022-08-01T23:00:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 0x7f846809ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 0x7f846809ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43910/0 (socket says 172.21.15.32:43910) 2022-08-01T23:00:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 -- 172.21.15.32:0/1845434663 learned_addr learned my addr 172.21.15.32:0/1845434663 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846d5d4700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b3070 0x7f846814a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846ddd5700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846814b040 0x7f846809f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 -- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b3070 msgr2=0x7f846814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b3070 0x7f846814a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 -- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846814b040 msgr2=0x7f846809f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846814b040 0x7f846809f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 -- 172.21.15.32:0/1845434663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009530 con 0x7f84680b7840 2022-08-01T23:00:48.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846cdd3700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 0x7f846809ed50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f84580032f0 tx=0x7f8458016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:48.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8458021440 con 0x7f84680b7840 2022-08-01T23:00:48.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84580215a0 con 0x7f84680b7840 2022-08-01T23:00:48.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84680aef20 con 0x7f84680b7840 2022-08-01T23:00:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8458021900 con 0x7f84680b7840 2022-08-01T23:00:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.407+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84680ae210 con 0x7f84680b7840 2022-08-01T23:00:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.411+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84680b0090 con 0x7f84680b7840 2022-08-01T23:00:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.411+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8458029070 con 0x7f84680b7840 2022-08-01T23:00:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.411+0000 7f84667fc700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8448053a70 0x7f8448055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.411+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f845806d1f0 con 0x7f84680b7840 2022-08-01T23:00:48.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.411+0000 7f846d5d4700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8448053a70 0x7f8448055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.415+0000 7f846d5d4700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8448053a70 0x7f8448055f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f845c00c000 tx=0x7f845c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:48.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.419+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84580442d0 con 0x7f84680b7840 2022-08-01T23:00:48.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84680609e0 con 0x7f84680b7840 2022-08-01T23:00:48.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f84667fc700 1 -- 172.21.15.32:0/1845434663 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f8458015070 con 0x7f84680b7840 2022-08-01T23:00:48.588 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:48.588 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2209588153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2209588153},{"type":"v1","addr":"172.21.15.55:6835","nonce":2209588153}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[]} 2022-08-01T23:00:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8448053a70 msgr2=0x7f8448055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8448053a70 0x7f8448055f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f845c00c000 tx=0x7f845c008040).stop 2022-08-01T23:00:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 msgr2=0x7f846809ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 0x7f846809ed50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f84580032f0 tx=0x7f8458016000).stop 2022-08-01T23:00:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 shutdown_connections 2022-08-01T23:00:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8448053a70 0x7f8448055f30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846814b040 0x7f846809f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84680b7840 0x7f846809ed50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 --2- 172.21.15.32:0/1845434663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84680b3070 0x7f846814a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 >> 172.21.15.32:0/1845434663 conn(0x7f846805f840 msgr2=0x7f84680b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 shutdown_connections 2022-08-01T23:00:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.587+0000 7f846e5d6700 1 -- 172.21.15.32:0/1845434663 wait complete. 2022-08-01T23:00:48.593 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2022-08-01T23:00:48.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:48.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.695+0000 7f5b90cb3700 1 Processor -- start 2022-08-01T23:00:48.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- start start 2022-08-01T23:00:48.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b8c0b13f0 0x7f5b8c0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c0adf10 con 0x7f5b8c0b2710 2022-08-01T23:00:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c0ae1b0 con 0x7f5b8c0b1920 2022-08-01T23:00:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8c0a9520 con 0x7f5b8c0b13f0 2022-08-01T23:00:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b8c0b13f0 0x7f5b8c0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b8c0b13f0 0x7f5b8c0afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43918/0 (socket says 172.21.15.32:43918) 2022-08-01T23:00:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 -- 172.21.15.32:0/4175903507 learned_addr learned my addr 172.21.15.32:0/4175903507 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 -- 172.21.15.32:0/4175903507 >> v1:172.21.15.55:6789/0 conn(0x7f5b8c0b1920 legacy=0x7f5b8c14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b8affd700 1 -- 172.21.15.32:0/4175903507 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466469568 0 0) 0x7f5b8c0adf10 con 0x7f5b8c0b2710 2022-08-01T23:00:48.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 -- 172.21.15.32:0/4175903507 >> v1:172.21.15.32:6789/0 conn(0x7f5b8c0b2710 legacy=0x7f5b8c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 -- 172.21.15.32:0/4175903507 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0a99c0 con 0x7f5b8c0b13f0 2022-08-01T23:00:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b83fff700 1 --2- 172.21.15.32:0/4175903507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b13f0 0x7f5b8c0afd60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f5b74002f70 tx=0x7f5b74006f40).ready entity=mon.0 client_cookie=2e77619385716219 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b8affd700 1 -- 172.21.15.32:0/4175903507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b7400dc70 con 0x7f5b8c0b13f0 2022-08-01T23:00:48.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b8affd700 1 -- 172.21.15.32:0/4175903507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74000e90 con 0x7f5b8c0b13f0 2022-08-01T23:00:48.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b8affd700 1 -- 172.21.15.32:0/4175903507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b7400be90 con 0x7f5b8c0b13f0 2022-08-01T23:00:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- 172.21.15.32:0/4175903507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b13f0 msgr2=0x7f5b8c0afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/4175903507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b13f0 0x7f5b8c0afd60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f5b74002f70 tx=0x7f5b74006f40).stop 2022-08-01T23:00:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- 172.21.15.32:0/4175903507 shutdown_connections 2022-08-01T23:00:48.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/4175903507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b13f0 0x7f5b8c0afd60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- 172.21.15.32:0/4175903507 >> 172.21.15.32:0/4175903507 conn(0x7f5b8c060310 msgr2=0x7f5b8c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:48.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- 172.21.15.32:0/4175903507 shutdown_connections 2022-08-01T23:00:48.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.699+0000 7f5b90cb3700 1 -- 172.21.15.32:0/4175903507 wait complete. 2022-08-01T23:00:48.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.703+0000 7f5b90cb3700 1 Processor -- start 2022-08-01T23:00:48.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.703+0000 7f5b90cb3700 1 -- start start 2022-08-01T23:00:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b8c0b13f0 0x7f5b8c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 0x7f5b8c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b8c0b2710 0x7f5b8c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c09f410 con 0x7f5b8c0b1920 2022-08-01T23:00:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c09f640 con 0x7f5b8c0b2710 2022-08-01T23:00:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0ab670 con 0x7f5b8c0b13f0 2022-08-01T23:00:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 0x7f5b8c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 0x7f5b8c152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43924/0 (socket says 172.21.15.32:43924) 2022-08-01T23:00:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 -- 172.21.15.32:0/1288282442 learned_addr learned my addr 172.21.15.32:0/1288282442 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b8b7fe700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b8c0b13f0 0x7f5b8c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b8bfff700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b8c0b2710 0x7f5b8c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 -- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b8c0b13f0 msgr2=0x7f5b8c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b8c0b13f0 0x7f5b8c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 -- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b8c0b2710 msgr2=0x7f5b8c152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b8c0b2710 0x7f5b8c152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 -- 172.21.15.32:0/1288282442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7400b460 con 0x7f5b8c0b1920 2022-08-01T23:00:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b83fff700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 0x7f5b8c152610 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f5b74002f70 tx=0x7f5b74006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b7401d070 con 0x7f5b8c0b1920 2022-08-01T23:00:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74022410 con 0x7f5b8c0b1920 2022-08-01T23:00:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8c0ab910 con 0x7f5b8c0b1920 2022-08-01T23:00:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b74022770 con 0x7f5b8c0b1920 2022-08-01T23:00:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.707+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8c150160 con 0x7f5b8c0b1920 2022-08-01T23:00:48.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.711+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b74029070 con 0x7f5b8c0b1920 2022-08-01T23:00:48.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.711+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8c0abad0 con 0x7f5b8c0b1920 2022-08-01T23:00:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.711+0000 7f5b897fa700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b70053a70 0x7f5b70055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.711+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5b7406d190 con 0x7f5b8c0b1920 2022-08-01T23:00:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.711+0000 7f5b8b7fe700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b70053a70 0x7f5b70055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.711+0000 7f5b8b7fe700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b70053a70 0x7f5b70055f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f5b7c00d000 tx=0x7f5b7c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.719+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b740441b0 con 0x7f5b8c0b1920 2022-08-01T23:00:48.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.879+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b8c0574e0 con 0x7f5b8c0b1920 2022-08-01T23:00:48.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.879+0000 7f5b897fa700 1 -- 172.21.15.32:0/1288282442 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f5b740477d0 con 0x7f5b8c0b1920 2022-08-01T23:00:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:48.881 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2209588153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2209588153},{"type":"v1","addr":"172.21.15.55:6835","nonce":2209588153}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3889590865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3889590865},{"type":"v1","addr":"172.21.15.32:6833","nonce":3889590865}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1633036605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1633036605},{"type":"v1","addr":"172.21.15.32:6835","nonce":1633036605}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[]} 2022-08-01T23:00:48.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.879+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b70053a70 msgr2=0x7f5b70055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b70053a70 0x7f5b70055f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f5b7c00d000 tx=0x7f5b7c009040).stop 2022-08-01T23:00:48.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 msgr2=0x7f5b8c152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:48.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 0x7f5b8c152610 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f5b74002f70 tx=0x7f5b74006f40).stop 2022-08-01T23:00:48.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 shutdown_connections 2022-08-01T23:00:48.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5b70053a70 0x7f5b70055f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b8c0b2710 0x7f5b8c152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b8c0b1920 0x7f5b8c152610 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 --2- 172.21.15.32:0/1288282442 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b8c0b13f0 0x7f5b8c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:48.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 >> 172.21.15.32:0/1288282442 conn(0x7f5b8c060310 msgr2=0x7f5b8c05c150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:48.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 shutdown_connections 2022-08-01T23:00:48.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:48.883+0000 7f5b90cb3700 1 -- 172.21.15.32:0/1288282442 wait complete. 2022-08-01T23:00:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2022-08-01T23:00:48.894 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:00:48.894 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:00:48.894 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:00:48.895 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:00:48.895 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:00:48.895 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:00:48.896 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:00:48.896 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:00:48.896 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:00:48.897 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:00:48.897 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:00:48.897 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:00:48.904 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:00:48.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:00:48.907 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:00:48.907 DEBUG:teuthology.parallel:result is None 2022-08-01T23:00:48.908 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:00:48.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:00:48.914 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:00:48.914 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:00:48.917 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:00:48.917 DEBUG:teuthology.parallel:result is None 2022-08-01T23:00:48.952 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:00:48.952 DEBUG:teuthology.parallel:result is None 2022-08-01T23:00:48.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:00:48.983 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:00:48.995 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:00:49.023 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:00:49.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078759e700 1 Processor -- start 2022-08-01T23:00:49.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078759e700 1 -- start start 2022-08-01T23:00:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f078805fb80 0x7f078813ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880afa50 con 0x7f07880b2050 2022-08-01T23:00:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078805ff60 con 0x7f07880ae3c0 2022-08-01T23:00:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0788140b90 con 0x7f078805fb80 2022-08-01T23:00:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f078659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f078805fb80 0x7f078813ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f0785d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07880b2050 0x7f0788140480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59726/0 (socket says 172.21.15.32:59726) 2022-08-01T23:00:49.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.043+0000 7f0785d9b700 1 -- 172.21.15.32:0/4241379038 learned_addr learned my addr 172.21.15.32:0/4241379038 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:49.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078659c700 1 -- 172.21.15.32:0/4241379038 >> v1:172.21.15.55:6789/0 conn(0x7f07880ae3c0 legacy=0x7f078813cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:49.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078659c700 1 -- 172.21.15.32:0/4241379038 >> v1:172.21.15.32:6789/0 conn(0x7f07880b2050 legacy=0x7f0788140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:49.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078559a700 1 -- 172.21.15.32:0/4241379038 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292783197 0 0) 0x7f07880afa50 con 0x7f07880b2050 2022-08-01T23:00:49.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078659c700 1 -- 172.21.15.32:0/4241379038 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880ad1c0 con 0x7f078805fb80 2022-08-01T23:00:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078659c700 1 --2- 172.21.15.32:0/4241379038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f078813ff90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f0778002f70 tx=0x7f0778006df0).ready entity=mon.0 client_cookie=70f648a82e60b4ad server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078559a700 1 -- 172.21.15.32:0/4241379038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077800d610 con 0x7f078805fb80 2022-08-01T23:00:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078559a700 1 -- 172.21.15.32:0/4241379038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778000e90 con 0x7f078805fb80 2022-08-01T23:00:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078559a700 1 -- 172.21.15.32:0/4241379038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077800d960 con 0x7f078805fb80 2022-08-01T23:00:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 -- 172.21.15.32:0/4241379038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 msgr2=0x7f078813ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 --2- 172.21.15.32:0/4241379038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f078813ff90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f0778002f70 tx=0x7f0778006df0).stop 2022-08-01T23:00:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 -- 172.21.15.32:0/4241379038 shutdown_connections 2022-08-01T23:00:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 --2- 172.21.15.32:0/4241379038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f078813ff90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 -- 172.21.15.32:0/4241379038 >> 172.21.15.32:0/4241379038 conn(0x7f07880613c0 msgr2=0x7f0788062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 -- 172.21.15.32:0/4241379038 shutdown_connections 2022-08-01T23:00:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 -- 172.21.15.32:0/4241379038 wait complete. 2022-08-01T23:00:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 Processor -- start 2022-08-01T23:00:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.047+0000 7f078759e700 1 -- start start 2022-08-01T23:00:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f0788064470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07880ae3c0 0x7f0788142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07880b2050 0x7f0788142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788143cc0 con 0x7f078805fb80 2022-08-01T23:00:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788129950 con 0x7f07880b2050 2022-08-01T23:00:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0788142110 con 0x7f07880ae3c0 2022-08-01T23:00:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f0788064470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f0788064470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43958/0 (socket says 172.21.15.32:43958) 2022-08-01T23:00:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 -- 172.21.15.32:0/2679182423 learned_addr learned my addr 172.21.15.32:0/2679182423 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f0786d9d700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07880b2050 0x7f0788142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f0785d9b700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07880ae3c0 0x7f0788142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 -- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07880ae3c0 msgr2=0x7f0788142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07880ae3c0 0x7f0788142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 -- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07880b2050 msgr2=0x7f0788142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07880b2050 0x7f0788142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 -- 172.21.15.32:0/2679182423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077800b180 con 0x7f078805fb80 2022-08-01T23:00:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f0786d9d700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07880b2050 0x7f0788142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078659c700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f0788064470 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f0778002f70 tx=0x7f0778015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077800d290 con 0x7f078805fb80 2022-08-01T23:00:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778011ce0 con 0x7f078805fb80 2022-08-01T23:00:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0788141490 con 0x7f078805fb80 2022-08-01T23:00:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077801e650 con 0x7f078805fb80 2022-08-01T23:00:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078812af80 con 0x7f078805fb80 2022-08-01T23:00:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0788142280 con 0x7f078805fb80 2022-08-01T23:00:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.051+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0778025030 con 0x7f078805fb80 2022-08-01T23:00:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.055+0000 7f077f7fe700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f076c053950 0x7f076c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.055+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f077806d140 con 0x7f078805fb80 2022-08-01T23:00:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.055+0000 7f0785d9b700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f076c053950 0x7f076c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.055+0000 7f0785d9b700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f076c053950 0x7f076c055e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f077000b000 tx=0x7f0770009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.059+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07780441e0 con 0x7f078805fb80 2022-08-01T23:00:49.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:49.195+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f07880afc80 con 0x7f078805fb80 2022-08-01T23:00:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.039+0000 7f077f7fe700 1 -- 172.21.15.32:0/2679182423 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0778023080 con 0x7f078805fb80 2022-08-01T23:00:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f076c053950 msgr2=0x7f076c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f076c053950 0x7f076c055e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f077000b000 tx=0x7f0770009040).stop 2022-08-01T23:00:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 msgr2=0x7f0788064470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f0788064470 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f0778002f70 tx=0x7f0778015000).stop 2022-08-01T23:00:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 shutdown_connections 2022-08-01T23:00:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f076c053950 0x7f076c055e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07880b2050 0x7f0788142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07880ae3c0 0x7f0788142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 --2- 172.21.15.32:0/2679182423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078805fb80 0x7f0788064470 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 >> 172.21.15.32:0/2679182423 conn(0x7f07880613c0 msgr2=0x7f07880b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 shutdown_connections 2022-08-01T23:00:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.043+0000 7f078759e700 1 -- 172.21.15.32:0/2679182423 wait complete. 2022-08-01T23:00:50.049 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:00:50.059 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:00:50.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:00:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.155+0000 7f9e2f175700 1 Processor -- start 2022-08-01T23:00:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.155+0000 7f9e2f175700 1 -- start start 2022-08-01T23:00:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e280b1bc0 0x7f9e2805ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2805f8e0 con 0x7f9e28140e40 2022-08-01T23:00:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e280b1630 con 0x7f9e280b1bc0 2022-08-01T23:00:50.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2805fb10 con 0x7f9e28066780 2022-08-01T23:00:50.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e974700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e28140e40 0x7f9e2813e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59744/0 (socket says 172.21.15.32:59744) 2022-08-01T23:00:50.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e974700 1 -- 172.21.15.32:0/3610529905 learned_addr learned my addr 172.21.15.32:0/3610529905 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:50.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d972700 1 --2- 172.21.15.32:0/3610529905 >> v2:172.21.15.55:3300/0 conn(0x7f9e280b1bc0 0x7f9e2805ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3610529905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d171700 1 -- 172.21.15.32:0/3610529905 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252782366 0 0) 0x7f9e2805f8e0 con 0x7f9e28140e40 2022-08-01T23:00:50.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d171700 1 -- 172.21.15.32:0/3610529905 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e10003260 con 0x7f9e28140e40 2022-08-01T23:00:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e173700 1 -- 172.21.15.32:0/3610529905 >> v2:172.21.15.55:3300/0 conn(0x7f9e280b1bc0 msgr2=0x7f9e2805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3610529905 >> v2:172.21.15.55:3300/0 conn(0x7f9e280b1bc0 0x7f9e2805ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e173700 1 -- 172.21.15.32:0/3610529905 >> v1:172.21.15.32:6789/0 conn(0x7f9e28140e40 legacy=0x7f9e2813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e173700 1 -- 172.21.15.32:0/3610529905 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e280604a0 con 0x7f9e28066780 2022-08-01T23:00:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d972700 1 --2- 172.21.15.32:0/3610529905 >> v2:172.21.15.55:3300/0 conn(0x7f9e280b1bc0 0x7f9e2805ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3610529905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28066b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9e24002f70 tx=0x7f9e24006df0).ready entity=mon.2 client_cookie=db0e549fa775c770 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d171700 1 -- 172.21.15.32:0/3610529905 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2400d4a0 con 0x7f9e28066780 2022-08-01T23:00:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d171700 1 -- 172.21.15.32:0/3610529905 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24000e90 con 0x7f9e28066780 2022-08-01T23:00:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2d171700 1 -- 172.21.15.32:0/3610529905 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2400d7f0 con 0x7f9e28066780 2022-08-01T23:00:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 -- 172.21.15.32:0/3610529905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 msgr2=0x7f9e28066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.159+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3610529905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28066b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9e24002f70 tx=0x7f9e24006df0).stop 2022-08-01T23:00:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- 172.21.15.32:0/3610529905 shutdown_connections 2022-08-01T23:00:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3610529905 >> v2:172.21.15.55:3300/0 conn(0x7f9e280b1bc0 0x7f9e2805ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3610529905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28066b60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- 172.21.15.32:0/3610529905 >> 172.21.15.32:0/3610529905 conn(0x7f9e28063870 msgr2=0x7f9e28063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- 172.21.15.32:0/3610529905 shutdown_connections 2022-08-01T23:00:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- 172.21.15.32:0/3610529905 wait complete. 2022-08-01T23:00:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 Processor -- start 2022-08-01T23:00:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- start start 2022-08-01T23:00:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28064aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 0x7f9e281527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e28140e40 0x7f9e28152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28152150 con 0x7f9e280b1bc0 2022-08-01T23:00:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280af450 con 0x7f9e28140e40 2022-08-01T23:00:50.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28142380 con 0x7f9e28066780 2022-08-01T23:00:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 0x7f9e281527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 0x7f9e281527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43974/0 (socket says 172.21.15.32:43974) 2022-08-01T23:00:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 -- 172.21.15.32:0/3974281776 learned_addr learned my addr 172.21.15.32:0/3974281776 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2e974700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e28140e40 0x7f9e28152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28064aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 -- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 msgr2=0x7f9e28064aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28064aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 -- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e28140e40 msgr2=0x7f9e28152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e28140e40 0x7f9e28152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 -- 172.21.15.32:0/3974281776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1800b180 con 0x7f9e280b1bc0 2022-08-01T23:00:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28064aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:00:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2d972700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 0x7f9e281527e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f9e1800c000 tx=0x7f9e18006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e1800ca40 con 0x7f9e280b1bc0 2022-08-01T23:00:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1800cba0 con 0x7f9e280b1bc0 2022-08-01T23:00:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2400b180 con 0x7f9e280b1bc0 2022-08-01T23:00:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e180123f0 con 0x7f9e280b1bc0 2022-08-01T23:00:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.163+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e28152340 con 0x7f9e280b1bc0 2022-08-01T23:00:50.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.167+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e28142820 con 0x7f9e280b1bc0 2022-08-01T23:00:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.167+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e18012550 con 0x7f9e280b1bc0 2022-08-01T23:00:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.167+0000 7f9e177fe700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e08053a70 0x7f9e08055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.167+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9e1806c490 con 0x7f9e280b1bc0 2022-08-01T23:00:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.167+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e08053a70 0x7f9e08055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.167+0000 7f9e2e173700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e08053a70 0x7f9e08055f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f9e24015000 tx=0x7f9e24014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.175+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e18043490 con 0x7f9e280b1bc0 2022-08-01T23:00:50.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.339+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9e281524d0 con 0x7f9e280b1bc0 2022-08-01T23:00:50.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.339+0000 7f9e177fe700 1 -- 172.21.15.32:0/3974281776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f9e18046ab0 con 0x7f9e280b1bc0 2022-08-01T23:00:50.341 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:50.341 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:00:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:00:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:00:50.342 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:00:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:00:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:00:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:00:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:00:50.345 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:00:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:00:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:00:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:00:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:00:50.349 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:00:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:00:50.351 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:00:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:00:50.353 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:00:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:00:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:00:50.357 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:00:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.358 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:00:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:00:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.360 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:00:50.361 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:00:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.362 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:00:50.363 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:00:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.364 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.365 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:00:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:00:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:00:50.366 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:00:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:00:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:00:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:00:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.368 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:00:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:00:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:00:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:00:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:00:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.370 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:00:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:00:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:00:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:00:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:00:50.372 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:00:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:00:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:00:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:00:50.374 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:00:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:00:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:00:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:00:50.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:50.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:00:50.376 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:00:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e08053a70 msgr2=0x7f9e08055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e08053a70 0x7f9e08055f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f9e24015000 tx=0x7f9e24014000).stop 2022-08-01T23:00:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 msgr2=0x7f9e281527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 0x7f9e281527e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f9e1800c000 tx=0x7f9e18006df0).stop 2022-08-01T23:00:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 shutdown_connections 2022-08-01T23:00:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e08053a70 0x7f9e08055f30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e28140e40 0x7f9e28152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280b1bc0 0x7f9e281527e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 --2- 172.21.15.32:0/3974281776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e28066780 0x7f9e28064aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 >> 172.21.15.32:0/3974281776 conn(0x7f9e28063870 msgr2=0x7f9e2813e310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 shutdown_connections 2022-08-01T23:00:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.343+0000 7f9e2f175700 1 -- 172.21.15.32:0/3974281776 wait complete. 2022-08-01T23:00:50.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:00:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f295cd38700 1 Processor -- start 2022-08-01T23:00:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f295cd38700 1 -- start start 2022-08-01T23:00:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f295cd38700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29581466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f295cd38700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29580aee80 con 0x7f29580614b0 2022-08-01T23:00:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f295cd38700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29580b70b0 con 0x7f29580ad750 2022-08-01T23:00:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f295cd38700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580af0b0 con 0x7f2958061890 2022-08-01T23:00:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f29577fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29580614b0 0x7f29580b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59754/0 (socket says 172.21.15.32:59754) 2022-08-01T23:00:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.487+0000 7f29577fe700 1 -- 172.21.15.32:0/1209251173 learned_addr learned my addr 172.21.15.32:0/1209251173 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f2957fff700 1 --2- 172.21.15.32:0/1209251173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29581466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542331219 0 0) 0x7f29580aee80 con 0x7f29580614b0 2022-08-01T23:00:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2938003260 con 0x7f29580614b0 2022-08-01T23:00:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16255817 0 0) 0x7f29580b70b0 con 0x7f29580ad750 2022-08-01T23:00:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29580aee80 con 0x7f29580ad750 2022-08-01T23:00:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f2957fff700 1 -- 172.21.15.32:0/1209251173 >> v1:172.21.15.55:6789/0 conn(0x7f29580ad750 legacy=0x7f29581405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3154217121 0 0) 0x7f2938003260 con 0x7f29580614b0 2022-08-01T23:00:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f2957fff700 1 -- 172.21.15.32:0/1209251173 >> v1:172.21.15.32:6789/0 conn(0x7f29580614b0 legacy=0x7f29580b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f2957fff700 1 -- 172.21.15.32:0/1209251173 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580af350 con 0x7f2958061890 2022-08-01T23:00:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f2957fff700 1 --2- 172.21.15.32:0/1209251173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29581466b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2948002f70 tx=0x7f2948006df0).ready entity=mon.2 client_cookie=ccafd2d4a3b9115f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f294800d5c0 con 0x7f2958061890 2022-08-01T23:00:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948000e90 con 0x7f2958061890 2022-08-01T23:00:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f29567fc700 1 -- 172.21.15.32:0/1209251173 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f294800d910 con 0x7f2958061890 2022-08-01T23:00:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 -- 172.21.15.32:0/1209251173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 msgr2=0x7f29581466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 --2- 172.21.15.32:0/1209251173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29581466b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2948002f70 tx=0x7f2948006df0).stop 2022-08-01T23:00:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 -- 172.21.15.32:0/1209251173 shutdown_connections 2022-08-01T23:00:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 --2- 172.21.15.32:0/1209251173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29581466b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 -- 172.21.15.32:0/1209251173 >> 172.21.15.32:0/1209251173 conn(0x7f2958066f70 msgr2=0x7f2958067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 -- 172.21.15.32:0/1209251173 shutdown_connections 2022-08-01T23:00:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.491+0000 7f295cd38700 1 -- 172.21.15.32:0/1209251173 wait complete. 2022-08-01T23:00:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 Processor -- start 2022-08-01T23:00:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 -- start start 2022-08-01T23:00:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29580614b0 0x7f2958146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29580b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 0x7f29580b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29580b3fb0 con 0x7f29580ad750 2022-08-01T23:00:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29580abbe0 con 0x7f29580614b0 2022-08-01T23:00:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580abe80 con 0x7f2958061890 2022-08-01T23:00:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 0x7f29580b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 0x7f29580b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43990/0 (socket says 172.21.15.32:43990) 2022-08-01T23:00:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 -- 172.21.15.32:0/3079022424 learned_addr learned my addr 172.21.15.32:0/3079022424 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2956ffd700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29580b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f29577fe700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29580614b0 0x7f2958146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 -- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 msgr2=0x7f29580b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29580b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 -- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29580614b0 msgr2=0x7f2958146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29580614b0 0x7f2958146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 -- 172.21.15.32:0/3079022424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294800b180 con 0x7f29580ad750 2022-08-01T23:00:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f29577fe700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29580614b0 0x7f2958146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:00:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2957fff700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 0x7f29580b0d50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f2948002f70 tx=0x7f2948014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f294800de70 con 0x7f29580ad750 2022-08-01T23:00:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294801e400 con 0x7f29580ad750 2022-08-01T23:00:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f294801e760 con 0x7f29580ad750 2022-08-01T23:00:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29580b3080 con 0x7f29580ad750 2022-08-01T23:00:50.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.495+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580ac120 con 0x7f29580ad750 2022-08-01T23:00:50.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.499+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29580b0af0 con 0x7f29580ad750 2022-08-01T23:00:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.499+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f294801c070 con 0x7f29580ad750 2022-08-01T23:00:50.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.499+0000 7f2954ff9700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f293c053950 0x7f293c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.499+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f294806c800 con 0x7f29580ad750 2022-08-01T23:00:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.499+0000 7f29577fe700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f293c053950 0x7f293c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.499+0000 7f29577fe700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f293c053950 0x7f293c055e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f2950008000 tx=0x7f2950009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.503+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29480437f0 con 0x7f29580ad750 2022-08-01T23:00:50.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.631+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29580574e0 con 0x7f29580ad750 2022-08-01T23:00:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.631+0000 7f2954ff9700 1 -- 172.21.15.32:0/3079022424 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f2948023020 con 0x7f29580ad750 2022-08-01T23:00:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:50.635 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":61,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:00:50.037312+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:00:50.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f293c053950 msgr2=0x7f293c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f293c053950 0x7f293c055e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f2950008000 tx=0x7f2950009040).stop 2022-08-01T23:00:50.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 msgr2=0x7f29580b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 0x7f29580b0d50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f2948002f70 tx=0x7f2948014000).stop 2022-08-01T23:00:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 shutdown_connections 2022-08-01T23:00:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f293c053950 0x7f293c055e10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29580ad750 0x7f29580b0d50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2958061890 0x7f29580b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 --2- 172.21.15.32:0/3079022424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29580614b0 0x7f2958146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 >> 172.21.15.32:0/3079022424 conn(0x7f2958066f70 msgr2=0x7f295805c820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 shutdown_connections 2022-08-01T23:00:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.635+0000 7f295cd38700 1 -- 172.21.15.32:0/3079022424 wait complete. 2022-08-01T23:00:50.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:00:50.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:00:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.755+0000 7f0eb81db700 1 Processor -- start 2022-08-01T23:00:50.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.755+0000 7f0eb81db700 1 -- start start 2022-08-01T23:00:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0eb0146ae0 0x7f0eb01403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0eb00b4310 0x7f0eb01408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb0143cf0 con 0x7f0eb00b4820 2022-08-01T23:00:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb01476a0 con 0x7f0eb00b4310 2022-08-01T23:00:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb0143f90 con 0x7f0eb0146ae0 2022-08-01T23:00:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0eb00b4310 0x7f0eb01408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb6778700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0eb00b4820 0x7f0eb00b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59770/0 (socket says 172.21.15.32:59770) 2022-08-01T23:00:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0eb00b4310 0x7f0eb01408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43996/0 (socket says 172.21.15.32:43996) 2022-08-01T23:00:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb6778700 1 -- 172.21.15.32:0/3364563129 learned_addr learned my addr 172.21.15.32:0/3364563129 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5f77700 1 --2- 172.21.15.32:0/3364563129 >> v2:172.21.15.55:3300/0 conn(0x7f0eb0146ae0 0x7f0eb01403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 -- 172.21.15.32:0/3364563129 >> v2:172.21.15.55:3300/0 conn(0x7f0eb0146ae0 msgr2=0x7f0eb01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 --2- 172.21.15.32:0/3364563129 >> v2:172.21.15.55:3300/0 conn(0x7f0eb0146ae0 0x7f0eb01403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb4f75700 1 -- 172.21.15.32:0/3364563129 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802357749 0 0) 0x7f0eb0143cf0 con 0x7f0eb00b4820 2022-08-01T23:00:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 -- 172.21.15.32:0/3364563129 >> v1:172.21.15.32:6789/0 conn(0x7f0eb00b4820 legacy=0x7f0eb00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 -- 172.21.15.32:0/3364563129 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00b0e20 con 0x7f0eb00b4310 2022-08-01T23:00:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb5776700 1 --2- 172.21.15.32:0/3364563129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb01408b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f0ea0002f70 tx=0x7f0ea0006df0).ready entity=mon.0 client_cookie=13c34457a26addb7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb4f75700 1 -- 172.21.15.32:0/3364563129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea000d570 con 0x7f0eb00b4310 2022-08-01T23:00:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb4f75700 1 -- 172.21.15.32:0/3364563129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0000e90 con 0x7f0eb00b4310 2022-08-01T23:00:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb4f75700 1 -- 172.21.15.32:0/3364563129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea000d8c0 con 0x7f0eb00b4310 2022-08-01T23:00:50.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- 172.21.15.32:0/3364563129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 msgr2=0x7f0eb01408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 --2- 172.21.15.32:0/3364563129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb01408b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f0ea0002f70 tx=0x7f0ea0006df0).stop 2022-08-01T23:00:50.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- 172.21.15.32:0/3364563129 shutdown_connections 2022-08-01T23:00:50.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 --2- 172.21.15.32:0/3364563129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb01408b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 --2- 172.21.15.32:0/3364563129 >> v2:172.21.15.55:3300/0 conn(0x7f0eb0146ae0 0x7f0eb01403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- 172.21.15.32:0/3364563129 >> 172.21.15.32:0/3364563129 conn(0x7f0eb005c620 msgr2=0x7f0eb005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- 172.21.15.32:0/3364563129 shutdown_connections 2022-08-01T23:00:50.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- 172.21.15.32:0/3364563129 wait complete. 2022-08-01T23:00:50.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 Processor -- start 2022-08-01T23:00:50.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.759+0000 7f0eb81db700 1 -- start start 2022-08-01T23:00:50.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb00b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb00b4820 0x7f0eb00b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0eb0146ae0 0x7f0eb01522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00b8ba0 con 0x7f0eb00b4310 2022-08-01T23:00:50.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0156280 con 0x7f0eb00b4820 2022-08-01T23:00:50.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00aecd0 con 0x7f0eb0146ae0 2022-08-01T23:00:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb00b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb00b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44000/0 (socket says 172.21.15.32:44000) 2022-08-01T23:00:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 -- 172.21.15.32:0/2587943260 learned_addr learned my addr 172.21.15.32:0/2587943260 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5776700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb00b4820 0x7f0eb00b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb6778700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0eb0146ae0 0x7f0eb01522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 -- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0eb0146ae0 msgr2=0x7f0eb01522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0eb0146ae0 0x7f0eb01522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 -- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb00b4820 msgr2=0x7f0eb00b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb00b4820 0x7f0eb00b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 -- 172.21.15.32:0/2587943260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea000b180 con 0x7f0eb00b4310 2022-08-01T23:00:50.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb5f77700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb00b81c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f0eac00c000 tx=0x7f0eac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac010780 con 0x7f0eb00b4310 2022-08-01T23:00:50.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac0108e0 con 0x7f0eb00b4310 2022-08-01T23:00:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac010c40 con 0x7f0eb00b4310 2022-08-01T23:00:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb00ad820 con 0x7f0eb00b4310 2022-08-01T23:00:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb01566e0 con 0x7f0eb00b4310 2022-08-01T23:00:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.763+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb00574e0 con 0x7f0eb00b4310 2022-08-01T23:00:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.767+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0eac01c460 con 0x7f0eb00b4310 2022-08-01T23:00:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.767+0000 7f0e9effd700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0e90053a70 0x7f0e90055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.767+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0eac06c340 con 0x7f0eb00b4310 2022-08-01T23:00:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.767+0000 7f0eb5776700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0e90053a70 0x7f0e90055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.767+0000 7f0eb5776700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0e90053a70 0x7f0e90055f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0ea0018000 tx=0x7f0ea0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.771+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0eac0432d0 con 0x7f0eb00b4310 2022-08-01T23:00:50.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.899+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0eb0156440 con 0x7f0eb00b4310 2022-08-01T23:00:50.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0e9effd700 1 -- 172.21.15.32:0/2587943260 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v61) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0eac0468f0 con 0x7f0eb00b4310 2022-08-01T23:00:50.903 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:00:50.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0e90053a70 msgr2=0x7f0e90055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0e90053a70 0x7f0e90055f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0ea0018000 tx=0x7f0ea0014000).stop 2022-08-01T23:00:50.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 msgr2=0x7f0eb00b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:50.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb00b81c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f0eac00c000 tx=0x7f0eac006df0).stop 2022-08-01T23:00:50.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 shutdown_connections 2022-08-01T23:00:50.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0e90053a70 0x7f0e90055f30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0eb0146ae0 0x7f0eb01522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb00b4820 0x7f0eb00b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 --2- 172.21.15.32:0/2587943260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eb00b4310 0x7f0eb00b81c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:50.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 >> 172.21.15.32:0/2587943260 conn(0x7f0eb005c620 msgr2=0x7f0eb013b310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:50.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 shutdown_connections 2022-08-01T23:00:50.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:50.903+0000 7f0eb81db700 1 -- 172.21.15.32:0/2587943260 wait complete. 2022-08-01T23:00:50.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:00:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.019+0000 7f9ea488d700 1 Processor -- start 2022-08-01T23:00:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.019+0000 7f9ea488d700 1 -- start start 2022-08-01T23:00:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ea01405d0 0x7f9ea0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00aee80 con 0x7f9ea00614b0 2022-08-01T23:00:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00b70b0 con 0x7f9ea01405d0 2022-08-01T23:00:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00af0b0 con 0x7f9ea00ad750 2022-08-01T23:00:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ea01405d0 0x7f9ea0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ea01405d0 0x7f9ea0140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53796/0 (socket says 172.21.15.32:53796) 2022-08-01T23:00:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3556154234 learned_addr learned my addr 172.21.15.32:0/3556154234 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55748/0 (socket says 172.21.15.32:55748) 2022-08-01T23:00:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9ed9d700 1 --1- 172.21.15.32:0/3556154234 >> v1:172.21.15.55:6789/0 conn(0x7f9ea00614b0 0x7f9ea00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52456/0 (socket says 172.21.15.32:52456) 2022-08-01T23:00:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 msgr2=0x7f9ea0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 --2- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3556154234 >> v1:172.21.15.55:6789/0 conn(0x7f9ea00614b0 legacy=0x7f9ea00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3556154234 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00af350 con 0x7f9ea01405d0 2022-08-01T23:00:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9e59c700 1 --2- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9f59e700 1 --2- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea01405d0 0x7f9ea0140a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f9e980032f0 tx=0x7f9e98009290).ready entity=mon.1 client_cookie=81e600463ea23705 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9dd9b700 1 -- 172.21.15.32:0/3556154234 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9800b8a0 con 0x7f9ea01405d0 2022-08-01T23:00:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9dd9b700 1 -- 172.21.15.32:0/3556154234 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9800ba00 con 0x7f9ea01405d0 2022-08-01T23:00:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9e9dd9b700 1 -- 172.21.15.32:0/3556154234 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9800bb70 con 0x7f9ea01405d0 2022-08-01T23:00:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea01405d0 msgr2=0x7f9ea0140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea01405d0 0x7f9ea0140a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f9e980032f0 tx=0x7f9e98009290).stop 2022-08-01T23:00:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- 172.21.15.32:0/3556154234 shutdown_connections 2022-08-01T23:00:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea01405d0 0x7f9ea0140a80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3556154234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- 172.21.15.32:0/3556154234 >> 172.21.15.32:0/3556154234 conn(0x7f9ea0066f70 msgr2=0x7f9ea0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- 172.21.15.32:0/3556154234 shutdown_connections 2022-08-01T23:00:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.023+0000 7f9ea488d700 1 -- 172.21.15.32:0/3556154234 wait complete. 2022-08-01T23:00:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 Processor -- start 2022-08-01T23:00:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 -- start start 2022-08-01T23:00:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea00614b0 0x7f9ea00a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 0x7f9ea01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00ad0d0 con 0x7f9ea01405d0 2022-08-01T23:00:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01565e0 con 0x7f9ea00614b0 2022-08-01T23:00:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9ea488d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0156880 con 0x7f9ea00ad750 2022-08-01T23:00:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 0x7f9ea01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea00614b0 0x7f9ea00a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 0x7f9ea01524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44018/0 (socket says 172.21.15.32:44018) 2022-08-01T23:00:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3628254625 learned_addr learned my addr 172.21.15.32:0/3628254625 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9e59c700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 msgr2=0x7f9ea0143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea00614b0 msgr2=0x7f9ea00a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea00614b0 0x7f9ea00a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 -- 172.21.15.32:0/3628254625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8800b180 con 0x7f9ea01405d0 2022-08-01T23:00:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.027+0000 7f9e9f59e700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 0x7f9ea01524f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f9e980032f0 tx=0x7f9e98016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9800b8a0 con 0x7f9ea01405d0 2022-08-01T23:00:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98021440 con 0x7f9ea01405d0 2022-08-01T23:00:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e98009530 con 0x7f9ea01405d0 2022-08-01T23:00:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e980217a0 con 0x7f9ea01405d0 2022-08-01T23:00:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea00acd50 con 0x7f9ea01405d0 2022-08-01T23:00:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea00af5f0 con 0x7f9ea01405d0 2022-08-01T23:00:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e98027020 con 0x7f9ea01405d0 2022-08-01T23:00:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9e97fff700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e84053950 0x7f9e84055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.031+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9e9806d1b0 con 0x7f9ea01405d0 2022-08-01T23:00:51.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.035+0000 7f9e9ed9d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e84053950 0x7f9e84055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.035+0000 7f9e9ed9d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e84053950 0x7f9e84055e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f9e9000d000 tx=0x7f9e90007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.039+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e98044290 con 0x7f9ea01405d0 2022-08-01T23:00:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ea00574e0 con 0x7f9ea01405d0 2022-08-01T23:00:51.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9e97fff700 1 -- 172.21.15.32:0/3628254625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9e98015020 con 0x7f9ea01405d0 2022-08-01T23:00:51.169 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:00:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e84053950 msgr2=0x7f9e84055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e84053950 0x7f9e84055e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f9e9000d000 tx=0x7f9e90007b80).stop 2022-08-01T23:00:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 msgr2=0x7f9ea01524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 0x7f9ea01524f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f9e980032f0 tx=0x7f9e98016000).stop 2022-08-01T23:00:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 shutdown_connections 2022-08-01T23:00:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9e84053950 0x7f9e84055e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea01405d0 0x7f9ea01524f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ea00ad750 0x7f9ea0143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 --2- 172.21.15.32:0/3628254625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea00614b0 0x7f9ea00a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 >> 172.21.15.32:0/3628254625 conn(0x7f9ea0066f70 msgr2=0x7f9ea00b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.167+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 shutdown_connections 2022-08-01T23:00:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.171+0000 7f9ea488d700 1 -- 172.21.15.32:0/3628254625 wait complete. 2022-08-01T23:00:51.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:00:51.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 Processor -- start 2022-08-01T23:00:51.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- start start 2022-08-01T23:00:51.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5a40b6ee0 0x7fc5a40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a4061d90 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40ba5a0 con 0x7fc5a40619b0 2022-08-01T23:00:51.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a4143fa0 con 0x7fc5a40b6ee0 2022-08-01T23:00:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5a40b6ee0 0x7fc5a40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5a40b6ee0 0x7fc5a40b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44024/0 (socket says 172.21.15.32:44024) 2022-08-01T23:00:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5aac71700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5a40b5ed0 0x7fc5a40b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59800/0 (socket says 172.21.15.32:59800) 2022-08-01T23:00:51.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/717554521 learned_addr learned my addr 172.21.15.32:0/717554521 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:51.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a946e700 1 -- 172.21.15.32:0/717554521 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631243551 0 0) 0x7fc5a40ba5a0 con 0x7fc5a40619b0 2022-08-01T23:00:51.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a946e700 1 -- 172.21.15.32:0/717554521 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc584003260 con 0x7fc5a40619b0 2022-08-01T23:00:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/717554521 >> v1:172.21.15.55:6789/0 conn(0x7fc5a40619b0 legacy=0x7fc5a4143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/717554521 >> v1:172.21.15.32:6789/0 conn(0x7fc5a40b5ed0 legacy=0x7fc5a40b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/717554521 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4143150 con 0x7fc5a40b6ee0 2022-08-01T23:00:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a9c6f700 1 --2- 172.21.15.32:0/717554521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40b48a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c006eb0).ready entity=mon.0 client_cookie=d6c894a6ba0fda14 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a946e700 1 -- 172.21.15.32:0/717554521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00d610 con 0x7fc5a40b6ee0 2022-08-01T23:00:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a946e700 1 -- 172.21.15.32:0/717554521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c000e90 con 0x7fc5a40b6ee0 2022-08-01T23:00:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5a946e700 1 -- 172.21.15.32:0/717554521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00d960 con 0x7fc5a40b6ee0 2022-08-01T23:00:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- 172.21.15.32:0/717554521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b6ee0 msgr2=0x7fc5a40b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 --2- 172.21.15.32:0/717554521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40b48a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c006eb0).stop 2022-08-01T23:00:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- 172.21.15.32:0/717554521 shutdown_connections 2022-08-01T23:00:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 --2- 172.21.15.32:0/717554521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40b48a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.283+0000 7fc5ab472700 1 -- 172.21.15.32:0/717554521 >> 172.21.15.32:0/717554521 conn(0x7fc5a406af80 msgr2=0x7fc5a405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 -- 172.21.15.32:0/717554521 shutdown_connections 2022-08-01T23:00:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 -- 172.21.15.32:0/717554521 wait complete. 2022-08-01T23:00:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 Processor -- start 2022-08-01T23:00:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 -- start start 2022-08-01T23:00:51.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5a40619b0 0x7fc5a413c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 0x7fc5a40b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b79a0 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b8ef0 con 0x7fc5a40b6ee0 2022-08-01T23:00:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5ab472700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40ad910 con 0x7fc5a40619b0 2022-08-01T23:00:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 0x7fc5a40b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 0x7fc5a40b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44032/0 (socket says 172.21.15.32:44032) 2022-08-01T23:00:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/1675569598 learned_addr learned my addr 172.21.15.32:0/1675569598 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5aac71700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5aa470700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5a40619b0 0x7fc5a413c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5a40619b0 msgr2=0x7fc5a413c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5a40619b0 0x7fc5a413c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a40b6ee0 msgr2=0x7fc5a40ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 -- 172.21.15.32:0/1675569598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c00b1e0 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.287+0000 7fc5a9c6f700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 0x7fc5a40b8710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00d990 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c011b40 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a40b9120 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c011ea0 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a40adb70 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a40b9b40 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc58c025030 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc588053a70 0x7fc588055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc58c06ce20 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.291+0000 7fc5aa470700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc588053a70 0x7fc588055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.295+0000 7fc5aa470700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc588053a70 0x7fc588055f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc59c00d000 tx=0x7fc59c007da0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.299+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc58c043f60 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.427+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc5a40574e0 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.427+0000 7fc5a37fe700 1 -- 172.21.15.32:0/1675569598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc58c047540 con 0x7fc5a40b5ed0 2022-08-01T23:00:51.431 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:51.431 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:00:51.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:51.432 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:00:51.432 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:00:51.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:00:51.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:00:51.433 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:00:51.433 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:00:51.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:00:51.433 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:00:51.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc588053a70 msgr2=0x7fc588055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc588053a70 0x7fc588055f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc59c00d000 tx=0x7fc59c007da0).stop 2022-08-01T23:00:51.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 msgr2=0x7fc5a40b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 0x7fc5a40b8710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c015000).stop 2022-08-01T23:00:51.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 shutdown_connections 2022-08-01T23:00:51.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc588053a70 0x7fc588055f30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a40b6ee0 0x7fc5a40ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40b5ed0 0x7fc5a40b8710 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 --2- 172.21.15.32:0/1675569598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc5a40619b0 0x7fc5a413c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 >> 172.21.15.32:0/1675569598 conn(0x7fc5a406af80 msgr2=0x7fc5a405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:51.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 shutdown_connections 2022-08-01T23:00:51.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.431+0000 7fc5ab472700 1 -- 172.21.15.32:0/1675569598 wait complete. 2022-08-01T23:00:51.449 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:00:51.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:00:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.555+0000 7f7ee1aa5700 1 Processor -- start 2022-08-01T23:00:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.555+0000 7f7ee1aa5700 1 -- start start 2022-08-01T23:00:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee1aa5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc1489f0 0x7f7edc14ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee1aa5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc067030 con 0x7f7edc0b31d0 2022-08-01T23:00:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee1aa5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0b35b0 con 0x7f7edc0a9da0 2022-08-01T23:00:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee1aa5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc064a70 con 0x7f7edc1489f0 2022-08-01T23:00:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee0aa3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7edc0b31d0 0x7f7edc066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59810/0 (socket says 172.21.15.32:59810) 2022-08-01T23:00:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee0aa3700 1 -- 172.21.15.32:0/2591532260 learned_addr learned my addr 172.21.15.32:0/2591532260 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:51.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee12a4700 1 --2- 172.21.15.32:0/2591532260 >> v2:172.21.15.32:3300/0 conn(0x7f7edc1489f0 0x7f7edc14ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2591532260 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62452724 0 0) 0x7f7edc067030 con 0x7f7edc0b31d0 2022-08-01T23:00:51.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2591532260 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec8003260 con 0x7f7edc0b31d0 2022-08-01T23:00:51.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee12a4700 1 -- 172.21.15.32:0/2591532260 >> v1:172.21.15.55:6789/0 conn(0x7f7edc0a9da0 legacy=0x7f7edc060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee12a4700 1 -- 172.21.15.32:0/2591532260 >> v1:172.21.15.32:6789/0 conn(0x7f7edc0b31d0 legacy=0x7f7edc066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2591532260 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769197205 0 0) 0x7f7edc0b35b0 con 0x7f7edc0a9da0 2022-08-01T23:00:51.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee12a4700 1 -- 172.21.15.32:0/2591532260 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc066af0 con 0x7f7edc1489f0 2022-08-01T23:00:51.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7ee12a4700 1 --2- 172.21.15.32:0/2591532260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc1489f0 0x7f7edc14ade0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f7ed40032f0 tx=0x7f7ed40091d0).ready entity=mon.0 client_cookie=9cfee65ed193a10b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2591532260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed400b6c0 con 0x7f7edc1489f0 2022-08-01T23:00:51.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2591532260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed400b820 con 0x7f7edc1489f0 2022-08-01T23:00:51.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.559+0000 7f7edb7fe700 1 -- 172.21.15.32:0/2591532260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed400b990 con 0x7f7edc1489f0 2022-08-01T23:00:51.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2591532260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc1489f0 msgr2=0x7f7edc14ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2591532260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc1489f0 0x7f7edc14ade0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f7ed40032f0 tx=0x7f7ed40091d0).stop 2022-08-01T23:00:51.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2591532260 shutdown_connections 2022-08-01T23:00:51.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2591532260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc1489f0 0x7f7edc14ade0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2591532260 >> 172.21.15.32:0/2591532260 conn(0x7f7edc063870 msgr2=0x7f7edc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:51.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2591532260 shutdown_connections 2022-08-01T23:00:51.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2591532260 wait complete. 2022-08-01T23:00:51.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 Processor -- start 2022-08-01T23:00:51.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- start start 2022-08-01T23:00:51.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0a9da0 0x7f7edc0abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 0x7f7edc153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7edc1489f0 0x7f7edc153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc1533d0 con 0x7f7edc0b31d0 2022-08-01T23:00:51.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc1470e0 con 0x7f7edc0a9da0 2022-08-01T23:00:51.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc1462a0 con 0x7f7edc1489f0 2022-08-01T23:00:51.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 0x7f7edc153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 0x7f7edc153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44046/0 (socket says 172.21.15.32:44046) 2022-08-01T23:00:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 -- 172.21.15.32:0/2162269865 learned_addr learned my addr 172.21.15.32:0/2162269865 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee12a4700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7edc1489f0 0x7f7edc153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee0aa3700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0a9da0 0x7f7edc0abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 -- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7edc1489f0 msgr2=0x7f7edc153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7edc1489f0 0x7f7edc153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 -- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0a9da0 msgr2=0x7f7edc0abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0a9da0 0x7f7edc0abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 -- 172.21.15.32:0/2162269865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4009470 con 0x7f7edc0b31d0 2022-08-01T23:00:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7edbfff700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 0x7f7edc153920 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f7edc0aab90 tx=0x7f7ed0007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed0012400 con 0x7f7edc0b31d0 2022-08-01T23:00:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0012560 con 0x7f7edc0b31d0 2022-08-01T23:00:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed00128c0 con 0x7f7edc0b31d0 2022-08-01T23:00:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc146500 con 0x7f7edc0b31d0 2022-08-01T23:00:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.563+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc145d50 con 0x7f7edc0b31d0 2022-08-01T23:00:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.567+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc066d90 con 0x7f7edc0b31d0 2022-08-01T23:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.567+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ed0019030 con 0x7f7edc0b31d0 2022-08-01T23:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.567+0000 7f7ed9ffb700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ebc053a70 0x7f7ebc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.567+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7ed006af30 con 0x7f7edc0b31d0 2022-08-01T23:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.567+0000 7f7ee0aa3700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ebc053a70 0x7f7ebc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.571+0000 7f7ee0aa3700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ebc053a70 0x7f7ebc055f30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f7ecc00d000 tx=0x7f7ecc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:51.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.575+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ed00420e0 con 0x7f7edc0b31d0 2022-08-01T23:00:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:51.699+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f7edc0574e0 con 0x7f7edc0b31d0 2022-08-01T23:00:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.167+0000 7f7ed9ffb700 1 -- 172.21.15.32:0/2162269865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v62) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7ed0045700 con 0x7f7edc0b31d0 2022-08-01T23:00:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ebc053a70 msgr2=0x7f7ebc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ebc053a70 0x7f7ebc055f30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f7ecc00d000 tx=0x7f7ecc009040).stop 2022-08-01T23:00:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 msgr2=0x7f7edc153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 0x7f7edc153920 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f7edc0aab90 tx=0x7f7ed0007a90).stop 2022-08-01T23:00:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 shutdown_connections 2022-08-01T23:00:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ebc053a70 0x7f7ebc055f30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7edc1489f0 0x7f7edc153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0b31d0 0x7f7edc153920 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 --2- 172.21.15.32:0/2162269865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0a9da0 0x7f7edc0abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 >> 172.21.15.32:0/2162269865 conn(0x7f7edc063870 msgr2=0x7f7edc0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 shutdown_connections 2022-08-01T23:00:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.171+0000 7f7ee1aa5700 1 -- 172.21.15.32:0/2162269865 wait complete. 2022-08-01T23:00:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:00:52.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:00:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.303+0000 7f3eed530700 1 Processor -- start 2022-08-01T23:00:52.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.303+0000 7f3eed530700 1 -- start start 2022-08-01T23:00:52.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.303+0000 7f3eed530700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3ee8066b60 0x7f3ee813e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.303+0000 7f3eed530700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee805f8e0 con 0x7f3ee8066780 2022-08-01T23:00:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.303+0000 7f3eed530700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80b1630 con 0x7f3ee80b1bc0 2022-08-01T23:00:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee805fb10 con 0x7f3ee8066b60 2022-08-01T23:00:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ee8066780 0x7f3ee805ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59824/0 (socket says 172.21.15.32:59824) 2022-08-01T23:00:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 -- 172.21.15.32:0/2714696491 learned_addr learned my addr 172.21.15.32:0/2714696491 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eecd2f700 1 --2- 172.21.15.32:0/2714696491 >> v2:172.21.15.55:3300/0 conn(0x7f3ee8066b60 0x7f3ee813e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941247837 0 0) 0x7f3ee805f8e0 con 0x7f3ee8066780 2022-08-01T23:00:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed4003260 con 0x7f3ee8066780 2022-08-01T23:00:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731305041 0 0) 0x7f3ed4003260 con 0x7f3ee8066780 2022-08-01T23:00:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 >> v2:172.21.15.55:3300/0 conn(0x7f3ee8066b60 msgr2=0x7f3ee813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 --2- 172.21.15.32:0/2714696491 >> v2:172.21.15.55:3300/0 conn(0x7f3ee8066b60 0x7f3ee813e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 >> v1:172.21.15.55:6789/0 conn(0x7f3ee80b1bc0 legacy=0x7f3ee8140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80b18d0 con 0x7f3ee8066780 2022-08-01T23:00:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f3ed8001660 con 0x7f3ee8066780 2022-08-01T23:00:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 >> v1:172.21.15.32:6789/0 conn(0x7f3ee8066780 legacy=0x7f3ee805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ed4003750 0x7f3ed4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4004500 0x7f3ed4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ed4004e70 0x7f3ed400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee805fb10 con 0x7f3ed4003750 2022-08-01T23:00:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed400bd20 con 0x7f3ed4004500 2022-08-01T23:00:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed400bee0 con 0x7f3ed4004e70 2022-08-01T23:00:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ed4003750 0x7f3ed4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee77fe700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4004500 0x7f3ed4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eecd2f700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ed4004e70 0x7f3ed400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 -- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ed4004e70 msgr2=0x7f3ed400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ed4004e70 0x7f3ed400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 -- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4004500 msgr2=0x7f3ed4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4004500 0x7f3ed4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eecd2f700 1 -- 172.21.15.32:0/2714696491 reap_dead start 2022-08-01T23:00:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 -- 172.21.15.32:0/2714696491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4003260 con 0x7f3ed4003750 2022-08-01T23:00:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ed4003750 0x7f3ed4003b70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f3ed8001370 tx=0x7f3ee80610c0).ready entity=mon.0 client_cookie=769f89576d673bb7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ed8013070 con 0x7f3ed4003750 2022-08-01T23:00:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed800e300 con 0x7f3ed4003750 2022-08-01T23:00:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3ee6ffd700 1 -- 172.21.15.32:0/2714696491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ed800edf0 con 0x7f3ed4003750 2022-08-01T23:00:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 -- 172.21.15.32:0/2714696491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ed4003750 msgr2=0x7f3ed4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ed4003750 0x7f3ed4003b70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f3ed8001370 tx=0x7f3ee80610c0).stop 2022-08-01T23:00:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 -- 172.21.15.32:0/2714696491 shutdown_connections 2022-08-01T23:00:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 --2- 172.21.15.32:0/2714696491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ed4003750 0x7f3ed4003b70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 -- 172.21.15.32:0/2714696491 >> 172.21.15.32:0/2714696491 conn(0x7f3ee8063870 msgr2=0x7f3ee8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 -- 172.21.15.32:0/2714696491 shutdown_connections 2022-08-01T23:00:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.307+0000 7f3eed530700 1 -- 172.21.15.32:0/2714696491 wait complete. 2022-08-01T23:00:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 Processor -- start 2022-08-01T23:00:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 -- start start 2022-08-01T23:00:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4003750 0x7f3ee8139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 0x7f3ee81513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ee8140fc0 0x7f3ee81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee814eba0 con 0x7f3ee8144d20 2022-08-01T23:00:52.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee81445e0 con 0x7f3ed4003750 2022-08-01T23:00:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee8144880 con 0x7f3ee8140fc0 2022-08-01T23:00:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 0x7f3ee81513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 0x7f3ee81513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44062/0 (socket says 172.21.15.32:44062) 2022-08-01T23:00:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 -- 172.21.15.32:0/1024274880 learned_addr learned my addr 172.21.15.32:0/1024274880 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4003750 0x7f3ee8139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eecd2f700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ee8140fc0 0x7f3ee81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 -- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ee8140fc0 msgr2=0x7f3ee81518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ee8140fc0 0x7f3ee81518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 -- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4003750 msgr2=0x7f3ee8139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4003750 0x7f3ee8139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 -- 172.21.15.32:0/1024274880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee814e8d0 con 0x7f3ee8144d20 2022-08-01T23:00:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee77fe700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 0x7f3ee81513d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f3edc00d000 tx=0x7f3edc007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3edc0133f0 con 0x7f3ee8144d20 2022-08-01T23:00:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc013550 con 0x7f3ee8144d20 2022-08-01T23:00:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee813a030 con 0x7f3ee8144d20 2022-08-01T23:00:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3edc0138b0 con 0x7f3ee8144d20 2022-08-01T23:00:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.311+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee813e850 con 0x7f3ee8144d20 2022-08-01T23:00:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.315+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee80b12b0 con 0x7f3ee8144d20 2022-08-01T23:00:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.315+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3edc01a030 con 0x7f3ee8144d20 2022-08-01T23:00:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.315+0000 7f3ee57fa700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ec80539e0 0x7f3ec8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.315+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f3edc06c060 con 0x7f3ee8144d20 2022-08-01T23:00:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.315+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ec80539e0 0x7f3ec8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:52.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.315+0000 7f3ee7fff700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ec80539e0 0x7f3ec8055ea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f3ed8020000 tx=0x7f3ed8019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:52.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.319+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3edc042f90 con 0x7f3ee8144d20 2022-08-01T23:00:52.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:52.447+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f3ee80574e0 con 0x7f3ee8144d20 2022-08-01T23:00:53.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.039+0000 7f3ee57fa700 1 -- 172.21.15.32:0/1024274880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v63) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3edc022070 con 0x7f3ee8144d20 2022-08-01T23:00:53.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.043+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ec80539e0 msgr2=0x7f3ec8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.043+0000 7f3eed530700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ec80539e0 0x7f3ec8055ea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f3ed8020000 tx=0x7f3ed8019000).stop 2022-08-01T23:00:53.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.043+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 msgr2=0x7f3ee81513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.043+0000 7f3eed530700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 0x7f3ee81513d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f3edc00d000 tx=0x7f3edc007a00).stop 2022-08-01T23:00:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 shutdown_connections 2022-08-01T23:00:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ec80539e0 0x7f3ec8055ea0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3ee8140fc0 0x7f3ee81518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee8144d20 0x7f3ee81513d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 --2- 172.21.15.32:0/1024274880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ed4003750 0x7f3ee8139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 >> 172.21.15.32:0/1024274880 conn(0x7f3ee8063870 msgr2=0x7f3ee813a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 shutdown_connections 2022-08-01T23:00:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.047+0000 7f3eed530700 1 -- 172.21.15.32:0/1024274880 wait complete. 2022-08-01T23:00:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:00:53.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:00:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 Processor -- start 2022-08-01T23:00:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 -- start start 2022-08-01T23:00:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63d00b08a0 0x7f63d0142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d01433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f63d00ad000 0x7f63d005f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d00b0c80 con 0x7f63d00b08a0 2022-08-01T23:00:53.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d00ae810 con 0x7f63d00ad000 2022-08-01T23:00:53.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d596f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00aeab0 con 0x7f63d00b3260 2022-08-01T23:00:53.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63d00b08a0 0x7f63d0142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63d00b08a0 0x7f63d0142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44070/0 (socket says 172.21.15.32:44070) 2022-08-01T23:00:53.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 -- 172.21.15.32:0/1764544772 learned_addr learned my addr 172.21.15.32:0/1764544772 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:53.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d516e700 1 --2- 172.21.15.32:0/1764544772 >> v2:172.21.15.55:3300/0 conn(0x7f63d00ad000 0x7f63d005f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63cffff700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d01433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 -- 172.21.15.32:0/1764544772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 msgr2=0x7f63d01433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:53.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d01433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 -- 172.21.15.32:0/1764544772 >> v2:172.21.15.55:3300/0 conn(0x7f63d00ad000 msgr2=0x7f63d005f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:53.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 --2- 172.21.15.32:0/1764544772 >> v2:172.21.15.55:3300/0 conn(0x7f63d00ad000 0x7f63d005f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 -- 172.21.15.32:0/1764544772 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00aed50 con 0x7f63d00b08a0 2022-08-01T23:00:53.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63cffff700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d01433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:53.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63d496d700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00b08a0 0x7f63d0142eb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f63c4002f70 tx=0x7f63c4006f40).ready entity=mon.0 client_cookie=fddbd8fc6199c505 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63cf7fe700 1 -- 172.21.15.32:0/1764544772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63c400da80 con 0x7f63d00b08a0 2022-08-01T23:00:53.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63cf7fe700 1 -- 172.21.15.32:0/1764544772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4000e90 con 0x7f63d00b08a0 2022-08-01T23:00:53.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.171+0000 7f63cf7fe700 1 -- 172.21.15.32:0/1764544772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63c400de10 con 0x7f63d00b08a0 2022-08-01T23:00:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 -- 172.21.15.32:0/1764544772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00b08a0 msgr2=0x7f63d0142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00b08a0 0x7f63d0142eb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f63c4002f70 tx=0x7f63c4006f40).stop 2022-08-01T23:00:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 -- 172.21.15.32:0/1764544772 shutdown_connections 2022-08-01T23:00:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 --2- 172.21.15.32:0/1764544772 >> v2:172.21.15.55:3300/0 conn(0x7f63d00ad000 0x7f63d005f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d01433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 --2- 172.21.15.32:0/1764544772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00b08a0 0x7f63d0142eb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 -- 172.21.15.32:0/1764544772 >> 172.21.15.32:0/1764544772 conn(0x7f63d00612d0 msgr2=0x7f63d0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 -- 172.21.15.32:0/1764544772 shutdown_connections 2022-08-01T23:00:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 -- 172.21.15.32:0/1764544772 wait complete. 2022-08-01T23:00:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 Processor -- start 2022-08-01T23:00:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.175+0000 7f63d596f700 1 -- start start 2022-08-01T23:00:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 0x7f63d00b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d00b08a0 0x7f63d014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d009efd0 con 0x7f63d00ad000 2022-08-01T23:00:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0152200 con 0x7f63d00b08a0 2022-08-01T23:00:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0066120 con 0x7f63d00b3260 2022-08-01T23:00:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 0x7f63d00b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 0x7f63d00b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44076/0 (socket says 172.21.15.32:44076) 2022-08-01T23:00:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 -- 172.21.15.32:0/3142650779 learned_addr learned my addr 172.21.15.32:0/3142650779 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63cffff700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d00b08a0 0x7f63d014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d516e700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 -- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 msgr2=0x7f63d014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 -- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d00b08a0 msgr2=0x7f63d014e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d00b08a0 0x7f63d014e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 -- 172.21.15.32:0/3142650779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c400b460 con 0x7f63d00ad000 2022-08-01T23:00:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d496d700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 0x7f63d00b3b00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f63c4002f70 tx=0x7f63d00a95b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63c400de40 con 0x7f63d00ad000 2022-08-01T23:00:53.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4006c00 con 0x7f63d00ad000 2022-08-01T23:00:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d0066c70 con 0x7f63d00ad000 2022-08-01T23:00:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63c40245f0 con 0x7f63d00ad000 2022-08-01T23:00:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d014eec0 con 0x7f63d00ad000 2022-08-01T23:00:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.179+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d0152580 con 0x7f63d00ad000 2022-08-01T23:00:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.183+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f63c4021020 con 0x7f63d00ad000 2022-08-01T23:00:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.183+0000 7f63cdffb700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63b0053aa0 0x7f63b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.183+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f63c406fbd0 con 0x7f63d00ad000 2022-08-01T23:00:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.183+0000 7f63cffff700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63b0053aa0 0x7f63b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.183+0000 7f63cffff700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63b0053aa0 0x7f63b0055f60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f63bc00c000 tx=0x7f63bc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.191+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f63c40468a0 con 0x7f63d00ad000 2022-08-01T23:00:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.359+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f63d0152050 con 0x7f63d00ad000 2022-08-01T23:00:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.367+0000 7f63cdffb700 1 -- 172.21.15.32:0/3142650779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v27) v1 ==== 189+0+0 (secure 0 0 0) 0x7f63c4029030 con 0x7f63d00ad000 2022-08-01T23:00:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63b0053aa0 msgr2=0x7f63b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63b0053aa0 0x7f63b0055f60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f63bc00c000 tx=0x7f63bc008040).stop 2022-08-01T23:00:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 msgr2=0x7f63d00b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 0x7f63d00b3b00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f63c4002f70 tx=0x7f63d00a95b0).stop 2022-08-01T23:00:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 shutdown_connections 2022-08-01T23:00:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f63b0053aa0 0x7f63b0055f60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f63d00b3260 0x7f63d014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.371+0000 7f63d596f700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d00b08a0 0x7f63d014e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.375+0000 7f63d596f700 1 --2- 172.21.15.32:0/3142650779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63d00ad000 0x7f63d00b3b00 secure :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0x7f63c4002f70 tx=0x7f63d00a95b0).stop 2022-08-01T23:00:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.375+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 >> 172.21.15.32:0/3142650779 conn(0x7f63d00612d0 msgr2=0x7f63d00b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.375+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 shutdown_connections 2022-08-01T23:00:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.375+0000 7f63d596f700 1 -- 172.21.15.32:0/3142650779 wait complete. 2022-08-01T23:00:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 11 and data pool 12 2022-08-01T23:00:53.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:00:53.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 Processor -- start 2022-08-01T23:00:53.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 -- start start 2022-08-01T23:00:53.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa1300b4550 0x7fa1300b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1300b6af0 0x7fa13012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa130144240 con 0x7fa1300b6710 2022-08-01T23:00:53.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa130143d00 con 0x7fa1300b6af0 2022-08-01T23:00:53.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.479+0000 7fa1373b4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa130143fa0 con 0x7fa1300b4550 2022-08-01T23:00:53.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1300b6af0 0x7fa13012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1300b6af0 0x7fa13012bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44090/0 (socket says 172.21.15.32:44090) 2022-08-01T23:00:53.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 -- 172.21.15.32:0/1295274099 learned_addr learned my addr 172.21.15.32:0/1295274099 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:53.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa13494f700 1 --2- 172.21.15.32:0/1295274099 >> v2:172.21.15.55:3300/0 conn(0x7fa1300b4550 0x7fa1300b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 -- 172.21.15.32:0/1295274099 >> v2:172.21.15.55:3300/0 conn(0x7fa1300b4550 msgr2=0x7fa1300b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:53.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 --2- 172.21.15.32:0/1295274099 >> v2:172.21.15.55:3300/0 conn(0x7fa1300b4550 0x7fa1300b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 -- 172.21.15.32:0/1295274099 >> v1:172.21.15.55:6789/0 conn(0x7fa1300b6710 legacy=0x7fa130061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 -- 172.21.15.32:0/1295274099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa130061780 con 0x7fa1300b6af0 2022-08-01T23:00:53.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa135951700 1 --2- 172.21.15.32:0/1295274099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6af0 0x7fa13012bb90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fa1301432b0 tx=0x7fa12c006eb0).ready entity=mon.0 client_cookie=2fb7ef2c6a8f9c9c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa127fff700 1 -- 172.21.15.32:0/1295274099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c00d610 con 0x7fa1300b6af0 2022-08-01T23:00:53.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa127fff700 1 -- 172.21.15.32:0/1295274099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c000e90 con 0x7fa1300b6af0 2022-08-01T23:00:53.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa127fff700 1 -- 172.21.15.32:0/1295274099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c00d9a0 con 0x7fa1300b6af0 2022-08-01T23:00:53.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 -- 172.21.15.32:0/1295274099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6af0 msgr2=0x7fa13012bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 --2- 172.21.15.32:0/1295274099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6af0 0x7fa13012bb90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fa1301432b0 tx=0x7fa12c006eb0).stop 2022-08-01T23:00:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 -- 172.21.15.32:0/1295274099 shutdown_connections 2022-08-01T23:00:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 --2- 172.21.15.32:0/1295274099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6af0 0x7fa13012bb90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 --2- 172.21.15.32:0/1295274099 >> v2:172.21.15.55:3300/0 conn(0x7fa1300b4550 0x7fa1300b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 -- 172.21.15.32:0/1295274099 >> 172.21.15.32:0/1295274099 conn(0x7fa13005c620 msgr2=0x7fa13005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 -- 172.21.15.32:0/1295274099 shutdown_connections 2022-08-01T23:00:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 -- 172.21.15.32:0/1295274099 wait complete. 2022-08-01T23:00:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 Processor -- start 2022-08-01T23:00:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.483+0000 7fa1373b4700 1 -- start start 2022-08-01T23:00:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1300b4550 0x7fa130129e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 0x7fa13012b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1300b6af0 0x7fa13012bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300b38c0 con 0x7fa1300b6710 2022-08-01T23:00:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13012c180 con 0x7fa1300b6af0 2022-08-01T23:00:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa130125580 con 0x7fa1300b4550 2022-08-01T23:00:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 0x7fa13012b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 0x7fa13012b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44096/0 (socket says 172.21.15.32:44096) 2022-08-01T23:00:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 -- 172.21.15.32:0/702875395 learned_addr learned my addr 172.21.15.32:0/702875395 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa135150700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1300b4550 0x7fa130129e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa135951700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1300b6af0 0x7fa13012bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 -- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1300b4550 msgr2=0x7fa130129e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1300b4550 0x7fa130129e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 -- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1300b6af0 msgr2=0x7fa13012bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1300b6af0 0x7fa13012bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 -- 172.21.15.32:0/702875395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c00b1e0 con 0x7fa1300b6710 2022-08-01T23:00:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa13494f700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 0x7fa13012b7a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fa12000c000 tx=0x7fa120006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa120010590 con 0x7fa1300b6710 2022-08-01T23:00:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1200106f0 con 0x7fa1300b6710 2022-08-01T23:00:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa130125820 con 0x7fa1300b6710 2022-08-01T23:00:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa120010a50 con 0x7fa1300b6710 2022-08-01T23:00:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.487+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa130153300 con 0x7fa1300b6710 2022-08-01T23:00:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.491+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa130061a20 con 0x7fa1300b6710 2022-08-01T23:00:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.491+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa120010bb0 con 0x7fa1300b6710 2022-08-01T23:00:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.491+0000 7fa1267fc700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa118053b00 0x7fa118055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.491+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa12006bfe0 con 0x7fa1300b6710 2022-08-01T23:00:53.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.491+0000 7fa135150700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa118053b00 0x7fa118055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.491+0000 7fa135150700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa118053b00 0x7fa118055fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fa12800d000 tx=0x7fa128007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.499+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa120042da0 con 0x7fa1300b6710 2022-08-01T23:00:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.627+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1300574e0 con 0x7fa1300b6710 2022-08-01T23:00:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.627+0000 7fa1267fc700 1 -- 172.21.15.32:0/702875395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fa1200463c0 con 0x7fa1300b6710 2022-08-01T23:00:53.630 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:53.630 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:00:53.363795+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:00:51.757733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T23:00:52.502708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:00:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa118053b00 msgr2=0x7fa118055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa118053b00 0x7fa118055fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fa12800d000 tx=0x7fa128007f10).stop 2022-08-01T23:00:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 msgr2=0x7fa13012b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 0x7fa13012b7a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fa12000c000 tx=0x7fa120006df0).stop 2022-08-01T23:00:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 shutdown_connections 2022-08-01T23:00:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa118053b00 0x7fa118055fc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1300b6af0 0x7fa13012bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1300b6710 0x7fa13012b7a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 --2- 172.21.15.32:0/702875395 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1300b4550 0x7fa130129e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 >> 172.21.15.32:0/702875395 conn(0x7fa13005c620 msgr2=0x7fa130061de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 shutdown_connections 2022-08-01T23:00:53.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.631+0000 7fa1373b4700 1 -- 172.21.15.32:0/702875395 wait complete. 2022-08-01T23:00:53.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:00:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 Processor -- start 2022-08-01T23:00:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 -- start start 2022-08-01T23:00:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ddc0b08a0 0x7f6ddc142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc143ab0 con 0x7f6ddc0b3260 2022-08-01T23:00:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ddc0b0c80 con 0x7f6ddc0b08a0 2022-08-01T23:00:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de324b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0ae810 con 0x7f6ddc0ad000 2022-08-01T23:00:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de1a48700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ddc0b3260 0x7f6ddc1433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59876/0 (socket says 172.21.15.32:59876) 2022-08-01T23:00:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de1a48700 1 -- 172.21.15.32:0/1003077412 learned_addr learned my addr 172.21.15.32:0/1003077412 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de2249700 1 --2- 172.21.15.32:0/1003077412 >> v2:172.21.15.32:3300/0 conn(0x7f6ddc0b08a0 0x7f6ddc142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de2a4a700 1 --2- 172.21.15.32:0/1003077412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de1247700 1 -- 172.21.15.32:0/1003077412 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063898785 0 0) 0x7f6ddc143ab0 con 0x7f6ddc0b3260 2022-08-01T23:00:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de1247700 1 -- 172.21.15.32:0/1003077412 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dc4003260 con 0x7f6ddc0b3260 2022-08-01T23:00:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.747+0000 7f6de2249700 1 -- 172.21.15.32:0/1003077412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 msgr2=0x7f6ddc13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de2249700 1 --2- 172.21.15.32:0/1003077412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de2249700 1 -- 172.21.15.32:0/1003077412 >> v1:172.21.15.32:6789/0 conn(0x7f6ddc0b3260 legacy=0x7f6ddc1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de2249700 1 -- 172.21.15.32:0/1003077412 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc0aeab0 con 0x7f6ddc0b08a0 2022-08-01T23:00:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de2249700 1 --2- 172.21.15.32:0/1003077412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc142eb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f6dd8002f70 tx=0x7f6dd8006df0).ready entity=mon.0 client_cookie=25bb69a9cf3ef17f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de1247700 1 -- 172.21.15.32:0/1003077412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dd800d4a0 con 0x7f6ddc0b08a0 2022-08-01T23:00:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de1247700 1 -- 172.21.15.32:0/1003077412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd8000e90 con 0x7f6ddc0b08a0 2022-08-01T23:00:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 -- 172.21.15.32:0/1003077412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b08a0 msgr2=0x7f6ddc142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 --2- 172.21.15.32:0/1003077412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc142eb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f6dd8002f70 tx=0x7f6dd8006df0).stop 2022-08-01T23:00:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 -- 172.21.15.32:0/1003077412 shutdown_connections 2022-08-01T23:00:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 --2- 172.21.15.32:0/1003077412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 --2- 172.21.15.32:0/1003077412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc142eb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 -- 172.21.15.32:0/1003077412 >> 172.21.15.32:0/1003077412 conn(0x7f6ddc0612d0 msgr2=0x7f6ddc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 -- 172.21.15.32:0/1003077412 shutdown_connections 2022-08-01T23:00:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 -- 172.21.15.32:0/1003077412 wait complete. 2022-08-01T23:00:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 Processor -- start 2022-08-01T23:00:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.751+0000 7f6de324b700 1 -- start start 2022-08-01T23:00:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc0657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 0x7f6ddc0640d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc066800 con 0x7f6ddc0b3260 2022-08-01T23:00:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1402b0 con 0x7f6ddc0b08a0 2022-08-01T23:00:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc140550 con 0x7f6ddc0ad000 2022-08-01T23:00:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 0x7f6ddc0640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 0x7f6ddc0640d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44112/0 (socket says 172.21.15.32:44112) 2022-08-01T23:00:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 -- 172.21.15.32:0/3226262572 learned_addr learned my addr 172.21.15.32:0/3226262572 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de1a48700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2249700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc0657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 -- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 msgr2=0x7f6ddc0657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc0657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 -- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ddc0b08a0 msgr2=0x7f6ddc0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 -- 172.21.15.32:0/3226262572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd800b180 con 0x7f6ddc0b3260 2022-08-01T23:00:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de2a4a700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 0x7f6ddc0640d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f6dd4006e00 tx=0x7f6dd40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dd400c640 con 0x7f6ddc0b3260 2022-08-01T23:00:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc064ec0 con 0x7f6ddc0b3260 2022-08-01T23:00:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd400c7a0 con 0x7f6ddc0b3260 2022-08-01T23:00:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6dd400cb00 con 0x7f6ddc0b3260 2022-08-01T23:00:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.755+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc067030 con 0x7f6ddc0b3260 2022-08-01T23:00:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.759+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc140750 con 0x7f6ddc0b3260 2022-08-01T23:00:53.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.759+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6dd400cc60 con 0x7f6ddc0b3260 2022-08-01T23:00:53.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.759+0000 7f6dcb7fe700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dbc053a70 0x7f6dbc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:53.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.759+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6dd406c2f0 con 0x7f6ddc0b3260 2022-08-01T23:00:53.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.759+0000 7f6de2249700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dbc053a70 0x7f6dbc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:53.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.759+0000 7f6de2249700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dbc053a70 0x7f6dbc055f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f6dd801a000 tx=0x7f6dd8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:53.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.763+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6dd4043010 con 0x7f6ddc0b3260 2022-08-01T23:00:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.887+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ddc0574e0 con 0x7f6ddc0b3260 2022-08-01T23:00:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6dcb7fe700 1 -- 172.21.15.32:0/3226262572 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f6dd4046630 con 0x7f6ddc0b3260 2022-08-01T23:00:53.892 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:53.893 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:00:53.363795+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:00:51.757733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T23:00:52.502708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:00:53.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dbc053a70 msgr2=0x7f6dbc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dbc053a70 0x7f6dbc055f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f6dd801a000 tx=0x7f6dd8014000).stop 2022-08-01T23:00:53.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 msgr2=0x7f6ddc0640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:53.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 0x7f6ddc0640d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f6dd4006e00 tx=0x7f6dd40091d0).stop 2022-08-01T23:00:53.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 shutdown_connections 2022-08-01T23:00:53.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dbc053a70 0x7f6dbc055f30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc0b3260 0x7f6ddc0640d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ddc0b08a0 0x7f6ddc0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 --2- 172.21.15.32:0/3226262572 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ddc0ad000 0x7f6ddc0657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:53.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.891+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 >> 172.21.15.32:0/3226262572 conn(0x7f6ddc0612d0 msgr2=0x7f6ddc0b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.895+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 shutdown_connections 2022-08-01T23:00:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:53.895+0000 7f6de324b700 1 -- 172.21.15.32:0/3226262572 wait complete. 2022-08-01T23:00:53.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:00:54.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:54.016 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:54.018 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.019+0000 7ff9325cb700 1 Processor -- start 2022-08-01T23:00:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.019+0000 7ff9325cb700 1 -- start start 2022-08-01T23:00:54.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:00:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.019+0000 7ff9325cb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 0x7ff92c142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.019+0000 7ff9325cb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff92c0ad000 0x7ff92c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.019+0000 7ff9325cb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c143ab0 con 0x7ff92c0b3260 2022-08-01T23:00:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c0b0c80 con 0x7ff92c0ad000 2022-08-01T23:00:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ae810 con 0x7ff92c0b08a0 2022-08-01T23:00:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9315c9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 0x7ff92c142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff931dca700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff92c0ad000 0x7ff92c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff930dc8700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff92c0b3260 0x7ff92c1433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:52566/0 (socket says 172.21.15.32:52566) 2022-08-01T23:00:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9315c9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 0x7ff92c142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:55854/0 (socket says 172.21.15.32:55854) 2022-08-01T23:00:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff930dc8700 1 -- 172.21.15.32:0/2469095537 learned_addr learned my addr 172.21.15.32:0/2469095537 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff931dca700 1 -- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 msgr2=0x7ff92c142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff931dca700 1 --2- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 0x7ff92c142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff931dca700 1 -- 172.21.15.32:0/2469095537 >> v1:172.21.15.55:6789/0 conn(0x7ff92c0b3260 legacy=0x7ff92c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff91bfff700 1 -- 172.21.15.32:0/2469095537 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218325770 0 0) 0x7ff92c143ab0 con 0x7ff92c0b3260 2022-08-01T23:00:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff931dca700 1 -- 172.21.15.32:0/2469095537 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0aeab0 con 0x7ff92c0ad000 2022-08-01T23:00:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9315c9700 1 --2- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 0x7ff92c142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff931dca700 1 --2- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c13fa30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7ff9240032f0 tx=0x7ff9240091d0).ready entity=mon.1 client_cookie=f25498485664e92a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff91bfff700 1 -- 172.21.15.32:0/2469095537 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff92400b6c0 con 0x7ff92c0ad000 2022-08-01T23:00:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff91bfff700 1 -- 172.21.15.32:0/2469095537 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92400b820 con 0x7ff92c0ad000 2022-08-01T23:00:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff91bfff700 1 -- 172.21.15.32:0/2469095537 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff92400b990 con 0x7ff92c0ad000 2022-08-01T23:00:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 msgr2=0x7ff92c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c13fa30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7ff9240032f0 tx=0x7ff9240091d0).stop 2022-08-01T23:00:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- 172.21.15.32:0/2469095537 shutdown_connections 2022-08-01T23:00:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c13fa30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2469095537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b08a0 0x7ff92c142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- 172.21.15.32:0/2469095537 >> 172.21.15.32:0/2469095537 conn(0x7ff92c0612d0 msgr2=0x7ff92c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- 172.21.15.32:0/2469095537 shutdown_connections 2022-08-01T23:00:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.023+0000 7ff9325cb700 1 -- 172.21.15.32:0/2469095537 wait complete. 2022-08-01T23:00:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 Processor -- start 2022-08-01T23:00:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 -- start start 2022-08-01T23:00:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 0x7ff92c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b3260 0x7ff92c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c155b00 con 0x7ff92c0b08a0 2022-08-01T23:00:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c156960 con 0x7ff92c0ad000 2022-08-01T23:00:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0abf40 con 0x7ff92c0b3260 2022-08-01T23:00:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 0x7ff92c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 0x7ff92c0b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44124/0 (socket says 172.21.15.32:44124) 2022-08-01T23:00:54.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 -- 172.21.15.32:0/2596630641 learned_addr learned my addr 172.21.15.32:0/2596630641 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:54.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9315c9700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff931dca700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b3260 0x7ff92c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 -- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b3260 msgr2=0x7ff92c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b3260 0x7ff92c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 -- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 msgr2=0x7ff92c156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 -- 172.21.15.32:0/2596630641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92800b180 con 0x7ff92c0b08a0 2022-08-01T23:00:54.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff930dc8700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 0x7ff92c0b4ec0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7ff92c156af0 tx=0x7ff91c007cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff91c0070d0 con 0x7ff92c0b08a0 2022-08-01T23:00:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c007230 con 0x7ff92c0b08a0 2022-08-01T23:00:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff924009470 con 0x7ff92c0b08a0 2022-08-01T23:00:54.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff91c005610 con 0x7ff92c0b08a0 2022-08-01T23:00:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.027+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c0b53b0 con 0x7ff92c0b08a0 2022-08-01T23:00:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.031+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c152c40 con 0x7ff92c0b08a0 2022-08-01T23:00:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.031+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff91c010070 con 0x7ff92c0b08a0 2022-08-01T23:00:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.031+0000 7ff91a7fc700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff90c053aa0 0x7ff90c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.031+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff91c06b800 con 0x7ff92c0b08a0 2022-08-01T23:00:54.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.031+0000 7ff9315c9700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff90c053aa0 0x7ff90c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.031+0000 7ff9315c9700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff90c053aa0 0x7ff90c055f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7ff92800b000 tx=0x7ff928006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.035+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff91c042430 con 0x7ff92c0b08a0 2022-08-01T23:00:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.199+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff92c0574e0 con 0x7ff92c0b08a0 2022-08-01T23:00:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.383+0000 7ff91a7fc700 1 -- 172.21.15.32:0/2596630641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff91c045a50 con 0x7ff92c0b08a0 2022-08-01T23:00:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff90c053aa0 msgr2=0x7ff90c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff90c053aa0 0x7ff90c055f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7ff92800b000 tx=0x7ff928006df0).stop 2022-08-01T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 msgr2=0x7ff92c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 0x7ff92c0b4ec0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7ff92c156af0 tx=0x7ff91c007cb0).stop 2022-08-01T23:00:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 shutdown_connections 2022-08-01T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff90c053aa0 0x7ff90c055f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff92c0b3260 0x7ff92c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92c0b08a0 0x7ff92c0b4ec0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 --2- 172.21.15.32:0/2596630641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff92c0ad000 0x7ff92c156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 >> 172.21.15.32:0/2596630641 conn(0x7ff92c0612d0 msgr2=0x7ff92c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 shutdown_connections 2022-08-01T23:00:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.387+0000 7ff9325cb700 1 -- 172.21.15.32:0/2596630641 wait complete. 2022-08-01T23:00:54.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:54.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f31058c9700 1 Processor -- start 2022-08-01T23:00:54.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f31058c9700 1 -- start start 2022-08-01T23:00:54.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f31058c9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31000b4550 0x7f31000b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f31058c9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3100144240 con 0x7f31000b6710 2022-08-01T23:00:54.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f31058c9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3100143d00 con 0x7f31000b6af0 2022-08-01T23:00:54.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f31058c9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3100143fa0 con 0x7f31000b4550 2022-08-01T23:00:54.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30feffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31000b6710 0x7f3100061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59904/0 (socket says 172.21.15.32:59904) 2022-08-01T23:00:54.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30feffd700 1 -- 172.21.15.32:0/766040522 learned_addr learned my addr 172.21.15.32:0/766040522 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fe7fc700 1 --2- 172.21.15.32:0/766040522 >> v2:172.21.15.55:3300/0 conn(0x7f31000b4550 0x7f31000b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743548459 0 0) 0x7f3100143d00 con 0x7f31000b6af0 2022-08-01T23:00:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e4003260 con 0x7f31000b6af0 2022-08-01T23:00:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730749922 0 0) 0x7f3100144240 con 0x7f31000b6710 2022-08-01T23:00:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3100143d00 con 0x7f31000b6710 2022-08-01T23:00:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fe7fc700 1 -- 172.21.15.32:0/766040522 >> v1:172.21.15.55:6789/0 conn(0x7f31000b6af0 legacy=0x7f310012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fe7fc700 1 -- 172.21.15.32:0/766040522 >> v1:172.21.15.32:6789/0 conn(0x7f31000b6710 legacy=0x7f3100061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fe7fc700 1 -- 172.21.15.32:0/766040522 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3100061780 con 0x7f31000b4550 2022-08-01T23:00:54.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.519+0000 7f30fe7fc700 1 --2- 172.21.15.32:0/766040522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f31000b5d80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f30f8001b40 tx=0x7f30f4006df0).ready entity=mon.1 client_cookie=a44935d273b6fc0e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30f80019e0 con 0x7f31000b4550 2022-08-01T23:00:54.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f4000e90 con 0x7f31000b4550 2022-08-01T23:00:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f30fdffb700 1 -- 172.21.15.32:0/766040522 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30f4011400 con 0x7f31000b4550 2022-08-01T23:00:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 -- 172.21.15.32:0/766040522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 msgr2=0x7f31000b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 --2- 172.21.15.32:0/766040522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f31000b5d80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f30f8001b40 tx=0x7f30f4006df0).stop 2022-08-01T23:00:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 -- 172.21.15.32:0/766040522 shutdown_connections 2022-08-01T23:00:54.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 --2- 172.21.15.32:0/766040522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f31000b5d80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 -- 172.21.15.32:0/766040522 >> 172.21.15.32:0/766040522 conn(0x7f310005c620 msgr2=0x7f310005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 -- 172.21.15.32:0/766040522 shutdown_connections 2022-08-01T23:00:54.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.523+0000 7f31058c9700 1 -- 172.21.15.32:0/766040522 wait complete. 2022-08-01T23:00:54.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 Processor -- start 2022-08-01T23:00:54.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 -- start start 2022-08-01T23:00:54.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f3100125510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 0x7f3100129030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31000b6af0 0x7f3100146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100148050 con 0x7f31000b6710 2022-08-01T23:00:54.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100146a60 con 0x7f31000b4550 2022-08-01T23:00:54.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3100125c90 con 0x7f31000b6af0 2022-08-01T23:00:54.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 0x7f3100129030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 0x7f3100129030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44138/0 (socket says 172.21.15.32:44138) 2022-08-01T23:00:54.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 -- 172.21.15.32:0/1325839776 learned_addr learned my addr 172.21.15.32:0/1325839776 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:54.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30feffd700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f3100125510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30ff7fe700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31000b6af0 0x7f3100146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 -- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31000b6af0 msgr2=0x7f3100146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31000b6af0 0x7f3100146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 -- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 msgr2=0x7f3100125510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f3100125510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 -- 172.21.15.32:0/1325839776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f400b180 con 0x7f31000b6710 2022-08-01T23:00:54.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30fe7fc700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 0x7f3100129030 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f3100125bb0 tx=0x7f3100143e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30f400d920 con 0x7f31000b6710 2022-08-01T23:00:54.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f400da80 con 0x7f31000b6710 2022-08-01T23:00:54.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30f400dde0 con 0x7f31000b6710 2022-08-01T23:00:54.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3100125f30 con 0x7f31000b6710 2022-08-01T23:00:54.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3100149560 con 0x7f31000b6710 2022-08-01T23:00:54.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.527+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f30f401b070 con 0x7f31000b6710 2022-08-01T23:00:54.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.531+0000 7f30effff700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f30dc053b30 0x7f30dc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.531+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f30f406c540 con 0x7f31000b6710 2022-08-01T23:00:54.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.531+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3100061a20 con 0x7f31000b6710 2022-08-01T23:00:54.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.531+0000 7f30feffd700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f30dc053b30 0x7f30dc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.531+0000 7f30feffd700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f30dc053b30 0x7f30dc055ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f30f000d000 tx=0x7f30f0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.535+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30f4022070 con 0x7f31000b6710 2022-08-01T23:00:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.691+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31000574e0 con 0x7f31000b6710 2022-08-01T23:00:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.691+0000 7f30effff700 1 -- 172.21.15.32:0/1325839776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f30f4043300 con 0x7f31000b6710 2022-08-01T23:00:54.693 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:54.693 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:00:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.691+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f30dc053b30 msgr2=0x7f30dc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.691+0000 7f31058c9700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f30dc053b30 0x7f30dc055ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f30f000d000 tx=0x7f30f0009040).stop 2022-08-01T23:00:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.691+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 msgr2=0x7f3100129030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.691+0000 7f31058c9700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 0x7f3100129030 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f3100125bb0 tx=0x7f3100143e90).stop 2022-08-01T23:00:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 shutdown_connections 2022-08-01T23:00:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f30dc053b30 0x7f30dc055ff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31000b6af0 0x7f3100146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31000b6710 0x7f3100129030 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 --2- 172.21.15.32:0/1325839776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31000b4550 0x7f3100125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 >> 172.21.15.32:0/1325839776 conn(0x7f310005c620 msgr2=0x7f3100141fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 shutdown_connections 2022-08-01T23:00:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.695+0000 7f31058c9700 1 -- 172.21.15.32:0/1325839776 wait complete. 2022-08-01T23:00:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:00:54.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:00:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 Processor -- start 2022-08-01T23:00:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 -- start start 2022-08-01T23:00:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba840614b0 0x7fba84061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba841405d0 0x7fba84140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba840aee80 con 0x7fba840ad750 2022-08-01T23:00:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840b70b0 con 0x7fba841405d0 2022-08-01T23:00:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba8a799700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840af0b0 con 0x7fba840614b0 2022-08-01T23:00:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba88f96700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba840ad750 0x7fba840b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59918/0 (socket says 172.21.15.32:59918) 2022-08-01T23:00:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba88f96700 1 -- 172.21.15.32:0/874411194 learned_addr learned my addr 172.21.15.32:0/874411194 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba89f98700 1 --2- 172.21.15.32:0/874411194 >> v2:172.21.15.32:3300/0 conn(0x7fba841405d0 0x7fba84140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba89797700 1 --2- 172.21.15.32:0/874411194 >> v2:172.21.15.55:3300/0 conn(0x7fba840614b0 0x7fba84061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba77fff700 1 -- 172.21.15.32:0/874411194 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346555594 0 0) 0x7fba840aee80 con 0x7fba840ad750 2022-08-01T23:00:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.811+0000 7fba77fff700 1 -- 172.21.15.32:0/874411194 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba68003260 con 0x7fba840ad750 2022-08-01T23:00:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba89f98700 1 -- 172.21.15.32:0/874411194 >> v2:172.21.15.55:3300/0 conn(0x7fba840614b0 msgr2=0x7fba84061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba89f98700 1 --2- 172.21.15.32:0/874411194 >> v2:172.21.15.55:3300/0 conn(0x7fba840614b0 0x7fba84061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba89f98700 1 -- 172.21.15.32:0/874411194 >> v1:172.21.15.32:6789/0 conn(0x7fba840ad750 legacy=0x7fba840b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba89f98700 1 -- 172.21.15.32:0/874411194 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840af350 con 0x7fba841405d0 2022-08-01T23:00:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba89f98700 1 --2- 172.21.15.32:0/874411194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84140a60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fba840aed30 tx=0x7fba7c0091d0).ready entity=mon.0 client_cookie=33179cb5da052651 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba77fff700 1 -- 172.21.15.32:0/874411194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7c00b5a0 con 0x7fba841405d0 2022-08-01T23:00:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba77fff700 1 -- 172.21.15.32:0/874411194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00b700 con 0x7fba841405d0 2022-08-01T23:00:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba77fff700 1 -- 172.21.15.32:0/874411194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7c00b870 con 0x7fba841405d0 2022-08-01T23:00:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 -- 172.21.15.32:0/874411194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 msgr2=0x7fba84140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 --2- 172.21.15.32:0/874411194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84140a60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fba840aed30 tx=0x7fba7c0091d0).stop 2022-08-01T23:00:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 -- 172.21.15.32:0/874411194 shutdown_connections 2022-08-01T23:00:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 --2- 172.21.15.32:0/874411194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84140a60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 --2- 172.21.15.32:0/874411194 >> v2:172.21.15.55:3300/0 conn(0x7fba840614b0 0x7fba84061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 -- 172.21.15.32:0/874411194 >> 172.21.15.32:0/874411194 conn(0x7fba84066f70 msgr2=0x7fba84067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 -- 172.21.15.32:0/874411194 shutdown_connections 2022-08-01T23:00:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 -- 172.21.15.32:0/874411194 wait complete. 2022-08-01T23:00:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 Processor -- start 2022-08-01T23:00:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.815+0000 7fba8a799700 1 -- start start 2022-08-01T23:00:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba840614b0 0x7fba84145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba840ad750 0x7fba840b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba84156900 con 0x7fba841405d0 2022-08-01T23:00:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8409edc0 con 0x7fba840ad750 2022-08-01T23:00:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840b1d80 con 0x7fba840614b0 2022-08-01T23:00:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44154/0 (socket says 172.21.15.32:44154) 2022-08-01T23:00:54.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 -- 172.21.15.32:0/1420893647 learned_addr learned my addr 172.21.15.32:0/1420893647 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:54.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89797700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba840614b0 0x7fba84145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba88f96700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba840ad750 0x7fba840b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 -- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba840614b0 msgr2=0x7fba84145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba840614b0 0x7fba84145670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 -- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba840ad750 msgr2=0x7fba840b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba840ad750 0x7fba840b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 -- 172.21.15.32:0/1420893647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c009470 con 0x7fba841405d0 2022-08-01T23:00:54.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba89f98700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84141f90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fba840b22a0 tx=0x7fba7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7c011c00 con 0x7fba841405d0 2022-08-01T23:00:54.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c011d60 con 0x7fba841405d0 2022-08-01T23:00:54.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba840b2020 con 0x7fba841405d0 2022-08-01T23:00:54.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7c01f5e0 con 0x7fba841405d0 2022-08-01T23:00:54.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba84155a50 con 0x7fba841405d0 2022-08-01T23:00:54.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fba7c026030 con 0x7fba841405d0 2022-08-01T23:00:54.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.819+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba840af5f0 con 0x7fba841405d0 2022-08-01T23:00:54.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.823+0000 7fba767fc700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba6c053950 0x7fba6c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:54.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.823+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fba7c06caf0 con 0x7fba841405d0 2022-08-01T23:00:54.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.823+0000 7fba89797700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba6c053950 0x7fba6c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:54.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.823+0000 7fba89797700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba6c053950 0x7fba6c055e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fba7800b000 tx=0x7fba78006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:54.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.827+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba7c0438b0 con 0x7fba841405d0 2022-08-01T23:00:54.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.955+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba840574e0 con 0x7fba841405d0 2022-08-01T23:00:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.955+0000 7fba767fc700 1 -- 172.21.15.32:0/1420893647 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fba7c024090 con 0x7fba841405d0 2022-08-01T23:00:54.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:54.959 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:00:54.367770+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:00:51.757733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T23:00:52.502708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:00:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba6c053950 msgr2=0x7fba6c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba6c053950 0x7fba6c055e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fba7800b000 tx=0x7fba78006df0).stop 2022-08-01T23:00:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 msgr2=0x7fba84141f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84141f90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fba840b22a0 tx=0x7fba7c016000).stop 2022-08-01T23:00:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 shutdown_connections 2022-08-01T23:00:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba6c053950 0x7fba6c055e10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba841405d0 0x7fba84141f90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba840ad750 0x7fba840b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 --2- 172.21.15.32:0/1420893647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba840614b0 0x7fba84145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 >> 172.21.15.32:0/1420893647 conn(0x7fba84066f70 msgr2=0x7fba8405ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 shutdown_connections 2022-08-01T23:00:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:54.959+0000 7fba8a799700 1 -- 172.21.15.32:0/1420893647 wait complete. 2022-08-01T23:00:54.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:00:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 Processor -- start 2022-08-01T23:00:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 -- start start 2022-08-01T23:00:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe4061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 0x7fbbe4061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe40b2100 con 0x7fbbe4147d80 2022-08-01T23:00:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40b1630 con 0x7fbbe40b41f0 2022-08-01T23:00:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe8cee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4061fe0 con 0x7fbbe40b65c0 2022-08-01T23:00:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe4061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 0x7fbbe4061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 0x7fbbe4061af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53946/0 (socket says 172.21.15.32:53946) 2022-08-01T23:00:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe2ffd700 1 -- 172.21.15.32:0/2696266628 learned_addr learned my addr 172.21.15.32:0/2696266628 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe37fe700 1 -- 172.21.15.32:0/2696266628 >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 msgr2=0x7fbbe4061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe37fe700 1 --2- 172.21.15.32:0/2696266628 >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 0x7fbbe4061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe37fe700 1 -- 172.21.15.32:0/2696266628 >> v1:172.21.15.55:6789/0 conn(0x7fbbe4147d80 legacy=0x7fbbe4144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe27fc700 1 -- 172.21.15.32:0/2696266628 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273522744 0 0) 0x7fbbe40b2100 con 0x7fbbe4147d80 2022-08-01T23:00:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe37fe700 1 -- 172.21.15.32:0/2696266628 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40b69a0 con 0x7fbbe40b65c0 2022-08-01T23:00:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe2ffd700 1 --2- 172.21.15.32:0/2696266628 >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 0x7fbbe4061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe37fe700 1 --2- 172.21.15.32:0/2696266628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe4061600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbbd4002f70 tx=0x7fbbd4006df0).ready entity=mon.2 client_cookie=b9c5b5e0df3b81c9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe27fc700 1 -- 172.21.15.32:0/2696266628 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd400d570 con 0x7fbbe40b65c0 2022-08-01T23:00:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe27fc700 1 -- 172.21.15.32:0/2696266628 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4000e90 con 0x7fbbe40b65c0 2022-08-01T23:00:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.095+0000 7fbbe27fc700 1 -- 172.21.15.32:0/2696266628 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd400d8c0 con 0x7fbbe40b65c0 2022-08-01T23:00:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 -- 172.21.15.32:0/2696266628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 msgr2=0x7fbbe4061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/2696266628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe4061600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbbd4002f70 tx=0x7fbbd4006df0).stop 2022-08-01T23:00:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 -- 172.21.15.32:0/2696266628 shutdown_connections 2022-08-01T23:00:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/2696266628 >> v2:172.21.15.55:3300/0 conn(0x7fbbe40b41f0 0x7fbbe4061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/2696266628 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe4061600 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 -- 172.21.15.32:0/2696266628 >> 172.21.15.32:0/2696266628 conn(0x7fbbe405c620 msgr2=0x7fbbe405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 -- 172.21.15.32:0/2696266628 shutdown_connections 2022-08-01T23:00:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 -- 172.21.15.32:0/2696266628 wait complete. 2022-08-01T23:00:55.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 Processor -- start 2022-08-01T23:00:55.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.099+0000 7fbbe8cee700 1 -- start start 2022-08-01T23:00:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 0x7fbbe4156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe40b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbe4147d80 0x7fbbe40b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40b93b0 con 0x7fbbe40b41f0 2022-08-01T23:00:55.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe414b240 con 0x7fbbe4147d80 2022-08-01T23:00:55.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe414b4e0 con 0x7fbbe40b65c0 2022-08-01T23:00:55.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 0x7fbbe4156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 0x7fbbe4156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44164/0 (socket says 172.21.15.32:44164) 2022-08-01T23:00:55.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 -- 172.21.15.32:0/4094907371 learned_addr learned my addr 172.21.15.32:0/4094907371 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:55.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe2ffd700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe40b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe3fff700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbe4147d80 0x7fbbe40b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 -- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 msgr2=0x7fbbe40b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe40b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 -- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbe4147d80 msgr2=0x7fbbe40b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbe4147d80 0x7fbbe40b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 -- 172.21.15.32:0/4094907371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd800b1e0 con 0x7fbbe40b41f0 2022-08-01T23:00:55.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe37fe700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 0x7fbbe4156730 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fbbd4002f70 tx=0x7fbbd4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd4016070 con 0x7fbbe40b41f0 2022-08-01T23:00:55.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4013a70 con 0x7fbbe40b41f0 2022-08-01T23:00:55.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd400b180 con 0x7fbbe40b41f0 2022-08-01T23:00:55.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd4013dd0 con 0x7fbbe40b41f0 2022-08-01T23:00:55.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.103+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4156c20 con 0x7fbbe40b41f0 2022-08-01T23:00:55.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.107+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbbd4025030 con 0x7fbbe40b41f0 2022-08-01T23:00:55.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.107+0000 7fbbe0ff9700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbc4053aa0 0x7fbbc4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.107+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbbd406d040 con 0x7fbbe40b41f0 2022-08-01T23:00:55.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.107+0000 7fbbe2ffd700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbc4053aa0 0x7fbbc4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.107+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe41482a0 con 0x7fbbe40b41f0 2022-08-01T23:00:55.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.107+0000 7fbbe2ffd700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbc4053aa0 0x7fbbc4055f60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fbbd800b000 tx=0x7fbbd8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.115+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbbd4023020 con 0x7fbbe40b41f0 2022-08-01T23:00:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.283+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbbe40574e0 con 0x7fbbe40b41f0 2022-08-01T23:00:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.291+0000 7fbbe0ff9700 1 -- 172.21.15.32:0/4094907371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbbd4023020 con 0x7fbbe40b41f0 2022-08-01T23:00:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbc4053aa0 msgr2=0x7fbbc4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbc4053aa0 0x7fbbc4055f60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fbbd800b000 tx=0x7fbbd8008040).stop 2022-08-01T23:00:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 msgr2=0x7fbbe4156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 0x7fbbe4156730 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fbbd4002f70 tx=0x7fbbd4014000).stop 2022-08-01T23:00:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 shutdown_connections 2022-08-01T23:00:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbc4053aa0 0x7fbbc4055f60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbe4147d80 0x7fbbe40b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbe40b65c0 0x7fbbe40b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 --2- 172.21.15.32:0/4094907371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe40b41f0 0x7fbbe4156730 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 >> 172.21.15.32:0/4094907371 conn(0x7fbbe405c620 msgr2=0x7fbbe4062120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 shutdown_connections 2022-08-01T23:00:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.295+0000 7fbbe8cee700 1 -- 172.21.15.32:0/4094907371 wait complete. 2022-08-01T23:00:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:00:55.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:00:55.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 Processor -- start 2022-08-01T23:00:55.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- start start 2022-08-01T23:00:55.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76740b5d80 0x7f76740b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7674144240 con 0x7f76740b6710 2022-08-01T23:00:55.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76740b6fe0 con 0x7f76740b4550 2022-08-01T23:00:55.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7674143d00 con 0x7f76740b5d80 2022-08-01T23:00:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76737fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76740b6710 0x7f7674061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59946/0 (socket says 172.21.15.32:59946) 2022-08-01T23:00:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76737fe700 1 -- 172.21.15.32:0/2052939587 learned_addr learned my addr 172.21.15.32:0/2052939587 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7673fff700 1 --2- 172.21.15.32:0/2052939587 >> v2:172.21.15.55:3300/0 conn(0x7f76740b5d80 0x7f76740b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595369719 0 0) 0x7f7674144240 con 0x7f76740b6710 2022-08-01T23:00:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7660003260 con 0x7f76740b6710 2022-08-01T23:00:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108377979 0 0) 0x7f76740b6fe0 con 0x7f76740b4550 2022-08-01T23:00:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7674144240 con 0x7f76740b4550 2022-08-01T23:00:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7673fff700 1 -- 172.21.15.32:0/2052939587 >> v1:172.21.15.55:6789/0 conn(0x7f76740b4550 legacy=0x7f767412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7673fff700 1 -- 172.21.15.32:0/2052939587 >> v1:172.21.15.32:6789/0 conn(0x7f76740b6710 legacy=0x7f7674061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7673fff700 1 -- 172.21.15.32:0/2052939587 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674143fa0 con 0x7f76740b5d80 2022-08-01T23:00:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7673fff700 1 --2- 172.21.15.32:0/2052939587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f76740b6af0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f766c0032f0 tx=0x7f766c0091d0).ready entity=mon.1 client_cookie=b7c693693cd862b2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766c00b600 con 0x7f76740b5d80 2022-08-01T23:00:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00b760 con 0x7f76740b5d80 2022-08-01T23:00:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f76727fc700 1 -- 172.21.15.32:0/2052939587 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766c00b8d0 con 0x7f76740b5d80 2022-08-01T23:00:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- 172.21.15.32:0/2052939587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 msgr2=0x7f76740b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 --2- 172.21.15.32:0/2052939587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f76740b6af0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f766c0032f0 tx=0x7f766c0091d0).stop 2022-08-01T23:00:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- 172.21.15.32:0/2052939587 shutdown_connections 2022-08-01T23:00:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 --2- 172.21.15.32:0/2052939587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f76740b6af0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- 172.21.15.32:0/2052939587 >> 172.21.15.32:0/2052939587 conn(0x7f767405c620 msgr2=0x7f767405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- 172.21.15.32:0/2052939587 shutdown_connections 2022-08-01T23:00:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.415+0000 7f7679dba700 1 -- 172.21.15.32:0/2052939587 wait complete. 2022-08-01T23:00:55.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 Processor -- start 2022-08-01T23:00:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 -- start start 2022-08-01T23:00:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76740b4550 0x7f7674129db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f7674126850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 0x7f767413c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7674140b70 con 0x7f76740b6710 2022-08-01T23:00:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7674125050 con 0x7f76740b5d80 2022-08-01T23:00:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76741252f0 con 0x7f76740b4550 2022-08-01T23:00:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 0x7f767413c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 0x7f767413c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44182/0 (socket says 172.21.15.32:44182) 2022-08-01T23:00:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 -- 172.21.15.32:0/3753787712 learned_addr learned my addr 172.21.15.32:0/3753787712 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7672ffd700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f7674126850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f76737fe700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76740b4550 0x7f7674129db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 -- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76740b4550 msgr2=0x7f7674129db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76740b4550 0x7f7674129db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 -- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 msgr2=0x7f7674126850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f7674126850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 -- 172.21.15.32:0/3753787712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c009470 con 0x7f76740b6710 2022-08-01T23:00:55.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7672ffd700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f7674126850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:55.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7673fff700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 0x7f767413c750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f766c0032f0 tx=0x7f766c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766c00b600 con 0x7f76740b6710 2022-08-01T23:00:55.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00bec0 con 0x7f76740b6710 2022-08-01T23:00:55.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7674126d40 con 0x7f76740b6710 2022-08-01T23:00:55.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766c01f5e0 con 0x7f76740b6710 2022-08-01T23:00:55.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.419+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767413d790 con 0x7f76740b6710 2022-08-01T23:00:55.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.423+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7674140ce0 con 0x7f76740b6710 2022-08-01T23:00:55.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.423+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f766c026030 con 0x7f76740b6710 2022-08-01T23:00:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.423+0000 7f7670ff9700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7654053b00 0x7f7654055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.423+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f766c06cc70 con 0x7f76740b6710 2022-08-01T23:00:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.423+0000 7f76737fe700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7654053b00 0x7f7654055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.423+0000 7f76737fe700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7654053b00 0x7f7654055fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f766400b000 tx=0x7f7664009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.431+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f766c043a30 con 0x7f76740b6710 2022-08-01T23:00:55.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.595+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7674061780 con 0x7f76740b6710 2022-08-01T23:00:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.603+0000 7f7670ff9700 1 -- 172.21.15.32:0/3753787712 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f766c0240e0 con 0x7f76740b6710 2022-08-01T23:00:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7654053b00 msgr2=0x7f7654055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7654053b00 0x7f7654055fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f766400b000 tx=0x7f7664009040).stop 2022-08-01T23:00:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 msgr2=0x7f767413c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 0x7f767413c750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f766c0032f0 tx=0x7f766c016000).stop 2022-08-01T23:00:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 shutdown_connections 2022-08-01T23:00:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7654053b00 0x7f7654055fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76740b6710 0x7f767413c750 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76740b5d80 0x7f7674126850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 --2- 172.21.15.32:0/3753787712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76740b4550 0x7f7674129db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 >> 172.21.15.32:0/3753787712 conn(0x7f767405c620 msgr2=0x7f7674061e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 shutdown_connections 2022-08-01T23:00:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.607+0000 7f7679dba700 1 -- 172.21.15.32:0/3753787712 wait complete. 2022-08-01T23:00:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:00:55.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43fcee2700 1 Processor -- start 2022-08-01T23:00:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43fcee2700 1 -- start start 2022-08-01T23:00:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43fcee2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43f80b01a0 0x7f43f80a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43fcee2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80a7250 con 0x7f43f80afd60 2022-08-01T23:00:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43fcee2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80a7480 con 0x7f43f80b2710 2022-08-01T23:00:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43fcee2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f80a7720 con 0x7f43f80b01a0 2022-08-01T23:00:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43f7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43f80afd60 0x7f43f8151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59964/0 (socket says 172.21.15.32:59964) 2022-08-01T23:00:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43f7fff700 1 -- 172.21.15.32:0/3453544056 learned_addr learned my addr 172.21.15.32:0/3453544056 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:00:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.723+0000 7f43f6ffd700 1 --2- 172.21.15.32:0/3453544056 >> v2:172.21.15.55:3300/0 conn(0x7f43f80b01a0 0x7f43f80a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088506585 0 0) 0x7f43f80a7250 con 0x7f43f80afd60 2022-08-01T23:00:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e4003260 con 0x7f43f80afd60 2022-08-01T23:00:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030271054 0 0) 0x7f43e4003260 con 0x7f43f80afd60 2022-08-01T23:00:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 >> v2:172.21.15.55:3300/0 conn(0x7f43f80b01a0 msgr2=0x7f43f80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 --2- 172.21.15.32:0/3453544056 >> v2:172.21.15.55:3300/0 conn(0x7f43f80b01a0 0x7f43f80a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 >> v1:172.21.15.55:6789/0 conn(0x7f43f80b2710 legacy=0x7f43f814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80a79c0 con 0x7f43f80afd60 2022-08-01T23:00:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f43f00019e0 con 0x7f43f80afd60 2022-08-01T23:00:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 >> v1:172.21.15.32:6789/0 conn(0x7f43f80afd60 legacy=0x7f43f8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4003750 0x7f43e4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43e4004500 0x7f43e4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4004e70 0x7f43e400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80a7720 con 0x7f43e4003750 2022-08-01T23:00:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e400bd20 con 0x7f43e4004e70 2022-08-01T23:00:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e400bee0 con 0x7f43e4004500 2022-08-01T23:00:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4003750 0x7f43e4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f7fff700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4004e70 0x7f43e400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f6ffd700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43e4004500 0x7f43e4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 -- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43e4004500 msgr2=0x7f43e4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43e4004500 0x7f43e4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 -- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4004e70 msgr2=0x7f43e400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4004e70 0x7f43e400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f7fff700 1 -- 172.21.15.32:0/3453544056 reap_dead start 2022-08-01T23:00:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 -- 172.21.15.32:0/3453544056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4003260 con 0x7f43e4003750 2022-08-01T23:00:55.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f77fe700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4003750 0x7f43e4003b70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f43f8061c10 tx=0x7f43ec007a00).ready entity=mon.0 client_cookie=733c345f9e35c08d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43ec007d10 con 0x7f43e4003750 2022-08-01T23:00:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec007030 con 0x7f43e4003750 2022-08-01T23:00:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43f67fc700 1 -- 172.21.15.32:0/3453544056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43ec007390 con 0x7f43e4003750 2022-08-01T23:00:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 -- 172.21.15.32:0/3453544056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4003750 msgr2=0x7f43e4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4003750 0x7f43e4003b70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f43f8061c10 tx=0x7f43ec007a00).stop 2022-08-01T23:00:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 -- 172.21.15.32:0/3453544056 shutdown_connections 2022-08-01T23:00:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 --2- 172.21.15.32:0/3453544056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43e4003750 0x7f43e4003b70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 -- 172.21.15.32:0/3453544056 >> 172.21.15.32:0/3453544056 conn(0x7f43f8060310 msgr2=0x7f43f80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 -- 172.21.15.32:0/3453544056 shutdown_connections 2022-08-01T23:00:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.727+0000 7f43fcee2700 1 -- 172.21.15.32:0/3453544056 wait complete. 2022-08-01T23:00:55.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 Processor -- start 2022-08-01T23:00:55.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 -- start start 2022-08-01T23:00:55.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4003750 0x7f43f8145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43f8145b80 0x7f43f8066970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 0x7f43f80a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f8157510 con 0x7f43f8066e60 2022-08-01T23:00:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80a9710 con 0x7f43e4003750 2022-08-01T23:00:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43fcee2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f8157230 con 0x7f43f8145b80 2022-08-01T23:00:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 0x7f43f80a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 0x7f43f80a9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44202/0 (socket says 172.21.15.32:44202) 2022-08-01T23:00:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43f7fff700 1 -- 172.21.15.32:0/1272531498 learned_addr learned my addr 172.21.15.32:0/1272531498 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43f6ffd700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43f8145b80 0x7f43f8066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.731+0000 7f43f77fe700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4003750 0x7f43f8145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f7fff700 1 -- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43f8145b80 msgr2=0x7f43f8066970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f7fff700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43f8145b80 0x7f43f8066970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f7fff700 1 -- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4003750 msgr2=0x7f43f8145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f7fff700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4003750 0x7f43f8145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f7fff700 1 -- 172.21.15.32:0/1272531498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8146000 con 0x7f43f8066e60 2022-08-01T23:00:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f7fff700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 0x7f43f80a9bb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f43f0001490 tx=0x7f43f80a7610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43f0011780 con 0x7f43f8066e60 2022-08-01T23:00:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f00118e0 con 0x7f43f8066e60 2022-08-01T23:00:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f80ab380 con 0x7f43f8066e60 2022-08-01T23:00:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43f0011c40 con 0x7f43f8066e60 2022-08-01T23:00:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f8067480 con 0x7f43f8066e60 2022-08-01T23:00:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f80a98d0 con 0x7f43f8066e60 2022-08-01T23:00:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.735+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f43f001b030 con 0x7f43f8066e60 2022-08-01T23:00:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.739+0000 7f43f4ff9700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43d8053b30 0x7f43d8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.739+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f43f006b8a0 con 0x7f43f8066e60 2022-08-01T23:00:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.739+0000 7f43f77fe700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43d8053b30 0x7f43d8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.739+0000 7f43f77fe700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43d8053b30 0x7f43d8055ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f43ec01b000 tx=0x7f43ec015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.743+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43f0042660 con 0x7f43f8066e60 2022-08-01T23:00:55.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43f80574e0 con 0x7f43f8066e60 2022-08-01T23:00:55.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43f4ff9700 1 -- 172.21.15.32:0/1272531498 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f43f0019020 con 0x7f43f8066e60 2022-08-01T23:00:55.907 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:00:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43d8053b30 msgr2=0x7f43d8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43d8053b30 0x7f43d8055ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f43ec01b000 tx=0x7f43ec015000).stop 2022-08-01T23:00:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 msgr2=0x7f43f80a9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 0x7f43f80a9bb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f43f0001490 tx=0x7f43f80a7610).stop 2022-08-01T23:00:55.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 shutdown_connections 2022-08-01T23:00:55.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43d8053b30 0x7f43d8055ff0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8066e60 0x7f43f80a9bb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43f8145b80 0x7f43f8066970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 --2- 172.21.15.32:0/1272531498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e4003750 0x7f43f8145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:55.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 >> 172.21.15.32:0/1272531498 conn(0x7f43f8060310 msgr2=0x7f43f80af460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:55.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 shutdown_connections 2022-08-01T23:00:55.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:55.907+0000 7f43fcee2700 1 -- 172.21.15.32:0/1272531498 wait complete. 2022-08-01T23:00:55.913 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:00:56.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:00:53.364281+0000', 'modified': '2022-08-01T23:00:54.368935+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5934}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5934': {'gid': 5934, 'name': 'b', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/590504455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 590504455}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 590504455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:00:56.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:00:56.225 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T23:00:56.339 INFO:teuthology.orchestra.run.smithi055.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5934,"want_state":"up:active","state":"up:active","rank_uptime":2.9661423660000001,"mdsmap_epoch":28,"osdmap_epoch":67,"osdmap_epoch_barrier":66,"uptime":7.3330790889999999} 2022-08-01T23:00:56.340 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5934, "mdsmap_epoch": 28, "osdmap_epoch": 67, "osdmap_epoch_barrier": 66, "rank_uptime": 2.966142366, "state": "up:active", "uptime": 7.333079089, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:00:56.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:00:56.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 Processor -- start 2022-08-01T23:00:56.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 -- start start 2022-08-01T23:00:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb1780b01a0 0x7fb17814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1780a7250 con 0x7fb1780afd60 2022-08-01T23:00:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780a7480 con 0x7fb1780b01a0 2022-08-01T23:00:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17e462700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780a7720 con 0x7fb1780b2710 2022-08-01T23:00:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17cc5f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb1780b01a0 0x7fb17814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17cc5f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb1780b01a0 0x7fb17814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:53992/0 (socket says 172.21.15.32:53992) 2022-08-01T23:00:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17cc5f700 1 -- 172.21.15.32:0/2175732810 learned_addr learned my addr 172.21.15.32:0/2175732810 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.451+0000 7fb17d460700 1 --2- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17cc5f700 1 -- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 msgr2=0x7fb17814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17cc5f700 1 --2- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17cc5f700 1 -- 172.21.15.32:0/2175732810 >> v1:172.21.15.55:6789/0 conn(0x7fb1780afd60 legacy=0x7fb1780a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17cc5f700 1 -- 172.21.15.32:0/2175732810 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780a79c0 con 0x7fb1780b01a0 2022-08-01T23:00:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17cc5f700 1 --2- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb17814b4c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fb174002f70 tx=0x7fb174006eb0).ready entity=mon.1 client_cookie=7db07b86a43b9332 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb16ffff700 1 -- 172.21.15.32:0/2175732810 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb17400d570 con 0x7fb1780b01a0 2022-08-01T23:00:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb16ffff700 1 -- 172.21.15.32:0/2175732810 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174000e90 con 0x7fb1780b01a0 2022-08-01T23:00:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb16ffff700 1 -- 172.21.15.32:0/2175732810 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb17400d900 con 0x7fb1780b01a0 2022-08-01T23:00:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 -- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 msgr2=0x7fb17814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 --2- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb17814b4c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fb174002f70 tx=0x7fb174006eb0).stop 2022-08-01T23:00:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 -- 172.21.15.32:0/2175732810 shutdown_connections 2022-08-01T23:00:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 --2- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb17814b4c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 --2- 172.21.15.32:0/2175732810 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 -- 172.21.15.32:0/2175732810 >> 172.21.15.32:0/2175732810 conn(0x7fb178060310 msgr2=0x7fb1780b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 -- 172.21.15.32:0/2175732810 shutdown_connections 2022-08-01T23:00:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 -- 172.21.15.32:0/2175732810 wait complete. 2022-08-01T23:00:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 Processor -- start 2022-08-01T23:00:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.455+0000 7fb17e462700 1 -- start start 2022-08-01T23:00:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 0x7fb178148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb1780b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b40c0 con 0x7fb1780afd60 2022-08-01T23:00:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b42f0 con 0x7fb1780b01a0 2022-08-01T23:00:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b4590 con 0x7fb1780b2710 2022-08-01T23:00:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 0x7fb178148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 0x7fb178148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44212/0 (socket says 172.21.15.32:44212) 2022-08-01T23:00:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17dc61700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 -- 172.21.15.32:0/1333840227 learned_addr learned my addr 172.21.15.32:0/1333840227 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17cc5f700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb1780b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 -- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 msgr2=0x7fb17815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 -- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 msgr2=0x7fb1780b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb1780b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 -- 172.21.15.32:0/1333840227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17400b1e0 con 0x7fb1780afd60 2022-08-01T23:00:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17d460700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 0x7fb178148b90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fb16800c000 tx=0x7fb168006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb168011750 con 0x7fb1780afd60 2022-08-01T23:00:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1680118b0 con 0x7fb1780afd60 2022-08-01T23:00:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb178159b40 con 0x7fb1780afd60 2022-08-01T23:00:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb168011c10 con 0x7fb1780afd60 2022-08-01T23:00:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.459+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1780b5fd0 con 0x7fb1780afd60 2022-08-01T23:00:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.463+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb168011d70 con 0x7fb1780afd60 2022-08-01T23:00:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.463+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1780b5680 con 0x7fb1780afd60 2022-08-01T23:00:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.463+0000 7fb16e7fc700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb160053b30 0x7fb160055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.463+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb16806c3e0 con 0x7fb1780afd60 2022-08-01T23:00:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.463+0000 7fb17cc5f700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb160053b30 0x7fb160055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.463+0000 7fb17cc5f700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb160053b30 0x7fb160055ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fb17401c000 tx=0x7fb174015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.471+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1780b5680 con 0x7fb1780afd60 2022-08-01T23:00:56.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.603+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1780b5680 con 0x7fb1780afd60 2022-08-01T23:00:56.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.603+0000 7fb16e7fc700 1 -- 172.21.15.32:0/1333840227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fb1780b5680 con 0x7fb1780afd60 2022-08-01T23:00:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:56.606 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:00:55.384929+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:00:51.757733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T23:00:52.502708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:00:56.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb160053b30 msgr2=0x7fb160055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb160053b30 0x7fb160055ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fb17401c000 tx=0x7fb174015000).stop 2022-08-01T23:00:56.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 msgr2=0x7fb178148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 0x7fb178148b90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fb16800c000 tx=0x7fb168006df0).stop 2022-08-01T23:00:56.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 shutdown_connections 2022-08-01T23:00:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb160053b30 0x7fb160055ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb1780b2710 0x7fb17815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1780b01a0 0x7fb1780b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 --2- 172.21.15.32:0/1333840227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1780afd60 0x7fb178148b90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 >> 172.21.15.32:0/1333840227 conn(0x7fb178060310 msgr2=0x7fb17814a210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 shutdown_connections 2022-08-01T23:00:56.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.607+0000 7fb17e462700 1 -- 172.21.15.32:0/1333840227 wait complete. 2022-08-01T23:00:56.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:00:51.757733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:00:52.502708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:00:56.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:00:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 Processor -- start 2022-08-01T23:00:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 -- start start 2022-08-01T23:00:56.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23340ad750 0x7f2334061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f23341405d0 0x7f2334140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340aee80 con 0x7f23340614b0 2022-08-01T23:00:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23340b70b0 con 0x7f23340ad750 2022-08-01T23:00:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f233359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23340af0b0 con 0x7f23341405d0 2022-08-01T23:00:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23340ad750 0x7f2334061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23340ad750 0x7f2334061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44220/0 (socket says 172.21.15.32:44220) 2022-08-01T23:00:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 -- 172.21.15.32:0/2631718861 learned_addr learned my addr 172.21.15.32:0/2631718861 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2332d9d700 1 --2- 172.21.15.32:0/2631718861 >> v2:172.21.15.55:3300/0 conn(0x7f23341405d0 0x7f2334140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 -- 172.21.15.32:0/2631718861 >> v2:172.21.15.55:3300/0 conn(0x7f23341405d0 msgr2=0x7f2334140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 --2- 172.21.15.32:0/2631718861 >> v2:172.21.15.55:3300/0 conn(0x7f23341405d0 0x7f2334140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 -- 172.21.15.32:0/2631718861 >> v1:172.21.15.55:6789/0 conn(0x7f23340614b0 legacy=0x7f23340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 -- 172.21.15.32:0/2631718861 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340af350 con 0x7f23340ad750 2022-08-01T23:00:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.731+0000 7f2331d9b700 1 --2- 172.21.15.32:0/2631718861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334061890 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f2328002f70 tx=0x7f2328006eb0).ready entity=mon.0 client_cookie=f529f995d8ddb4f9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233159a700 1 -- 172.21.15.32:0/2631718861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f232800d5c0 con 0x7f23340ad750 2022-08-01T23:00:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233159a700 1 -- 172.21.15.32:0/2631718861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328000e90 con 0x7f23340ad750 2022-08-01T23:00:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233159a700 1 -- 172.21.15.32:0/2631718861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f232800d950 con 0x7f23340ad750 2022-08-01T23:00:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- 172.21.15.32:0/2631718861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 msgr2=0x7f2334061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 --2- 172.21.15.32:0/2631718861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334061890 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f2328002f70 tx=0x7f2328006eb0).stop 2022-08-01T23:00:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- 172.21.15.32:0/2631718861 shutdown_connections 2022-08-01T23:00:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 --2- 172.21.15.32:0/2631718861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334061890 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 --2- 172.21.15.32:0/2631718861 >> v2:172.21.15.55:3300/0 conn(0x7f23341405d0 0x7f2334140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- 172.21.15.32:0/2631718861 >> 172.21.15.32:0/2631718861 conn(0x7f2334066f70 msgr2=0x7f2334067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:56.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- 172.21.15.32:0/2631718861 shutdown_connections 2022-08-01T23:00:56.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- 172.21.15.32:0/2631718861 wait complete. 2022-08-01T23:00:56.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 Processor -- start 2022-08-01T23:00:56.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- start start 2022-08-01T23:00:56.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23340614b0 0x7f23340a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23341405d0 0x7f2334155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340ad1f0 con 0x7f23340ad750 2022-08-01T23:00:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340ad420 con 0x7f23341405d0 2022-08-01T23:00:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2334152800 con 0x7f23340614b0 2022-08-01T23:00:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44226/0 (socket says 172.21.15.32:44226) 2022-08-01T23:00:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 -- 172.21.15.32:0/1247671522 learned_addr learned my addr 172.21.15.32:0/1247671522 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2332d9d700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23341405d0 0x7f2334155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f233259c700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23340614b0 0x7f23340a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 -- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23340614b0 msgr2=0x7f23340a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23340614b0 0x7f23340a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 -- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23341405d0 msgr2=0x7f2334155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23341405d0 0x7f2334155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 -- 172.21.15.32:0/1247671522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232800b1e0 con 0x7f23340ad750 2022-08-01T23:00:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.735+0000 7f2331d9b700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334152ca0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f2328002f70 tx=0x7f2328015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2328011860 con 0x7f23340ad750 2022-08-01T23:00:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23280119c0 con 0x7f23340ad750 2022-08-01T23:00:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23341524f0 con 0x7f23340ad750 2022-08-01T23:00:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2328011d20 con 0x7f23340ad750 2022-08-01T23:00:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23340a1c10 con 0x7f23340ad750 2022-08-01T23:00:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2328025030 con 0x7f23340ad750 2022-08-01T23:00:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f23237fe700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23100539e0 0x7f2310055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f232806d360 con 0x7f23340ad750 2022-08-01T23:00:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23340af5f0 con 0x7f23340ad750 2022-08-01T23:00:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f233259c700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23100539e0 0x7f2310055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.739+0000 7f233259c700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23100539e0 0x7f2310055ea0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f232400b000 tx=0x7f2324009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.747+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2328023020 con 0x7f23340ad750 2022-08-01T23:00:56.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f23340574e0 con 0x7f23340ad750 2022-08-01T23:00:56.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f23237fe700 1 -- 172.21.15.32:0/1247671522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v67) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2328044010 con 0x7f23340ad750 2022-08-01T23:00:56.876 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:00:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23100539e0 msgr2=0x7f2310055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23100539e0 0x7f2310055ea0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f232400b000 tx=0x7f2324009040).stop 2022-08-01T23:00:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 msgr2=0x7f2334152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334152ca0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f2328002f70 tx=0x7f2328015000).stop 2022-08-01T23:00:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 shutdown_connections 2022-08-01T23:00:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23100539e0 0x7f2310055ea0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23341405d0 0x7f2334155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23340ad750 0x7f2334152ca0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 --2- 172.21.15.32:0/1247671522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23340614b0 0x7f23340a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:56.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 >> 172.21.15.32:0/1247671522 conn(0x7f2334066f70 msgr2=0x7f233413fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:56.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 shutdown_connections 2022-08-01T23:00:56.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.875+0000 7f233359e700 1 -- 172.21.15.32:0/1247671522 wait complete. 2022-08-01T23:00:56.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:00:56.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 Processor -- start 2022-08-01T23:00:56.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- start start 2022-08-01T23:00:57.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81ec0b2730 0x7f81ec0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0654c0 con 0x7f81ec0b3190 2022-08-01T23:00:57.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0b3570 con 0x7f81ec066ba0 2022-08-01T23:00:57.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec062190 con 0x7f81ec0b2730 2022-08-01T23:00:57.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81ec0b2730 0x7f81ec0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81ec0b2730 0x7f81ec0666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44236/0 (socket says 172.21.15.32:44236) 2022-08-01T23:00:57.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 -- 172.21.15.32:0/2596484643 learned_addr learned my addr 172.21.15.32:0/2596484643 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f10e2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81ec0b3190 0x7f81ec060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60008/0 (socket says 172.21.15.32:60008) 2022-08-01T23:00:57.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 -- 172.21.15.32:0/2596484643 >> v1:172.21.15.55:6789/0 conn(0x7f81ec066ba0 legacy=0x7f81ec14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 -- 172.21.15.32:0/2596484643 >> v1:172.21.15.32:6789/0 conn(0x7f81ec0b3190 legacy=0x7f81ec060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 -- 172.21.15.32:0/2596484643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec05fbf0 con 0x7f81ec0b2730 2022-08-01T23:00:57.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f08e1700 1 --2- 172.21.15.32:0/2596484643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec0b2730 0x7f81ec0666b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f81dc002f70 tx=0x7f81dc006eb0).ready entity=mon.0 client_cookie=5d2b6a48260caa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81dbfff700 1 -- 172.21.15.32:0/2596484643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81dc00d660 con 0x7f81ec0b2730 2022-08-01T23:00:57.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81dbfff700 1 -- 172.21.15.32:0/2596484643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc000e90 con 0x7f81ec0b2730 2022-08-01T23:00:57.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81dbfff700 1 -- 172.21.15.32:0/2596484643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81dc00d9f0 con 0x7f81ec0b2730 2022-08-01T23:00:57.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- 172.21.15.32:0/2596484643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec0b2730 msgr2=0x7f81ec0666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 --2- 172.21.15.32:0/2596484643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec0b2730 0x7f81ec0666b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f81dc002f70 tx=0x7f81dc006eb0).stop 2022-08-01T23:00:57.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- 172.21.15.32:0/2596484643 shutdown_connections 2022-08-01T23:00:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 --2- 172.21.15.32:0/2596484643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec0b2730 0x7f81ec0666b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- 172.21.15.32:0/2596484643 >> 172.21.15.32:0/2596484643 conn(0x7f81ec063870 msgr2=0x7f81ec063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- 172.21.15.32:0/2596484643 shutdown_connections 2022-08-01T23:00:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:56.999+0000 7f81f20e4700 1 -- 172.21.15.32:0/2596484643 wait complete. 2022-08-01T23:00:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 Processor -- start 2022-08-01T23:00:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 -- start start 2022-08-01T23:00:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 0x7f81ec0b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81ec0b2730 0x7f81ec155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f81ec0b3190 0x7f81ec155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0aa350 con 0x7f81ec066ba0 2022-08-01T23:00:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0aa5f0 con 0x7f81ec0b2730 2022-08-01T23:00:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0aa890 con 0x7f81ec0b3190 2022-08-01T23:00:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 0x7f81ec0b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 0x7f81ec0b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44240/0 (socket says 172.21.15.32:44240) 2022-08-01T23:00:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 -- 172.21.15.32:0/3586289791 learned_addr learned my addr 172.21.15.32:0/3586289791 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f08e1700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81ec0b2730 0x7f81ec155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f18e3700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f81ec0b3190 0x7f81ec155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 -- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f81ec0b3190 msgr2=0x7f81ec155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f81ec0b3190 0x7f81ec155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 -- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81ec0b2730 msgr2=0x7f81ec155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81ec0b2730 0x7f81ec155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 -- 172.21.15.32:0/3586289791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc00b1e0 con 0x7f81ec066ba0 2022-08-01T23:00:57.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f18e3700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f81ec0b3190 0x7f81ec155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:00:57.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f10e2700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 0x7f81ec0b6260 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f81ec0634e0 tx=0x7f81e8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e80070d0 con 0x7f81ec066ba0 2022-08-01T23:00:57.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8007230 con 0x7f81ec066ba0 2022-08-01T23:00:57.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.003+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec0aab30 con 0x7f81ec066ba0 2022-08-01T23:00:57.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e8005550 con 0x7f81ec066ba0 2022-08-01T23:00:57.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec0aafd0 con 0x7f81ec066ba0 2022-08-01T23:00:57.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec0aad10 con 0x7f81ec066ba0 2022-08-01T23:00:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f81e801a060 con 0x7f81ec066ba0 2022-08-01T23:00:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81da7fc700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81cc053b00 0x7f81cc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f81e806bc90 con 0x7f81ec066ba0 2022-08-01T23:00:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81f08e1700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81cc053b00 0x7f81cc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.007+0000 7f81f08e1700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81cc053b00 0x7f81cc055fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f81dc01c000 tx=0x7f81dc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.015+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f81e8042940 con 0x7f81ec066ba0 2022-08-01T23:00:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.139+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f81ec0aacc0 con 0x7f81ec066ba0 2022-08-01T23:00:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.139+0000 7f81da7fc700 1 -- 172.21.15.32:0/3586289791 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v67) v1 ==== 92+0+10 (secure 0 0 0) 0x7f81e8018030 con 0x7f81ec066ba0 2022-08-01T23:00:57.140 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:00:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81cc053b00 msgr2=0x7f81cc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81cc053b00 0x7f81cc055fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f81dc01c000 tx=0x7f81dc015000).stop 2022-08-01T23:00:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 msgr2=0x7f81ec0b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 0x7f81ec0b6260 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f81ec0634e0 tx=0x7f81e8007c40).stop 2022-08-01T23:00:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 shutdown_connections 2022-08-01T23:00:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81cc053b00 0x7f81cc055fc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f81ec0b3190 0x7f81ec155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81ec0b2730 0x7f81ec155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 --2- 172.21.15.32:0/3586289791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81ec066ba0 0x7f81ec0b6260 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 >> 172.21.15.32:0/3586289791 conn(0x7f81ec063870 msgr2=0x7f81ec146170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 shutdown_connections 2022-08-01T23:00:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.143+0000 7f81f20e4700 1 -- 172.21.15.32:0/3586289791 wait complete. 2022-08-01T23:00:57.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:00:57.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 Processor -- start 2022-08-01T23:00:57.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- start start 2022-08-01T23:00:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5180b7500 0x7fd5180b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd5180aadb0 0x7fd518064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180b2510 con 0x7fd5180b31f0 2022-08-01T23:00:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5181443e0 con 0x7fd5180b7500 2022-08-01T23:00:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180b3700 con 0x7fd5180aadb0 2022-08-01T23:00:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5180b7500 0x7fd5180b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5180b7500 0x7fd5180b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44248/0 (socket says 172.21.15.32:44248) 2022-08-01T23:00:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 -- 172.21.15.32:0/2654106083 learned_addr learned my addr 172.21.15.32:0/2654106083 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd517fff700 1 --1- 172.21.15.32:0/2654106083 >> v1:172.21.15.32:6789/0 conn(0x7fd5180b31f0 0x7fd51813e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60026/0 (socket says 172.21.15.32:60026) 2022-08-01T23:00:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51d44d700 1 --2- 172.21.15.32:0/2654106083 >> v2:172.21.15.55:3300/0 conn(0x7fd5180aadb0 0x7fd518064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 -- 172.21.15.32:0/2654106083 >> v2:172.21.15.55:3300/0 conn(0x7fd5180aadb0 msgr2=0x7fd518064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 --2- 172.21.15.32:0/2654106083 >> v2:172.21.15.55:3300/0 conn(0x7fd5180aadb0 0x7fd518064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 -- 172.21.15.32:0/2654106083 >> v1:172.21.15.32:6789/0 conn(0x7fd5180b31f0 legacy=0x7fd51813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 -- 172.21.15.32:0/2654106083 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180b2740 con 0x7fd5180b7500 2022-08-01T23:00:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51cc4c700 1 --2- 172.21.15.32:0/2654106083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180b7500 0x7fd5180b78e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fd5000032f0 tx=0x7fd500009250).ready entity=mon.0 client_cookie=4a66e8902ac5506 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd5177fe700 1 -- 172.21.15.32:0/2654106083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd500009ea0 con 0x7fd5180b7500 2022-08-01T23:00:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd5177fe700 1 -- 172.21.15.32:0/2654106083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000b870 con 0x7fd5180b7500 2022-08-01T23:00:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd5177fe700 1 -- 172.21.15.32:0/2654106083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd50000b9e0 con 0x7fd5180b7500 2022-08-01T23:00:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- 172.21.15.32:0/2654106083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180b7500 msgr2=0x7fd5180b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/2654106083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180b7500 0x7fd5180b78e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fd5000032f0 tx=0x7fd500009250).stop 2022-08-01T23:00:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- 172.21.15.32:0/2654106083 shutdown_connections 2022-08-01T23:00:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/2654106083 >> v2:172.21.15.55:3300/0 conn(0x7fd5180aadb0 0x7fd518064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/2654106083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180b7500 0x7fd5180b78e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- 172.21.15.32:0/2654106083 >> 172.21.15.32:0/2654106083 conn(0x7fd51805f840 msgr2=0x7fd51805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.263+0000 7fd51dc4e700 1 -- 172.21.15.32:0/2654106083 shutdown_connections 2022-08-01T23:00:57.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 -- 172.21.15.32:0/2654106083 wait complete. 2022-08-01T23:00:57.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 Processor -- start 2022-08-01T23:00:57.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 -- start start 2022-08-01T23:00:57.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 0x7fd518138740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5180b31f0 0x7fd518138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5180b7500 0x7fd5180afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180aedc0 con 0x7fd5180aadb0 2022-08-01T23:00:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5181466d0 con 0x7fd5180b7500 2022-08-01T23:00:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51dc4e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180ae880 con 0x7fd5180b31f0 2022-08-01T23:00:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 0x7fd518138740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 0x7fd518138740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44254/0 (socket says 172.21.15.32:44254) 2022-08-01T23:00:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 -- 172.21.15.32:0/3641649827 learned_addr learned my addr 172.21.15.32:0/3641649827 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd517fff700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5180b31f0 0x7fd518138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51d44d700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5180b7500 0x7fd5180afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 -- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5180b31f0 msgr2=0x7fd518138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5180b31f0 0x7fd518138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 -- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5180b7500 msgr2=0x7fd5180afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5180b7500 0x7fd5180afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 -- 172.21.15.32:0/3641649827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000094d0 con 0x7fd5180aadb0 2022-08-01T23:00:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.267+0000 7fd51cc4c700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 0x7fd518138740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fd518146860 tx=0x7fd500016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd500006510 con 0x7fd5180aadb0 2022-08-01T23:00:57.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd500006670 con 0x7fd5180aadb0 2022-08-01T23:00:57.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5000069d0 con 0x7fd5180aadb0 2022-08-01T23:00:57.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180aeb20 con 0x7fd5180aadb0 2022-08-01T23:00:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518139120 con 0x7fd5180aadb0 2022-08-01T23:00:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5180adbb0 con 0x7fd5180aadb0 2022-08-01T23:00:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd500029070 con 0x7fd5180aadb0 2022-08-01T23:00:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd515ffb700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4f0053a70 0x7fd4f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.271+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd50006ce70 con 0x7fd5180aadb0 2022-08-01T23:00:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.275+0000 7fd517fff700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4f0053a70 0x7fd4f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.275+0000 7fd517fff700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4f0053a70 0x7fd4f0055f30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fd51000d000 tx=0x7fd510007b40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.279+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd500043c30 con 0x7fd5180aadb0 2022-08-01T23:00:57.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5180574e0 con 0x7fd5180aadb0 2022-08-01T23:00:57.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd515ffb700 1 -- 172.21.15.32:0/3641649827 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd5000150a0 con 0x7fd5180aadb0 2022-08-01T23:00:57.408 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:00:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4f0053a70 msgr2=0x7fd4f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4f0053a70 0x7fd4f0055f30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fd51000d000 tx=0x7fd510007b40).stop 2022-08-01T23:00:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 msgr2=0x7fd518138740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 0x7fd518138740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fd518146860 tx=0x7fd500016000).stop 2022-08-01T23:00:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 shutdown_connections 2022-08-01T23:00:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4f0053a70 0x7fd4f0055f30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5180b7500 0x7fd5180afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5180b31f0 0x7fd518138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 --2- 172.21.15.32:0/3641649827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5180aadb0 0x7fd518138740 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.407+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 >> 172.21.15.32:0/3641649827 conn(0x7fd51805f840 msgr2=0x7fd51805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.411+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 shutdown_connections 2022-08-01T23:00:57.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.411+0000 7fd51dc4e700 1 -- 172.21.15.32:0/3641649827 wait complete. 2022-08-01T23:00:57.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:00:57.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.527+0000 7f77299ca700 1 Processor -- start 2022-08-01T23:00:57.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.527+0000 7f77299ca700 1 -- start start 2022-08-01T23:00:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77240b31d0 0x7f7724066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77241402d0 0x7f7724140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7724067030 con 0x7f77240a9da0 2022-08-01T23:00:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7724064a70 con 0x7f77241402d0 2022-08-01T23:00:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7724061f60 con 0x7f77240b31d0 2022-08-01T23:00:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77241402d0 0x7f7724140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77241402d0 0x7f7724140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44266/0 (socket says 172.21.15.32:44266) 2022-08-01T23:00:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 -- 172.21.15.32:0/2159338749 learned_addr learned my addr 172.21.15.32:0/2159338749 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77289c8700 1 --2- 172.21.15.32:0/2159338749 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77240b31d0 0x7f7724066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 -- 172.21.15.32:0/2159338749 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77240b31d0 msgr2=0x7f7724066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 --2- 172.21.15.32:0/2159338749 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77240b31d0 0x7f7724066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 -- 172.21.15.32:0/2159338749 >> v1:172.21.15.55:6789/0 conn(0x7f77240a9da0 legacy=0x7f7724060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 -- 172.21.15.32:0/2159338749 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240b35b0 con 0x7f77241402d0 2022-08-01T23:00:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77291c9700 1 --2- 172.21.15.32:0/2159338749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77241402d0 0x7f7724140780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f771c0032f0 tx=0x7f771c00b530).ready entity=mon.0 client_cookie=4fa3b9a6cd557790 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77237fe700 1 -- 172.21.15.32:0/2159338749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f771c010660 con 0x7f77241402d0 2022-08-01T23:00:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77237fe700 1 -- 172.21.15.32:0/2159338749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c0107c0 con 0x7f77241402d0 2022-08-01T23:00:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77237fe700 1 -- 172.21.15.32:0/2159338749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f771c010b20 con 0x7f77241402d0 2022-08-01T23:00:57.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- 172.21.15.32:0/2159338749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77241402d0 msgr2=0x7f7724140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 --2- 172.21.15.32:0/2159338749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77241402d0 0x7f7724140780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f771c0032f0 tx=0x7f771c00b530).stop 2022-08-01T23:00:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- 172.21.15.32:0/2159338749 shutdown_connections 2022-08-01T23:00:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 --2- 172.21.15.32:0/2159338749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77241402d0 0x7f7724140780 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 --2- 172.21.15.32:0/2159338749 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77240b31d0 0x7f7724066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- 172.21.15.32:0/2159338749 >> 172.21.15.32:0/2159338749 conn(0x7f7724063870 msgr2=0x7f7724063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- 172.21.15.32:0/2159338749 shutdown_connections 2022-08-01T23:00:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.531+0000 7f77299ca700 1 -- 172.21.15.32:0/2159338749 wait complete. 2022-08-01T23:00:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 Processor -- start 2022-08-01T23:00:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 -- start start 2022-08-01T23:00:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77240a9da0 0x7f7724141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 0x7f772414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77241402d0 0x7f772414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240aa430 con 0x7f77240b31d0 2022-08-01T23:00:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240b4f50 con 0x7f77240a9da0 2022-08-01T23:00:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77240aa660 con 0x7f77241402d0 2022-08-01T23:00:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 0x7f772414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 0x7f772414e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44268/0 (socket says 172.21.15.32:44268) 2022-08-01T23:00:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 -- 172.21.15.32:0/1358783878 learned_addr learned my addr 172.21.15.32:0/1358783878 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77291c9700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77241402d0 0x7f772414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77289c8700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77240a9da0 0x7f7724141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 -- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77241402d0 msgr2=0x7f772414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77241402d0 0x7f772414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 -- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77240a9da0 msgr2=0x7f7724141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77240a9da0 0x7f7724141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 -- 172.21.15.32:0/1358783878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c00b7d0 con 0x7f77240b31d0 2022-08-01T23:00:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7723fff700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 0x7f772414e200 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7724065790 tx=0x7f7710007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f771000ce30 con 0x7f77240b31d0 2022-08-01T23:00:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771000f470 con 0x7f77240b31d0 2022-08-01T23:00:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77240aa900 con 0x7f77240b31d0 2022-08-01T23:00:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f771000f7d0 con 0x7f77240b31d0 2022-08-01T23:00:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.535+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77240aae40 con 0x7f77240b31d0 2022-08-01T23:00:57.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.539+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7724066af0 con 0x7f77240b31d0 2022-08-01T23:00:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.539+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7710019030 con 0x7f77240b31d0 2022-08-01T23:00:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.539+0000 7f7721ffb700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7704053950 0x7f7704055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.539+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f771006b480 con 0x7f77240b31d0 2022-08-01T23:00:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.539+0000 7f77289c8700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7704053950 0x7f7704055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.539+0000 7f77289c8700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7704053950 0x7f7704055e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f771800c000 tx=0x7f7718008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.543+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77100421e0 con 0x7f77240b31d0 2022-08-01T23:00:57.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.671+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77240574e0 con 0x7f77240b31d0 2022-08-01T23:00:57.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.671+0000 7f7721ffb700 1 -- 172.21.15.32:0/1358783878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7710045800 con 0x7f77240b31d0 2022-08-01T23:00:57.674 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:00:57.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7704053950 msgr2=0x7f7704055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7704053950 0x7f7704055e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f771800c000 tx=0x7f7718008040).stop 2022-08-01T23:00:57.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 msgr2=0x7f772414e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 0x7f772414e200 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7724065790 tx=0x7f7710007c20).stop 2022-08-01T23:00:57.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 shutdown_connections 2022-08-01T23:00:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7704053950 0x7f7704055e10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77241402d0 0x7f772414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77240b31d0 0x7f772414e200 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 --2- 172.21.15.32:0/1358783878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77240a9da0 0x7f7724141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 >> 172.21.15.32:0/1358783878 conn(0x7f7724063870 msgr2=0x7f77241406e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 shutdown_connections 2022-08-01T23:00:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.675+0000 7f77299ca700 1 -- 172.21.15.32:0/1358783878 wait complete. 2022-08-01T23:00:57.687 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:00:57.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:00:57.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.791+0000 7f4b31a6d700 1 Processor -- start 2022-08-01T23:00:57.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.791+0000 7f4b31a6d700 1 -- start start 2022-08-01T23:00:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.791+0000 7f4b31a6d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b7500 0x7f4b2c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b2c0b31f0 0x7f4b2c13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b2c13e8a0 0x7f4b2c064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c0b2510 con 0x7f4b2c13e8a0 2022-08-01T23:00:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c1443e0 con 0x7f4b2c0b31f0 2022-08-01T23:00:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0b3700 con 0x7f4b2c0b7500 2022-08-01T23:00:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b2c13e8a0 0x7f4b2c064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b2c13e8a0 0x7f4b2c064d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44280/0 (socket says 172.21.15.32:44280) 2022-08-01T23:00:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 -- 172.21.15.32:0/2453587205 learned_addr learned my addr 172.21.15.32:0/2453587205 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b2bfff700 1 --2- 172.21.15.32:0/2453587205 >> v2:172.21.15.55:3300/0 conn(0x7f4b2c0b31f0 0x7f4b2c13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b30a6b700 1 --2- 172.21.15.32:0/2453587205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b7500 0x7f4b2c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 -- 172.21.15.32:0/2453587205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b7500 msgr2=0x7f4b2c0b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 --2- 172.21.15.32:0/2453587205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b7500 0x7f4b2c0b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 -- 172.21.15.32:0/2453587205 >> v2:172.21.15.55:3300/0 conn(0x7f4b2c0b31f0 msgr2=0x7f4b2c13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:00:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 --2- 172.21.15.32:0/2453587205 >> v2:172.21.15.55:3300/0 conn(0x7f4b2c0b31f0 0x7f4b2c13e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 -- 172.21.15.32:0/2453587205 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0b2740 con 0x7f4b2c13e8a0 2022-08-01T23:00:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b3126c700 1 --2- 172.21.15.32:0/2453587205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c064d30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f4b1c002f70 tx=0x7f4b1c006eb0).ready entity=mon.0 client_cookie=1ff85a2fd3f10452 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b2b7fe700 1 -- 172.21.15.32:0/2453587205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1c00d570 con 0x7f4b2c13e8a0 2022-08-01T23:00:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b2b7fe700 1 -- 172.21.15.32:0/2453587205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c000e90 con 0x7f4b2c13e8a0 2022-08-01T23:00:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b2b7fe700 1 -- 172.21.15.32:0/2453587205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1c00d900 con 0x7f4b2c13e8a0 2022-08-01T23:00:57.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2453587205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c13e8a0 msgr2=0x7f4b2c064d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2453587205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c064d30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f4b1c002f70 tx=0x7f4b1c006eb0).stop 2022-08-01T23:00:57.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2453587205 shutdown_connections 2022-08-01T23:00:57.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2453587205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c064d30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2453587205 >> v2:172.21.15.55:3300/0 conn(0x7f4b2c0b31f0 0x7f4b2c13e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2453587205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b7500 0x7f4b2c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2453587205 >> 172.21.15.32:0/2453587205 conn(0x7f4b2c05f840 msgr2=0x7f4b2c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2453587205 shutdown_connections 2022-08-01T23:00:57.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.795+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2453587205 wait complete. 2022-08-01T23:00:57.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 Processor -- start 2022-08-01T23:00:57.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 -- start start 2022-08-01T23:00:57.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b31f0 0x7f4b2c09ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 0x7f4b2c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0b17d0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1537e0 con 0x7f4b2c13e8a0 2022-08-01T23:00:57.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0b19c0 con 0x7f4b2c0b31f0 2022-08-01T23:00:57.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 0x7f4b2c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 0x7f4b2c152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44284/0 (socket says 172.21.15.32:44284) 2022-08-01T23:00:57.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 -- 172.21.15.32:0/2124208217 learned_addr learned my addr 172.21.15.32:0/2124208217 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:00:57.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b3126c700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b30a6b700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b31f0 0x7f4b2c09ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 -- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b31f0 msgr2=0x7f4b2c09ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b31f0 0x7f4b2c09ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 -- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b2c13e8a0 msgr2=0x7f4b2c1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 -- 172.21.15.32:0/2124208217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c00b1e0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b3126c700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:00:57.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b2bfff700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 0x7f4b2c152e00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f4b2c153920 tx=0x7f4b14006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b14010780 con 0x7f4b2c0b7500 2022-08-01T23:00:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b140108e0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c1529a0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b14010c40 con 0x7f4b2c0b7500 2022-08-01T23:00:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.799+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c09f430 con 0x7f4b2c0b7500 2022-08-01T23:00:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.803+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4b1401c460 con 0x7f4b2c0b7500 2022-08-01T23:00:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.803+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c1423f0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.803+0000 7f4b29ffb700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4b10053a70 0x7f4b10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:00:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.803+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4b1406c5e0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.803+0000 7f4b30a6b700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4b10053a70 0x7f4b10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:00:57.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.803+0000 7f4b30a6b700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4b10053a70 0x7f4b10055f30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f4b2400b000 tx=0x7f4b240091a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:00:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.807+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b2c1423f0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b2c1423f0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b29ffb700 1 -- 172.21.15.32:0/2124208217 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f4b2c1423f0 con 0x7f4b2c0b7500 2022-08-01T23:00:57.972 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:00:57.972 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:00:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4b10053a70 msgr2=0x7f4b10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4b10053a70 0x7f4b10055f30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f4b2400b000 tx=0x7f4b240091a0).stop 2022-08-01T23:00:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 msgr2=0x7f4b2c152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:00:57.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 0x7f4b2c152e00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f4b2c153920 tx=0x7f4b14006df0).stop 2022-08-01T23:00:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 shutdown_connections 2022-08-01T23:00:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4b10053a70 0x7f4b10055f30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b2c13e8a0 0x7f4b2c1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b2c0b7500 0x7f4b2c152e00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 --2- 172.21.15.32:0/2124208217 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4b2c0b31f0 0x7f4b2c09ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:00:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 >> 172.21.15.32:0/2124208217 conn(0x7f4b2c05f840 msgr2=0x7f4b2c05cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:00:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 shutdown_connections 2022-08-01T23:00:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:00:57.971+0000 7f4b31a6d700 1 -- 172.21.15.32:0/2124208217 wait complete. 2022-08-01T23:00:57.977 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:00:58.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:00:53.364281+0000', 'modified': '2022-08-01T23:00:54.368935+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5934}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5934': {'gid': 5934, 'name': 'b', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/590504455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 590504455}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 590504455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:00:58.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:00:58.288 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T23:00:58.403 INFO:teuthology.orchestra.run.smithi055.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":5934,"want_state":"up:active","state":"up:active","rank_uptime":5.0305628039999997,"mdsmap_epoch":28,"osdmap_epoch":67,"osdmap_epoch_barrier":66,"uptime":9.3974998200000002} 2022-08-01T23:00:58.404 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 5934, "mdsmap_epoch": 28, "osdmap_epoch": 67, "osdmap_epoch_barrier": 66, "rank_uptime": 5.030562804, "state": "up:active", "uptime": 9.39749982, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:00:58.405 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:00:58.405 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:00:58.405 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:00:58.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:58.406 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:58.415 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:00:58.432 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:00:58.433 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:00:58.443 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:00:58.444 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:00:58.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:58.446 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:00:58.495 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:00:58.524 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:00:58.527 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:00:58.542 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:00:58.542 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:00:58.601 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:00:58.602 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:00:58.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:58.603 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:00:58.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:00:58.605+0000 7fbc520eb080 -1 init, newargv = 0x5584b2f4fe90 newargc=9 2022-08-01T23:00:58.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22873]: starting ceph client 2022-08-01T23:00:58.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22873]: starting fuse 2022-08-01T23:00:59.713 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:00:59.731 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:00:59.732 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:00:59.788 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:00:59.789 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:00:59.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:59.791 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:00:59.796 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:00:59.796 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:00:59.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:59.797 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:00:59.797 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:00:59.798 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:00:59.798 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:00:59.799 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:00:59.799 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:00:59.799 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:00:59.800 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:00:59.800 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:00:59.801 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:00:59.801 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:00:59.802 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:00:59.802 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:00:59.802 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:00:59.803 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:00:59.803 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:00:59.804 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:00:59.804 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:00:59.804 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:00:59.805 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:00:59.805 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:00:59.806 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:00:59.806 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:00:59.846 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22873.asok 2022-08-01T23:00:59.851 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22873.asok 2022-08-01T23:00:59.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:59.852 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22873.asok status 2022-08-01T23:00:59.987 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:00:59.987 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:00:59.988 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:00:59.988 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:00:59.988 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:00:59.988 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:00:59.989 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:00:59.989 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22873", 2022-08-01T23:00:59.989 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:00:59.989 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:00:59.989 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:00:59.990 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:00:59.990 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 6096, 2022-08-01T23:00:59.990 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:00:59.990 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:00:59.991 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:00:59.991 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 6096 2022-08-01T23:00:59.991 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:00:59.991 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:00:59.991 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:00:59.992 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:00:59.992 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 4234835569 2022-08-01T23:00:59.992 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:00:59.992 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:00:59.993 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:00:59.993 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:00:59.993 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:00:59.993 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 4234835569 2022-08-01T23:00:59.993 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:00:59.994 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.6096 172.21.15.130:0/4234835569", 2022-08-01T23:00:59.994 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/4234835569", 2022-08-01T23:00:59.994 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:00:59.994 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 28, 2022-08-01T23:00:59.994 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 67, 2022-08-01T23:00:59.995 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:00:59.995 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:00:59.995 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:00:59.995 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:00:59.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:00:59.997 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:00:59.998 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:00.004 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:01:00.005 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:01:00.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:00.006 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:00.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:00.070 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:00.070 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:00.078 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:01:00.079 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:01:00.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T23:01:00.080 DEBUG:teuthology.orchestra.run.smithi130:> import sys 2022-08-01T23:01:00.080 DEBUG:teuthology.orchestra.run.smithi130:> import time 2022-08-01T23:01:00.081 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:01:00.081 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:00.082 DEBUG:teuthology.orchestra.run.smithi130:> n = 1000 2022-08-01T23:01:00.082 DEBUG:teuthology.orchestra.run.smithi130:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T23:01:00.082 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:00.083 DEBUG:teuthology.orchestra.run.smithi130:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T23:01:00.083 DEBUG:teuthology.orchestra.run.smithi130:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T23:01:00.084 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:00.084 DEBUG:teuthology.orchestra.run.smithi130:> for i in range(0, n): 2022-08-01T23:01:00.084 DEBUG:teuthology.orchestra.run.smithi130:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T23:01:00.085 DEBUG:teuthology.orchestra.run.smithi130:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T23:01:00.085 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content'"'"') 2022-08-01T23:01:00.086 DEBUG:teuthology.orchestra.run.smithi130:> if True: 2022-08-01T23:01:00.086 DEBUG:teuthology.orchestra.run.smithi130:> f.flush() 2022-08-01T23:01:00.086 DEBUG:teuthology.orchestra.run.smithi130:> os.fsync(f.fileno()) 2022-08-01T23:01:00.087 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:01:10.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 Processor -- start 2022-08-01T23:01:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 -- start start 2022-08-01T23:01:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4584148530 0x7f45840b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45840a7250 0x7f45841448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45840b04e0 con 0x7f45840b1f40 2022-08-01T23:01:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45840a9250 con 0x7f4584148530 2022-08-01T23:01:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f458a20f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45840b0780 con 0x7f45840a7250 2022-08-01T23:01:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45837fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45840b1f40 0x7f45840b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60064/0 (socket says 172.21.15.32:60064) 2022-08-01T23:01:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45837fe700 1 -- 172.21.15.32:0/1118455989 learned_addr learned my addr 172.21.15.32:0/1118455989 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4582ffd700 1 --2- 172.21.15.32:0/1118455989 >> v2:172.21.15.55:3300/0 conn(0x7f4584148530 0x7f45840b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4583fff700 1 --2- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45840a7250 0x7f45841448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45827fc700 1 -- 172.21.15.32:0/1118455989 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072772304 0 0) 0x7f45840b04e0 con 0x7f45840b1f40 2022-08-01T23:01:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45827fc700 1 -- 172.21.15.32:0/1118455989 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4568003260 con 0x7f45840b1f40 2022-08-01T23:01:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4582ffd700 1 -- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45840a7250 msgr2=0x7f45841448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4582ffd700 1 --2- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45840a7250 0x7f45841448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4582ffd700 1 -- 172.21.15.32:0/1118455989 >> v1:172.21.15.32:6789/0 conn(0x7f45840b1f40 legacy=0x7f45840b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45827fc700 1 -- 172.21.15.32:0/1118455989 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3062068877 0 0) 0x7f4568003260 con 0x7f45840b1f40 2022-08-01T23:01:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4582ffd700 1 -- 172.21.15.32:0/1118455989 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45840b0a20 con 0x7f4584148530 2022-08-01T23:01:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f4582ffd700 1 --2- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4584148530 0x7f45840b1160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f456c001800 tx=0x7f45740091d0).ready entity=mon.1 client_cookie=bcadf0670c4f0180 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45827fc700 1 -- 172.21.15.32:0/1118455989 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f456c001660 con 0x7f4584148530 2022-08-01T23:01:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45827fc700 1 -- 172.21.15.32:0/1118455989 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457400f510 con 0x7f4584148530 2022-08-01T23:01:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.515+0000 7f45827fc700 1 -- 172.21.15.32:0/1118455989 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f457400f680 con 0x7f4584148530 2022-08-01T23:01:10.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 -- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4584148530 msgr2=0x7f45840b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 --2- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4584148530 0x7f45840b1160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f456c001800 tx=0x7f45740091d0).stop 2022-08-01T23:01:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 -- 172.21.15.32:0/1118455989 shutdown_connections 2022-08-01T23:01:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 --2- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45840a7250 0x7f45841448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 --2- 172.21.15.32:0/1118455989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4584148530 0x7f45840b1160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 -- 172.21.15.32:0/1118455989 >> 172.21.15.32:0/1118455989 conn(0x7f458405f840 msgr2=0x7f458405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 -- 172.21.15.32:0/1118455989 shutdown_connections 2022-08-01T23:01:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 -- 172.21.15.32:0/1118455989 wait complete. 2022-08-01T23:01:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 Processor -- start 2022-08-01T23:01:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.519+0000 7f458a20f700 1 -- start start 2022-08-01T23:01:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 0x7f45840ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45840b1f40 0x7f45840af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4584148530 0x7f4584141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45840ad9e0 con 0x7f45840a7250 2022-08-01T23:01:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45840ad720 con 0x7f45840b1f40 2022-08-01T23:01:10.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45840adbd0 con 0x7f4584148530 2022-08-01T23:01:10.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 0x7f45840ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 0x7f45840ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44296/0 (socket says 172.21.15.32:44296) 2022-08-01T23:01:10.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 -- 172.21.15.32:0/2424211926 learned_addr learned my addr 172.21.15.32:0/2424211926 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:10.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f4583fff700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4584148530 0x7f4584141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f4582ffd700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45840b1f40 0x7f45840af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 -- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4584148530 msgr2=0x7f4584141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4584148530 0x7f4584141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 -- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45840b1f40 msgr2=0x7f45840af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45840b1f40 0x7f45840af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 -- 172.21.15.32:0/2424211926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4574009470 con 0x7f45840a7250 2022-08-01T23:01:10.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f45837fe700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 0x7f45840ae180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f456c001430 tx=0x7f456c001ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:10.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f456c007170 con 0x7f45840a7250 2022-08-01T23:01:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45840afb00 con 0x7f45840a7250 2022-08-01T23:01:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45840af820 con 0x7f45840a7250 2022-08-01T23:01:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c0072d0 con 0x7f45840a7250 2022-08-01T23:01:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f456c004920 con 0x7f45840a7250 2022-08-01T23:01:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4560003410 con 0x7f45840a7250 2022-08-01T23:01:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.523+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f456c01b070 con 0x7f45840a7250 2022-08-01T23:01:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.527+0000 7f4580ff9700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f455c053a70 0x7f455c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.527+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f456c06b990 con 0x7f45840a7250 2022-08-01T23:01:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.527+0000 7f4582ffd700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f455c053a70 0x7f455c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.527+0000 7f4582ffd700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f455c053a70 0x7f455c055f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f457401a000 tx=0x7f4574016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.531+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f456c0426b0 con 0x7f45840a7250 2022-08-01T23:01:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.691+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4560003c40 con 0x7f45840a7250 2022-08-01T23:01:10.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f4580ff9700 1 -- 172.21.15.32:0/2424211926 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f456c045cd0 con 0x7f45840a7250 2022-08-01T23:01:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:10.695 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:01:10.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f455c053a70 msgr2=0x7f455c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f455c053a70 0x7f455c055f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f457401a000 tx=0x7f4574016000).stop 2022-08-01T23:01:10.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 msgr2=0x7f45840ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 0x7f45840ae180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f456c001430 tx=0x7f456c001ae0).stop 2022-08-01T23:01:10.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 shutdown_connections 2022-08-01T23:01:10.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f455c053a70 0x7f455c055f30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4584148530 0x7f4584141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45840b1f40 0x7f45840af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 --2- 172.21.15.32:0/2424211926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45840a7250 0x7f45840ae180 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 >> 172.21.15.32:0/2424211926 conn(0x7f458405f840 msgr2=0x7f45840b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 shutdown_connections 2022-08-01T23:01:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.695+0000 7f458a20f700 1 -- 172.21.15.32:0/2424211926 wait complete. 2022-08-01T23:01:10.700 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:01:10.707 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-08-01T23:01:10.814 INFO:teuthology.orchestra.run.smithi055.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T23:01:10.815 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T23:01:10.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:10.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f254215a700 1 Processor -- start 2022-08-01T23:01:10.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f254215a700 1 -- start start 2022-08-01T23:01:10.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f254215a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f253c0b01a0 0x7f253c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f254215a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0a7250 con 0x7f253c0b2710 2022-08-01T23:01:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f254215a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0a7480 con 0x7f253c0afd60 2022-08-01T23:01:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f254215a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0a7720 con 0x7f253c0b01a0 2022-08-01T23:01:10.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f2541158700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f253c0b2710 0x7f253c14afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60078/0 (socket says 172.21.15.32:60078) 2022-08-01T23:01:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f2541158700 1 -- 172.21.15.32:0/3871861154 learned_addr learned my addr 172.21.15.32:0/3871861154 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f2540957700 1 --2- 172.21.15.32:0/3871861154 >> v2:172.21.15.55:3300/0 conn(0x7f253c0b01a0 0x7f253c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772964725 0 0) 0x7f253c0a7250 con 0x7f253c0b2710 2022-08-01T23:01:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.923+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003260 con 0x7f253c0b2710 2022-08-01T23:01:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822291544 0 0) 0x7f253c0a7480 con 0x7f253c0afd60 2022-08-01T23:01:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f253c0a7250 con 0x7f253c0afd60 2022-08-01T23:01:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962173267 0 0) 0x7f2524003260 con 0x7f253c0b2710 2022-08-01T23:01:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 >> v2:172.21.15.55:3300/0 conn(0x7f253c0b01a0 msgr2=0x7f253c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 --2- 172.21.15.32:0/3871861154 >> v2:172.21.15.55:3300/0 conn(0x7f253c0b01a0 0x7f253c0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 >> v1:172.21.15.55:6789/0 conn(0x7f253c0afd60 legacy=0x7f253c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0a79c0 con 0x7f253c0b2710 2022-08-01T23:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f2538001660 con 0x7f253c0b2710 2022-08-01T23:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 >> v1:172.21.15.32:6789/0 conn(0x7f253c0b2710 legacy=0x7f253c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2524003930 0x7f2524003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2524004500 0x7f2524004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524004e70 0x7f252400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0a7720 con 0x7f2524003930 2022-08-01T23:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252400bd20 con 0x7f2524004e70 2022-08-01T23:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252400bee0 con 0x7f2524004500 2022-08-01T23:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2524003930 0x7f2524003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541959700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524004e70 0x7f252400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 -- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2524004500 msgr2=0x7f2524004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2524004500 0x7f2524004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T23:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 -- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524004e70 msgr2=0x7f252400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524004e70 0x7f252400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 -- 172.21.15.32:0/3871861154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524003260 con 0x7f2524003930 2022-08-01T23:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541959700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524004e70 0x7f252400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541959700 1 -- 172.21.15.32:0/3871861154 reap_dead start 2022-08-01T23:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f2541158700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2524003930 0x7f2524003d30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2538001430 tx=0x7f253c0a7610).ready entity=mon.0 client_cookie=9b5147a5fd2b12d8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25380070d0 con 0x7f2524003930 2022-08-01T23:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538007230 con 0x7f2524003930 2022-08-01T23:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f252bfff700 1 -- 172.21.15.32:0/3871861154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f253800c8f0 con 0x7f2524003930 2022-08-01T23:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 -- 172.21.15.32:0/3871861154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2524003930 msgr2=0x7f2524003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2524003930 0x7f2524003d30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2538001430 tx=0x7f253c0a7610).stop 2022-08-01T23:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 -- 172.21.15.32:0/3871861154 shutdown_connections 2022-08-01T23:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 --2- 172.21.15.32:0/3871861154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2524003930 0x7f2524003d30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 -- 172.21.15.32:0/3871861154 >> 172.21.15.32:0/3871861154 conn(0x7f253c060310 msgr2=0x7f253c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 -- 172.21.15.32:0/3871861154 shutdown_connections 2022-08-01T23:01:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.927+0000 7f254215a700 1 -- 172.21.15.32:0/3871861154 wait complete. 2022-08-01T23:01:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 Processor -- start 2022-08-01T23:01:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 -- start start 2022-08-01T23:01:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524003930 0x7f253c066bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 0x7f253c148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f253c063520 0x7f253c0a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0abc00 con 0x7f253c064450 2022-08-01T23:01:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0ac610 con 0x7f2524003930 2022-08-01T23:01:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f254215a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0ac0d0 con 0x7f253c063520 2022-08-01T23:01:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 0x7f253c148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 0x7f253c148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44316/0 (socket says 172.21.15.32:44316) 2022-08-01T23:01:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 -- 172.21.15.32:0/920246860 learned_addr learned my addr 172.21.15.32:0/920246860 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2541158700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524003930 0x7f253c066bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2541959700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f253c063520 0x7f253c0a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 -- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f253c063520 msgr2=0x7f253c0a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f253c063520 0x7f253c0a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 -- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524003930 msgr2=0x7f253c066bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524003930 0x7f253c066bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.931+0000 7f2540957700 1 -- 172.21.15.32:0/920246860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0ac370 con 0x7f253c064450 2022-08-01T23:01:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f2540957700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 0x7f253c148e20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f252c00c000 tx=0x7f252c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f252c016070 con 0x7f253c064450 2022-08-01T23:01:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c000e90 con 0x7f253c064450 2022-08-01T23:01:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c0ab880 con 0x7f253c064450 2022-08-01T23:01:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f252c010970 con 0x7f253c064450 2022-08-01T23:01:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c0639a0 con 0x7f253c064450 2022-08-01T23:01:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c0574e0 con 0x7f253c064450 2022-08-01T23:01:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f252c010ad0 con 0x7f253c064450 2022-08-01T23:01:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f252a7fc700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f251c053a70 0x7f251c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f252c06c0c0 con 0x7f253c064450 2022-08-01T23:01:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.935+0000 7f2541158700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f251c053a70 0x7f251c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.939+0000 7f2541158700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f251c053a70 0x7f251c055f30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f2538019000 tx=0x7f2538013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:10.943+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f252c042cf0 con 0x7f253c064450 2022-08-01T23:01:11.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.099+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f253c148b50 con 0x7f253c064450 2022-08-01T23:01:11.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f252a7fc700 1 -- 172.21.15.32:0/920246860 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f252c046310 con 0x7f253c064450 2022-08-01T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:11.103 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:01:11.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f251c053a70 msgr2=0x7f251c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f251c053a70 0x7f251c055f30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f2538019000 tx=0x7f2538013000).stop 2022-08-01T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 msgr2=0x7f253c148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 0x7f253c148e20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f252c00c000 tx=0x7f252c006df0).stop 2022-08-01T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 shutdown_connections 2022-08-01T23:01:11.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f251c053a70 0x7f251c055f30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f253c063520 0x7f253c0a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253c064450 0x7f253c148e20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 --2- 172.21.15.32:0/920246860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2524003930 0x7f253c066bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 >> 172.21.15.32:0/920246860 conn(0x7f253c060310 msgr2=0x7f253c0676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 shutdown_connections 2022-08-01T23:01:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.103+0000 7f254215a700 1 -- 172.21.15.32:0/920246860 wait complete. 2022-08-01T23:01:11.107 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:01:11.115 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T23:01:11.229 INFO:teuthology.orchestra.run.smithi055.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T23:01:11.229 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T23:01:11.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.339+0000 7fcd43fff700 1 Processor -- start 2022-08-01T23:01:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.339+0000 7fcd43fff700 1 -- start start 2022-08-01T23:01:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd440b2710 0x7fcd4414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440a7250 con 0x7fcd440a8c50 2022-08-01T23:01:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440a7480 con 0x7fcd440b01a0 2022-08-01T23:01:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440a7720 con 0x7fcd440b2710 2022-08-01T23:01:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd437fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcd440a8c50 0x7fcd44151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60100/0 (socket says 172.21.15.32:60100) 2022-08-01T23:01:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd437fe700 1 -- 172.21.15.32:0/3735961552 learned_addr learned my addr 172.21.15.32:0/3735961552 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd42ffd700 1 --2- 172.21.15.32:0/3735961552 >> v2:172.21.15.55:3300/0 conn(0x7fcd440b2710 0x7fcd4414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd427fc700 1 -- 172.21.15.32:0/3735961552 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749181170 0 0) 0x7fcd440a7250 con 0x7fcd440a8c50 2022-08-01T23:01:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd427fc700 1 -- 172.21.15.32:0/3735961552 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd24003260 con 0x7fcd440a8c50 2022-08-01T23:01:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd42ffd700 1 -- 172.21.15.32:0/3735961552 >> v1:172.21.15.55:6789/0 conn(0x7fcd440b01a0 legacy=0x7fcd440a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd42ffd700 1 -- 172.21.15.32:0/3735961552 >> v1:172.21.15.32:6789/0 conn(0x7fcd440a8c50 legacy=0x7fcd44151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd42ffd700 1 -- 172.21.15.32:0/3735961552 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440a79c0 con 0x7fcd440b2710 2022-08-01T23:01:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd42ffd700 1 --2- 172.21.15.32:0/3735961552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd4414afd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fcd34002f70 tx=0x7fcd34006df0).ready entity=mon.1 client_cookie=64709b8ba8270818 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd427fc700 1 -- 172.21.15.32:0/3735961552 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd3400d610 con 0x7fcd440b2710 2022-08-01T23:01:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd427fc700 1 -- 172.21.15.32:0/3735961552 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34000e90 con 0x7fcd440b2710 2022-08-01T23:01:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd427fc700 1 -- 172.21.15.32:0/3735961552 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd3400d960 con 0x7fcd440b2710 2022-08-01T23:01:11.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- 172.21.15.32:0/3735961552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 msgr2=0x7fcd4414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 --2- 172.21.15.32:0/3735961552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd4414afd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fcd34002f70 tx=0x7fcd34006df0).stop 2022-08-01T23:01:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- 172.21.15.32:0/3735961552 shutdown_connections 2022-08-01T23:01:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 --2- 172.21.15.32:0/3735961552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd4414afd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- 172.21.15.32:0/3735961552 >> 172.21.15.32:0/3735961552 conn(0x7fcd44060310 msgr2=0x7fcd440b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- 172.21.15.32:0/3735961552 shutdown_connections 2022-08-01T23:01:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.343+0000 7fcd43fff700 1 -- 172.21.15.32:0/3735961552 wait complete. 2022-08-01T23:01:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 Processor -- start 2022-08-01T23:01:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 -- start start 2022-08-01T23:01:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd440a8c50 0x7fcd440af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 0x7fcd440b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd440b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440b3d40 con 0x7fcd440b01a0 2022-08-01T23:01:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440b3ae0 con 0x7fcd440b2710 2022-08-01T23:01:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440b3070 con 0x7fcd440a8c50 2022-08-01T23:01:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 0x7fcd440b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 0x7fcd440b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44330/0 (socket says 172.21.15.32:44330) 2022-08-01T23:01:11.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 -- 172.21.15.32:0/252396389 learned_addr learned my addr 172.21.15.32:0/252396389 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:11.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd437fe700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd440b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd42ffd700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd440a8c50 0x7fcd440af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 -- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd440a8c50 msgr2=0x7fcd440af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd440a8c50 0x7fcd440af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 -- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 msgr2=0x7fcd440b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd440b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 -- 172.21.15.32:0/252396389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3400b180 con 0x7fcd440b01a0 2022-08-01T23:01:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd437fe700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd440b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd3bfff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 0x7fcd440b4a10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fcd44061c10 tx=0x7fcd2c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2c00ce30 con 0x7fcd440b01a0 2022-08-01T23:01:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c007240 con 0x7fcd440b01a0 2022-08-01T23:01:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4414be30 con 0x7fcd440b01a0 2022-08-01T23:01:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2c006700 con 0x7fcd440b01a0 2022-08-01T23:01:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.347+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd440b40a0 con 0x7fcd440b01a0 2022-08-01T23:01:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.351+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd440574e0 con 0x7fcd440b01a0 2022-08-01T23:01:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.351+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd2c010030 con 0x7fcd440b01a0 2022-08-01T23:01:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.351+0000 7fcd40ff9700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd280539e0 0x7fcd28055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.351+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcd2c06bd80 con 0x7fcd440b01a0 2022-08-01T23:01:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.351+0000 7fcd42ffd700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd280539e0 0x7fcd28055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.351+0000 7fcd42ffd700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd280539e0 0x7fcd28055ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fcd34018000 tx=0x7fcd34014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.359+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd2c042aa0 con 0x7fcd440b01a0 2022-08-01T23:01:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.523+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd44069a70 con 0x7fcd440b01a0 2022-08-01T23:01:11.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.523+0000 7fcd40ff9700 1 -- 172.21.15.32:0/252396389 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fcd2c06d030 con 0x7fcd440b01a0 2022-08-01T23:01:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:11.525 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:01:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd280539e0 msgr2=0x7fcd28055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd280539e0 0x7fcd28055ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fcd34018000 tx=0x7fcd34014000).stop 2022-08-01T23:01:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 msgr2=0x7fcd440b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 0x7fcd440b4a10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fcd44061c10 tx=0x7fcd2c007bb0).stop 2022-08-01T23:01:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 shutdown_connections 2022-08-01T23:01:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd280539e0 0x7fcd28055ea0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd440b2710 0x7fcd440b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd440b01a0 0x7fcd440b4a10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 --2- 172.21.15.32:0/252396389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd440a8c50 0x7fcd440af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 >> 172.21.15.32:0/252396389 conn(0x7fcd44060310 msgr2=0x7fcd4414a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 shutdown_connections 2022-08-01T23:01:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.527+0000 7fcd43fff700 1 -- 172.21.15.32:0/252396389 wait complete. 2022-08-01T23:01:11.529 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:01:11.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 10 2022-08-01T23:01:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 Processor -- start 2022-08-01T23:01:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 -- start start 2022-08-01T23:01:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8d00b6510 0x7fd8d0140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8d0061480 0x7fd8d00b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d01467f0 con 0x7fd8d00b2aa0 2022-08-01T23:01:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d0062140 con 0x7fd8d0061480 2022-08-01T23:01:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.647+0000 7fd8d5587700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d00656b0 con 0x7fd8d00b6510 2022-08-01T23:01:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8c77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd8d00b2aa0 0x7fd8d00b0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60110/0 (socket says 172.21.15.32:60110) 2022-08-01T23:01:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d4d86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8d0061480 0x7fd8d00b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8c77fe700 1 -- 172.21.15.32:0/1574402078 learned_addr learned my addr 172.21.15.32:0/1574402078 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1574402078 >> v2:172.21.15.55:3300/0 conn(0x7fd8d00b6510 0x7fd8d0140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d4d86700 1 -- 172.21.15.32:0/1574402078 >> v2:172.21.15.55:3300/0 conn(0x7fd8d00b6510 msgr2=0x7fd8d0140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1574402078 >> v2:172.21.15.55:3300/0 conn(0x7fd8d00b6510 0x7fd8d0140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d4d86700 1 -- 172.21.15.32:0/1574402078 >> v1:172.21.15.32:6789/0 conn(0x7fd8d00b2aa0 legacy=0x7fd8d00b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d4d86700 1 -- 172.21.15.32:0/1574402078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d01476a0 con 0x7fd8d0061480 2022-08-01T23:01:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1574402078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00b1570 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fd8c80032f0 tx=0x7fd8c80091d0).ready entity=mon.0 client_cookie=6e961aed472d0ad5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cf7fe700 1 -- 172.21.15.32:0/1574402078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c800b680 con 0x7fd8d0061480 2022-08-01T23:01:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cf7fe700 1 -- 172.21.15.32:0/1574402078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800b7e0 con 0x7fd8d0061480 2022-08-01T23:01:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cf7fe700 1 -- 172.21.15.32:0/1574402078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c800b990 con 0x7fd8d0061480 2022-08-01T23:01:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- 172.21.15.32:0/1574402078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 msgr2=0x7fd8d00b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1574402078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00b1570 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7fd8c80032f0 tx=0x7fd8c80091d0).stop 2022-08-01T23:01:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- 172.21.15.32:0/1574402078 shutdown_connections 2022-08-01T23:01:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1574402078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00b1570 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1574402078 >> v2:172.21.15.55:3300/0 conn(0x7fd8d00b6510 0x7fd8d0140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- 172.21.15.32:0/1574402078 >> 172.21.15.32:0/1574402078 conn(0x7fd8d0063800 msgr2=0x7fd8d0063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- 172.21.15.32:0/1574402078 shutdown_connections 2022-08-01T23:01:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- 172.21.15.32:0/1574402078 wait complete. 2022-08-01T23:01:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 Processor -- start 2022-08-01T23:01:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- start start 2022-08-01T23:01:11.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd8d00b2aa0 0x7fd8d0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8d00b6510 0x7fd8d0156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0143b70 con 0x7fd8d0061480 2022-08-01T23:01:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0142bc0 con 0x7fd8d00b6510 2022-08-01T23:01:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8d5587700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00ad040 con 0x7fd8d00b2aa0 2022-08-01T23:01:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44340/0 (socket says 172.21.15.32:44340) 2022-08-01T23:01:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.651+0000 7fd8cffff700 1 -- 172.21.15.32:0/1933828770 learned_addr learned my addr 172.21.15.32:0/1933828770 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:11.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd8d00b2aa0 0x7fd8d0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8d00b6510 0x7fd8d0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cffff700 1 -- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd8d00b2aa0 msgr2=0x7fd8d0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd8d00b2aa0 0x7fd8d0155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cffff700 1 -- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8d00b6510 msgr2=0x7fd8d0156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8d00b6510 0x7fd8d0156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cffff700 1 -- 172.21.15.32:0/1933828770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8009470 con 0x7fd8d0061480 2022-08-01T23:01:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8d00b6510 0x7fd8d0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd8d00b2aa0 0x7fd8d0155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00ac750 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fd8c000c000 tx=0x7fd8c0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cdffb700 1 -- 172.21.15.32:0/1933828770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c0010780 con 0x7fd8d0061480 2022-08-01T23:01:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cdffb700 1 -- 172.21.15.32:0/1933828770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c00108e0 con 0x7fd8d0061480 2022-08-01T23:01:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00ae260 con 0x7fd8d0061480 2022-08-01T23:01:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cdffb700 1 -- 172.21.15.32:0/1933828770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c0010c40 con 0x7fd8d0061480 2022-08-01T23:01:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d0144140 con 0x7fd8d0061480 2022-08-01T23:01:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cdffb700 1 -- 172.21.15.32:0/1933828770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd8c001c460 con 0x7fd8d0061480 2022-08-01T23:01:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.655+0000 7fd8cdffb700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd8b4053b00 0x7fd8b4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8cdffb700 1 -- 172.21.15.32:0/1933828770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd8c006c570 con 0x7fd8d0061480 2022-08-01T23:01:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd8b4053b00 0x7fd8b4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd8b4053b00 0x7fd8b4055fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fd8b800b000 tx=0x7fd8b8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 Processor -- start 2022-08-01T23:01:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 -- start start 2022-08-01T23:01:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x1118f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x111adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 0x111e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x107a3c0 con 0x111b2b0 2022-08-01T23:01:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x108e050 con 0x111a940 2022-08-01T23:01:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x106e290 con 0x1118b40 2022-08-01T23:01:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 0x111e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 0x111e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44352/0 (socket says 172.21.15.32:44352) 2022-08-01T23:01:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 -- 172.21.15.32:0/2813291549 learned_addr learned my addr 172.21.15.32:0/2813291549 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x111adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8cffff700 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x1118f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 -- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 msgr2=0x1118f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x1118f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 -- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 msgr2=0x111adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x111adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 -- 172.21.15.32:0/2813291549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10c5e90 con 0x111b2b0 2022-08-01T23:01:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.659+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 0x111e940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fd8c8019000 tx=0x7fd8c8016000).ready entity=mon.0 client_cookie=720c2d6f32c6ac50 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8d5587700 1 -- 172.21.15.32:0/2813291549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c800bd70 con 0x111b2b0 2022-08-01T23:01:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8d5587700 1 -- 172.21.15.32:0/2813291549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800bed0 con 0x111b2b0 2022-08-01T23:01:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8d5587700 1 -- 172.21.15.32:0/2813291549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c80205f0 con 0x111b2b0 2022-08-01T23:01:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2813291549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 msgr2=0x111e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 0x111e940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fd8c8019000 tx=0x7fd8c8016000).stop 2022-08-01T23:01:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2813291549 shutdown_connections 2022-08-01T23:01:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111b2b0 0x111e940 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x111adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2813291549 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x1118f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2813291549 >> 172.21.15.32:0/2813291549 conn(0x1156930 msgr2=0x1156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2813291549 shutdown_connections 2022-08-01T23:01:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2813291549 wait complete. 2022-08-01T23:01:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- start start 2022-08-01T23:01:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 Processor -- start 2022-08-01T23:01:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x1165670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111a940 0x1165b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x11211c0 con 0x1118b40 2022-08-01T23:01:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x1121460 con 0x111b2b0 2022-08-01T23:01:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x11651d0 con 0x111a940 2022-08-01T23:01:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8e0a40740 10 client.0 ll_register_callbacks cb 0x10d5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T23:01:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x1165670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x1165670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44354/0 (socket says 172.21.15.32:44354) 2022-08-01T23:01:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 -- 172.21.15.32:0/2241743456 learned_addr learned my addr 172.21.15.32:0/2241743456 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111a940 0x1165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 -- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111a940 msgr2=0x1165b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111a940 0x1165b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 -- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 msgr2=0x1161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 -- 172.21.15.32:0/2241743456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d01476a0 con 0x1118b40 2022-08-01T23:01:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8cffff700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x1165670 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd8c0025000 tx=0x7fd8c0014000).ready entity=mon.0 client_cookie=f20528ae195296af server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.663+0000 7fd8aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T23:01:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 1 -- 172.21.15.32:0/2241743456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c006abe0 con 0x1118b40 2022-08-01T23:01:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 1 -- 172.21.15.32:0/2241743456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd8c006ad40 con 0x1118b40 2022-08-01T23:01:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1084000 con 0x1118b40 2022-08-01T23:01:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 1 -- 172.21.15.32:0/2241743456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c00463e0 con 0x1118b40 2022-08-01T23:01:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 10 client.6129 fetch_fsmap learned FSMap version 28 2022-08-01T23:01:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x10609f0 con 0x1118b40 2022-08-01T23:01:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 1 -- 172.21.15.32:0/2241743456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1084000 con 0x1118b40 2022-08-01T23:01:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 1 -- 172.21.15.32:0/2241743456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fd8c006c0a0 con 0x1118b40 2022-08-01T23:01:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 10 client.6129 fetch_fsmap finished waiting for FSMap version 28 2022-08-01T23:01:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 10 client.6129 resolve_mds: resolved ID 'b' to GID 5934 2022-08-01T23:01:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 20 client.6129 populate_metadata read hostname 'smithi032' 2022-08-01T23:01:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x1161af0 0x111d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 4 client.6129 mds_command: new command op to 5934 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T23:01:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 --> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x107c080 con 0x1161af0 2022-08-01T23:01:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x1161af0 0x111d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 10 client.6129 ms_handle_connect on v2:172.21.15.55:6834/590504455 2022-08-01T23:01:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x1161af0 0x111d8b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ad9d60fdb822810d server_cookie=3d3708b1db5c5b8b in_seq=0 out_seq=0 2022-08-01T23:01:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 1 -- 172.21.15.32:0/2241743456 <== mds.0 v2:172.21.15.55:6834/590504455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x107c080 con 0x1161af0 2022-08-01T23:01:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 10 client.6129 handle_command_reply: tid=0 2022-08-01T23:01:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 client.6129 shutdown 2022-08-01T23:01:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 msgr2=0x1165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x1165670 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fd8c0025000 tx=0x7fd8c0014000).stop 2022-08-01T23:01:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 shutdown_connections 2022-08-01T23:01:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x1161af0 0x111d8b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8aeffd700 0 client.6129 ms_handle_reset on v2:172.21.15.55:6834/590504455 2022-08-01T23:01:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111a940 0x1165b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/2241743456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x1165670 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 >> 172.21.15.32:0/2241743456 conn(0x1156930 msgr2=0x111b710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 shutdown_connections 2022-08-01T23:01:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 1 -- 172.21.15.32:0/2241743456 wait complete. 2022-08-01T23:01:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.667+0000 7fd8e0a40740 20 client.6129 trim_cache size 0 max 16384 2022-08-01T23:01:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 Processor -- start 2022-08-01T23:01:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 -- start start 2022-08-01T23:01:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x111ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 0x1122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x11234c0 con 0x111a940 2022-08-01T23:01:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x11205b0 con 0x111b2b0 2022-08-01T23:01:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x10ba9d0 con 0x1118b40 2022-08-01T23:01:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 0x1122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 0x1122ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44364/0 (socket says 172.21.15.32:44364) 2022-08-01T23:01:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8c77fe700 1 -- 172.21.15.32:0/1811027299 learned_addr learned my addr 172.21.15.32:0/1811027299 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x111ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8d4d86700 1 -- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 msgr2=0x111ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x111ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8d4d86700 1 -- 172.21.15.32:0/1811027299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 msgr2=0x1122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 0x1122ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8d4d86700 1 -- 172.21.15.32:0/1811027299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1119280 con 0x111b2b0 2022-08-01T23:01:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.683+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 0x1122ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1122fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x1085a70 tx=0x7fd8c801e000).ready entity=mon.1 client_cookie=6aa1d55322f2d89d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8c57fa700 1 -- 172.21.15.32:0/1811027299 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c800b680 con 0x111b2b0 2022-08-01T23:01:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8c57fa700 1 -- 172.21.15.32:0/1811027299 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800b950 con 0x111b2b0 2022-08-01T23:01:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8c57fa700 1 -- 172.21.15.32:0/1811027299 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c800bcb0 con 0x111b2b0 2022-08-01T23:01:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 msgr2=0x1122fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1122fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x1085a70 tx=0x7fd8c801e000).stop 2022-08-01T23:01:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1811027299 shutdown_connections 2022-08-01T23:01:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111b2b0 0x1122fd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x111a940 0x1122ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1811027299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x1118b40 0x111ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1811027299 >> 172.21.15.32:0/1811027299 conn(0x1156930 msgr2=0x1117bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1811027299 shutdown_connections 2022-08-01T23:01:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1811027299 wait complete. 2022-08-01T23:01:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 -- start start 2022-08-01T23:01:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.687+0000 7fd8e0a40740 1 Processor -- start 2022-08-01T23:01:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x11b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x11b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111b2b0 0x11b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x11243a0 con 0x1118b40 2022-08-01T23:01:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x111f6e0 con 0x111a940 2022-08-01T23:01:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x1124170 con 0x111b2b0 2022-08-01T23:01:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x11b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 10 client.0 ll_register_callbacks cb 0x7fd8c002a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T23:01:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x11b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44368/0 (socket says 172.21.15.32:44368) 2022-08-01T23:01:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 -- 172.21.15.32:0/1021468898 learned_addr learned my addr 172.21.15.32:0/1021468898 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111b2b0 0x11b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 -- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111b2b0 msgr2=0x11b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x11b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111b2b0 0x11b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 -- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 msgr2=0x11b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x11b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 -- 172.21.15.32:0/1021468898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b800f040 con 0x1118b40 2022-08-01T23:01:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8d4d86700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111b2b0 0x11b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8cffff700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x11b1f20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x11246b0 tx=0x7fd8c006c030).ready entity=mon.0 client_cookie=dd9c61f515586546 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2022-08-01T23:01:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 1 -- 172.21.15.32:0/1021468898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8c006add0 con 0x1118b40 2022-08-01T23:01:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 1 -- 172.21.15.32:0/1021468898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd8c00463e0 con 0x1118b40 2022-08-01T23:01:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8c800bed0 con 0x1118b40 2022-08-01T23:01:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 1 -- 172.21.15.32:0/1021468898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c00465a0 con 0x1118b40 2022-08-01T23:01:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 10 client.6135 fetch_fsmap learned FSMap version 28 2022-08-01T23:01:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x10c5e90 con 0x1118b40 2022-08-01T23:01:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 1 -- 172.21.15.32:0/1021468898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800bed0 con 0x1118b40 2022-08-01T23:01:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 1 -- 172.21.15.32:0/1021468898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fd8c0042ce0 con 0x1118b40 2022-08-01T23:01:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 10 client.6135 fetch_fsmap finished waiting for FSMap version 28 2022-08-01T23:01:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 10 client.6135 resolve_mds: resolved ID 'b' to GID 5934 2022-08-01T23:01:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 20 client.6135 populate_metadata read hostname 'smithi032' 2022-08-01T23:01:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x11b5330 0x11b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 4 client.6135 mds_command: new command op to 5934 tid=0[{"prefix": "cache drop", "timeout": 10}] 2022-08-01T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 --> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0xf715e0 con 0x11b5330 2022-08-01T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x11b5330 0x11b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8c77fe700 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x11b5330 0x11b8410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a796fb18b424d7ad server_cookie=63b4c196d151a785 in_seq=0 out_seq=0 2022-08-01T23:01:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:11.691+0000 7fd8affff700 10 client.6135 ms_handle_connect on v2:172.21.15.55:6834/590504455 2022-08-01T23:01:21.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8affff700 1 -- 172.21.15.32:0/1021468898 <== mds.0 v2:172.21.15.55:6834/590504455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0xf715e0 con 0x11b5330 2022-08-01T23:01:21.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8affff700 10 client.6135 handle_command_reply: tid=0 2022-08-01T23:01:21.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 client.6135 shutdown 2022-08-01T23:01:21.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 msgr2=0x11b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:21.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x11b1f20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x11246b0 tx=0x7fd8c006c030).stop 2022-08-01T23:01:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 shutdown_connections 2022-08-01T23:01:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:6834/590504455,v1:172.21.15.55:6835/590504455] conn(0x11b5330 0x11b8410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x111b2b0 0x11b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x111a940 0x11b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8affff700 0 client.6135 ms_handle_reset on v2:172.21.15.55:6834/590504455 2022-08-01T23:01:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 --2- 172.21.15.32:0/1021468898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1118b40 0x11b1f20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 >> 172.21.15.32:0/1021468898 conn(0x1156930 msgr2=0x11532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 shutdown_connections 2022-08-01T23:01:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 1 -- 172.21.15.32:0/1021468898 wait complete. 2022-08-01T23:01:21.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8e0a40740 20 client.6135 trim_cache size 0 max 16384 2022-08-01T23:01:21.968 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:01:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_recall": { 2022-08-01T23:01:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": -110, 2022-08-01T23:01:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "(110) Connection timed out", 2022-08-01T23:01:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "recalled": 240 2022-08-01T23:01:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flush_journal": { 2022-08-01T23:01:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2022-08-01T23:01:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "" 2022-08-01T23:01:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "trim_cache": { 2022-08-01T23:01:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "trimmed": 220 2022-08-01T23:01:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2022-08-01T23:01:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": { 2022-08-01T23:01:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "items": 15412, 2022-08-01T23:01:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes": 2428176 2022-08-01T23:01:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 10.268875794 2022-08-01T23:01:21.973 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:01:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd8b4053b00 msgr2=0x7fd8b4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd8b4053b00 0x7fd8b4055fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fd8b800b000 tx=0x7fd8b8009040).stop 2022-08-01T23:01:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 msgr2=0x7fd8d00ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00ac750 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fd8c000c000 tx=0x7fd8c0006df0).stop 2022-08-01T23:01:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 shutdown_connections 2022-08-01T23:01:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd8b4053b00 0x7fd8b4055fc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8d00b6510 0x7fd8d0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd8d00b2aa0 0x7fd8d0155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 --2- 172.21.15.32:0/1933828770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d0061480 0x7fd8d00ac750 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 >> 172.21.15.32:0/1933828770 conn(0x7fd8d0063800 msgr2=0x7fd8d013fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 shutdown_connections 2022-08-01T23:01:21.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:21.962+0000 7fd8d5587700 1 -- 172.21.15.32:0/1933828770 wait complete. 2022-08-01T23:01:21.980 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:01:21.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:21.996 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:22.015 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:22.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:22.058 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:01:22.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:22.059 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:22.059 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:22.067 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:01:22.069 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:01:22.069 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:01:22.069 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:01:22.070 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:22.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22873]: fuse finished with error 0 and tester_r 0 2022-08-01T23:01:22.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:22.129 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:22.130 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:22.137 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:01:22.138 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:01:22.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:01:22.228 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:22.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:22.236 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:22.237 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:22.244 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:01:22.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:01:22.245 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:01:22.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:01:22.247 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:22.293 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:01:22.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:01:22.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:01:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 Processor -- start 2022-08-01T23:01:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- start start 2022-08-01T23:01:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 0x7fee0c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee0c0ac880 0x7fee0c1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0c067a10 con 0x7fee0c0b0270 2022-08-01T23:01:22.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c0ad740 con 0x7fee0c0ac880 2022-08-01T23:01:22.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c067cb0 con 0x7fee0c0b2710 2022-08-01T23:01:22.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee0c0ac880 0x7fee0c1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 0x7fee0c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 0x7fee0c1401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:56114/0 (socket says 172.21.15.32:56114) 2022-08-01T23:01:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 -- 172.21.15.32:0/2298361985 learned_addr learned my addr 172.21.15.32:0/2298361985 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee0c0ac880 0x7fee0c1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44380/0 (socket says 172.21.15.32:44380) 2022-08-01T23:01:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0affd700 1 -- 172.21.15.32:0/2298361985 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634492716 0 0) 0x7fee0c067a10 con 0x7fee0c0b0270 2022-08-01T23:01:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 -- 172.21.15.32:0/2298361985 >> v2:172.21.15.32:3300/0 conn(0x7fee0c0ac880 msgr2=0x7fee0c1406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 --2- 172.21.15.32:0/2298361985 >> v2:172.21.15.32:3300/0 conn(0x7fee0c0ac880 0x7fee0c1406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 -- 172.21.15.32:0/2298361985 >> v1:172.21.15.55:6789/0 conn(0x7fee0c0b0270 legacy=0x7fee0c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 -- 172.21.15.32:0/2298361985 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c147160 con 0x7fee0c0b2710 2022-08-01T23:01:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0b7fe700 1 --2- 172.21.15.32:0/2298361985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 0x7fee0c1401e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fedfc002f70 tx=0x7fedfc006eb0).ready entity=mon.2 client_cookie=53f3a3baf13761c4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0affd700 1 -- 172.21.15.32:0/2298361985 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedfc00d570 con 0x7fee0c0b2710 2022-08-01T23:01:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0affd700 1 -- 172.21.15.32:0/2298361985 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc000e90 con 0x7fee0c0b2710 2022-08-01T23:01:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee0affd700 1 -- 172.21.15.32:0/2298361985 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedfc00d8c0 con 0x7fee0c0b2710 2022-08-01T23:01:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- 172.21.15.32:0/2298361985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 msgr2=0x7fee0c1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 --2- 172.21.15.32:0/2298361985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 0x7fee0c1401e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fedfc002f70 tx=0x7fedfc006eb0).stop 2022-08-01T23:01:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- 172.21.15.32:0/2298361985 shutdown_connections 2022-08-01T23:01:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 --2- 172.21.15.32:0/2298361985 >> v2:172.21.15.32:3300/0 conn(0x7fee0c0ac880 0x7fee0c1406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 --2- 172.21.15.32:0/2298361985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b2710 0x7fee0c1401e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- 172.21.15.32:0/2298361985 >> 172.21.15.32:0/2298361985 conn(0x7fee0c060310 msgr2=0x7fee0c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- 172.21.15.32:0/2298361985 shutdown_connections 2022-08-01T23:01:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.406+0000 7fee1099a700 1 -- 172.21.15.32:0/2298361985 wait complete. 2022-08-01T23:01:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 Processor -- start 2022-08-01T23:01:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 -- start start 2022-08-01T23:01:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee0c0ac880 0x7fee0c09f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b0270 0x7fee0c0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 0x7fee0c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c156730 con 0x7fee0c141880 2022-08-01T23:01:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1569d0 con 0x7fee0c0ac880 2022-08-01T23:01:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee1099a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c09eb20 con 0x7fee0c0b0270 2022-08-01T23:01:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 0x7fee0c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee02ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b0270 0x7fee0c0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 0x7fee0c152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44386/0 (socket says 172.21.15.32:44386) 2022-08-01T23:01:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 -- 172.21.15.32:0/4048184027 learned_addr learned my addr 172.21.15.32:0/4048184027 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0b7fe700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee0c0ac880 0x7fee0c09f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 -- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b0270 msgr2=0x7fee0c0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b0270 0x7fee0c0b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 -- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee0c0ac880 msgr2=0x7fee0c09f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee0c0ac880 0x7fee0c09f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.410+0000 7fee0bfff700 1 -- 172.21.15.32:0/4048184027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc00b1e0 con 0x7fee0c141880 2022-08-01T23:01:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee0bfff700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 0x7fee0c152640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fee0400c000 tx=0x7fee040091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee04012600 con 0x7fee0c141880 2022-08-01T23:01:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee04012760 con 0x7fee0c141880 2022-08-01T23:01:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c09edc0 con 0x7fee0c141880 2022-08-01T23:01:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee04012ac0 con 0x7fee0c141880 2022-08-01T23:01:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c140ea0 con 0x7fee0c141880 2022-08-01T23:01:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c141180 con 0x7fee0c141880 2022-08-01T23:01:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee04022070 con 0x7fee0c141880 2022-08-01T23:01:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee097fa700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fedf00539e0 0x7fedf0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee0406c6e0 con 0x7fee0c141880 2022-08-01T23:01:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.414+0000 7fee0b7fe700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fedf00539e0 0x7fedf0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.418+0000 7fee0b7fe700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fedf00539e0 0x7fedf0055ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fedfc01c000 tx=0x7fedfc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.422+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee04043400 con 0x7fee0c141880 2022-08-01T23:01:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.550+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fee0c09ef50 con 0x7fee0c141880 2022-08-01T23:01:22.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee097fa700 1 -- 172.21.15.32:0/4048184027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v67) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fee04016020 con 0x7fee0c141880 2022-08-01T23:01:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:22.557 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:01:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 67, 2022-08-01T23:01:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:01:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:01:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:00:55.384929+0000", 2022-08-01T23:01:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:01:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:01:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:01:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:01:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:01:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:01:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:01:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:01:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:01:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:01:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:01:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:01:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:01:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:01:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 12, 2022-08-01T23:01:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:01:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:01:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:01:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:01:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:01:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:01:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:01:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:01:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:01:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:01:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:01:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:01:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:01:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:01:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:01:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:01:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:01:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:01:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:01:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:01:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:01:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:01:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:01:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:01:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:01:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:01:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:01:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:01:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:01:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:01:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:01:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:01:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:01:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:01:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:01:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:01:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:01:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:01:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:01:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:01:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:01:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:01:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:01:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:01:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:01:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:01:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:01:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:01:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:01:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:01:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:01:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:01:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:01:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:01:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:01:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:01:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:01:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:01:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:01:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:01:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:01:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:01:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:01:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:01:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:01:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:01:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:01:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:01:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:01:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:01:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:01:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:01:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:01:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:01:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:01:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:01:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:01:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:01:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:01:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:01:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:01:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:01:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:01:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:01:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:01:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:01:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:01:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:01:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:01:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:01:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:01:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:01:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:01:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:01:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:01:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:01:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:01:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:01:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:01:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:01:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:01:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:01:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:01:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:01:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:01:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:01:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:01:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:01:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:01:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:01:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:01:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:01:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:01:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:01:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:01:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:01:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:01:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:01:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:01:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:01:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:01:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:01:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:01:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:01:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:01:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:01:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:01:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:01:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:01:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:01:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:01:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:01:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:01:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:01:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 11, 2022-08-01T23:01:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:01:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:00:51.757733+0000", 2022-08-01T23:01:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:01:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:01:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:01:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:01:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:01:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:01:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:01:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:01:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:01:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:01:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:01:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:01:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:01:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:01:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:01:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:01:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:01:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:01:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:01:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:01:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "65", 2022-08-01T23:01:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:01:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:01:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:01:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:01:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:01:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:01:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:01:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:01:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:01:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:01:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:01:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:01:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:01:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:01:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:01:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:01:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:01:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:01:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:01:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:01:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:01:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:01:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:01:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:01:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:01:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:01:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:01:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:01:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:01:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:01:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:01:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:01:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:01:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:01:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:01:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:01:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:01:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:01:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:01:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:01:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:01:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:01:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:01:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:01:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 12, 2022-08-01T23:01:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:01:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:00:52.502708+0000", 2022-08-01T23:01:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:01:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:01:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:01:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:01:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:01:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:01:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:01:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:01:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:01:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:01:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:01:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:01:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:01:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:01:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:01:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:01:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:01:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:01:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:01:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:01:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "67", 2022-08-01T23:01:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:01:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:01:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:01:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:01:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:01:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:01:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:01:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:01:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:01:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:01:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:01:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:01:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:01:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:01:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:01:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:01:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:01:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:01:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:01:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:01:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:01:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:01:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:01:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:01:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:01:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:01:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:01:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:01:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:01:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:01:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:01:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:01:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:01:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:01:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:01:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:01:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:01:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:01:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:01:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:01:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:01:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:01:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:01:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:01:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:01:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:01:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:01:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:01:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:01:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:01:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:01:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:01:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:01:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:01:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:01:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:01:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:01:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:01:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:01:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:01:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:01:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:01:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:01:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:01:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:01:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:01:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:01:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:01:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:01:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:01:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:01:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:01:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:01:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:01:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 62, 2022-08-01T23:01:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:01:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:01:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:01:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:01:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:01:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:01:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:01:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:01:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:01:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2022-08-01T23:01:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:01:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:01:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:01:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:01:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:01:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:01:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:01:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:01:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:01:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:01:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:01:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:01:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:01:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:01:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 65, 2022-08-01T23:01:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:01:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:01:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:01:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:01:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:01:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:01:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:01:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:01:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:01:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:01:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:01:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:01:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:01:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:01:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 62, 2022-08-01T23:01:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:01:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:01:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:01:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:01:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:01:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:01:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:01:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:01:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:01:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:01:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:01:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:01:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:01:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:01:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:01:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:01:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:01:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:01:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:01:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:01:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:01:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:01:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:01:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:01:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:01:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:01:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:01:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:01:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:01:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 62, 2022-08-01T23:01:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:01:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:01:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:01:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:01:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:01:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:01:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:01:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:01:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:01:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:01:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:01:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:01:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:01:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:01:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:01:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:01:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:01:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:01:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:01:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:01:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:01:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:01:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:01:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:01:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:01:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:01:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:01:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:01:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:01:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:01:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:01:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:01:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:01:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:01:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:01:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:01:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:01:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:01:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:01:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:01:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:01:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:01:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:01:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:01:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:01:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:01:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:01:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:01:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:01:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:01:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:01:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:01:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:01:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:01:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:01:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:01:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:01:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:01:22.782 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:01:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fedf00539e0 msgr2=0x7fedf0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fedf00539e0 0x7fedf0055ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fedfc01c000 tx=0x7fedfc015000).stop 2022-08-01T23:01:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 msgr2=0x7fee0c152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 0x7fee0c152640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fee0400c000 tx=0x7fee040091d0).stop 2022-08-01T23:01:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 shutdown_connections 2022-08-01T23:01:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fedf00539e0 0x7fedf0055ea0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c141880 0x7fee0c152640 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee0c0b0270 0x7fee0c0b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 --2- 172.21.15.32:0/4048184027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee0c0ac880 0x7fee0c09f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 >> 172.21.15.32:0/4048184027 conn(0x7fee0c060310 msgr2=0x7fee0c144660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 shutdown_connections 2022-08-01T23:01:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.554+0000 7fee1099a700 1 -- 172.21.15.32:0/4048184027 wait complete. 2022-08-01T23:01:22.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 Processor -- start 2022-08-01T23:01:22.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- start start 2022-08-01T23:01:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0b1f40 0x7fab9c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab9c148530 0x7fab9c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0b04e0 con 0x7fab9c0a7250 2022-08-01T23:01:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c0a9250 con 0x7fab9c148530 2022-08-01T23:01:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0b0780 con 0x7fab9c0b1f40 2022-08-01T23:01:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba15e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab9c148530 0x7fab9c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba25e7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab9c0a7250 0x7fab9c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60168/0 (socket says 172.21.15.32:60168) 2022-08-01T23:01:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba25e7700 1 -- 172.21.15.32:0/2610107758 learned_addr learned my addr 172.21.15.32:0/2610107758 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba1de6700 1 --2- 172.21.15.32:0/2610107758 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0b1f40 0x7fab9c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba1de6700 1 -- 172.21.15.32:0/2610107758 >> v2:172.21.15.32:3300/0 conn(0x7fab9c148530 msgr2=0x7fab9c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba1de6700 1 --2- 172.21.15.32:0/2610107758 >> v2:172.21.15.32:3300/0 conn(0x7fab9c148530 0x7fab9c0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba1de6700 1 -- 172.21.15.32:0/2610107758 >> v1:172.21.15.32:6789/0 conn(0x7fab9c0a7250 legacy=0x7fab9c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba1de6700 1 -- 172.21.15.32:0/2610107758 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0b0a20 con 0x7fab9c0b1f40 2022-08-01T23:01:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba1de6700 1 --2- 172.21.15.32:0/2610107758 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0b1f40 0x7fab9c0b1160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fab8c0032f0 tx=0x7fab8c0091d0).ready entity=mon.2 client_cookie=d9df8771dcefeced server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba0de4700 1 -- 172.21.15.32:0/2610107758 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab8c00b6c0 con 0x7fab9c0b1f40 2022-08-01T23:01:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba0de4700 1 -- 172.21.15.32:0/2610107758 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00b820 con 0x7fab9c0b1f40 2022-08-01T23:01:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba0de4700 1 -- 172.21.15.32:0/2610107758 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab8c00b990 con 0x7fab9c0b1f40 2022-08-01T23:01:22.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- 172.21.15.32:0/2610107758 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0b1f40 msgr2=0x7fab9c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 --2- 172.21.15.32:0/2610107758 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0b1f40 0x7fab9c0b1160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fab8c0032f0 tx=0x7fab8c0091d0).stop 2022-08-01T23:01:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- 172.21.15.32:0/2610107758 shutdown_connections 2022-08-01T23:01:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 --2- 172.21.15.32:0/2610107758 >> v2:172.21.15.32:3300/0 conn(0x7fab9c148530 0x7fab9c0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 --2- 172.21.15.32:0/2610107758 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0b1f40 0x7fab9c0b1160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- 172.21.15.32:0/2610107758 >> 172.21.15.32:0/2610107758 conn(0x7fab9c05f840 msgr2=0x7fab9c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- 172.21.15.32:0/2610107758 shutdown_connections 2022-08-01T23:01:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.898+0000 7faba404a700 1 -- 172.21.15.32:0/2610107758 wait complete. 2022-08-01T23:01:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 Processor -- start 2022-08-01T23:01:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 -- start start 2022-08-01T23:01:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0a7250 0x7fab9c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9c0b1f40 0x7fab9c14b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 0x7fab9c0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c14a520 con 0x7fab9c148530 2022-08-01T23:01:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0ab2b0 con 0x7fab9c0b1f40 2022-08-01T23:01:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0ab550 con 0x7fab9c0a7250 2022-08-01T23:01:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 0x7fab9c0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba1de6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0a7250 0x7fab9c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 0x7fab9c0abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44400/0 (socket says 172.21.15.32:44400) 2022-08-01T23:01:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 -- 172.21.15.32:0/387684904 learned_addr learned my addr 172.21.15.32:0/387684904 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba15e5700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9c0b1f40 0x7fab9c14b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 -- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0a7250 msgr2=0x7fab9c14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0a7250 0x7fab9c14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 -- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9c0b1f40 msgr2=0x7fab9c14b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9c0b1f40 0x7fab9c14b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 -- 172.21.15.32:0/387684904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c009470 con 0x7fab9c148530 2022-08-01T23:01:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba15e5700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9c0b1f40 0x7fab9c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba25e7700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 0x7fab9c0abc90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fab9c061530 tx=0x7fab94007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab940070d0 con 0x7fab9c148530 2022-08-01T23:01:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab94007230 con 0x7fab9c148530 2022-08-01T23:01:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0ab7f0 con 0x7fab9c148530 2022-08-01T23:01:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab94005610 con 0x7fab9c148530 2022-08-01T23:01:22.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.902+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c149dc0 con 0x7fab9c148530 2022-08-01T23:01:22.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.906+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c0b0cc0 con 0x7fab9c148530 2022-08-01T23:01:22.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.906+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab9401a070 con 0x7fab9c148530 2022-08-01T23:01:22.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.906+0000 7fab9affd700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab7c053a70 0x7fab7c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:22.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.906+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fab9406b3a0 con 0x7fab9c148530 2022-08-01T23:01:22.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.906+0000 7faba1de6700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab7c053a70 0x7fab7c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:22.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.910+0000 7faba1de6700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab7c053a70 0x7fab7c055f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fab8c01b000 tx=0x7fab8c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:22.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:22.914+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab94045e60 con 0x7fab9c148530 2022-08-01T23:01:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.078+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab9c0574e0 con 0x7fab9c148530 2022-08-01T23:01:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.078+0000 7fab9affd700 1 -- 172.21.15.32:0/387684904 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fab94041f70 con 0x7fab9c148530 2022-08-01T23:01:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:23.082 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:00:54.368935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5934":{"gid":5934,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/590504455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":590504455},{"type":"v1","addr":"172.21.15.55:6835","nonce":590504455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:01:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab7c053a70 msgr2=0x7fab7c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab7c053a70 0x7fab7c055f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fab8c01b000 tx=0x7fab8c016000).stop 2022-08-01T23:01:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 msgr2=0x7fab9c0abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 0x7fab9c0abc90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fab9c061530 tx=0x7fab94007bc0).stop 2022-08-01T23:01:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 shutdown_connections 2022-08-01T23:01:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fab7c053a70 0x7fab7c055f30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9c148530 0x7fab9c0abc90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9c0b1f40 0x7fab9c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 --2- 172.21.15.32:0/387684904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fab9c0a7250 0x7fab9c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 >> 172.21.15.32:0/387684904 conn(0x7fab9c05f840 msgr2=0x7fab9c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 shutdown_connections 2022-08-01T23:01:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.082+0000 7faba404a700 1 -- 172.21.15.32:0/387684904 wait complete. 2022-08-01T23:01:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2022-08-01T23:01:23.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:01:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 Processor -- start 2022-08-01T23:01:23.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 -- start start 2022-08-01T23:01:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c9c0b13c0 0x7f3c9c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c9c0afd80 0x7f3c9c142910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c0ac520 con 0x7f3c9c142e00 2022-08-01T23:01:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9c143200 con 0x7f3c9c0afd80 2022-08-01T23:01:23.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca2389700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9c0b17a0 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c9c0afd80 0x7f3c9c142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c9c0afd80 0x7f3c9c142910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44404/0 (socket says 172.21.15.32:44404) 2022-08-01T23:01:23.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 -- 172.21.15.32:0/1919492346 learned_addr learned my addr 172.21.15.32:0/1919492346 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:23.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca1387700 1 --2- 172.21.15.32:0/1919492346 >> v2:172.21.15.55:3300/0 conn(0x7f3c9c0b13c0 0x7f3c9c0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 -- 172.21.15.32:0/1919492346 >> v2:172.21.15.55:3300/0 conn(0x7f3c9c0b13c0 msgr2=0x7f3c9c0b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:23.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 --2- 172.21.15.32:0/1919492346 >> v2:172.21.15.55:3300/0 conn(0x7f3c9c0b13c0 0x7f3c9c0b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 -- 172.21.15.32:0/1919492346 >> v1:172.21.15.55:6789/0 conn(0x7f3c9c142e00 legacy=0x7f3c9c0ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 -- 172.21.15.32:0/1919492346 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0b0450 con 0x7f3c9c0afd80 2022-08-01T23:01:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3ca0b86700 1 --2- 172.21.15.32:0/1919492346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c142910 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f3c98002f70 tx=0x7f3c98006eb0).ready entity=mon.0 client_cookie=a6a05c0cb977b938 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.202+0000 7f3c93fff700 1 -- 172.21.15.32:0/1919492346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9800d5c0 con 0x7f3c9c0afd80 2022-08-01T23:01:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3c93fff700 1 -- 172.21.15.32:0/1919492346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98000e90 con 0x7f3c9c0afd80 2022-08-01T23:01:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3c93fff700 1 -- 172.21.15.32:0/1919492346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9800d950 con 0x7f3c9c0afd80 2022-08-01T23:01:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/1919492346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0afd80 msgr2=0x7f3c9c142910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 --2- 172.21.15.32:0/1919492346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c142910 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f3c98002f70 tx=0x7f3c98006eb0).stop 2022-08-01T23:01:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/1919492346 shutdown_connections 2022-08-01T23:01:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 --2- 172.21.15.32:0/1919492346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c142910 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 --2- 172.21.15.32:0/1919492346 >> v2:172.21.15.55:3300/0 conn(0x7f3c9c0b13c0 0x7f3c9c0b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/1919492346 >> 172.21.15.32:0/1919492346 conn(0x7f3c9c0626f0 msgr2=0x7f3c9c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/1919492346 shutdown_connections 2022-08-01T23:01:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/1919492346 wait complete. 2022-08-01T23:01:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 Processor -- start 2022-08-01T23:01:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- start start 2022-08-01T23:01:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 0x7f3c9c146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c9c142e00 0x7f3c9c146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c09fa40 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c145de0 con 0x7f3c9c0afd80 2022-08-01T23:01:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c1458a0 con 0x7f3c9c142e00 2022-08-01T23:01:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 0x7f3c9c146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 0x7f3c9c146ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44410/0 (socket says 172.21.15.32:44410) 2022-08-01T23:01:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 -- 172.21.15.32:0/4234313108 learned_addr learned my addr 172.21.15.32:0/4234313108 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca1387700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca1b88700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c9c142e00 0x7f3c9c146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 -- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c9c142e00 msgr2=0x7f3c9c146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c9c142e00 0x7f3c9c146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 -- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c9c0afd80 msgr2=0x7f3c9c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 -- 172.21.15.32:0/4234313108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9800b1e0 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca0b86700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 0x7f3c9c146ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3c98002f70 tx=0x7f3c98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c98011760 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c980118c0 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c145b40 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c98011c20 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.206+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9c140e70 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.210+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9c09ebe0 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.210+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3c98025030 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.210+0000 7f3c927fc700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c84053a70 0x7f3c84055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.210+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c9806d1a0 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.210+0000 7f3ca1387700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c84053a70 0x7f3c84055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.210+0000 7f3ca1387700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c84053a70 0x7f3c84055f30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3c8c00b000 tx=0x7f3c8c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.218+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c98043e50 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.382+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3c9c0b06f0 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.385 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:01:23.382+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:01:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.398+0000 7f3c927fc700 1 -- 172.21.15.32:0/4234313108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3c98023090 con 0x7f3c9c0b13c0 2022-08-01T23:01:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c84053a70 msgr2=0x7f3c84055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c84053a70 0x7f3c84055f30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3c8c00b000 tx=0x7f3c8c006df0).stop 2022-08-01T23:01:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 msgr2=0x7f3c9c146ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 0x7f3c9c146ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3c98002f70 tx=0x7f3c98015000).stop 2022-08-01T23:01:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 shutdown_connections 2022-08-01T23:01:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c84053a70 0x7f3c84055f30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3c9c142e00 0x7f3c9c146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c9c0b13c0 0x7f3c9c146ab0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 --2- 172.21.15.32:0/4234313108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c9c0afd80 0x7f3c9c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 >> 172.21.15.32:0/4234313108 conn(0x7f3c9c0626f0 msgr2=0x7f3c9c148cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:23.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 shutdown_connections 2022-08-01T23:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.402+0000 7f3ca2389700 1 -- 172.21.15.32:0/4234313108 wait complete. 2022-08-01T23:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:01:23.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:23.437 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:01:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 Processor -- start 2022-08-01T23:01:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- start start 2022-08-01T23:01:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1100b7840 0x7fc110065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1100b3070 0x7fc110061120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1100b2c20 con 0x7fc11014b040 2022-08-01T23:01:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc110065c10 con 0x7fc1100b7840 2022-08-01T23:01:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1100b7c20 con 0x7fc1100b3070 2022-08-01T23:01:23.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1100b7840 0x7fc110065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1100b7840 0x7fc110065720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44416/0 (socket says 172.21.15.32:44416) 2022-08-01T23:01:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 -- 172.21.15.32:0/986401414 learned_addr learned my addr 172.21.15.32:0/986401414 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc10ffff700 1 --2- 172.21.15.32:0/986401414 >> v2:172.21.15.55:3300/0 conn(0x7fc1100b3070 0x7fc110061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 -- 172.21.15.32:0/986401414 >> v2:172.21.15.55:3300/0 conn(0x7fc1100b3070 msgr2=0x7fc110061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:23.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 --2- 172.21.15.32:0/986401414 >> v2:172.21.15.55:3300/0 conn(0x7fc1100b3070 0x7fc110061120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 -- 172.21.15.32:0/986401414 >> v1:172.21.15.55:6789/0 conn(0x7fc11014b040 legacy=0x7fc11014b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 -- 172.21.15.32:0/986401414 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100b2980 con 0x7fc1100b7840 2022-08-01T23:01:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc114a72700 1 --2- 172.21.15.32:0/986401414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc110065720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fc100002f70 tx=0x7fc100006e50).ready entity=mon.0 client_cookie=9c528daa6e5d9ead server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc10f7fe700 1 -- 172.21.15.32:0/986401414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc10000d610 con 0x7fc1100b7840 2022-08-01T23:01:23.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc10f7fe700 1 -- 172.21.15.32:0/986401414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc100000e90 con 0x7fc1100b7840 2022-08-01T23:01:23.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc10f7fe700 1 -- 172.21.15.32:0/986401414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc10000d9a0 con 0x7fc1100b7840 2022-08-01T23:01:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- 172.21.15.32:0/986401414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 msgr2=0x7fc110065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 --2- 172.21.15.32:0/986401414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc110065720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fc100002f70 tx=0x7fc100006e50).stop 2022-08-01T23:01:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- 172.21.15.32:0/986401414 shutdown_connections 2022-08-01T23:01:23.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 --2- 172.21.15.32:0/986401414 >> v2:172.21.15.55:3300/0 conn(0x7fc1100b3070 0x7fc110061120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 --2- 172.21.15.32:0/986401414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc110065720 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.518+0000 7fc115a74700 1 -- 172.21.15.32:0/986401414 >> 172.21.15.32:0/986401414 conn(0x7fc11005f840 msgr2=0x7fc11005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- 172.21.15.32:0/986401414 shutdown_connections 2022-08-01T23:01:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- 172.21.15.32:0/986401414 wait complete. 2022-08-01T23:01:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 Processor -- start 2022-08-01T23:01:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- start start 2022-08-01T23:01:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc1100b3070 0x7fc1100b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc1100b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11014b040 0x7fc1100b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11009f4e0 con 0x7fc1100b7840 2022-08-01T23:01:23.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100acb60 con 0x7fc11014b040 2022-08-01T23:01:23.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100ace00 con 0x7fc1100b3070 2022-08-01T23:01:23.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc1100b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc1100b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44422/0 (socket says 172.21.15.32:44422) 2022-08-01T23:01:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 -- 172.21.15.32:0/2087977732 learned_addr learned my addr 172.21.15.32:0/2087977732 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc114a72700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc1100b3070 0x7fc1100b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115273700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11014b040 0x7fc1100b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 -- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc1100b3070 msgr2=0x7fc1100b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc1100b3070 0x7fc1100b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 -- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11014b040 msgr2=0x7fc1100b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11014b040 0x7fc1100b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 -- 172.21.15.32:0/2087977732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10000b1e0 con 0x7fc1100b7840 2022-08-01T23:01:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115273700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11014b040 0x7fc1100b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10ffff700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc1100b0740 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc1040030b0 tx=0x7fc104008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc104011780 con 0x7fc1100b7840 2022-08-01T23:01:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1040118e0 con 0x7fc1100b7840 2022-08-01T23:01:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1100ad0a0 con 0x7fc1100b7840 2022-08-01T23:01:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.522+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc104011c40 con 0x7fc1100b7840 2022-08-01T23:01:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.526+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11014a340 con 0x7fc1100b7840 2022-08-01T23:01:23.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.526+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1100609e0 con 0x7fc1100b7840 2022-08-01T23:01:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.526+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc104012420 con 0x7fc1100b7840 2022-08-01T23:01:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.526+0000 7fc10dffb700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.526+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc10406be20 con 0x7fc1100b7840 2022-08-01T23:01:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.526+0000 7fc114a72700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.530+0000 7fc114a72700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc10001c000 tx=0x7fc100015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.534+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc104042b30 con 0x7fc1100b7840 2022-08-01T23:01:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.694+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1100574e0 con 0x7fc1100b7840 2022-08-01T23:01:23.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.694+0000 7fc10dffb700 1 -- 172.21.15.32:0/2087977732 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2132 (secure 0 0 0) 0x7fc104046150 con 0x7fc1100b7840 2022-08-01T23:01:23.697 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:23.697 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:00:53.364281+0000","modified":"2022-08-01T23:01:23.389674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T23:01:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 msgr2=0x7fc0f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc10001c000 tx=0x7fc100015000).stop 2022-08-01T23:01:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 msgr2=0x7fc1100b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc1100b0740 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc1040030b0 tx=0x7fc104008040).stop 2022-08-01T23:01:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 shutdown_connections 2022-08-01T23:01:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11014b040 0x7fc1100b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1100b7840 0x7fc1100b0740 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 --2- 172.21.15.32:0/2087977732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc1100b3070 0x7fc1100b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 >> 172.21.15.32:0/2087977732 conn(0x7fc11005f840 msgr2=0x7fc110066790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 shutdown_connections 2022-08-01T23:01:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.698+0000 7fc115a74700 1 -- 172.21.15.32:0/2087977732 wait complete. 2022-08-01T23:01:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2022-08-01T23:01:23.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:01:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.814+0000 7f1963fff700 1 Processor -- start 2022-08-01T23:01:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.814+0000 7f1963fff700 1 -- start start 2022-08-01T23:01:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19640af550 0x7f19640b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964060330 0x7f19641445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1964144ad0 0x7f19640a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640b15e0 con 0x7f19640af550 2022-08-01T23:01:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640aa630 con 0x7f1964144ad0 2022-08-01T23:01:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196405f840 con 0x7f1964060330 2022-08-01T23:01:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19640af550 0x7f19640b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19640af550 0x7f19640b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44428/0 (socket says 172.21.15.32:44428) 2022-08-01T23:01:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 -- 172.21.15.32:0/2029199532 learned_addr learned my addr 172.21.15.32:0/2029199532 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f19637fe700 1 --2- 172.21.15.32:0/2029199532 >> v2:172.21.15.55:3300/0 conn(0x7f1964144ad0 0x7f19640a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f19627fc700 1 --2- 172.21.15.32:0/2029199532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964060330 0x7f19641445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 -- 172.21.15.32:0/2029199532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964060330 msgr2=0x7f19641445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 --2- 172.21.15.32:0/2029199532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964060330 0x7f19641445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 -- 172.21.15.32:0/2029199532 >> v2:172.21.15.55:3300/0 conn(0x7f1964144ad0 msgr2=0x7f19640a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 --2- 172.21.15.32:0/2029199532 >> v2:172.21.15.55:3300/0 conn(0x7f1964144ad0 0x7f19640a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 -- 172.21.15.32:0/2029199532 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196405fd10 con 0x7f19640af550 2022-08-01T23:01:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1962ffd700 1 --2- 172.21.15.32:0/2029199532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19640af550 0x7f19640b0e40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f1954002f70 tx=0x7f1954006eb0).ready entity=mon.0 client_cookie=5d9bca4e47900ccf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1961ffb700 1 -- 172.21.15.32:0/2029199532 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195400d570 con 0x7f19640af550 2022-08-01T23:01:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1961ffb700 1 -- 172.21.15.32:0/2029199532 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1954000e90 con 0x7f19640af550 2022-08-01T23:01:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1961ffb700 1 -- 172.21.15.32:0/2029199532 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195400d900 con 0x7f19640af550 2022-08-01T23:01:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- 172.21.15.32:0/2029199532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19640af550 msgr2=0x7f19640b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- 172.21.15.32:0/2029199532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19640af550 0x7f19640b0e40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f1954002f70 tx=0x7f1954006eb0).stop 2022-08-01T23:01:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- 172.21.15.32:0/2029199532 shutdown_connections 2022-08-01T23:01:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- 172.21.15.32:0/2029199532 >> v2:172.21.15.55:3300/0 conn(0x7f1964144ad0 0x7f19640a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- 172.21.15.32:0/2029199532 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964060330 0x7f19641445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 --2- 172.21.15.32:0/2029199532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19640af550 0x7f19640b0e40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- 172.21.15.32:0/2029199532 >> 172.21.15.32:0/2029199532 conn(0x7f19640612d0 msgr2=0x7f1964062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- 172.21.15.32:0/2029199532 shutdown_connections 2022-08-01T23:01:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.818+0000 7f1963fff700 1 -- 172.21.15.32:0/2029199532 wait complete. 2022-08-01T23:01:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 Processor -- start 2022-08-01T23:01:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 -- start start 2022-08-01T23:01:23.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1964060330 0x7f1964065740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19640af550 0x7f1964067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964144ad0 0x7f196413ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964066510 con 0x7f1964060330 2022-08-01T23:01:23.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964067800 con 0x7f19640af550 2022-08-01T23:01:23.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1963fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1964140ea0 con 0x7f1964144ad0 2022-08-01T23:01:23.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1962ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1964060330 0x7f1964065740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1962ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1964060330 0x7f1964065740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44434/0 (socket says 172.21.15.32:44434) 2022-08-01T23:01:23.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f1962ffd700 1 -- 172.21.15.32:0/1328900281 learned_addr learned my addr 172.21.15.32:0/1328900281 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:23.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f19637fe700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964144ad0 0x7f196413ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.822+0000 7f19627fc700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19640af550 0x7f1964067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f19637fe700 1 -- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19640af550 msgr2=0x7f1964067310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f19637fe700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19640af550 0x7f1964067310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f19637fe700 1 -- 172.21.15.32:0/1328900281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1964060330 msgr2=0x7f1964065740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:23.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f19637fe700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1964060330 0x7f1964065740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:23.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f19637fe700 1 -- 172.21.15.32:0/1328900281 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195400b1e0 con 0x7f1964144ad0 2022-08-01T23:01:23.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f19637fe700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964144ad0 0x7f196413ef30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1964064430 tx=0x7f195c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195c00c070 con 0x7f1964144ad0 2022-08-01T23:01:23.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c0133f0 con 0x7f1964144ad0 2022-08-01T23:01:23.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1964064160 con 0x7f1964144ad0 2022-08-01T23:01:23.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195c013750 con 0x7f1964144ad0 2022-08-01T23:01:23.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1964067cd0 con 0x7f1964144ad0 2022-08-01T23:01:23.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.826+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19640574e0 con 0x7f1964144ad0 2022-08-01T23:01:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.830+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f195c0138b0 con 0x7f1964144ad0 2022-08-01T23:01:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.830+0000 7f1953fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1940053950 0x7f1940055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.830+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f195c06c8e0 con 0x7f1964144ad0 2022-08-01T23:01:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.830+0000 7f1962ffd700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1940053950 0x7f1940055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.830+0000 7f1962ffd700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1940053950 0x7f1940055e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f195401c000 tx=0x7f1954015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.834+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f195c043650 con 0x7f1964144ad0 2022-08-01T23:01:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:23.998+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1964069a70 con 0x7f1964144ad0 2022-08-01T23:01:24.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:24.097 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:24.100 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:24.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.418+0000 7f1953fff700 1 -- 172.21.15.32:0/1328900281 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f195c046c70 con 0x7f1964144ad0 2022-08-01T23:01:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1940053950 msgr2=0x7f1940055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1940053950 0x7f1940055e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f195401c000 tx=0x7f1954015000).stop 2022-08-01T23:01:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964144ad0 msgr2=0x7f196413ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964144ad0 0x7f196413ef30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1964064430 tx=0x7f195c008040).stop 2022-08-01T23:01:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 shutdown_connections 2022-08-01T23:01:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1940053950 0x7f1940055e10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1964144ad0 0x7f196413ef30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19640af550 0x7f1964067310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 --2- 172.21.15.32:0/1328900281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1964060330 0x7f1964065740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 >> 172.21.15.32:0/1328900281 conn(0x7f19640612d0 msgr2=0x7f19640b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 shutdown_connections 2022-08-01T23:01:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.422+0000 7f1963fff700 1 -- 172.21.15.32:0/1328900281 wait complete. 2022-08-01T23:01:24.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:01:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.550+0000 7fc2c9ac2700 1 Processor -- start 2022-08-01T23:01:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.550+0000 7fc2c9ac2700 1 -- start start 2022-08-01T23:01:24.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2c40b6710 0x7fc2c40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:24.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2c40b4550 0x7fc2c40b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:24.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c4144240 con 0x7fc2c4061ec0 2022-08-01T23:01:24.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40b6fe0 con 0x7fc2c40b6710 2022-08-01T23:01:24.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c4143d00 con 0x7fc2c40b4550 2022-08-01T23:01:24.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2c40b6710 0x7fc2c40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:24.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2c4061ec0 0x7fc2c4146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60220/0 (socket says 172.21.15.32:60220) 2022-08-01T23:01:24.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c3fff700 1 -- 172.21.15.32:0/1135442812 learned_addr learned my addr 172.21.15.32:0/1135442812 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:24.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c2ffd700 1 --2- 172.21.15.32:0/1135442812 >> v2:172.21.15.55:3300/0 conn(0x7fc2c40b4550 0x7fc2c40b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:24.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1135442812 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541069521 0 0) 0x7fc2c4144240 con 0x7fc2c4061ec0 2022-08-01T23:01:24.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1135442812 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a4003260 con 0x7fc2c4061ec0 2022-08-01T23:01:24.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c37fe700 1 -- 172.21.15.32:0/1135442812 >> v2:172.21.15.55:3300/0 conn(0x7fc2c40b4550 msgr2=0x7fc2c40b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:24.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c37fe700 1 --2- 172.21.15.32:0/1135442812 >> v2:172.21.15.55:3300/0 conn(0x7fc2c40b4550 0x7fc2c40b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1135442812 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108845239 0 0) 0x7fc2a4003260 con 0x7fc2c4061ec0 2022-08-01T23:01:24.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c37fe700 1 -- 172.21.15.32:0/1135442812 >> v1:172.21.15.32:6789/0 conn(0x7fc2c4061ec0 legacy=0x7fc2c4146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:24.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c37fe700 1 -- 172.21.15.32:0/1135442812 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c4143fa0 con 0x7fc2c40b6710 2022-08-01T23:01:24.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c2ffd700 1 --2- 172.21.15.32:0/1135442812 >> v2:172.21.15.55:3300/0 conn(0x7fc2c40b4550 0x7fc2c40b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:24.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c37fe700 1 --2- 172.21.15.32:0/1135442812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c40b5d80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fc2bc001b80 tx=0x7fc2ac006df0).ready entity=mon.0 client_cookie=76245a1dc0a1281a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:24.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1135442812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc0019e0 con 0x7fc2c40b6710 2022-08-01T23:01:24.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1135442812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac000e90 con 0x7fc2c40b6710 2022-08-01T23:01:24.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1135442812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac012460 con 0x7fc2c40b6710 2022-08-01T23:01:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1135442812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 msgr2=0x7fc2c40b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1135442812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c40b5d80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fc2bc001b80 tx=0x7fc2ac006df0).stop 2022-08-01T23:01:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1135442812 shutdown_connections 2022-08-01T23:01:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1135442812 >> v2:172.21.15.55:3300/0 conn(0x7fc2c40b4550 0x7fc2c40b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1135442812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c40b5d80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1135442812 >> 172.21.15.32:0/1135442812 conn(0x7fc2c405c620 msgr2=0x7fc2c405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1135442812 shutdown_connections 2022-08-01T23:01:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.554+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1135442812 wait complete. 2022-08-01T23:01:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 Processor -- start 2022-08-01T23:01:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 -- start start 2022-08-01T23:01:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2c4061ec0 0x7fc2c4128d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:24.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c40b4550 0x7fc2c412ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:24.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c4148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:24.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41471c0 con 0x7fc2c40b6710 2022-08-01T23:01:24.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4148220 con 0x7fc2c40b4550 2022-08-01T23:01:24.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c9ac2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4148fc0 con 0x7fc2c4061ec0 2022-08-01T23:01:24.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c4148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:24.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c4148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44452/0 (socket says 172.21.15.32:44452) 2022-08-01T23:01:24.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 -- 172.21.15.32:0/1567671020 learned_addr learned my addr 172.21.15.32:0/1567671020 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c2ffd700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c40b4550 0x7fc2c412ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c37fe700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2c4061ec0 0x7fc2c4128d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 -- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2c4061ec0 msgr2=0x7fc2c4128d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2c4061ec0 0x7fc2c4128d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 -- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c40b4550 msgr2=0x7fc2c412ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c40b4550 0x7fc2c412ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 -- 172.21.15.32:0/1567671020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b400b180 con 0x7fc2c40b6710 2022-08-01T23:01:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c37fe700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2c4061ec0 0x7fc2c4128d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:24.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.558+0000 7fc2c3fff700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c4148650 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fc2c41497d0 tx=0x7fc2bc002100).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc004ab0 con 0x7fc2c40b6710 2022-08-01T23:01:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc004c10 con 0x7fc2c40b6710 2022-08-01T23:01:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac00b180 con 0x7fc2c40b6710 2022-08-01T23:01:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc007170 con 0x7fc2c40b6710 2022-08-01T23:01:24.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c412c690 con 0x7fc2c40b6710 2022-08-01T23:01:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc2bc01b070 con 0x7fc2c40b6710 2022-08-01T23:01:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c4149370 con 0x7fc2c40b6710 2022-08-01T23:01:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c0ff9700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2a8053b30 0x7fc2a8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc2bc06ba90 con 0x7fc2c40b6710 2022-08-01T23:01:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c37fe700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2a8053b30 0x7fc2a8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.562+0000 7fc2c37fe700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2a8053b30 0x7fc2a8055ff0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fc2ac018000 tx=0x7fc2ac014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.566+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2bc042650 con 0x7fc2c40b6710 2022-08-01T23:01:24.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:24.690+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc2c4061780 con 0x7fc2c40b6710 2022-08-01T23:01:25.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.474+0000 7fc2c0ff9700 1 -- 172.21.15.32:0/1567671020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc2bc045c70 con 0x7fc2c40b6710 2022-08-01T23:01:25.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.478+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2a8053b30 msgr2=0x7fc2a8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:25.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.478+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2a8053b30 0x7fc2a8055ff0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fc2ac018000 tx=0x7fc2ac014000).stop 2022-08-01T23:01:25.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 msgr2=0x7fc2c4148650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:25.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c4148650 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fc2c41497d0 tx=0x7fc2bc002100).stop 2022-08-01T23:01:25.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 shutdown_connections 2022-08-01T23:01:25.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2a8053b30 0x7fc2a8055ff0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c40b6710 0x7fc2c4148650 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c40b4550 0x7fc2c412ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 --2- 172.21.15.32:0/1567671020 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2c4061ec0 0x7fc2c4128d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 >> 172.21.15.32:0/1567671020 conn(0x7fc2c405c620 msgr2=0x7fc2c4126f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:25.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 shutdown_connections 2022-08-01T23:01:25.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.482+0000 7fc2c9ac2700 1 -- 172.21.15.32:0/1567671020 wait complete. 2022-08-01T23:01:25.483 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:01:25.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:01:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.606+0000 7f7592802700 1 Processor -- start 2022-08-01T23:01:25.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.606+0000 7f7592802700 1 -- start start 2022-08-01T23:01:25.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9da0 0x7f758c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f758c1402d0 0x7f758c140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c067030 con 0x7f758c0b31d0 2022-08-01T23:01:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c064a70 con 0x7f758c1402d0 2022-08-01T23:01:25.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c061f60 con 0x7f758c0a9da0 2022-08-01T23:01:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f758c1402d0 0x7f758c140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7591800700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f758c0b31d0 0x7f758c066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60230/0 (socket says 172.21.15.32:60230) 2022-08-01T23:01:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7591800700 1 -- 172.21.15.32:0/2426348006 learned_addr learned my addr 172.21.15.32:0/2426348006 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:25.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 --2- 172.21.15.32:0/2426348006 >> v2:172.21.15.32:3300/0 conn(0x7f758c1402d0 0x7f758c140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44460/0 (socket says 172.21.15.32:44460) 2022-08-01T23:01:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7590fff700 1 --2- 172.21.15.32:0/2426348006 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9da0 0x7f758c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 -- 172.21.15.32:0/2426348006 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9da0 msgr2=0x7f758c060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 --2- 172.21.15.32:0/2426348006 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9da0 0x7f758c060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 -- 172.21.15.32:0/2426348006 >> v1:172.21.15.32:6789/0 conn(0x7f758c0b31d0 legacy=0x7f758c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 -- 172.21.15.32:0/2426348006 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0b35b0 con 0x7f758c1402d0 2022-08-01T23:01:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f757ffff700 1 -- 172.21.15.32:0/2426348006 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181188451 0 0) 0x7f758c067030 con 0x7f758c0b31d0 2022-08-01T23:01:25.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592001700 1 --2- 172.21.15.32:0/2426348006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c140760 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f7584002f70 tx=0x7f7584006e80).ready entity=mon.0 client_cookie=da8201113023b42f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f757ffff700 1 -- 172.21.15.32:0/2426348006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f758400b980 con 0x7f758c1402d0 2022-08-01T23:01:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f757ffff700 1 -- 172.21.15.32:0/2426348006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584000e90 con 0x7f758c1402d0 2022-08-01T23:01:25.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f757ffff700 1 -- 172.21.15.32:0/2426348006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f758400bcd0 con 0x7f758c1402d0 2022-08-01T23:01:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- 172.21.15.32:0/2426348006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 msgr2=0x7f758c140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 --2- 172.21.15.32:0/2426348006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c140760 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f7584002f70 tx=0x7f7584006e80).stop 2022-08-01T23:01:25.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- 172.21.15.32:0/2426348006 shutdown_connections 2022-08-01T23:01:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 --2- 172.21.15.32:0/2426348006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c140760 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 --2- 172.21.15.32:0/2426348006 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0a9da0 0x7f758c060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- 172.21.15.32:0/2426348006 >> 172.21.15.32:0/2426348006 conn(0x7f758c063870 msgr2=0x7f758c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:25.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- 172.21.15.32:0/2426348006 shutdown_connections 2022-08-01T23:01:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.610+0000 7f7592802700 1 -- 172.21.15.32:0/2426348006 wait complete. 2022-08-01T23:01:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 Processor -- start 2022-08-01T23:01:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 -- start start 2022-08-01T23:01:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9da0 0x7f758c13ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:25.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b31d0 0x7f758c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:25.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:25.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c060400 con 0x7f758c1402d0 2022-08-01T23:01:25.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c156320 con 0x7f758c0a9da0 2022-08-01T23:01:25.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592802700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0ab570 con 0x7f758c0b31d0 2022-08-01T23:01:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44466/0 (socket says 172.21.15.32:44466) 2022-08-01T23:01:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 -- 172.21.15.32:0/2784654453 learned_addr learned my addr 172.21.15.32:0/2784654453 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:25.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7591800700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9da0 0x7f758c13ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7590fff700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b31d0 0x7f758c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 -- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b31d0 msgr2=0x7f758c152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b31d0 0x7f758c152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 -- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9da0 msgr2=0x7f758c13ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9da0 0x7f758c13ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.614+0000 7f7592001700 1 -- 172.21.15.32:0/2784654453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758400d410 con 0x7f758c1402d0 2022-08-01T23:01:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f7592001700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c152580 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f7584009040 tx=0x7f758c067360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7584010d20 con 0x7f758c1402d0 2022-08-01T23:01:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584010e80 con 0x7f758c1402d0 2022-08-01T23:01:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0ab810 con 0x7f758c1402d0 2022-08-01T23:01:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75840185f0 con 0x7f758c1402d0 2022-08-01T23:01:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c13c1a0 con 0x7f758c1402d0 2022-08-01T23:01:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c156520 con 0x7f758c1402d0 2022-08-01T23:01:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f75840243f0 con 0x7f758c1402d0 2022-08-01T23:01:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f757e7fc700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7574053950 0x7f7574055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f758406d4f0 con 0x7f758c1402d0 2022-08-01T23:01:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.618+0000 7f7591800700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7574053950 0x7f7574055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.622+0000 7f7591800700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7574053950 0x7f7574055e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f758000d000 tx=0x7f7580007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.626+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f758c156520 con 0x7f758c1402d0 2022-08-01T23:01:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:25.750+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f758c1564e0 con 0x7f758c1402d0 2022-08-01T23:01:26.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.478+0000 7f757e7fc700 1 -- 172.21.15.32:0/2784654453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 173+0+0 (secure 0 0 0) 0x7f75840442a0 con 0x7f758c1402d0 2022-08-01T23:01:26.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7574053950 msgr2=0x7f7574055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7574053950 0x7f7574055e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f758000d000 tx=0x7f7580007a00).stop 2022-08-01T23:01:26.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 msgr2=0x7f758c152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c152580 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f7584009040 tx=0x7f758c067360).stop 2022-08-01T23:01:26.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 shutdown_connections 2022-08-01T23:01:26.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7574053950 0x7f7574055e10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c1402d0 0x7f758c152580 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f758c0b31d0 0x7f758c152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 --2- 172.21.15.32:0/2784654453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f758c0a9da0 0x7f758c13ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 >> 172.21.15.32:0/2784654453 conn(0x7f758c063870 msgr2=0x7f758c060a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 shutdown_connections 2022-08-01T23:01:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.482+0000 7f7592802700 1 -- 172.21.15.32:0/2784654453 wait complete. 2022-08-01T23:01:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:01:26.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:01:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f28448e4700 1 Processor -- start 2022-08-01T23:01:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f28448e4700 1 -- start start 2022-08-01T23:01:26.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f28448e4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28400b0270 0x7f2840143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f28448e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2840067a10 con 0x7f28400b2710 2022-08-01T23:01:26.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f28448e4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2840067cb0 con 0x7f28400ac880 2022-08-01T23:01:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f28448e4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28400ad740 con 0x7f28400b0270 2022-08-01T23:01:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f28400b2710 0x7f28401401e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60244/0 (socket says 172.21.15.32:60244) 2022-08-01T23:01:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283ed9d700 1 -- 172.21.15.32:0/718585106 learned_addr learned my addr 172.21.15.32:0/718585106 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f2837fff700 1 --2- 172.21.15.32:0/718585106 >> v2:172.21.15.55:3300/0 conn(0x7f28400b0270 0x7f2840143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266983594 0 0) 0x7f2840067a10 con 0x7f28400b2710 2022-08-01T23:01:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2820003260 con 0x7f28400b2710 2022-08-01T23:01:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124883385 0 0) 0x7f2820003260 con 0x7f28400b2710 2022-08-01T23:01:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 >> v2:172.21.15.55:3300/0 conn(0x7f28400b0270 msgr2=0x7f2840143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 --2- 172.21.15.32:0/718585106 >> v2:172.21.15.55:3300/0 conn(0x7f28400b0270 0x7f2840143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 >> v1:172.21.15.55:6789/0 conn(0x7f28400ac880 legacy=0x7f28401434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840147160 con 0x7f28400b2710 2022-08-01T23:01:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f2830001660 con 0x7f28400b2710 2022-08-01T23:01:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.610+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 >> v1:172.21.15.32:6789/0 conn(0x7f28400b2710 legacy=0x7f28401401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f2820003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2820004500 0x7f2820004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2820004e70 0x7f282000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400ad740 con 0x7f2820003750 2022-08-01T23:01:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282000bca0 con 0x7f2820004500 2022-08-01T23:01:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282000be60 con 0x7f2820004e70 2022-08-01T23:01:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f2820003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f2837fff700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2820004500 0x7f2820004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283f59e700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2820004e70 0x7f282000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 -- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2820004e70 msgr2=0x7f282000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2820004e70 0x7f282000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 -- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2820004500 msgr2=0x7f2820004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2820004500 0x7f2820004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 -- 172.21.15.32:0/718585106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820003260 con 0x7f2820003750 2022-08-01T23:01:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283f59e700 1 -- 172.21.15.32:0/718585106 reap_dead start 2022-08-01T23:01:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283ed9d700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f2820003b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f2830001370 tx=0x7f2840067e40).ready entity=mon.0 client_cookie=1de802065e1acd1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28300070d0 con 0x7f2820003750 2022-08-01T23:01:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2830007230 con 0x7f2820003750 2022-08-01T23:01:26.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f283e59c700 1 -- 172.21.15.32:0/718585106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f283000c9c0 con 0x7f2820003750 2022-08-01T23:01:26.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 -- 172.21.15.32:0/718585106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 msgr2=0x7f2820003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f2820003b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f2830001370 tx=0x7f2840067e40).stop 2022-08-01T23:01:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 -- 172.21.15.32:0/718585106 shutdown_connections 2022-08-01T23:01:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 --2- 172.21.15.32:0/718585106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f2820003b50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 -- 172.21.15.32:0/718585106 >> 172.21.15.32:0/718585106 conn(0x7f2840060310 msgr2=0x7f28400b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 -- 172.21.15.32:0/718585106 shutdown_connections 2022-08-01T23:01:26.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.614+0000 7f28448e4700 1 -- 172.21.15.32:0/718585106 wait complete. 2022-08-01T23:01:26.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 Processor -- start 2022-08-01T23:01:26.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 -- start start 2022-08-01T23:01:26.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f284005c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2840140ea0 0x7f2840152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2840141320 0x7f2840152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284013e450 con 0x7f2820003750 2022-08-01T23:01:26.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2840152ff0 con 0x7f2840140ea0 2022-08-01T23:01:26.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28401520b0 con 0x7f2840141320 2022-08-01T23:01:26.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f284005c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f284005c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44482/0 (socket says 172.21.15.32:44482) 2022-08-01T23:01:26.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 -- 172.21.15.32:0/2072767529 learned_addr learned my addr 172.21.15.32:0/2072767529 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:26.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283f59e700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2840141320 0x7f2840152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f2837fff700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2840140ea0 0x7f2840152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 -- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2840141320 msgr2=0x7f2840152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2840141320 0x7f2840152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 -- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2840140ea0 msgr2=0x7f2840152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2840140ea0 0x7f2840152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 -- 172.21.15.32:0/2072767529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400aa9d0 con 0x7f2820003750 2022-08-01T23:01:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283ed9d700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f284005c770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f2840153180 tx=0x7f2830013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f283001b070 con 0x7f2820003750 2022-08-01T23:01:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2830004920 con 0x7f2820003750 2022-08-01T23:01:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28400aac70 con 0x7f2820003750 2022-08-01T23:01:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28300078b0 con 0x7f2820003750 2022-08-01T23:01:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.618+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28400ac210 con 0x7f2820003750 2022-08-01T23:01:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.622+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28400574e0 con 0x7f2820003750 2022-08-01T23:01:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.622+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2830010070 con 0x7f2820003750 2022-08-01T23:01:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.622+0000 7f283cd99700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2824053a70 0x7f2824055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.622+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f283006c0f0 con 0x7f2820003750 2022-08-01T23:01:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.622+0000 7f2837fff700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2824053a70 0x7f2824055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.622+0000 7f2837fff700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2824053a70 0x7f2824055f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f282800b000 tx=0x7f2828008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.634+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28300430f0 con 0x7f2820003750 2022-08-01T23:01:26.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.762+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f28400b1bb0 con 0x7f2820003750 2022-08-01T23:01:26.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f283cd99700 1 -- 172.21.15.32:0/2072767529 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2830046710 con 0x7f2820003750 2022-08-01T23:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2824053a70 msgr2=0x7f2824055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2824053a70 0x7f2824055f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f282800b000 tx=0x7f2828008040).stop 2022-08-01T23:01:26.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 msgr2=0x7f284005c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f284005c770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f2840153180 tx=0x7f2830013000).stop 2022-08-01T23:01:26.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 shutdown_connections 2022-08-01T23:01:26.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2824053a70 0x7f2824055f30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2840141320 0x7f2840152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2840140ea0 0x7f2840152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 --2- 172.21.15.32:0/2072767529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2820003750 0x7f284005c770 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 >> 172.21.15.32:0/2072767529 conn(0x7f2840060310 msgr2=0x7f284013eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:26.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 shutdown_connections 2022-08-01T23:01:26.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.770+0000 7f28448e4700 1 -- 172.21.15.32:0/2072767529 wait complete. 2022-08-01T23:01:26.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-08-01T23:01:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 Processor -- start 2022-08-01T23:01:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 -- start start 2022-08-01T23:01:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cd4064b10 0x7f7cd4144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4141a80 0x7f7cd4066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd4064ef0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd413ebf0 con 0x7f7cd4064b10 2022-08-01T23:01:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.886+0000 7f7cd8aa5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4066690 con 0x7f7cd4141a80 2022-08-01T23:01:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cd4064b10 0x7f7cd4144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cd4064b10 0x7f7cd4144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44490/0 (socket says 172.21.15.32:44490) 2022-08-01T23:01:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 -- 172.21.15.32:0/3511477777 learned_addr learned my addr 172.21.15.32:0/3511477777 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd3fff700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4141a80 0x7f7cd4066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 -- 172.21.15.32:0/3511477777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4141a80 msgr2=0x7f7cd4066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4141a80 0x7f7cd4066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 -- 172.21.15.32:0/3511477777 >> v1:172.21.15.55:6789/0 conn(0x7f7cd40b36e0 legacy=0x7f7cd4141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 -- 172.21.15.32:0/3511477777 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40b4850 con 0x7f7cd4064b10 2022-08-01T23:01:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd3fff700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4141a80 0x7f7cd4066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd37fe700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd4064b10 0x7f7cd4144e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f7ccc0032f0 tx=0x7f7ccc009290).ready entity=mon.0 client_cookie=4307d7e4e3bfa0e7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd27fc700 1 -- 172.21.15.32:0/3511477777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ccc00b8a0 con 0x7f7cd4064b10 2022-08-01T23:01:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd27fc700 1 -- 172.21.15.32:0/3511477777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc00ba00 con 0x7f7cd4064b10 2022-08-01T23:01:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd27fc700 1 -- 172.21.15.32:0/3511477777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ccc00bb70 con 0x7f7cd4064b10 2022-08-01T23:01:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/3511477777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd4064b10 msgr2=0x7f7cd4144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd4064b10 0x7f7cd4144e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f7ccc0032f0 tx=0x7f7ccc009290).stop 2022-08-01T23:01:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/3511477777 shutdown_connections 2022-08-01T23:01:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4141a80 0x7f7cd4066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/3511477777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd4064b10 0x7f7cd4144e60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/3511477777 >> 172.21.15.32:0/3511477777 conn(0x7f7cd405f840 msgr2=0x7f7cd405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/3511477777 shutdown_connections 2022-08-01T23:01:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/3511477777 wait complete. 2022-08-01T23:01:26.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 Processor -- start 2022-08-01T23:01:26.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.890+0000 7f7cd8aa5700 1 -- start start 2022-08-01T23:01:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4064b10 0x7f7cd40b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 0x7f7cd4150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cd4141a80 0x7f7cd4150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40a89f0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4151160 con 0x7f7cd4141a80 2022-08-01T23:01:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ae300 con 0x7f7cd4064b10 2022-08-01T23:01:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 0x7f7cd4150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 0x7f7cd4150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44498/0 (socket says 172.21.15.32:44498) 2022-08-01T23:01:26.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 -- 172.21.15.32:0/2155043166 learned_addr learned my addr 172.21.15.32:0/2155043166 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:26.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd37fe700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4064b10 0x7f7cd40b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd3fff700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cd4141a80 0x7f7cd4150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 -- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4064b10 msgr2=0x7f7cd40b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4064b10 0x7f7cd40b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 -- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cd4141a80 msgr2=0x7f7cd4150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cd4141a80 0x7f7cd4150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 -- 172.21.15.32:0/2155043166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc009530 con 0x7f7cd40b36e0 2022-08-01T23:01:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd2ffd700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 0x7f7cd4150780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f7cd40b74a0 tx=0x7f7cc8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cc8012400 con 0x7f7cd40b36e0 2022-08-01T23:01:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc8012560 con 0x7f7cd40b36e0 2022-08-01T23:01:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd413bad0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cc80128c0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd40adff0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd40b4af0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.894+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7cc8019030 con 0x7f7cd40b36e0 2022-08-01T23:01:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.898+0000 7f7cd0ff9700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cb40539e0 0x7f7cb4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.898+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7cc806b0a0 con 0x7f7cd40b36e0 2022-08-01T23:01:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.898+0000 7f7cd37fe700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cb40539e0 0x7f7cb4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.898+0000 7f7cd37fe700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cb40539e0 0x7f7cb4055ea0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f7ccc01a000 tx=0x7f7ccc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:26.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:26.906+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7cc80420e0 con 0x7f7cd40b36e0 2022-08-01T23:01:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.074+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f7cd4069d90 con 0x7f7cd40b36e0 2022-08-01T23:01:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.494+0000 7f7cd0ff9700 1 -- 172.21.15.32:0/2155043166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v244) v1 ==== 147+0+0 (secure 0 0 0) 0x7f7cc8045700 con 0x7f7cd40b36e0 2022-08-01T23:01:27.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cb40539e0 msgr2=0x7f7cb4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cb40539e0 0x7f7cb4055ea0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f7ccc01a000 tx=0x7f7ccc016000).stop 2022-08-01T23:01:27.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 msgr2=0x7f7cd4150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 0x7f7cd4150780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f7cd40b74a0 tx=0x7f7cc8007c50).stop 2022-08-01T23:01:27.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 shutdown_connections 2022-08-01T23:01:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7cb40539e0 0x7f7cb4055ea0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cd4141a80 0x7f7cd4150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd40b36e0 0x7f7cd4150780 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 --2- 172.21.15.32:0/2155043166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cd4064b10 0x7f7cd40b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 >> 172.21.15.32:0/2155043166 conn(0x7f7cd405f840 msgr2=0x7f7cd41402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 shutdown_connections 2022-08-01T23:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.498+0000 7f7cd8aa5700 1 -- 172.21.15.32:0/2155043166 wait complete. 2022-08-01T23:01:27.510 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T23:01:27.510 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:01:27.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-08-01T23:01:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3bfff700 1 Processor -- start 2022-08-01T23:01:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3bfff700 1 -- start start 2022-08-01T23:01:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fca3c064b10 0x7fca3c144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c064ef0 con 0x7fca3c141a80 2022-08-01T23:01:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c066690 con 0x7fca3c0b36e0 2022-08-01T23:01:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c13ebf0 con 0x7fca3c064b10 2022-08-01T23:01:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca3c141a80 0x7fca3c066ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60280/0 (socket says 172.21.15.32:60280) 2022-08-01T23:01:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3b7fe700 1 -- 172.21.15.32:0/3698287794 learned_addr learned my addr 172.21.15.32:0/3698287794 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3affd700 1 --2- 172.21.15.32:0/3698287794 >> v2:172.21.15.55:3300/0 conn(0x7fca3c064b10 0x7fca3c144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804900478 0 0) 0x7fca3c064ef0 con 0x7fca3c141a80 2022-08-01T23:01:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca1c003260 con 0x7fca3c141a80 2022-08-01T23:01:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423547305 0 0) 0x7fca1c003260 con 0x7fca3c141a80 2022-08-01T23:01:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 >> v2:172.21.15.55:3300/0 conn(0x7fca3c064b10 msgr2=0x7fca3c144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 --2- 172.21.15.32:0/3698287794 >> v2:172.21.15.55:3300/0 conn(0x7fca3c064b10 0x7fca3c144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 >> v1:172.21.15.55:6789/0 conn(0x7fca3c0b36e0 legacy=0x7fca3c141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0b4850 con 0x7fca3c141a80 2022-08-01T23:01:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fca340019e0 con 0x7fca3c141a80 2022-08-01T23:01:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 >> v1:172.21.15.32:6789/0 conn(0x7fca3c141a80 legacy=0x7fca3c066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c003750 0x7fca1c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca1c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca1c004e70 0x7fca1c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00bcf0 con 0x7fca1c004500 2022-08-01T23:01:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00be60 con 0x7fca1c004e70 2022-08-01T23:01:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00c020 con 0x7fca1c003750 2022-08-01T23:01:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca1c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3b7fe700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca1c004e70 0x7fca1c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3affd700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c003750 0x7fca1c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 -- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c003750 msgr2=0x7fca1c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca1c003750 0x7fca1c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 -- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca1c004e70 msgr2=0x7fca1c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca1c004e70 0x7fca1c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3b7fe700 1 -- 172.21.15.32:0/3698287794 reap_dead start 2022-08-01T23:01:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 -- 172.21.15.32:0/3698287794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c003260 con 0x7fca1c004500 2022-08-01T23:01:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca33fff700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca1c004980 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fca3c063dd0 tx=0x7fca24007a00).ready entity=mon.0 client_cookie=5138286436e832d0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca24007d10 con 0x7fca1c004500 2022-08-01T23:01:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca24007220 con 0x7fca1c004500 2022-08-01T23:01:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.614+0000 7fca3a7fc700 1 -- 172.21.15.32:0/3698287794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca24006510 con 0x7fca1c004500 2022-08-01T23:01:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 -- 172.21.15.32:0/3698287794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 msgr2=0x7fca1c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca1c004980 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fca3c063dd0 tx=0x7fca24007a00).stop 2022-08-01T23:01:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 -- 172.21.15.32:0/3698287794 shutdown_connections 2022-08-01T23:01:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 --2- 172.21.15.32:0/3698287794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca1c004980 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 -- 172.21.15.32:0/3698287794 >> 172.21.15.32:0/3698287794 conn(0x7fca3c05f840 msgr2=0x7fca3c05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 -- 172.21.15.32:0/3698287794 shutdown_connections 2022-08-01T23:01:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 -- 172.21.15.32:0/3698287794 wait complete. 2022-08-01T23:01:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 Processor -- start 2022-08-01T23:01:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.618+0000 7fca3bfff700 1 -- start start 2022-08-01T23:01:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca3c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca3c0b5440 0x7fca3c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca3c0b7630 0x7fca3c0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c14e420 con 0x7fca1c004500 2022-08-01T23:01:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b6a40 con 0x7fca3c0b5440 2022-08-01T23:01:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c14e180 con 0x7fca3c0b7630 2022-08-01T23:01:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca3c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca3c0af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44516/0 (socket says 172.21.15.32:44516) 2022-08-01T23:01:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 -- 172.21.15.32:0/1797128567 learned_addr learned my addr 172.21.15.32:0/1797128567 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca33fff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca3c0b5440 0x7fca3c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3b7fe700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca3c0b7630 0x7fca3c0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 -- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca3c0b7630 msgr2=0x7fca3c0b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca3c0b7630 0x7fca3c0b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 -- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca3c0b5440 msgr2=0x7fca3c0b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca3c0b5440 0x7fca3c0b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 -- 172.21.15.32:0/1797128567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0b6ca0 con 0x7fca1c004500 2022-08-01T23:01:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3b7fe700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca3c0b7630 0x7fca3c0b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3affd700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca3c0af810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fca2c00b000 tx=0x7fca2c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca2c00f940 con 0x7fca1c004500 2022-08-01T23:01:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c000e90 con 0x7fca1c004500 2022-08-01T23:01:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c0b1820 con 0x7fca1c004500 2022-08-01T23:01:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca2c00fcd0 con 0x7fca1c004500 2022-08-01T23:01:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c1390d0 con 0x7fca1c004500 2022-08-01T23:01:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.622+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c0b6e30 con 0x7fca1c004500 2022-08-01T23:01:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.626+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fca2c014440 con 0x7fca1c004500 2022-08-01T23:01:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.626+0000 7fca38ff9700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca20053b00 0x7fca20055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.626+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fca2c06c940 con 0x7fca1c004500 2022-08-01T23:01:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.626+0000 7fca33fff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca20053b00 0x7fca20055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.626+0000 7fca33fff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca20053b00 0x7fca20055fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fca2401d000 tx=0x7fca24017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.630+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca2c043870 con 0x7fca1c004500 2022-08-01T23:01:27.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:27.798+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fca3c0b4af0 con 0x7fca1c004500 2022-08-01T23:01:28.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.502+0000 7fca38ff9700 1 -- 172.21.15.32:0/1797128567 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v245) v1 ==== 142+0+0 (secure 0 0 0) 0x7fca2c018070 con 0x7fca1c004500 2022-08-01T23:01:28.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca20053b00 msgr2=0x7fca20055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca20053b00 0x7fca20055fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fca2401d000 tx=0x7fca24017000).stop 2022-08-01T23:01:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 msgr2=0x7fca3c0af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca3c0af810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fca2c00b000 tx=0x7fca2c006df0).stop 2022-08-01T23:01:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 shutdown_connections 2022-08-01T23:01:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca20053b00 0x7fca20055fc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fca3c0b7630 0x7fca3c0b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca3c0b5440 0x7fca3c0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 --2- 172.21.15.32:0/1797128567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca1c004500 0x7fca3c0af810 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 >> 172.21.15.32:0/1797128567 conn(0x7fca3c05f840 msgr2=0x7fca3c05be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 shutdown_connections 2022-08-01T23:01:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.506+0000 7fca3bfff700 1 -- 172.21.15.32:0/1797128567 wait complete. 2022-08-01T23:01:28.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:01:28.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.638+0000 7fccd0f1c700 1 Processor -- start 2022-08-01T23:01:28.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.638+0000 7fccd0f1c700 1 -- start start 2022-08-01T23:01:28.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcccc0b0d10 0x7fcccc146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc0b12d0 con 0x7fcccc0b0560 2022-08-01T23:01:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc0a9700 con 0x7fcccc0a9320 2022-08-01T23:01:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcccc0b1500 con 0x7fcccc0b0d10 2022-08-01T23:01:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcccc0b0560 0x7fcccc140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60302/0 (socket says 172.21.15.32:60302) 2022-08-01T23:01:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccbfff700 1 -- 172.21.15.32:0/3572572874 learned_addr learned my addr 172.21.15.32:0/3572572874 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccc3fff700 1 --2- 172.21.15.32:0/3572572874 >> v2:172.21.15.55:3300/0 conn(0x7fcccc0b0d10 0x7fcccc146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccaffd700 1 -- 172.21.15.32:0/3572572874 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979833652 0 0) 0x7fcccc0b12d0 con 0x7fcccc0b0560 2022-08-01T23:01:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccaffd700 1 -- 172.21.15.32:0/3572572874 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccac003260 con 0x7fcccc0b0560 2022-08-01T23:01:28.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccc3fff700 1 -- 172.21.15.32:0/3572572874 >> v1:172.21.15.55:6789/0 conn(0x7fcccc0a9320 legacy=0x7fcccc1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccc3fff700 1 -- 172.21.15.32:0/3572572874 >> v1:172.21.15.32:6789/0 conn(0x7fcccc0b0560 legacy=0x7fcccc140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccc3fff700 1 -- 172.21.15.32:0/3572572874 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0b17a0 con 0x7fcccc0b0d10 2022-08-01T23:01:28.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccc3fff700 1 --2- 172.21.15.32:0/3572572874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc146df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fccb4002f70 tx=0x7fccb4006df0).ready entity=mon.1 client_cookie=7e7eebce035c5d8b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:28.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccaffd700 1 -- 172.21.15.32:0/3572572874 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb400d5c0 con 0x7fcccc0b0d10 2022-08-01T23:01:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccaffd700 1 -- 172.21.15.32:0/3572572874 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4000e90 con 0x7fcccc0b0d10 2022-08-01T23:01:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fcccaffd700 1 -- 172.21.15.32:0/3572572874 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb400d910 con 0x7fcccc0b0d10 2022-08-01T23:01:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- 172.21.15.32:0/3572572874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 msgr2=0x7fcccc146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/3572572874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc146df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fccb4002f70 tx=0x7fccb4006df0).stop 2022-08-01T23:01:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- 172.21.15.32:0/3572572874 shutdown_connections 2022-08-01T23:01:28.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/3572572874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc146df0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- 172.21.15.32:0/3572572874 >> 172.21.15.32:0/3572572874 conn(0x7fcccc0612d0 msgr2=0x7fcccc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:28.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.642+0000 7fccd0f1c700 1 -- 172.21.15.32:0/3572572874 shutdown_connections 2022-08-01T23:01:28.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 -- 172.21.15.32:0/3572572874 wait complete. 2022-08-01T23:01:28.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 Processor -- start 2022-08-01T23:01:28.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 -- start start 2022-08-01T23:01:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 0x7fcccc13b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcccc0b0560 0x7fcccc0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0b37c0 con 0x7fcccc0a9320 2022-08-01T23:01:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc146010 con 0x7fcccc0b0d10 2022-08-01T23:01:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc145840 con 0x7fcccc0b0560 2022-08-01T23:01:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 0x7fcccc13b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 0x7fcccc13b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44530/0 (socket says 172.21.15.32:44530) 2022-08-01T23:01:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 -- 172.21.15.32:0/980845939 learned_addr learned my addr 172.21.15.32:0/980845939 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccbfff700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccc3fff700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcccc0b0560 0x7fcccc0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 -- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcccc0b0560 msgr2=0x7fcccc0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcccc0b0560 0x7fcccc0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 -- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 msgr2=0x7fcccc156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 -- 172.21.15.32:0/980845939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb400b180 con 0x7fcccc0a9320 2022-08-01T23:01:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccc3fff700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcccc0b0560 0x7fcccc0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fcccb7fe700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 0x7fcccc13b180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fcccc146150 tx=0x7fccbc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccbc012400 con 0x7fcccc0a9320 2022-08-01T23:01:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc012560 con 0x7fcccc0a9320 2022-08-01T23:01:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc156c70 con 0x7fcccc0a9320 2022-08-01T23:01:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.646+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccbc0128c0 con 0x7fcccc0a9320 2022-08-01T23:01:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc0b39b0 con 0x7fcccc0a9320 2022-08-01T23:01:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc145d50 con 0x7fcccc0a9320 2022-08-01T23:01:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fccbc010070 con 0x7fcccc0a9320 2022-08-01T23:01:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccc97fa700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fccb0053a70 0x7fccb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fccbc06ac10 con 0x7fcccc0a9320 2022-08-01T23:01:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccc3fff700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fccb0053a70 0x7fccb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.650+0000 7fccc3fff700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fccb0053a70 0x7fccb0055f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fccb4018000 tx=0x7fccb4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.658+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fccbc041c10 con 0x7fcccc0a9320 2022-08-01T23:01:28.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.786+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcccc156c70 con 0x7fcccc0a9320 2022-08-01T23:01:28.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.794+0000 7fccc97fa700 1 -- 172.21.15.32:0/980845939 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcccc156c70 con 0x7fcccc0a9320 2022-08-01T23:01:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fccb0053a70 msgr2=0x7fccb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fccb0053a70 0x7fccb0055f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fccb4018000 tx=0x7fccb4014000).stop 2022-08-01T23:01:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 msgr2=0x7fcccc13b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 0x7fcccc13b180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fcccc146150 tx=0x7fccbc007c50).stop 2022-08-01T23:01:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 shutdown_connections 2022-08-01T23:01:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fccb0053a70 0x7fccb0055f30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0b0d10 0x7fcccc156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcccc0b0560 0x7fcccc0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 --2- 172.21.15.32:0/980845939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcccc0a9320 0x7fcccc13b180 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 >> 172.21.15.32:0/980845939 conn(0x7fcccc0612d0 msgr2=0x7fcccc13c430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 shutdown_connections 2022-08-01T23:01:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:28.798+0000 7fccd0f1c700 1 -- 172.21.15.32:0/980845939 wait complete. 2022-08-01T23:01:28.809 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:01:28.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:28.824 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:28.846 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:28.867 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:01:28.888 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:01:28.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:28.889 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:28.889 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:28.915 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:01:28.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:01:28.917 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:01:28.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:28.918 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:28.918 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:28.925 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:01:28.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:01:28.927 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:01:28.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:01:29.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.038+0000 7f7973fff700 1 Processor -- start 2022-08-01T23:01:29.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.038+0000 7f7973fff700 1 -- start start 2022-08-01T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79740b2710 0x7f797414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740a7250 con 0x7f79740b01a0 2022-08-01T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740a7480 con 0x7f79740a8c50 2022-08-01T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79740a7720 con 0x7f79740b2710 2022-08-01T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f796bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79740b01a0 0x7f79740a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60312/0 (socket says 172.21.15.32:60312) 2022-08-01T23:01:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f796bfff700 1 -- 172.21.15.32:0/3305430698 learned_addr learned my addr 172.21.15.32:0/3305430698 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7972ffd700 1 --2- 172.21.15.32:0/3305430698 >> v2:172.21.15.32:3300/0 conn(0x7f79740b2710 0x7f797414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722292004 0 0) 0x7f79740a7250 con 0x7f79740b01a0 2022-08-01T23:01:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7954003260 con 0x7f79740b01a0 2022-08-01T23:01:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777053618 0 0) 0x7f79740a7480 con 0x7f79740a8c50 2022-08-01T23:01:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79740a7250 con 0x7f79740a8c50 2022-08-01T23:01:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7972ffd700 1 -- 172.21.15.32:0/3305430698 >> v1:172.21.15.55:6789/0 conn(0x7f79740a8c50 legacy=0x7f7974151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7972ffd700 1 -- 172.21.15.32:0/3305430698 >> v1:172.21.15.32:6789/0 conn(0x7f79740b01a0 legacy=0x7f79740a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7972ffd700 1 -- 172.21.15.32:0/3305430698 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740a79c0 con 0x7f79740b2710 2022-08-01T23:01:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7972ffd700 1 --2- 172.21.15.32:0/3305430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740b2710 0x7f797414afd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f7964002f70 tx=0x7f7964006df0).ready entity=mon.0 client_cookie=ad6137815d840ef8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796400d540 con 0x7f79740b2710 2022-08-01T23:01:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7964000e90 con 0x7f79740b2710 2022-08-01T23:01:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f79727fc700 1 -- 172.21.15.32:0/3305430698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f796400d890 con 0x7f79740b2710 2022-08-01T23:01:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- 172.21.15.32:0/3305430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740b2710 msgr2=0x7f797414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 --2- 172.21.15.32:0/3305430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740b2710 0x7f797414afd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f7964002f70 tx=0x7f7964006df0).stop 2022-08-01T23:01:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- 172.21.15.32:0/3305430698 shutdown_connections 2022-08-01T23:01:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 --2- 172.21.15.32:0/3305430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740b2710 0x7f797414afd0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- 172.21.15.32:0/3305430698 >> 172.21.15.32:0/3305430698 conn(0x7f7974060310 msgr2=0x7f79740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- 172.21.15.32:0/3305430698 shutdown_connections 2022-08-01T23:01:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.042+0000 7f7973fff700 1 -- 172.21.15.32:0/3305430698 wait complete. 2022-08-01T23:01:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 Processor -- start 2022-08-01T23:01:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 -- start start 2022-08-01T23:01:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 0x7f79740aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79740b01a0 0x7f79740b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79740b2710 0x7f79740b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79740ac3b0 con 0x7f79740a8c50 2022-08-01T23:01:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79740b3870 con 0x7f79740b2710 2022-08-01T23:01:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7973fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79740ab530 con 0x7f79740b01a0 2022-08-01T23:01:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.046+0000 7f7972ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 0x7f79740aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 0x7f79740aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44544/0 (socket says 172.21.15.32:44544) 2022-08-01T23:01:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 -- 172.21.15.32:0/2623288166 learned_addr learned my addr 172.21.15.32:0/2623288166 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f796bfff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79740b01a0 0x7f79740b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f79737fe700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79740b2710 0x7f79740b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 -- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79740b01a0 msgr2=0x7f79740b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79740b01a0 0x7f79740b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 -- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79740b2710 msgr2=0x7f79740b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79740b2710 0x7f79740b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 -- 172.21.15.32:0/2623288166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796400b180 con 0x7f79740a8c50 2022-08-01T23:01:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f796bfff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79740b01a0 0x7f79740b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f79737fe700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79740b2710 0x7f79740b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7972ffd700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 0x7f79740aa830 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f79740ab6c0 tx=0x7f7964014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7964016070 con 0x7f79740a8c50 2022-08-01T23:01:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7964013980 con 0x7f79740a8c50 2022-08-01T23:01:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79740ab290 con 0x7f79740a8c50 2022-08-01T23:01:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7964013ce0 con 0x7f79740a8c50 2022-08-01T23:01:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79740abff0 con 0x7f79740a8c50 2022-08-01T23:01:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79740574e0 con 0x7f79740a8c50 2022-08-01T23:01:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.050+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7964025030 con 0x7f79740a8c50 2022-08-01T23:01:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.054+0000 7f7970ff9700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7958053980 0x7f7958055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.054+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f796406cdf0 con 0x7f79740a8c50 2022-08-01T23:01:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.054+0000 7f796bfff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7958053980 0x7f7958055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.054+0000 7f796bfff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7958053980 0x7f7958055e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f795c00b000 tx=0x7f795c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.058+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7964043e90 con 0x7f79740a8c50 2022-08-01T23:01:29.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.189+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7974069a70 con 0x7f79740a8c50 2022-08-01T23:01:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.189+0000 7f7970ff9700 1 -- 172.21.15.32:0/2623288166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v70) v1 ==== 81+0+29218 (secure 0 0 0) 0x7f7964023020 con 0x7f79740a8c50 2022-08-01T23:01:29.191 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:29.191 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:01:29.191 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 70, 2022-08-01T23:01:29.192 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:01:29.192 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:01:29.192 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:01:26.472362+0000", 2022-08-01T23:01:29.192 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:01:29.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:01:29.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:01:29.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:01:29.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:01:29.194 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:01:29.194 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:01:29.194 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:01:29.194 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:01:29.195 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:29.195 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:01:29.195 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:01:29.195 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:01:29.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:01:29.196 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:01:29.196 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 12, 2022-08-01T23:01:29.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:01:29.196 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:01:29.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:01:29.197 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:01:29.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:01:29.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.198 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:01:29.198 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:01:29.198 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:01:29.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:01:29.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:01:29.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:01:29.199 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:01:29.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:01:29.200 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:01:29.200 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:01:29.200 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:01:29.200 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:01:29.200 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:01:29.201 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:01:29.201 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:01:29.201 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:01:29.201 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:01:29.202 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:01:29.203 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:01:29.204 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:01:29.205 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:01:29.205 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:01:29.205 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:01:29.205 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:01:29.206 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:01:29.206 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:01:29.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:01:29.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:01:29.207 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:01:29.207 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:01:29.207 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:01:29.207 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:01:29.207 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:01:29.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:01:29.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:01:29.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.252 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:01:29.252 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:01:29.252 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:01:29.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:01:29.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:01:29.253 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:01:29.253 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:01:29.254 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:01:29.254 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:01:29.254 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:01:29.254 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:01:29.255 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:01:29.255 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:01:29.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.255 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:01:29.256 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:01:29.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.256 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:01:29.257 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:01:29.257 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:01:29.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:01:29.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:01:29.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:01:29.258 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:01:29.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:01:29.258 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:01:29.259 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:01:29.259 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:01:29.259 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:01:29.259 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:01:29.259 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:01:29.260 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:01:29.260 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:01:29.260 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:01:29.260 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:01:29.261 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:01:29.261 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:01:29.261 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:01:29.261 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:01:29.261 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:01:29.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.262 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:01:29.262 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:01:29.262 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:01:29.263 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:01:29.263 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:01:29.263 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:01:29.263 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:01:29.264 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:01:29.264 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:01:29.264 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:01:29.264 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:01:29.264 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:01:29.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:01:29.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:01:29.265 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:01:29.265 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:01:29.266 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:01:29.266 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:01:29.266 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:01:29.266 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:01:29.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:01:29.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:01:29.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:01:29.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:01:29.267 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:01:29.268 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:01:29.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:01:29.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.268 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:01:29.269 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:01:29.269 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:01:29.269 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:01:29.269 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:01:29.270 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:01:29.270 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:01:29.270 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:01:29.270 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:01:29.270 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:01:29.271 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:01:29.271 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:01:29.271 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:01:29.271 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:01:29.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.272 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:29.272 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:01:29.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.273 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:01:29.273 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:01:29.273 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.273 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.274 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.274 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.275 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.275 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:01:29.275 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.275 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.276 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:01:29.277 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:01:29.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.279 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.279 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:01:29.280 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:01:29.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.281 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.282 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:01:29.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:01:29.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.285 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:01:29.286 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:01:29.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:01:29.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.288 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:01:29.288 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:01:29.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:01:29.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:01:29.289 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.289 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.291 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:01:29.291 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:01:29.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.291 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.291 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.292 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.292 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:01:29.293 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.293 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.293 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:01:29.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:01:29.295 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.296 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.296 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:01:29.297 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:01:29.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.299 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:01:29.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:01:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.302 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:01:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:01:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:01:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.305 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:01:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:01:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:01:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:01:29.306 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:01:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:01:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.308 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 62, 2022-08-01T23:01:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.310 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:01:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:01:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:01:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:01:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:01:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:01:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:01:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:01:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:01:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.322 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:01:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:01:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:01:29.323 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:01:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:01:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:01:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.326 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 52, 2022-08-01T23:01:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.327 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:01:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:01:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:01:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:01:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:01:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:01:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:01:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:01:29.339 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:01:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.340 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:01:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:01:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:01:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:01:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.341 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:01:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:01:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.343 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.344 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 65, 2022-08-01T23:01:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.345 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:01:29.346 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:01:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.348 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:01:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:01:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:01:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:01:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:01:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:01:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:01:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:01:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:01:29.358 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:01:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:01:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:01:29.360 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:01:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.361 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 62, 2022-08-01T23:01:29.362 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:01:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:01:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:01:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:01:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:01:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:01:29.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.372 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:01:29.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:01:29.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:01:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:01:29.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:01:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:01:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:01:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.376 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.377 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:01:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:01:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.379 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:01:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:01:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:01:29.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:01:29.384 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:01:29.385 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.386 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:01:29.387 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:01:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.389 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.390 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:01:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:01:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:01:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:01:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:01:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:01:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:01:29.393 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.394 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.394 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.395 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:01:29.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:01:29.395 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:01:29.395 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:01:29.396 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:01:29.396 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:01:29.396 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:01:29.396 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:01:29.397 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:01:29.397 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 62, 2022-08-01T23:01:29.397 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:01:29.397 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:01:29.397 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:01:29.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:01:29.399 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.400 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:01:29.400 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.401 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:01:29.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.402 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:01:29.402 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.403 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:01:29.403 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.403 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.404 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:01:29.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.405 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:01:29.405 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.406 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:01:29.406 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.407 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.407 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:01:29.407 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:01:29.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:01:29.408 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:01:29.408 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:01:29.409 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:01:29.410 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:01:29.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.411 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:01:29.411 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:01:29.411 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:01:29.411 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:01:29.411 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:01:29.412 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:01:29.412 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:01:29.412 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:29.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.413 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:29.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:01:29.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:01:29.413 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.414 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.414 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.414 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.414 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.415 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:01:29.415 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:01:29.416 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.416 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.416 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.416 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.417 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:01:29.417 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:01:29.418 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.418 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.419 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.419 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.419 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.419 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:01:29.419 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:01:29.420 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.421 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.421 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.421 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.421 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.422 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:01:29.422 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:01:29.423 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.423 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.423 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.424 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.424 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:01:29.424 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:01:29.425 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.425 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.426 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.426 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.426 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.426 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:01:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:01:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.428 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.428 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.428 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.428 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:01:29.429 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:29.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:01:29.430 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:01:29.430 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:01:29.430 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:01:29.430 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:01:29.431 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:01:29.431 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:01:29.431 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:01:29.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.431 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:01:29.432 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:01:29.432 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:01:29.432 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:01:29.432 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:01:29.433 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:01:29.433 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.55:6835/590504455": "2022-08-02T23:01:23.388193+0000", 2022-08-01T23:01:29.433 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.55:6834/590504455": "2022-08-02T23:01:23.388193+0000" 2022-08-01T23:01:29.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.434 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:01:29.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:01:29.434 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:01:29.434 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:01:29.435 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:01:29.435 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:01:29.435 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:01:29.435 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:01:29.435 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:01:29.436 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:01:29.436 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:01:29.436 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:01:29.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:01:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:01:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:01:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:01:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:01:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:01:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:01:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:01:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:01:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:01:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:01:29.440 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:01:29.440 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:01:29.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7958053980 msgr2=0x7f7958055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7958053980 0x7f7958055e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f795c00b000 tx=0x7f795c009040).stop 2022-08-01T23:01:29.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 msgr2=0x7f79740aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 0x7f79740aa830 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f79740ab6c0 tx=0x7f7964014000).stop 2022-08-01T23:01:29.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 shutdown_connections 2022-08-01T23:01:29.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7958053980 0x7f7958055e40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79740b2710 0x7f79740b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79740b01a0 0x7f79740b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 --2- 172.21.15.32:0/2623288166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79740a8c50 0x7f79740aa830 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 >> 172.21.15.32:0/2623288166 conn(0x7f7974060310 msgr2=0x7f7974149d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:29.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 shutdown_connections 2022-08-01T23:01:29.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.193+0000 7f7973fff700 1 -- 172.21.15.32:0/2623288166 wait complete. 2022-08-01T23:01:29.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:29.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.557+0000 7faea1ebc700 1 Processor -- start 2022-08-01T23:01:29.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.557+0000 7faea1ebc700 1 -- start start 2022-08-01T23:01:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae9c0b2710 0x7fae9c0abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae9c0b13f0 0x7fae9c0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c0adf10 con 0x7fae9c0b1920 2022-08-01T23:01:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0ae1b0 con 0x7fae9c0b13f0 2022-08-01T23:01:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0a9520 con 0x7fae9c0b2710 2022-08-01T23:01:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae9c0b13f0 0x7fae9c0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea16bb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fae9c0b1920 0x7fae9c0afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60328/0 (socket says 172.21.15.32:60328) 2022-08-01T23:01:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea16bb700 1 -- 172.21.15.32:0/1290566634 learned_addr learned my addr 172.21.15.32:0/1290566634 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea0eba700 1 --2- 172.21.15.32:0/1290566634 >> v2:172.21.15.55:3300/0 conn(0x7fae9c0b2710 0x7fae9c0abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9b7fe700 1 -- 172.21.15.32:0/1290566634 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646135482 0 0) 0x7fae9c0adf10 con 0x7fae9c0b1920 2022-08-01T23:01:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9bfff700 1 -- 172.21.15.32:0/1290566634 >> v2:172.21.15.55:3300/0 conn(0x7fae9c0b2710 msgr2=0x7fae9c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9bfff700 1 --2- 172.21.15.32:0/1290566634 >> v2:172.21.15.55:3300/0 conn(0x7fae9c0b2710 0x7fae9c0abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9bfff700 1 -- 172.21.15.32:0/1290566634 >> v1:172.21.15.32:6789/0 conn(0x7fae9c0b1920 legacy=0x7fae9c0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9bfff700 1 -- 172.21.15.32:0/1290566634 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0a99c0 con 0x7fae9c0b13f0 2022-08-01T23:01:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea0eba700 1 --2- 172.21.15.32:0/1290566634 >> v2:172.21.15.55:3300/0 conn(0x7fae9c0b2710 0x7fae9c0abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9bfff700 1 --2- 172.21.15.32:0/1290566634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b13f0 0x7fae9c0ac190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fae88002f70 tx=0x7fae88006df0).ready entity=mon.0 client_cookie=e17bfa5c9ef76daa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9b7fe700 1 -- 172.21.15.32:0/1290566634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8800d5c0 con 0x7fae9c0b13f0 2022-08-01T23:01:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9b7fe700 1 -- 172.21.15.32:0/1290566634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88000e90 con 0x7fae9c0b13f0 2022-08-01T23:01:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7fae9b7fe700 1 -- 172.21.15.32:0/1290566634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8800d910 con 0x7fae9c0b13f0 2022-08-01T23:01:29.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- 172.21.15.32:0/1290566634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b13f0 msgr2=0x7fae9c0ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1290566634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b13f0 0x7fae9c0ac190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fae88002f70 tx=0x7fae88006df0).stop 2022-08-01T23:01:29.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- 172.21.15.32:0/1290566634 shutdown_connections 2022-08-01T23:01:29.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1290566634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b13f0 0x7fae9c0ac190 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1290566634 >> v2:172.21.15.55:3300/0 conn(0x7fae9c0b2710 0x7fae9c0abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- 172.21.15.32:0/1290566634 >> 172.21.15.32:0/1290566634 conn(0x7fae9c060310 msgr2=0x7fae9c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- 172.21.15.32:0/1290566634 shutdown_connections 2022-08-01T23:01:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.561+0000 7faea1ebc700 1 -- 172.21.15.32:0/1290566634 wait complete. 2022-08-01T23:01:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.565+0000 7faea1ebc700 1 Processor -- start 2022-08-01T23:01:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.565+0000 7faea1ebc700 1 -- start start 2022-08-01T23:01:29.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae9c0b13f0 0x7fae9c153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 0x7fae9c066a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae9c0b2710 0x7fae9c05c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c09ea40 con 0x7fae9c0b1920 2022-08-01T23:01:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c150a20 con 0x7fae9c0b13f0 2022-08-01T23:01:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c063c40 con 0x7fae9c0b2710 2022-08-01T23:01:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 0x7fae9c066a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 0x7fae9c066a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44560/0 (socket says 172.21.15.32:44560) 2022-08-01T23:01:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 -- 172.21.15.32:0/1689036433 learned_addr learned my addr 172.21.15.32:0/1689036433 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea16bb700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae9c0b2710 0x7fae9c05c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea0eba700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae9c0b13f0 0x7fae9c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 -- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae9c0b2710 msgr2=0x7fae9c05c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae9c0b2710 0x7fae9c05c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 -- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae9c0b13f0 msgr2=0x7fae9c153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae9c0b13f0 0x7fae9c153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 -- 172.21.15.32:0/1689036433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8800b180 con 0x7fae9c0b1920 2022-08-01T23:01:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea0eba700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae9c0b13f0 0x7fae9c153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae9bfff700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 0x7fae9c066a20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fae88002f70 tx=0x7fae88014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8801e400 con 0x7fae9c0b1920 2022-08-01T23:01:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8801e560 con 0x7fae9c0b1920 2022-08-01T23:01:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c150e50 con 0x7fae9c0b1920 2022-08-01T23:01:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8801e8c0 con 0x7fae9c0b1920 2022-08-01T23:01:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c09ec70 con 0x7fae9c0b1920 2022-08-01T23:01:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c0574e0 con 0x7fae9c0b1920 2022-08-01T23:01:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.569+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fae88025030 con 0x7fae9c0b1920 2022-08-01T23:01:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.573+0000 7fae99ffb700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae7c053b30 0x7fae7c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.573+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fae8806cc90 con 0x7fae9c0b1920 2022-08-01T23:01:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.573+0000 7faea0eba700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae7c053b30 0x7fae7c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.573+0000 7faea0eba700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae7c053b30 0x7fae7c055ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fae9000c000 tx=0x7fae90008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.577+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fae88043c20 con 0x7fae9c0b1920 2022-08-01T23:01:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.737+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae9c069a70 con 0x7fae9c0b1920 2022-08-01T23:01:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.737+0000 7fae99ffb700 1 -- 172.21.15.32:0/1689036433 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fae88023070 con 0x7fae9c0b1920 2022-08-01T23:01:29.738 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:29.738 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":6121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/132377455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":132377455},{"type":"v1","addr":"172.21.15.55:6835","nonce":132377455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[]} 2022-08-01T23:01:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae7c053b30 msgr2=0x7fae7c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae7c053b30 0x7fae7c055ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fae9000c000 tx=0x7fae90008040).stop 2022-08-01T23:01:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 msgr2=0x7fae9c066a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 0x7fae9c066a20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fae88002f70 tx=0x7fae88014000).stop 2022-08-01T23:01:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 shutdown_connections 2022-08-01T23:01:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae7c053b30 0x7fae7c055ff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae9c0b2710 0x7fae9c05c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c0b1920 0x7fae9c066a20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 --2- 172.21.15.32:0/1689036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae9c0b13f0 0x7fae9c153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 >> 172.21.15.32:0/1689036433 conn(0x7fae9c060310 msgr2=0x7fae9c0673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 shutdown_connections 2022-08-01T23:01:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.741+0000 7faea1ebc700 1 -- 172.21.15.32:0/1689036433 wait complete. 2022-08-01T23:01:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 30 2022-08-01T23:01:29.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.853+0000 7f706f0cd700 1 Processor -- start 2022-08-01T23:01:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.853+0000 7f706f0cd700 1 -- start start 2022-08-01T23:01:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f70680b1f40 0x7f70680b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70680a7250 0x7f70681448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680b04e0 con 0x7f7068148530 2022-08-01T23:01:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70680a9250 con 0x7f70680a7250 2022-08-01T23:01:29.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70680b0780 con 0x7f70680b1f40 2022-08-01T23:01:29.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706d66a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70680a7250 0x7f70681448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f7067fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7068148530 0x7f70680b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60340/0 (socket says 172.21.15.32:60340) 2022-08-01T23:01:29.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f7067fff700 1 -- 172.21.15.32:0/2858068534 learned_addr learned my addr 172.21.15.32:0/2858068534 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:29.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706ce69700 1 --2- 172.21.15.32:0/2858068534 >> v2:172.21.15.55:3300/0 conn(0x7f70680b1f40 0x7f70680b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706d66a700 1 -- 172.21.15.32:0/2858068534 >> v2:172.21.15.55:3300/0 conn(0x7f70680b1f40 msgr2=0x7f70680b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:29.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706d66a700 1 --2- 172.21.15.32:0/2858068534 >> v2:172.21.15.55:3300/0 conn(0x7f70680b1f40 0x7f70680b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706d66a700 1 -- 172.21.15.32:0/2858068534 >> v1:172.21.15.32:6789/0 conn(0x7f7068148530 legacy=0x7f70680b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706d66a700 1 -- 172.21.15.32:0/2858068534 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680b0a20 con 0x7f70680a7250 2022-08-01T23:01:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706ce69700 1 --2- 172.21.15.32:0/2858068534 >> v2:172.21.15.55:3300/0 conn(0x7f70680b1f40 0x7f70680b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f70677fe700 1 -- 172.21.15.32:0/2858068534 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038792697 0 0) 0x7f70680b04e0 con 0x7f7068148530 2022-08-01T23:01:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706d66a700 1 --2- 172.21.15.32:0/2858068534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680a7250 0x7f70681448b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f70681478c0 tx=0x7f7060006df0).ready entity=mon.0 client_cookie=d017e02fdea2c3d1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f70677fe700 1 -- 172.21.15.32:0/2858068534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706000d5c0 con 0x7f70680a7250 2022-08-01T23:01:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f70677fe700 1 -- 172.21.15.32:0/2858068534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060000e90 con 0x7f70680a7250 2022-08-01T23:01:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f70677fe700 1 -- 172.21.15.32:0/2858068534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706000d910 con 0x7f70680a7250 2022-08-01T23:01:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- 172.21.15.32:0/2858068534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680a7250 msgr2=0x7f70681448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2858068534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680a7250 0x7f70681448b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f70681478c0 tx=0x7f7060006df0).stop 2022-08-01T23:01:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- 172.21.15.32:0/2858068534 shutdown_connections 2022-08-01T23:01:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2858068534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680a7250 0x7f70681448b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2858068534 >> v2:172.21.15.55:3300/0 conn(0x7f70680b1f40 0x7f70680b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- 172.21.15.32:0/2858068534 >> 172.21.15.32:0/2858068534 conn(0x7f706805f840 msgr2=0x7f706805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- 172.21.15.32:0/2858068534 shutdown_connections 2022-08-01T23:01:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.857+0000 7f706f0cd700 1 -- 172.21.15.32:0/2858068534 wait complete. 2022-08-01T23:01:29.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 Processor -- start 2022-08-01T23:01:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- start start 2022-08-01T23:01:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680a7250 0x7f7068146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 0x7f706814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7068148530 0x7f706814a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70680a9b20 con 0x7f70680b1f40 2022-08-01T23:01:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70680aa5a0 con 0x7f7068148530 2022-08-01T23:01:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706814ae50 con 0x7f70680a7250 2022-08-01T23:01:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 0x7f706814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 0x7f706814a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44572/0 (socket says 172.21.15.32:44572) 2022-08-01T23:01:29.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 -- 172.21.15.32:0/2575517991 learned_addr learned my addr 172.21.15.32:0/2575517991 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:29.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706ce69700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680a7250 0x7f7068146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706d66a700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7068148530 0x7f706814a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 -- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680a7250 msgr2=0x7f7068146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680a7250 0x7f7068146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 -- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7068148530 msgr2=0x7f706814a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7068148530 0x7f706814a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 -- 172.21.15.32:0/2575517991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706000b180 con 0x7f70680b1f40 2022-08-01T23:01:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7067fff700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 0x7f706814a470 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f7058001250 tx=0x7f70580013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7058005ba0 con 0x7f70680b1f40 2022-08-01T23:01:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7058005d00 con 0x7f70680b1f40 2022-08-01T23:01:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70580072c0 con 0x7f70680b1f40 2022-08-01T23:01:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70680aad50 con 0x7f70680b1f40 2022-08-01T23:01:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7068148fd0 con 0x7f70680b1f40 2022-08-01T23:01:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.861+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70680b0cc0 con 0x7f70680b1f40 2022-08-01T23:01:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.865+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7058010070 con 0x7f70680b1f40 2022-08-01T23:01:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.865+0000 7f7065ffb700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7048053950 0x7f7048055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.865+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f705806b750 con 0x7f70680b1f40 2022-08-01T23:01:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.865+0000 7f706ce69700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7048053950 0x7f7048055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.865+0000 7f706ce69700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7048053950 0x7f7048055e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f705c00b000 tx=0x7f705c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:29.869+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70580427f0 con 0x7f70680b1f40 2022-08-01T23:01:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.029+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70680aa710 con 0x7f70680b1f40 2022-08-01T23:01:30.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.029+0000 7f7065ffb700 1 -- 172.21.15.32:0/2575517991 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f7058045e10 con 0x7f70680b1f40 2022-08-01T23:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1958319749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1958319749},{"type":"v1","addr":"172.21.15.32:6835","nonce":1958319749}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/3613649438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3613649438},{"type":"v1","addr":"172.21.15.32:6833","nonce":3613649438}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":6121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/132377455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":132377455},{"type":"v1","addr":"172.21.15.55:6835","nonce":132377455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[]} 2022-08-01T23:01:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7048053950 msgr2=0x7f7048055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7048053950 0x7f7048055e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f705c00b000 tx=0x7f705c008040).stop 2022-08-01T23:01:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 msgr2=0x7f706814a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 0x7f706814a470 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f7058001250 tx=0x7f70580013f0).stop 2022-08-01T23:01:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 shutdown_connections 2022-08-01T23:01:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7048053950 0x7f7048055e10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7068148530 0x7f706814a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70680b1f40 0x7f706814a470 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 --2- 172.21.15.32:0/2575517991 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70680a7250 0x7f7068146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 >> 172.21.15.32:0/2575517991 conn(0x7f706805f840 msgr2=0x7f70680b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 shutdown_connections 2022-08-01T23:01:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.033+0000 7f706f0cd700 1 -- 172.21.15.32:0/2575517991 wait complete. 2022-08-01T23:01:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 30 2022-08-01T23:01:30.044 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:01:30.044 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:01:30.045 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:01:30.045 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:01:30.045 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:01:30.046 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:01:30.046 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:01:30.046 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:01:30.047 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:01:30.047 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:01:30.047 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:01:30.048 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:01:30.056 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:01:30.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:01:30.058 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:01:30.059 DEBUG:teuthology.parallel:result is None 2022-08-01T23:01:30.061 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:01:30.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:01:30.063 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:01:30.064 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:01:30.066 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:01:30.066 DEBUG:teuthology.parallel:result is None 2022-08-01T23:01:30.106 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:01:30.106 DEBUG:teuthology.parallel:result is None 2022-08-01T23:01:30.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:01:30.122 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:01:30.139 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:01:30.177 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:01:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 Processor -- start 2022-08-01T23:01:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 -- start start 2022-08-01T23:01:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0f40b07a0 0x7fd0f40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f4147730 0x7fd0f40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f40b5a10 con 0x7fd0f40b5630 2022-08-01T23:01:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f4145470 con 0x7fd0f40b07a0 2022-08-01T23:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40ae180 con 0x7fd0f4147730 2022-08-01T23:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0f40b5630 0x7fd0f4067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60376/0 (socket says 172.21.15.32:60376) 2022-08-01T23:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f259c700 1 -- 172.21.15.32:0/2732317837 learned_addr learned my addr 172.21.15.32:0/2732317837 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2732317837 >> v2:172.21.15.32:3300/0 conn(0x7fd0f40b07a0 0x7fd0f40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f2d9d700 1 --2- 172.21.15.32:0/2732317837 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f4147730 0x7fd0f40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f159a700 1 -- 172.21.15.32:0/2732317837 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814519135 0 0) 0x7fd0f40b5a10 con 0x7fd0f40b5630 2022-08-01T23:01:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f159a700 1 -- 172.21.15.32:0/2732317837 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d4003260 con 0x7fd0f40b5630 2022-08-01T23:01:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f1d9b700 1 -- 172.21.15.32:0/2732317837 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f4147730 msgr2=0x7fd0f40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2732317837 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f4147730 0x7fd0f40acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f1d9b700 1 -- 172.21.15.32:0/2732317837 >> v1:172.21.15.32:6789/0 conn(0x7fd0f40b5630 legacy=0x7fd0f4067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f1d9b700 1 -- 172.21.15.32:0/2732317837 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f40ac990 con 0x7fd0f40b07a0 2022-08-01T23:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2732317837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f40af6e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fd0dc002f70 tx=0x7fd0dc006df0).ready entity=mon.0 client_cookie=b874139a3798848e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f159a700 1 -- 172.21.15.32:0/2732317837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0dc00d4a0 con 0x7fd0f40b07a0 2022-08-01T23:01:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f159a700 1 -- 172.21.15.32:0/2732317837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc000e90 con 0x7fd0f40b07a0 2022-08-01T23:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f159a700 1 -- 172.21.15.32:0/2732317837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0dc00d830 con 0x7fd0f40b07a0 2022-08-01T23:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 -- 172.21.15.32:0/2732317837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 msgr2=0x7fd0f40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.205+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2732317837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f40af6e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fd0dc002f70 tx=0x7fd0dc006df0).stop 2022-08-01T23:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- 172.21.15.32:0/2732317837 shutdown_connections 2022-08-01T23:01:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2732317837 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f4147730 0x7fd0f40acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2732317837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f40af6e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- 172.21.15.32:0/2732317837 >> 172.21.15.32:0/2732317837 conn(0x7fd0f40626f0 msgr2=0x7fd0f4062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- 172.21.15.32:0/2732317837 shutdown_connections 2022-08-01T23:01:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- 172.21.15.32:0/2732317837 wait complete. 2022-08-01T23:01:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 Processor -- start 2022-08-01T23:01:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- start start 2022-08-01T23:01:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f40b5630 0x7fd0f414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0f4147730 0x7fd0f415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4149c20 con 0x7fd0f40b07a0 2022-08-01T23:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4149ec0 con 0x7fd0f4147730 2022-08-01T23:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40b3080 con 0x7fd0f40b5630 2022-08-01T23:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f414e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44608/0 (socket says 172.21.15.32:44608) 2022-08-01T23:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f259c700 1 -- 172.21.15.32:0/2692864478 learned_addr learned my addr 172.21.15.32:0/2692864478 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f40b5630 0x7fd0f414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.209+0000 7fd0f2d9d700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0f4147730 0x7fd0f415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f259c700 1 -- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f40b5630 msgr2=0x7fd0f414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f259c700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f40b5630 0x7fd0f414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f259c700 1 -- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0f4147730 msgr2=0x7fd0f415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f259c700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0f4147730 0x7fd0f415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f259c700 1 -- 172.21.15.32:0/2692864478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc00b180 con 0x7fd0f40b07a0 2022-08-01T23:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f40b5630 0x7fd0f414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f2d9d700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0f4147730 0x7fd0f415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f259c700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f414e990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fd0e4001490 tx=0x7fd0e4007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e4005ba0 con 0x7fd0f40b07a0 2022-08-01T23:01:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f41422f0 con 0x7fd0f40b07a0 2022-08-01T23:01:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e4005d00 con 0x7fd0f40b07a0 2022-08-01T23:01:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f41428f0 con 0x7fd0f40b07a0 2022-08-01T23:01:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0e4007300 con 0x7fd0f40b07a0 2022-08-01T23:01:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f414a180 con 0x7fd0f40b07a0 2022-08-01T23:01:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd0e4010030 con 0x7fd0f40b07a0 2022-08-01T23:01:30.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0eb7fe700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0d8053a10 0x7fd0d8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:30.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd0e406be10 con 0x7fd0f40b07a0 2022-08-01T23:01:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.213+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0d8053a10 0x7fd0d8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.217+0000 7fd0f1d9b700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0d8053a10 0x7fd0d8055ed0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fd0dc015000 tx=0x7fd0dc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.221+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0e4042e10 con 0x7fd0f40b07a0 2022-08-01T23:01:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:30.361+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fd0f4061020 con 0x7fd0f40b07a0 2022-08-01T23:01:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.185+0000 7fd0eb7fe700 1 -- 172.21.15.32:0/2692864478 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd0e4046430 con 0x7fd0f40b07a0 2022-08-01T23:01:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0d8053a10 msgr2=0x7fd0d8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0d8053a10 0x7fd0d8055ed0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fd0dc015000 tx=0x7fd0dc014000).stop 2022-08-01T23:01:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 msgr2=0x7fd0f414e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f414e990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7fd0e4001490 tx=0x7fd0e4007e50).stop 2022-08-01T23:01:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 shutdown_connections 2022-08-01T23:01:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0d8053a10 0x7fd0d8055ed0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0f4147730 0x7fd0f415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0f40b5630 0x7fd0f414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 --2- 172.21.15.32:0/2692864478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f40b07a0 0x7fd0f414e990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 >> 172.21.15.32:0/2692864478 conn(0x7fd0f40626f0 msgr2=0x7fd0f41447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 shutdown_connections 2022-08-01T23:01:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.189+0000 7fd0f359e700 1 -- 172.21.15.32:0/2692864478 wait complete. 2022-08-01T23:01:31.192 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:01:31.204 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:01:31.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:01:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 Processor -- start 2022-08-01T23:01:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 -- start start 2022-08-01T23:01:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40480b3630 0x7f4048143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40480aa0d0 0x7f40480af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40480b3a10 con 0x7f40480af6f0 2022-08-01T23:01:31.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40481443f0 con 0x7f40480aa0d0 2022-08-01T23:01:31.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.313+0000 7f404d105700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40480afb80 con 0x7f40480b3630 2022-08-01T23:01:31.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40480aa0d0 0x7f40480af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40480aa0d0 0x7f40480af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44620/0 (socket says 172.21.15.32:44620) 2022-08-01T23:01:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 -- 172.21.15.32:0/3263894505 learned_addr learned my addr 172.21.15.32:0/3263894505 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f4046d9d700 1 --2- 172.21.15.32:0/3263894505 >> v2:172.21.15.55:3300/0 conn(0x7f40480b3630 0x7f4048143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 -- 172.21.15.32:0/3263894505 >> v2:172.21.15.55:3300/0 conn(0x7f40480b3630 msgr2=0x7f4048143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 --2- 172.21.15.32:0/3263894505 >> v2:172.21.15.55:3300/0 conn(0x7f40480b3630 0x7f4048143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 -- 172.21.15.32:0/3263894505 >> v1:172.21.15.55:6789/0 conn(0x7f40480af6f0 legacy=0x7f404805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 -- 172.21.15.32:0/3263894505 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048146b40 con 0x7f40480aa0d0 2022-08-01T23:01:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404659c700 1 --2- 172.21.15.32:0/3263894505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480af200 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f403c002f70 tx=0x7f403c006eb0).ready entity=mon.0 client_cookie=df4b63dc1fe828c6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f4045d9b700 1 -- 172.21.15.32:0/3263894505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403c00d5c0 con 0x7f40480aa0d0 2022-08-01T23:01:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f4045d9b700 1 -- 172.21.15.32:0/3263894505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c000e90 con 0x7f40480aa0d0 2022-08-01T23:01:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f4045d9b700 1 -- 172.21.15.32:0/3263894505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403c00d950 con 0x7f40480aa0d0 2022-08-01T23:01:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 -- 172.21.15.32:0/3263894505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 msgr2=0x7f40480af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 --2- 172.21.15.32:0/3263894505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480af200 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f403c002f70 tx=0x7f403c006eb0).stop 2022-08-01T23:01:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 -- 172.21.15.32:0/3263894505 shutdown_connections 2022-08-01T23:01:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 --2- 172.21.15.32:0/3263894505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480af200 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 --2- 172.21.15.32:0/3263894505 >> v2:172.21.15.55:3300/0 conn(0x7f40480b3630 0x7f4048143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 -- 172.21.15.32:0/3263894505 >> 172.21.15.32:0/3263894505 conn(0x7f4048060950 msgr2=0x7f4048060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:31.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 -- 172.21.15.32:0/3263894505 shutdown_connections 2022-08-01T23:01:31.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.317+0000 7f404d105700 1 -- 172.21.15.32:0/3263894505 wait complete. 2022-08-01T23:01:31.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 Processor -- start 2022-08-01T23:01:31.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 -- start start 2022-08-01T23:01:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40480af6f0 0x7f40480b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40480b3630 0x7f4048141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40480b3c40 con 0x7f40480aa0d0 2022-08-01T23:01:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048063600 con 0x7f40480b3630 2022-08-01T23:01:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f404d105700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4048142fb0 con 0x7f40480af6f0 2022-08-01T23:01:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.321+0000 7f4046d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44624/0 (socket says 172.21.15.32:44624) 2022-08-01T23:01:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 -- 172.21.15.32:0/1657613077 learned_addr learned my addr 172.21.15.32:0/1657613077 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f404659c700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40480af6f0 0x7f40480b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f404759e700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40480b3630 0x7f4048141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 -- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40480af6f0 msgr2=0x7f40480b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40480af6f0 0x7f40480b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 -- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40480b3630 msgr2=0x7f4048141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40480b3630 0x7f4048141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 -- 172.21.15.32:0/1657613077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c00b1e0 con 0x7f40480aa0d0 2022-08-01T23:01:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4046d9d700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480b2550 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f40480b11a0 tx=0x7f4038006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403800c830 con 0x7f40480aa0d0 2022-08-01T23:01:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403800c990 con 0x7f40480aa0d0 2022-08-01T23:01:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4048148110 con 0x7f40480aa0d0 2022-08-01T23:01:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403800ccf0 con 0x7f40480aa0d0 2022-08-01T23:01:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40481431e0 con 0x7f40480aa0d0 2022-08-01T23:01:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f403801c460 con 0x7f40480aa0d0 2022-08-01T23:01:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4048146de0 con 0x7f40480aa0d0 2022-08-01T23:01:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4037fff700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4024053b00 0x7f4024055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f403806c2f0 con 0x7f40480aa0d0 2022-08-01T23:01:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.325+0000 7f404659c700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4024053b00 0x7f4024055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.329+0000 7f404659c700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4024053b00 0x7f4024055fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f403c01a000 tx=0x7f403c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.333+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40380432f0 con 0x7f40480aa0d0 2022-08-01T23:01:31.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.497+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f40480574e0 con 0x7f40480aa0d0 2022-08-01T23:01:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f4037fff700 1 -- 172.21.15.32:0/1657613077 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f4038046910 con 0x7f40480aa0d0 2022-08-01T23:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:01:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4024053b00 msgr2=0x7f4024055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4024053b00 0x7f4024055fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f403c01a000 tx=0x7f403c015000).stop 2022-08-01T23:01:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 msgr2=0x7f40480b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480b2550 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f40480b11a0 tx=0x7f4038006df0).stop 2022-08-01T23:01:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 shutdown_connections 2022-08-01T23:01:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4024053b00 0x7f4024055fc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40480b3630 0x7f4048141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40480af6f0 0x7f40480b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 --2- 172.21.15.32:0/1657613077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40480aa0d0 0x7f40480b2550 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 >> 172.21.15.32:0/1657613077 conn(0x7f4048060950 msgr2=0x7f4048140210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 shutdown_connections 2022-08-01T23:01:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.501+0000 7f404d105700 1 -- 172.21.15.32:0/1657613077 wait complete. 2022-08-01T23:01:31.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:01:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.637+0000 7f195f59e700 1 Processor -- start 2022-08-01T23:01:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.637+0000 7f195f59e700 1 -- start start 2022-08-01T23:01:31.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f19600b2710 0x7f196014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f196014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19600afd60 0x7f19600a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19600a7250 con 0x7f19600afd60 2022-08-01T23:01:31.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19600a7480 con 0x7f19600b2710 2022-08-01T23:01:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19600a7720 con 0x7f19600b01a0 2022-08-01T23:01:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19600afd60 0x7f19600a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19600afd60 0x7f19600a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44634/0 (socket says 172.21.15.32:44634) 2022-08-01T23:01:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 -- 172.21.15.32:0/2801851567 learned_addr learned my addr 172.21.15.32:0/2801851567 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195dd9b700 1 --2- 172.21.15.32:0/2801851567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f196014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195e59c700 1 --2- 172.21.15.32:0/2801851567 >> v2:172.21.15.55:3300/0 conn(0x7f19600b2710 0x7f196014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 -- 172.21.15.32:0/2801851567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 msgr2=0x7f196014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 --2- 172.21.15.32:0/2801851567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f196014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 -- 172.21.15.32:0/2801851567 >> v2:172.21.15.55:3300/0 conn(0x7f19600b2710 msgr2=0x7f196014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 --2- 172.21.15.32:0/2801851567 >> v2:172.21.15.55:3300/0 conn(0x7f19600b2710 0x7f196014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 -- 172.21.15.32:0/2801851567 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600a79c0 con 0x7f19600afd60 2022-08-01T23:01:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195e59c700 1 --2- 172.21.15.32:0/2801851567 >> v2:172.21.15.55:3300/0 conn(0x7f19600b2710 0x7f196014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195ed9d700 1 --2- 172.21.15.32:0/2801851567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600afd60 0x7f19600a8540 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f19580032f0 tx=0x7f1958009290).ready entity=mon.0 client_cookie=8ee38482ba957355 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195d59a700 1 -- 172.21.15.32:0/2801851567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195800b870 con 0x7f19600afd60 2022-08-01T23:01:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195d59a700 1 -- 172.21.15.32:0/2801851567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195800b9d0 con 0x7f19600afd60 2022-08-01T23:01:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195d59a700 1 -- 172.21.15.32:0/2801851567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195800bb40 con 0x7f19600afd60 2022-08-01T23:01:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- 172.21.15.32:0/2801851567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600afd60 msgr2=0x7f19600a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- 172.21.15.32:0/2801851567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600afd60 0x7f19600a8540 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f19580032f0 tx=0x7f1958009290).stop 2022-08-01T23:01:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- 172.21.15.32:0/2801851567 shutdown_connections 2022-08-01T23:01:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- 172.21.15.32:0/2801851567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600afd60 0x7f19600a8540 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- 172.21.15.32:0/2801851567 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f196014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 --2- 172.21.15.32:0/2801851567 >> v2:172.21.15.55:3300/0 conn(0x7f19600b2710 0x7f196014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- 172.21.15.32:0/2801851567 >> 172.21.15.32:0/2801851567 conn(0x7f1960060310 msgr2=0x7f19600b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:31.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- 172.21.15.32:0/2801851567 shutdown_connections 2022-08-01T23:01:31.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- 172.21.15.32:0/2801851567 wait complete. 2022-08-01T23:01:31.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 Processor -- start 2022-08-01T23:01:31.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.641+0000 7f195f59e700 1 -- start start 2022-08-01T23:01:31.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19600afd60 0x7f19600b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f19601432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 0x7f19601437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600a8d90 con 0x7f19600b2710 2022-08-01T23:01:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600b10a0 con 0x7f19600afd60 2022-08-01T23:01:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1960142bf0 con 0x7f19600b01a0 2022-08-01T23:01:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 0x7f19601437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 0x7f19601437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44642/0 (socket says 172.21.15.32:44642) 2022-08-01T23:01:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 -- 172.21.15.32:0/1629677591 learned_addr learned my addr 172.21.15.32:0/1629677591 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195e59c700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19600afd60 0x7f19600b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195dd9b700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f19601432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 -- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 msgr2=0x7f19601432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f19601432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 -- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19600afd60 msgr2=0x7f19600b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19600afd60 0x7f19600b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 -- 172.21.15.32:0/1629677591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009530 con 0x7f19600b2710 2022-08-01T23:01:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195dd9b700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f19601432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195ed9d700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 0x7f19601437b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f19600aa100 tx=0x7f1958016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f195800b870 con 0x7f19600b2710 2022-08-01T23:01:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958020620 con 0x7f19600b2710 2022-08-01T23:01:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1958020980 con 0x7f19600b2710 2022-08-01T23:01:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1960142e20 con 0x7f19600b2710 2022-08-01T23:01:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19600aab80 con 0x7f19600b2710 2022-08-01T23:01:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19600b3d70 con 0x7f19600b2710 2022-08-01T23:01:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.645+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1958026020 con 0x7f19600b2710 2022-08-01T23:01:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.649+0000 7f194f7fe700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f193c053b00 0x7f193c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.649+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f195806cba0 con 0x7f19600b2710 2022-08-01T23:01:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.649+0000 7f195e59c700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f193c053b00 0x7f193c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.649+0000 7f195e59c700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f193c053b00 0x7f193c055fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f195400c000 tx=0x7f1954008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.653+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1958043ca0 con 0x7f19600b2710 2022-08-01T23:01:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.777+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1960142fb0 con 0x7f19600b2710 2022-08-01T23:01:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f194f7fe700 1 -- 172.21.15.32:0/1629677591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f1958015070 con 0x7f19600b2710 2022-08-01T23:01:31.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:31.781 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:01:31.172662+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:01:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f193c053b00 msgr2=0x7f193c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f193c053b00 0x7f193c055fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f195400c000 tx=0x7f1954008040).stop 2022-08-01T23:01:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 msgr2=0x7f19601437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 0x7f19601437b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f19600aa100 tx=0x7f1958016000).stop 2022-08-01T23:01:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 shutdown_connections 2022-08-01T23:01:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f193c053b00 0x7f193c055fc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19600b2710 0x7f19601437b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f19600b01a0 0x7f19601432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 --2- 172.21.15.32:0/1629677591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19600afd60 0x7f19600b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 >> 172.21.15.32:0/1629677591 conn(0x7f1960060310 msgr2=0x7f1960148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 shutdown_connections 2022-08-01T23:01:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.781+0000 7f195f59e700 1 -- 172.21.15.32:0/1629677591 wait complete. 2022-08-01T23:01:31.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:01:31.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:01:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 Processor -- start 2022-08-01T23:01:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 -- start start 2022-08-01T23:01:31.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52140b01a0 0x7f52140a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52140a7250 con 0x7f52140b2710 2022-08-01T23:01:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52140a7480 con 0x7f52140b01a0 2022-08-01T23:01:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52198ee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140a7720 con 0x7f52140afd60 2022-08-01T23:01:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f5213fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52140b01a0 0x7f52140a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52188ec700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52140b2710 0x7f521414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60420/0 (socket says 172.21.15.32:60420) 2022-08-01T23:01:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52188ec700 1 -- 172.21.15.32:0/2933893257 learned_addr learned my addr 172.21.15.32:0/2933893257 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52190ed700 1 --2- 172.21.15.32:0/2933893257 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52137fe700 1 -- 172.21.15.32:0/2933893257 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169550329 0 0) 0x7f52140a7250 con 0x7f52140b2710 2022-08-01T23:01:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52137fe700 1 -- 172.21.15.32:0/2933893257 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5200003260 con 0x7f52140b2710 2022-08-01T23:01:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f5213fff700 1 -- 172.21.15.32:0/2933893257 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 msgr2=0x7f52140a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f5213fff700 1 --2- 172.21.15.32:0/2933893257 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f5213fff700 1 -- 172.21.15.32:0/2933893257 >> v1:172.21.15.32:6789/0 conn(0x7f52140b2710 legacy=0x7f521414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f5213fff700 1 -- 172.21.15.32:0/2933893257 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140a79c0 con 0x7f52140b01a0 2022-08-01T23:01:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f5213fff700 1 --2- 172.21.15.32:0/2933893257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140a8540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f5208002f70 tx=0x7f5208006df0).ready entity=mon.0 client_cookie=174cba73ff979b15 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52137fe700 1 -- 172.21.15.32:0/2933893257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f520800d4a0 con 0x7f52140b01a0 2022-08-01T23:01:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52137fe700 1 -- 172.21.15.32:0/2933893257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208000e90 con 0x7f52140b01a0 2022-08-01T23:01:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.909+0000 7f52137fe700 1 -- 172.21.15.32:0/2933893257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f520800d7f0 con 0x7f52140b01a0 2022-08-01T23:01:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- 172.21.15.32:0/2933893257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 msgr2=0x7f52140a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 --2- 172.21.15.32:0/2933893257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140a8540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f5208002f70 tx=0x7f5208006df0).stop 2022-08-01T23:01:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- 172.21.15.32:0/2933893257 shutdown_connections 2022-08-01T23:01:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 --2- 172.21.15.32:0/2933893257 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 --2- 172.21.15.32:0/2933893257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140a8540 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- 172.21.15.32:0/2933893257 >> 172.21.15.32:0/2933893257 conn(0x7f5214060310 msgr2=0x7f52140b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- 172.21.15.32:0/2933893257 shutdown_connections 2022-08-01T23:01:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- 172.21.15.32:0/2933893257 wait complete. 2022-08-01T23:01:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 Processor -- start 2022-08-01T23:01:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- start start 2022-08-01T23:01:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52140b2710 0x7f52140ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521415a410 con 0x7f52140b01a0 2022-08-01T23:01:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521415a6b0 con 0x7f52140b2710 2022-08-01T23:01:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52198ee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140a9450 con 0x7f52140afd60 2022-08-01T23:01:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44654/0 (socket says 172.21.15.32:44654) 2022-08-01T23:01:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 -- 172.21.15.32:0/2593491287 learned_addr learned my addr 172.21.15.32:0/2593491287 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52188ec700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f52190ed700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52140b2710 0x7f52140ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 -- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 msgr2=0x7f52140aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 -- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52140b2710 msgr2=0x7f52140ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:31.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52140b2710 0x7f52140ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 -- 172.21.15.32:0/2593491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520800b180 con 0x7f52140b01a0 2022-08-01T23:01:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.913+0000 7f5213fff700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140abc70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f5208002f70 tx=0x7f5208014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5208016070 con 0x7f52140b01a0 2022-08-01T23:01:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208013a40 con 0x7f52140b01a0 2022-08-01T23:01:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5208013da0 con 0x7f52140b01a0 2022-08-01T23:01:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52140a96f0 con 0x7f52140b01a0 2022-08-01T23:01:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52140b4630 con 0x7f52140b01a0 2022-08-01T23:01:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521415a8b0 con 0x7f52140b01a0 2022-08-01T23:01:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f520800f030 con 0x7f52140b01a0 2022-08-01T23:01:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f5211ffb700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51f4053b00 0x7f51f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f520806c960 con 0x7f52140b01a0 2022-08-01T23:01:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f52188ec700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51f4053b00 0x7f51f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:31.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.917+0000 7f52188ec700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51f4053b00 0x7f51f4055fc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f520400b000 tx=0x7f5204009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:31.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:31.921+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52080439c0 con 0x7f52140b01a0 2022-08-01T23:01:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f52140574e0 con 0x7f52140b01a0 2022-08-01T23:01:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f5211ffb700 1 -- 172.21.15.32:0/2593491287 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v72) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5208023020 con 0x7f52140b01a0 2022-08-01T23:01:32.049 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:01:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51f4053b00 msgr2=0x7f51f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51f4053b00 0x7f51f4055fc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f520400b000 tx=0x7f5204009040).stop 2022-08-01T23:01:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 msgr2=0x7f52140abc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140abc70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f5208002f70 tx=0x7f5208014000).stop 2022-08-01T23:01:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 shutdown_connections 2022-08-01T23:01:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51f4053b00 0x7f51f4055fc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52140b2710 0x7f52140ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140b01a0 0x7f52140abc70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 --2- 172.21.15.32:0/2593491287 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f52140afd60 0x7f52140aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 >> 172.21.15.32:0/2593491287 conn(0x7f5214060310 msgr2=0x7f5214144d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 shutdown_connections 2022-08-01T23:01:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.049+0000 7f52198ee700 1 -- 172.21.15.32:0/2593491287 wait complete. 2022-08-01T23:01:32.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:01:32.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.169+0000 7f7d278fc700 1 Processor -- start 2022-08-01T23:01:32.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.169+0000 7f7d278fc700 1 -- start start 2022-08-01T23:01:32.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d278fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d200aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d278fc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d20061b20 con 0x7f7d2013f920 2022-08-01T23:01:32.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d278fc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d20061dc0 con 0x7f7d20060560 2022-08-01T23:01:32.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d278fc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20060d20 con 0x7f7d200aa050 2022-08-01T23:01:32.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25e99700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d2013f920 0x7f7d2014a1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60438/0 (socket says 172.21.15.32:60438) 2022-08-01T23:01:32.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25e99700 1 -- 172.21.15.32:0/2859991070 learned_addr learned my addr 172.21.15.32:0/2859991070 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:32.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25698700 1 --2- 172.21.15.32:0/2859991070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d200aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165050003 0 0) 0x7f7d20061b20 con 0x7f7d2013f920 2022-08-01T23:01:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0c003260 con 0x7f7d2013f920 2022-08-01T23:01:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632558967 0 0) 0x7f7d20061dc0 con 0x7f7d20060560 2022-08-01T23:01:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d20061b20 con 0x7f7d20060560 2022-08-01T23:01:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25698700 1 -- 172.21.15.32:0/2859991070 >> v1:172.21.15.55:6789/0 conn(0x7f7d20060560 legacy=0x7f7d2013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25698700 1 -- 172.21.15.32:0/2859991070 >> v1:172.21.15.32:6789/0 conn(0x7f7d2013f920 legacy=0x7f7d2014a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618866186 0 0) 0x7f7d0c003260 con 0x7f7d2013f920 2022-08-01T23:01:32.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25698700 1 -- 172.21.15.32:0/2859991070 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20060fc0 con 0x7f7d200aa050 2022-08-01T23:01:32.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d25698700 1 --2- 172.21.15.32:0/2859991070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d200aa430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7d18001490 tx=0x7f7d10006df0).ready entity=mon.2 client_cookie=94cef127e93b91bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d180019e0 con 0x7f7d200aa050 2022-08-01T23:01:32.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10000e90 con 0x7f7d200aa050 2022-08-01T23:01:32.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.173+0000 7f7d1ffff700 1 -- 172.21.15.32:0/2859991070 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d10012460 con 0x7f7d200aa050 2022-08-01T23:01:32.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 -- 172.21.15.32:0/2859991070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 msgr2=0x7f7d200aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 --2- 172.21.15.32:0/2859991070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d200aa430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7d18001490 tx=0x7f7d10006df0).stop 2022-08-01T23:01:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 -- 172.21.15.32:0/2859991070 shutdown_connections 2022-08-01T23:01:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 --2- 172.21.15.32:0/2859991070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d200aa430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 -- 172.21.15.32:0/2859991070 >> 172.21.15.32:0/2859991070 conn(0x7f7d200b2b00 msgr2=0x7f7d200b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 -- 172.21.15.32:0/2859991070 shutdown_connections 2022-08-01T23:01:32.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 -- 172.21.15.32:0/2859991070 wait complete. 2022-08-01T23:01:32.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 Processor -- start 2022-08-01T23:01:32.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.177+0000 7f7d278fc700 1 -- start start 2022-08-01T23:01:32.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d20060560 0x7f7d20143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d2013b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 0x7f7d2013bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20061e70 con 0x7f7d2013f920 2022-08-01T23:01:32.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20065180 con 0x7f7d20060560 2022-08-01T23:01:32.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200afc20 con 0x7f7d200aa050 2022-08-01T23:01:32.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 0x7f7d2013bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 0x7f7d2013bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44670/0 (socket says 172.21.15.32:44670) 2022-08-01T23:01:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 -- 172.21.15.32:0/3030222234 learned_addr learned my addr 172.21.15.32:0/3030222234 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25698700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d20060560 0x7f7d20143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d24e97700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d2013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 -- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 msgr2=0x7f7d2013b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d2013b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 -- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d20060560 msgr2=0x7f7d20143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d20060560 0x7f7d20143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 -- 172.21.15.32:0/3030222234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1000b180 con 0x7f7d2013f920 2022-08-01T23:01:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d25e99700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 0x7f7d2013bc30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f7d18001bc0 tx=0x7f7d18001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d18004ab0 con 0x7f7d2013f920 2022-08-01T23:01:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d18004c10 con 0x7f7d2013f920 2022-08-01T23:01:32.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d200affc0 con 0x7f7d2013f920 2022-08-01T23:01:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d18007170 con 0x7f7d2013f920 2022-08-01T23:01:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d200b0420 con 0x7f7d2013f920 2022-08-01T23:01:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.181+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d20061260 con 0x7f7d2013f920 2022-08-01T23:01:32.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.185+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d1801b070 con 0x7f7d2013f920 2022-08-01T23:01:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.185+0000 7f7d1e7fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053a70 0x7f7d00055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.185+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7d1806b690 con 0x7f7d2013f920 2022-08-01T23:01:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.185+0000 7f7d25698700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053a70 0x7f7d00055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.185+0000 7f7d25698700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053a70 0x7f7d00055f30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7d1001a000 tx=0x7f7d10014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.193+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d18042680 con 0x7f7d2013f920 2022-08-01T23:01:32.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.317+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7d200b0150 con 0x7f7d2013f920 2022-08-01T23:01:32.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d1e7fc700 1 -- 172.21.15.32:0/3030222234 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d18045ca0 con 0x7f7d2013f920 2022-08-01T23:01:32.320 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053a70 msgr2=0x7f7d00055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053a70 0x7f7d00055f30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7d1001a000 tx=0x7f7d10014000).stop 2022-08-01T23:01:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 msgr2=0x7f7d2013bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 0x7f7d2013bc30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f7d18001bc0 tx=0x7f7d18001a40).stop 2022-08-01T23:01:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 shutdown_connections 2022-08-01T23:01:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053a70 0x7f7d00055f30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d2013f920 0x7f7d2013bc30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d200aa050 0x7f7d2013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 --2- 172.21.15.32:0/3030222234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d20060560 0x7f7d20143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 >> 172.21.15.32:0/3030222234 conn(0x7f7d200b2b00 msgr2=0x7f7d200b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:32.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 shutdown_connections 2022-08-01T23:01:32.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.321+0000 7f7d278fc700 1 -- 172.21.15.32:0/3030222234 wait complete. 2022-08-01T23:01:32.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:01:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.429+0000 7fde40d6f700 1 Processor -- start 2022-08-01T23:01:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.429+0000 7fde40d6f700 1 -- start start 2022-08-01T23:01:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c0b2340 0x7fde3c05f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fde3c143e20 0x7fde3c12c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c0b3dd0 con 0x7fde3c060eb0 2022-08-01T23:01:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c063f80 con 0x7fde3c143e20 2022-08-01T23:01:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0b2720 con 0x7fde3c0b2340 2022-08-01T23:01:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde33fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fde3c060eb0 0x7fde3c0b36c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60448/0 (socket says 172.21.15.32:60448) 2022-08-01T23:01:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde33fff700 1 -- 172.21.15.32:0/2391129803 learned_addr learned my addr 172.21.15.32:0/2391129803 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3bfff700 1 --2- 172.21.15.32:0/2391129803 >> v2:172.21.15.55:3300/0 conn(0x7fde3c143e20 0x7fde3c12c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3b7fe700 1 --2- 172.21.15.32:0/2391129803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c0b2340 0x7fde3c05f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3affd700 1 -- 172.21.15.32:0/2391129803 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767743548 0 0) 0x7fde3c0b3dd0 con 0x7fde3c060eb0 2022-08-01T23:01:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3affd700 1 -- 172.21.15.32:0/2391129803 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde1c003260 con 0x7fde3c060eb0 2022-08-01T23:01:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3b7fe700 1 -- 172.21.15.32:0/2391129803 >> v2:172.21.15.55:3300/0 conn(0x7fde3c143e20 msgr2=0x7fde3c12c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3b7fe700 1 --2- 172.21.15.32:0/2391129803 >> v2:172.21.15.55:3300/0 conn(0x7fde3c143e20 0x7fde3c12c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3b7fe700 1 -- 172.21.15.32:0/2391129803 >> v1:172.21.15.32:6789/0 conn(0x7fde3c060eb0 legacy=0x7fde3c0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3b7fe700 1 -- 172.21.15.32:0/2391129803 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0b2fc0 con 0x7fde3c0b2340 2022-08-01T23:01:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3bfff700 1 --2- 172.21.15.32:0/2391129803 >> v2:172.21.15.55:3300/0 conn(0x7fde3c143e20 0x7fde3c12c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3b7fe700 1 --2- 172.21.15.32:0/2391129803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c0b2340 0x7fde3c05f840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fde2c002f70 tx=0x7fde2c006df0).ready entity=mon.2 client_cookie=54e7f9963fc786b2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3affd700 1 -- 172.21.15.32:0/2391129803 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2c00d4a0 con 0x7fde3c0b2340 2022-08-01T23:01:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3affd700 1 -- 172.21.15.32:0/2391129803 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c000e90 con 0x7fde3c0b2340 2022-08-01T23:01:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde3affd700 1 -- 172.21.15.32:0/2391129803 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2c00d7f0 con 0x7fde3c0b2340 2022-08-01T23:01:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- 172.21.15.32:0/2391129803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c0b2340 msgr2=0x7fde3c05f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2391129803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c0b2340 0x7fde3c05f840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fde2c002f70 tx=0x7fde2c006df0).stop 2022-08-01T23:01:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- 172.21.15.32:0/2391129803 shutdown_connections 2022-08-01T23:01:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2391129803 >> v2:172.21.15.55:3300/0 conn(0x7fde3c143e20 0x7fde3c12c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2391129803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c0b2340 0x7fde3c05f840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- 172.21.15.32:0/2391129803 >> 172.21.15.32:0/2391129803 conn(0x7fde3c05c620 msgr2=0x7fde3c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- 172.21.15.32:0/2391129803 shutdown_connections 2022-08-01T23:01:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.433+0000 7fde40d6f700 1 -- 172.21.15.32:0/2391129803 wait complete. 2022-08-01T23:01:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 Processor -- start 2022-08-01T23:01:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 -- start start 2022-08-01T23:01:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c060eb0 0x7fde3c12ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde3c0b2340 0x7fde3c064f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 0x7fde3c0653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0645f0 con 0x7fde3c143e20 2022-08-01T23:01:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0b0190 con 0x7fde3c0b2340 2022-08-01T23:01:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c064cf0 con 0x7fde3c060eb0 2022-08-01T23:01:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 0x7fde3c0653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c060eb0 0x7fde3c12ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 0x7fde3c0653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44684/0 (socket says 172.21.15.32:44684) 2022-08-01T23:01:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde3c0b2340 0x7fde3c064f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 -- 172.21.15.32:0/2397184829 learned_addr learned my addr 172.21.15.32:0/2397184829 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 -- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c060eb0 msgr2=0x7fde3c12ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c060eb0 0x7fde3c12ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 -- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde3c0b2340 msgr2=0x7fde3c064f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde3c0b2340 0x7fde3c064f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 -- 172.21.15.32:0/2397184829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009470 con 0x7fde3c143e20 2022-08-01T23:01:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde3bfff700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 0x7fde3c0653f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fde3400c000 tx=0x7fde340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3400ee60 con 0x7fde3c143e20 2022-08-01T23:01:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3400c850 con 0x7fde3c143e20 2022-08-01T23:01:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2c00b180 con 0x7fde3c143e20 2022-08-01T23:01:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3400cbb0 con 0x7fde3c143e20 2022-08-01T23:01:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.437+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c064850 con 0x7fde3c143e20 2022-08-01T23:01:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.441+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c061d60 con 0x7fde3c143e20 2022-08-01T23:01:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.441+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fde34022070 con 0x7fde3c143e20 2022-08-01T23:01:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.441+0000 7fde397fa700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde20053b30 0x7fde20055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.441+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fde3406c3f0 con 0x7fde3c143e20 2022-08-01T23:01:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.441+0000 7fde3b7fe700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde20053b30 0x7fde20055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.445+0000 7fde3b7fe700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde20053b30 0x7fde20055ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fde2c01a000 tx=0x7fde2c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.449+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde340434f0 con 0x7fde3c143e20 2022-08-01T23:01:32.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.573+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fde3c067530 con 0x7fde3c143e20 2022-08-01T23:01:32.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.573+0000 7fde397fa700 1 -- 172.21.15.32:0/2397184829 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+143 (secure 0 0 0) 0x7fde34016090 con 0x7fde3c143e20 2022-08-01T23:01:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:32.575 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:01:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:01:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:01:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:01:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:01:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:01:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:01:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:01:32.577 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:01:32.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde20053b30 msgr2=0x7fde20055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde20053b30 0x7fde20055ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fde2c01a000 tx=0x7fde2c014000).stop 2022-08-01T23:01:32.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 msgr2=0x7fde3c0653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 0x7fde3c0653f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fde3400c000 tx=0x7fde340091d0).stop 2022-08-01T23:01:32.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 shutdown_connections 2022-08-01T23:01:32.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde20053b30 0x7fde20055ff0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde3c143e20 0x7fde3c0653f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde3c0b2340 0x7fde3c064f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 --2- 172.21.15.32:0/2397184829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde3c060eb0 0x7fde3c12ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 >> 172.21.15.32:0/2397184829 conn(0x7fde3c05c620 msgr2=0x7fde3c142f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:32.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 shutdown_connections 2022-08-01T23:01:32.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.577+0000 7fde40d6f700 1 -- 172.21.15.32:0/2397184829 wait complete. 2022-08-01T23:01:32.588 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:01:32.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:01:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 Processor -- start 2022-08-01T23:01:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 -- start start 2022-08-01T23:01:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 0x7f2fc80af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2fc813fa40 0x7f2fc813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc80af610 con 0x7f2fc80ad670 2022-08-01T23:01:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc80b0c40 con 0x7f2fc813fa40 2022-08-01T23:01:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcd13b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80b0ee0 con 0x7f2fc80b0570 2022-08-01T23:01:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fc7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fc80ad670 0x7f2fc80b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60462/0 (socket says 172.21.15.32:60462) 2022-08-01T23:01:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fc7fff700 1 -- 172.21.15.32:0/1542722399 learned_addr learned my addr 172.21.15.32:0/1542722399 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/1542722399 >> v2:172.21.15.55:3300/0 conn(0x7f2fc813fa40 0x7f2fc813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fc77fe700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 0x7f2fc80af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097615810 0 0) 0x7f2fb0001110 con 0x7f2fc80ad670 2022-08-01T23:01:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.693+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa8003260 con 0x7f2fc80ad670 2022-08-01T23:01:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013006325 0 0) 0x7f2fa8003260 con 0x7f2fc80ad670 2022-08-01T23:01:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 msgr2=0x7f2fc80af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 0x7f2fc80af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 >> v2:172.21.15.55:3300/0 conn(0x7f2fc813fa40 msgr2=0x7f2fc813fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 --2- 172.21.15.32:0/1542722399 >> v2:172.21.15.55:3300/0 conn(0x7f2fc813fa40 0x7f2fc813fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc80ae9e0 con 0x7f2fc80ad670 2022-08-01T23:01:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f2fb00017f0 con 0x7f2fc80ad670 2022-08-01T23:01:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 >> v1:172.21.15.32:6789/0 conn(0x7f2fc80ad670 legacy=0x7f2fc80b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 0x7f2fa8003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fa8003c80 0x7f2fa8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80049f0 0x7f2fa8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80b0c40 con 0x7f2fa80049f0 2022-08-01T23:01:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80b0ee0 con 0x7f2fa8003c80 2022-08-01T23:01:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa800b750 con 0x7f2fc80b0570 2022-08-01T23:01:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80049f0 0x7f2fa8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc7fff700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 0x7f2fa8003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc77fe700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fa8003c80 0x7f2fa8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 -- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 msgr2=0x7f2fa8003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fc80b0570 0x7f2fa8003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 -- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fa8003c80 msgr2=0x7f2fa8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fa8003c80 0x7f2fa8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 -- 172.21.15.32:0/1542722399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8003260 con 0x7f2fa80049f0 2022-08-01T23:01:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80049f0 0x7f2fa8004e70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f2fa80054d0 tx=0x7f2fc00091d0).ready entity=mon.0 client_cookie=2536cb79cbf6a717 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc000e7e0 con 0x7f2fa80049f0 2022-08-01T23:01:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc000e940 con 0x7f2fa80049f0 2022-08-01T23:01:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fc6ffd700 1 -- 172.21.15.32:0/1542722399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc000eca0 con 0x7f2fa80049f0 2022-08-01T23:01:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 -- 172.21.15.32:0/1542722399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80049f0 msgr2=0x7f2fa8004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/1542722399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80049f0 0x7f2fa8004e70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f2fa80054d0 tx=0x7f2fc00091d0).stop 2022-08-01T23:01:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcc93a700 1 -- 172.21.15.32:0/1542722399 reap_dead start 2022-08-01T23:01:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 -- 172.21.15.32:0/1542722399 shutdown_connections 2022-08-01T23:01:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 -- 172.21.15.32:0/1542722399 >> 172.21.15.32:0/1542722399 conn(0x7f2fc8061360 msgr2=0x7f2fc8062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 -- 172.21.15.32:0/1542722399 shutdown_connections 2022-08-01T23:01:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 -- 172.21.15.32:0/1542722399 wait complete. 2022-08-01T23:01:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 Processor -- start 2022-08-01T23:01:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.697+0000 7f2fcd13b700 1 -- start start 2022-08-01T23:01:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fa80049f0 0x7f2fc8134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fc80b6390 0x7f2fc80b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 0x7f2fc8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8067900 con 0x7f2fc8155a20 2022-08-01T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80a8d80 con 0x7f2fc80b6390 2022-08-01T23:01:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80b4b20 con 0x7f2fa80049f0 2022-08-01T23:01:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 0x7f2fc8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fc80b6390 0x7f2fc80b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 0x7f2fc8152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44704/0 (socket says 172.21.15.32:44704) 2022-08-01T23:01:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 -- 172.21.15.32:0/2238174177 learned_addr learned my addr 172.21.15.32:0/2238174177 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fc7fff700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fa80049f0 0x7f2fc8134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 -- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fa80049f0 msgr2=0x7f2fc8134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fa80049f0 0x7f2fc8134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 -- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fc80b6390 msgr2=0x7f2fc80b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fc80b6390 0x7f2fc80b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 -- 172.21.15.32:0/2238174177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8067b30 con 0x7f2fc8155a20 2022-08-01T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcc93a700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 0x7f2fc8152840 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f2fb00012a0 tx=0x7f2fc001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc0009600 con 0x7f2fc8155a20 2022-08-01T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc001db00 con 0x7f2fc8155a20 2022-08-01T23:01:32.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc001de60 con 0x7f2fc8155a20 2022-08-01T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc80b4dc0 con 0x7f2fc8155a20 2022-08-01T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc8155eb0 con 0x7f2fc8155a20 2022-08-01T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.701+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc8067860 con 0x7f2fc8155a20 2022-08-01T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.705+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2fc001b450 con 0x7f2fc8155a20 2022-08-01T23:01:32.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.705+0000 7f2fc57fa700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fac053b30 0x7f2fac055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.705+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2fc006d4f0 con 0x7f2fc8155a20 2022-08-01T23:01:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.705+0000 7f2fc7fff700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fac053b30 0x7f2fac055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.705+0000 7f2fc7fff700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fac053b30 0x7f2fac055ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f2fb000d000 tx=0x7f2fb0007040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.709+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fc0044590 con 0x7f2fc8155a20 2022-08-01T23:01:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:32.833+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f2fc80b54f0 con 0x7f2fc8155a20 2022-08-01T23:01:33.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.541+0000 7f2fc57fa700 1 -- 172.21.15.32:0/2238174177 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v73) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2fc0016020 con 0x7f2fc8155a20 2022-08-01T23:01:33.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fac053b30 msgr2=0x7f2fac055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fac053b30 0x7f2fac055ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f2fb000d000 tx=0x7f2fb0007040).stop 2022-08-01T23:01:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 msgr2=0x7f2fc8152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 0x7f2fc8152840 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f2fb00012a0 tx=0x7f2fc001a000).stop 2022-08-01T23:01:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 shutdown_connections 2022-08-01T23:01:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fac053b30 0x7f2fac055ff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc8155a20 0x7f2fc8152840 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fc80b6390 0x7f2fc80b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 --2- 172.21.15.32:0/2238174177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fa80049f0 0x7f2fc8134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 >> 172.21.15.32:0/2238174177 conn(0x7f2fc8061360 msgr2=0x7f2fc813fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 shutdown_connections 2022-08-01T23:01:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.545+0000 7f2fcd13b700 1 -- 172.21.15.32:0/2238174177 wait complete. 2022-08-01T23:01:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:01:33.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:01:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff1409d2700 1 Processor -- start 2022-08-01T23:01:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff1409d2700 1 -- start start 2022-08-01T23:01:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff1409d2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff13c0b3260 0x7ff13c1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff1409d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c143ab0 con 0x7ff13c0b08a0 2022-08-01T23:01:33.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff1409d2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0b0c80 con 0x7ff13c0ad000 2022-08-01T23:01:33.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff1409d2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c0ae810 con 0x7ff13c0b3260 2022-08-01T23:01:33.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff13c0b08a0 0x7ff13c142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60482/0 (socket says 172.21.15.32:60482) 2022-08-01T23:01:33.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13b7fe700 1 -- 172.21.15.32:0/3210892477 learned_addr learned my addr 172.21.15.32:0/3210892477 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13affd700 1 --2- 172.21.15.32:0/3210892477 >> v2:172.21.15.55:3300/0 conn(0x7ff13c0b3260 0x7ff13c1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573606666 0 0) 0x7ff13c143ab0 con 0x7ff13c0b08a0 2022-08-01T23:01:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c003260 con 0x7ff13c0b08a0 2022-08-01T23:01:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220212183 0 0) 0x7ff13c0b0c80 con 0x7ff13c0ad000 2022-08-01T23:01:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff13c143ab0 con 0x7ff13c0ad000 2022-08-01T23:01:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13affd700 1 -- 172.21.15.32:0/3210892477 >> v1:172.21.15.55:6789/0 conn(0x7ff13c0ad000 legacy=0x7ff13c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13affd700 1 -- 172.21.15.32:0/3210892477 >> v1:172.21.15.32:6789/0 conn(0x7ff13c0b08a0 legacy=0x7ff13c142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13affd700 1 -- 172.21.15.32:0/3210892477 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0aeab0 con 0x7ff13c0b3260 2022-08-01T23:01:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13affd700 1 --2- 172.21.15.32:0/3210892477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b3260 0x7ff13c1435c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7ff124002f70 tx=0x7ff124006df0).ready entity=mon.1 client_cookie=f2b14438dd2aae8e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.685+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12400d540 con 0x7ff13c0b3260 2022-08-01T23:01:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124000e90 con 0x7ff13c0b3260 2022-08-01T23:01:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff13a7fc700 1 -- 172.21.15.32:0/3210892477 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12400d890 con 0x7ff13c0b3260 2022-08-01T23:01:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- 172.21.15.32:0/3210892477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b3260 msgr2=0x7ff13c1435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 --2- 172.21.15.32:0/3210892477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b3260 0x7ff13c1435c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7ff124002f70 tx=0x7ff124006df0).stop 2022-08-01T23:01:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- 172.21.15.32:0/3210892477 shutdown_connections 2022-08-01T23:01:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 --2- 172.21.15.32:0/3210892477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b3260 0x7ff13c1435c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- 172.21.15.32:0/3210892477 >> 172.21.15.32:0/3210892477 conn(0x7ff13c0612d0 msgr2=0x7ff13c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- 172.21.15.32:0/3210892477 shutdown_connections 2022-08-01T23:01:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- 172.21.15.32:0/3210892477 wait complete. 2022-08-01T23:01:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 Processor -- start 2022-08-01T23:01:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- start start 2022-08-01T23:01:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 0x7ff13c13b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b08a0 0x7ff13c140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff13c0b3260 0x7ff13c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c064a90 con 0x7ff13c0ad000 2022-08-01T23:01:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c156890 con 0x7ff13c0b08a0 2022-08-01T23:01:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff1409d2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c15a8e0 con 0x7ff13c0b3260 2022-08-01T23:01:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff13b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 0x7ff13c13b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:33.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.689+0000 7ff13b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 0x7ff13c13b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44714/0 (socket says 172.21.15.32:44714) 2022-08-01T23:01:33.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 -- 172.21.15.32:0/656130937 learned_addr learned my addr 172.21.15.32:0/656130937 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:33.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13bfff700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff13c0b3260 0x7ff13c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:33.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13affd700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b08a0 0x7ff13c140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 -- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff13c0b3260 msgr2=0x7ff13c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff13c0b3260 0x7ff13c156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 -- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b08a0 msgr2=0x7ff13c140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b08a0 0x7ff13c140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 -- 172.21.15.32:0/656130937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12400b180 con 0x7ff13c0ad000 2022-08-01T23:01:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff13b7fe700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 0x7ff13c13b170 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7ff12c0011d0 tx=0x7ff12c007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12c0070d0 con 0x7ff13c0ad000 2022-08-01T23:01:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c007230 con 0x7ff13c0ad000 2022-08-01T23:01:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12c005610 con 0x7ff13c0ad000 2022-08-01T23:01:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c0aa8b0 con 0x7ff13c0ad000 2022-08-01T23:01:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c0ab730 con 0x7ff13c0ad000 2022-08-01T23:01:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c0574e0 con 0x7ff13c0ad000 2022-08-01T23:01:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.693+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff12c010070 con 0x7ff13c0ad000 2022-08-01T23:01:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.697+0000 7ff138ff9700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff120053950 0x7ff120055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.697+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5954+0+0 (secure 0 0 0) 0x7ff12c06b620 con 0x7ff13c0ad000 2022-08-01T23:01:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.697+0000 7ff13affd700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff120053950 0x7ff120055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.697+0000 7ff13affd700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff120053950 0x7ff120055e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff124018000 tx=0x7ff124014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.705+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff12c0424b0 con 0x7ff13c0ad000 2022-08-01T23:01:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:33.829+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff13c0b2e00 con 0x7ff13c0ad000 2022-08-01T23:01:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.141+0000 7ff138ff9700 1 -- 172.21.15.32:0/656130937 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v74) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff12c045ad0 con 0x7ff13c0ad000 2022-08-01T23:01:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff120053950 msgr2=0x7ff120055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff120053950 0x7ff120055e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff124018000 tx=0x7ff124014000).stop 2022-08-01T23:01:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 msgr2=0x7ff13c13b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 0x7ff13c13b170 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7ff12c0011d0 tx=0x7ff12c007b50).stop 2022-08-01T23:01:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 shutdown_connections 2022-08-01T23:01:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff120053950 0x7ff120055e10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff13c0b3260 0x7ff13c156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff13c0b08a0 0x7ff13c140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 --2- 172.21.15.32:0/656130937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13c0ad000 0x7ff13c13b170 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.149+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 >> 172.21.15.32:0/656130937 conn(0x7ff13c0612d0 msgr2=0x7ff13c142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.157+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 shutdown_connections 2022-08-01T23:01:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.157+0000 7ff1409d2700 1 -- 172.21.15.32:0/656130937 wait complete. 2022-08-01T23:01:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:01:34.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:01:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.273+0000 7fee4bfff700 1 Processor -- start 2022-08-01T23:01:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.273+0000 7fee4bfff700 1 -- start start 2022-08-01T23:01:34.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0a9320 0x7fee4c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee4c0b0d10 0x7fee4c146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee4c0b0560 0x7fee4c140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0b12d0 con 0x7fee4c0b0560 2022-08-01T23:01:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0a9700 con 0x7fee4c0b0d10 2022-08-01T23:01:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0b1500 con 0x7fee4c0a9320 2022-08-01T23:01:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee4c0b0560 0x7fee4c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee4c0b0560 0x7fee4c140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44722/0 (socket says 172.21.15.32:44722) 2022-08-01T23:01:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 -- 172.21.15.32:0/807542952 learned_addr learned my addr 172.21.15.32:0/807542952 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4a7fc700 1 --2- 172.21.15.32:0/807542952 >> v2:172.21.15.55:3300/0 conn(0x7fee4c0b0d10 0x7fee4c146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4affd700 1 --2- 172.21.15.32:0/807542952 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0a9320 0x7fee4c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 -- 172.21.15.32:0/807542952 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0a9320 msgr2=0x7fee4c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 --2- 172.21.15.32:0/807542952 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0a9320 0x7fee4c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 -- 172.21.15.32:0/807542952 >> v2:172.21.15.55:3300/0 conn(0x7fee4c0b0d10 msgr2=0x7fee4c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 --2- 172.21.15.32:0/807542952 >> v2:172.21.15.55:3300/0 conn(0x7fee4c0b0d10 0x7fee4c146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 -- 172.21.15.32:0/807542952 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0b17a0 con 0x7fee4c0b0560 2022-08-01T23:01:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4b7fe700 1 --2- 172.21.15.32:0/807542952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c140350 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fee440032f0 tx=0x7fee44009290).ready entity=mon.0 client_cookie=28743de4ad467e58 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee49ffb700 1 -- 172.21.15.32:0/807542952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4400b870 con 0x7fee4c0b0560 2022-08-01T23:01:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee49ffb700 1 -- 172.21.15.32:0/807542952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4400b9d0 con 0x7fee4c0b0560 2022-08-01T23:01:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee49ffb700 1 -- 172.21.15.32:0/807542952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4400bb40 con 0x7fee4c0b0560 2022-08-01T23:01:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- 172.21.15.32:0/807542952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 msgr2=0x7fee4c140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- 172.21.15.32:0/807542952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c140350 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fee440032f0 tx=0x7fee44009290).stop 2022-08-01T23:01:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- 172.21.15.32:0/807542952 shutdown_connections 2022-08-01T23:01:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- 172.21.15.32:0/807542952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c140350 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- 172.21.15.32:0/807542952 >> v2:172.21.15.55:3300/0 conn(0x7fee4c0b0d10 0x7fee4c146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 --2- 172.21.15.32:0/807542952 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0a9320 0x7fee4c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- 172.21.15.32:0/807542952 >> 172.21.15.32:0/807542952 conn(0x7fee4c0612d0 msgr2=0x7fee4c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- 172.21.15.32:0/807542952 shutdown_connections 2022-08-01T23:01:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.277+0000 7fee4bfff700 1 -- 172.21.15.32:0/807542952 wait complete. 2022-08-01T23:01:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 Processor -- start 2022-08-01T23:01:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 -- start start 2022-08-01T23:01:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4c0a9320 0x7fee4c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0b0d10 0x7fee4c13adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0b6010 con 0x7fee4c0b0560 2022-08-01T23:01:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0b62b0 con 0x7fee4c0a9320 2022-08-01T23:01:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c156600 con 0x7fee4c0b0d10 2022-08-01T23:01:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44730/0 (socket says 172.21.15.32:44730) 2022-08-01T23:01:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 -- 172.21.15.32:0/355349036 learned_addr learned my addr 172.21.15.32:0/355349036 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4affd700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4c0a9320 0x7fee4c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4b7fe700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0b0d10 0x7fee4c13adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 -- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0b0d10 msgr2=0x7fee4c13adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0b0d10 0x7fee4c13adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 -- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4c0a9320 msgr2=0x7fee4c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4c0a9320 0x7fee4c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 -- 172.21.15.32:0/355349036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44009530 con 0x7fee4c0b0560 2022-08-01T23:01:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.281+0000 7fee4a7fc700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c141aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fee400030b0 tx=0x7fee40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee40011780 con 0x7fee4c0b0560 2022-08-01T23:01:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee400118e0 con 0x7fee4c0b0560 2022-08-01T23:01:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee40011c40 con 0x7fee4c0b0560 2022-08-01T23:01:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c156aa0 con 0x7fee4c0b0560 2022-08-01T23:01:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c0b37c0 con 0x7fee4c0b0560 2022-08-01T23:01:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c156800 con 0x7fee4c0b0560 2022-08-01T23:01:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee4001c460 con 0x7fee4c0b0560 2022-08-01T23:01:34.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee3bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee28053980 0x7fee28055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fee4006c1c0 con 0x7fee4c0b0560 2022-08-01T23:01:34.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.285+0000 7fee4affd700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee28053980 0x7fee28055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.289+0000 7fee4affd700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee28053980 0x7fee28055e40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fee3c00c000 tx=0x7fee3c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.293+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee4c156800 con 0x7fee4c0b0560 2022-08-01T23:01:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.453+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fee4c156440 con 0x7fee4c0b0560 2022-08-01T23:01:34.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.465+0000 7fee3bfff700 1 -- 172.21.15.32:0/355349036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v32) v1 ==== 189+0+0 (secure 0 0 0) 0x7fee40042e90 con 0x7fee4c0b0560 2022-08-01T23:01:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee28053980 msgr2=0x7fee28055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee28053980 0x7fee28055e40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fee3c00c000 tx=0x7fee3c008040).stop 2022-08-01T23:01:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 msgr2=0x7fee4c141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c141aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fee400030b0 tx=0x7fee40008040).stop 2022-08-01T23:01:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 shutdown_connections 2022-08-01T23:01:34.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee28053980 0x7fee28055e40 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0x7fee3c00c000 tx=0x7fee3c008040).stop 2022-08-01T23:01:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee4c0b0d10 0x7fee4c13adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4c0b0560 0x7fee4c141aa0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 --2- 172.21.15.32:0/355349036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4c0a9320 0x7fee4c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 >> 172.21.15.32:0/355349036 conn(0x7fee4c0612d0 msgr2=0x7fee4c1469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 shutdown_connections 2022-08-01T23:01:34.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.469+0000 7fee4bfff700 1 -- 172.21.15.32:0/355349036 wait complete. 2022-08-01T23:01:34.471 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 13 and data pool 14 2022-08-01T23:01:34.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:01:34.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 Processor -- start 2022-08-01T23:01:34.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- start start 2022-08-01T23:01:34.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88500614b0 0x7f8850061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88500aee80 con 0x7f88501405d0 2022-08-01T23:01:34.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88500b70b0 con 0x7f88500ad750 2022-08-01T23:01:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88500af0b0 con 0x7f88500614b0 2022-08-01T23:01:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8855cde700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88500614b0 0x7f8850061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f88564df700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88501405d0 0x7f88501466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60526/0 (socket says 172.21.15.32:60526) 2022-08-01T23:01:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f88564df700 1 -- 172.21.15.32:0/954650457 learned_addr learned my addr 172.21.15.32:0/954650457 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8855cde700 1 -- 172.21.15.32:0/954650457 >> v1:172.21.15.55:6789/0 conn(0x7f88500ad750 legacy=0x7f88500b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8855cde700 1 -- 172.21.15.32:0/954650457 >> v1:172.21.15.32:6789/0 conn(0x7f88501405d0 legacy=0x7f88501466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8854cdc700 1 -- 172.21.15.32:0/954650457 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262585752 0 0) 0x7f88500aee80 con 0x7f88501405d0 2022-08-01T23:01:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8855cde700 1 -- 172.21.15.32:0/954650457 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500af350 con 0x7f88500614b0 2022-08-01T23:01:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8855cde700 1 --2- 172.21.15.32:0/954650457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500614b0 0x7f8850061890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f88500aed30 tx=0x7f884c006df0).ready entity=mon.0 client_cookie=ffa8dfa75147354c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8854cdc700 1 -- 172.21.15.32:0/954650457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884c00d610 con 0x7f88500614b0 2022-08-01T23:01:34.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8854cdc700 1 -- 172.21.15.32:0/954650457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c000e90 con 0x7f88500614b0 2022-08-01T23:01:34.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8854cdc700 1 -- 172.21.15.32:0/954650457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884c00d960 con 0x7f88500614b0 2022-08-01T23:01:34.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- 172.21.15.32:0/954650457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500614b0 msgr2=0x7f8850061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 --2- 172.21.15.32:0/954650457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500614b0 0x7f8850061890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f88500aed30 tx=0x7f884c006df0).stop 2022-08-01T23:01:34.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- 172.21.15.32:0/954650457 shutdown_connections 2022-08-01T23:01:34.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 --2- 172.21.15.32:0/954650457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500614b0 0x7f8850061890 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- 172.21.15.32:0/954650457 >> 172.21.15.32:0/954650457 conn(0x7f8850066f70 msgr2=0x7f8850067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.577+0000 7f8856ce0700 1 -- 172.21.15.32:0/954650457 shutdown_connections 2022-08-01T23:01:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.581+0000 7f8856ce0700 1 -- 172.21.15.32:0/954650457 wait complete. 2022-08-01T23:01:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.581+0000 7f8856ce0700 1 Processor -- start 2022-08-01T23:01:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.581+0000 7f8856ce0700 1 -- start start 2022-08-01T23:01:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88500614b0 0x7f88500aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 0x7f88500b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88501405d0 0x7f8850156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88500aa520 con 0x7f88500ad750 2022-08-01T23:01:34.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88500aa7c0 con 0x7f88501405d0 2022-08-01T23:01:34.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8850155cf0 con 0x7f88500614b0 2022-08-01T23:01:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 0x7f88500b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 0x7f88500b8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44756/0 (socket says 172.21.15.32:44756) 2022-08-01T23:01:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 -- 172.21.15.32:0/3460645805 learned_addr learned my addr 172.21.15.32:0/3460645805 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8855cde700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88500614b0 0x7f88500aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88564df700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88501405d0 0x7f8850156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 -- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88500614b0 msgr2=0x7f88500aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88500614b0 0x7f88500aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 -- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88501405d0 msgr2=0x7f8850156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88501405d0 0x7f8850156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 -- 172.21.15.32:0/3460645805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c00b180 con 0x7f88500ad750 2022-08-01T23:01:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8855cde700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88500614b0 0x7f88500aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88564df700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88501405d0 0x7f8850156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f88554dd700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 0x7f88500b8850 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f88500af970 tx=0x7f8840007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884000ce30 con 0x7f88500ad750 2022-08-01T23:01:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884000f470 con 0x7f88500ad750 2022-08-01T23:01:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885009f4a0 con 0x7f88500ad750 2022-08-01T23:01:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884000f7d0 con 0x7f88500ad750 2022-08-01T23:01:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88500aa1a0 con 0x7f88500ad750 2022-08-01T23:01:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8850155eb0 con 0x7f88500ad750 2022-08-01T23:01:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.585+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8840019030 con 0x7f88500ad750 2022-08-01T23:01:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.589+0000 7f883effd700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8830053aa0 0x7f8830055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.589+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f884006b540 con 0x7f88500ad750 2022-08-01T23:01:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.589+0000 7f8855cde700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8830053aa0 0x7f8830055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.589+0000 7f8855cde700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8830053aa0 0x7f8830055f60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f884c01a000 tx=0x7f884c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.593+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8840042190 con 0x7f88500ad750 2022-08-01T23:01:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.721+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88500af5f0 con 0x7f88500ad750 2022-08-01T23:01:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.721+0000 7f883effd700 1 -- 172.21.15.32:0/3460645805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f88400457b0 con 0x7f88500ad750 2022-08-01T23:01:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:34.722 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":76,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:01:34.459122+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:01:32.889134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T23:01:33.885837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:01:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8830053aa0 msgr2=0x7f8830055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8830053aa0 0x7f8830055f60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f884c01a000 tx=0x7f884c015000).stop 2022-08-01T23:01:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 msgr2=0x7f88500b8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 0x7f88500b8850 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f88500af970 tx=0x7f8840007bf0).stop 2022-08-01T23:01:34.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 shutdown_connections 2022-08-01T23:01:34.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8830053aa0 0x7f8830055f60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88501405d0 0x7f8850156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88500ad750 0x7f88500b8850 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 --2- 172.21.15.32:0/3460645805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88500614b0 0x7f88500aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 >> 172.21.15.32:0/3460645805 conn(0x7f8850066f70 msgr2=0x7f8850145500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 shutdown_connections 2022-08-01T23:01:34.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.725+0000 7f8856ce0700 1 -- 172.21.15.32:0/3460645805 wait complete. 2022-08-01T23:01:34.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:01:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 Processor -- start 2022-08-01T23:01:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 -- start start 2022-08-01T23:01:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d3c0a8c50 0x7f1d3c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c0a7250 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c0a7480 con 0x7f1d3c0a8c50 2022-08-01T23:01:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0a7720 con 0x7f1d3c0b2710 2022-08-01T23:01:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d3c0a8c50 0x7f1d3c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d3c0a8c50 0x7f1d3c14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44766/0 (socket says 172.21.15.32:44766) 2022-08-01T23:01:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 -- 172.21.15.32:0/2502527343 learned_addr learned my addr 172.21.15.32:0/2502527343 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3a59c700 1 --2- 172.21.15.32:0/2502527343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 -- 172.21.15.32:0/2502527343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 msgr2=0x7f1d3c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 --2- 172.21.15.32:0/2502527343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 -- 172.21.15.32:0/2502527343 >> v1:172.21.15.32:6789/0 conn(0x7f1d3c0b01a0 legacy=0x7f1d3c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 -- 172.21.15.32:0/2502527343 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c0a79c0 con 0x7f1d3c0a8c50 2022-08-01T23:01:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.837+0000 7f1d3ad9d700 1 --2- 172.21.15.32:0/2502527343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c14b4c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f1d34002f70 tx=0x7f1d34006eb0).ready entity=mon.0 client_cookie=a1d9eea54e201de7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3959a700 1 -- 172.21.15.32:0/2502527343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3400d5c0 con 0x7f1d3c0a8c50 2022-08-01T23:01:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3959a700 1 -- 172.21.15.32:0/2502527343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34000e90 con 0x7f1d3c0a8c50 2022-08-01T23:01:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3959a700 1 -- 172.21.15.32:0/2502527343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3400d950 con 0x7f1d3c0a8c50 2022-08-01T23:01:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 -- 172.21.15.32:0/2502527343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0a8c50 msgr2=0x7f1d3c14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/2502527343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c14b4c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f1d34002f70 tx=0x7f1d34006eb0).stop 2022-08-01T23:01:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 -- 172.21.15.32:0/2502527343 shutdown_connections 2022-08-01T23:01:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/2502527343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c14b4c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/2502527343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 -- 172.21.15.32:0/2502527343 >> 172.21.15.32:0/2502527343 conn(0x7f1d3c060310 msgr2=0x7f1d3c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 -- 172.21.15.32:0/2502527343 shutdown_connections 2022-08-01T23:01:34.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.841+0000 7f1d3b59e700 1 -- 172.21.15.32:0/2502527343 wait complete. 2022-08-01T23:01:34.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 Processor -- start 2022-08-01T23:01:34.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 -- start start 2022-08-01T23:01:34.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 0x7f1d3c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0ac7c0 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b3270 con 0x7f1d3c0a8c50 2022-08-01T23:01:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c0b3510 con 0x7f1d3c0b2710 2022-08-01T23:01:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 0x7f1d3c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 0x7f1d3c0b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44770/0 (socket says 172.21.15.32:44770) 2022-08-01T23:01:34.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 -- 172.21.15.32:0/43289352 learned_addr learned my addr 172.21.15.32:0/43289352 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:34.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3a59c700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3ad9d700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 -- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 msgr2=0x7f1d3c0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c0b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 -- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c0a8c50 msgr2=0x7f1d3c145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 -- 172.21.15.32:0/43289352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3400b1e0 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3ad9d700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c0b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d39d9b700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 0x7f1d3c0b4b00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1d3c061c10 tx=0x7f1d2c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2c006aa0 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c006c00 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c0b3b70 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2c007030 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.845+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c0b2f60 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.849+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c0574e0 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.849+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d2c010030 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.849+0000 7f1d2b7fe700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d18053b00 0x7f1d18055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.849+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d2c06c110 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.849+0000 7f1d3a59c700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d18053b00 0x7f1d18055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.849+0000 7f1d3a59c700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d18053b00 0x7f1d18055fc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f1d3000c000 tx=0x7f1d30008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.853+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d2c042d40 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.981+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d3c069a70 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.981+0000 7f1d2b7fe700 1 -- 172.21.15.32:0/43289352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f1d2c046360 con 0x7f1d3c0b01a0 2022-08-01T23:01:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:34.982 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":76,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:01:34.459122+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:01:32.889134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T23:01:33.885837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:01:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d18053b00 msgr2=0x7f1d18055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d18053b00 0x7f1d18055fc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f1d3000c000 tx=0x7f1d30008040).stop 2022-08-01T23:01:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 msgr2=0x7f1d3c0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 0x7f1d3c0b4b00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1d3c061c10 tx=0x7f1d2c007f90).stop 2022-08-01T23:01:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 shutdown_connections 2022-08-01T23:01:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d18053b00 0x7f1d18055fc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3c0b2710 0x7f1d3c0b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0b01a0 0x7f1d3c0b4b00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 --2- 172.21.15.32:0/43289352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d3c0a8c50 0x7f1d3c145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 >> 172.21.15.32:0/43289352 conn(0x7f1d3c060310 msgr2=0x7f1d3c14bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 shutdown_connections 2022-08-01T23:01:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:34.985+0000 7f1d3b59e700 1 -- 172.21.15.32:0/43289352 wait complete. 2022-08-01T23:01:35.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:01:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 Processor -- start 2022-08-01T23:01:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 -- start start 2022-08-01T23:01:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f69480ad750 0x7f6948061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69481405d0 0x7f6948140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480aee80 con 0x7f69480614b0 2022-08-01T23:01:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69480b70b0 con 0x7f69481405d0 2022-08-01T23:01:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.113+0000 7f694759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69480af0b0 con 0x7f69480ad750 2022-08-01T23:01:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f69480614b0 0x7f69480b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60550/0 (socket says 172.21.15.32:60550) 2022-08-01T23:01:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694659c700 1 -- 172.21.15.32:0/2492437416 learned_addr learned my addr 172.21.15.32:0/2492437416 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6946d9d700 1 --2- 172.21.15.32:0/2492437416 >> v2:172.21.15.32:3300/0 conn(0x7f69481405d0 0x7f6948140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6945d9b700 1 --2- 172.21.15.32:0/2492437416 >> v2:172.21.15.55:3300/0 conn(0x7f69480ad750 0x7f6948061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6946d9d700 1 -- 172.21.15.32:0/2492437416 >> v2:172.21.15.55:3300/0 conn(0x7f69480ad750 msgr2=0x7f6948061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6946d9d700 1 --2- 172.21.15.32:0/2492437416 >> v2:172.21.15.55:3300/0 conn(0x7f69480ad750 0x7f6948061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6946d9d700 1 -- 172.21.15.32:0/2492437416 >> v1:172.21.15.32:6789/0 conn(0x7f69480614b0 legacy=0x7f69480b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6946d9d700 1 -- 172.21.15.32:0/2492437416 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480af350 con 0x7f69481405d0 2022-08-01T23:01:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f6946d9d700 1 --2- 172.21.15.32:0/2492437416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f6948140a80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f69400032f0 tx=0x7f69400091d0).ready entity=mon.0 client_cookie=80dd957ea39255a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694559a700 1 -- 172.21.15.32:0/2492437416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f694000b680 con 0x7f69481405d0 2022-08-01T23:01:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694559a700 1 -- 172.21.15.32:0/2492437416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694000b7e0 con 0x7f69481405d0 2022-08-01T23:01:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694559a700 1 -- 172.21.15.32:0/2492437416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f694000b950 con 0x7f69481405d0 2022-08-01T23:01:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 -- 172.21.15.32:0/2492437416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 msgr2=0x7f6948140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 --2- 172.21.15.32:0/2492437416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f6948140a80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f69400032f0 tx=0x7f69400091d0).stop 2022-08-01T23:01:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 -- 172.21.15.32:0/2492437416 shutdown_connections 2022-08-01T23:01:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 --2- 172.21.15.32:0/2492437416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f6948140a80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 --2- 172.21.15.32:0/2492437416 >> v2:172.21.15.55:3300/0 conn(0x7f69480ad750 0x7f6948061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 -- 172.21.15.32:0/2492437416 >> 172.21.15.32:0/2492437416 conn(0x7f6948066f70 msgr2=0x7f6948067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 -- 172.21.15.32:0/2492437416 shutdown_connections 2022-08-01T23:01:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.117+0000 7f694759e700 1 -- 172.21.15.32:0/2492437416 wait complete. 2022-08-01T23:01:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 Processor -- start 2022-08-01T23:01:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 -- start start 2022-08-01T23:01:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69480614b0 0x7f6948152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69480ad750 0x7f6948153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f69480b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480ac8e0 con 0x7f69481405d0 2022-08-01T23:01:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480acad0 con 0x7f69480614b0 2022-08-01T23:01:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6948152a30 con 0x7f69480ad750 2022-08-01T23:01:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f69480b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f69480b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44786/0 (socket says 172.21.15.32:44786) 2022-08-01T23:01:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69480614b0 0x7f6948152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 -- 172.21.15.32:0/3651521506 learned_addr learned my addr 172.21.15.32:0/3651521506 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6945d9b700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69480ad750 0x7f6948153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 -- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69480ad750 msgr2=0x7f6948153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69480ad750 0x7f6948153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 -- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69480614b0 msgr2=0x7f6948152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69480614b0 0x7f6948152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 -- 172.21.15.32:0/3651521506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940009470 con 0x7f69481405d0 2022-08-01T23:01:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f6946d9d700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f69480b0af0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f69400032f0 tx=0x7f6940016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6940011e20 con 0x7f69481405d0 2022-08-01T23:01:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940021440 con 0x7f69481405d0 2022-08-01T23:01:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6948153680 con 0x7f69481405d0 2022-08-01T23:01:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69400217a0 con 0x7f69481405d0 2022-08-01T23:01:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.121+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69481527a0 con 0x7f69481405d0 2022-08-01T23:01:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.125+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69480af5f0 con 0x7f69481405d0 2022-08-01T23:01:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.125+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6940027020 con 0x7f69481405d0 2022-08-01T23:01:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.125+0000 7f69377fe700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6924053950 0x7f6924055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.125+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f694006d400 con 0x7f69481405d0 2022-08-01T23:01:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.125+0000 7f694659c700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6924053950 0x7f6924055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.125+0000 7f694659c700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6924053950 0x7f6924055e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f693800d000 tx=0x7f6938009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.129+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6940044200 con 0x7f69481405d0 2022-08-01T23:01:35.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.301+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f69480574e0 con 0x7f69481405d0 2022-08-01T23:01:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.465+0000 7f69377fe700 1 -- 172.21.15.32:0/3651521506 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f69400150b0 con 0x7f69481405d0 2022-08-01T23:01:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6924053950 msgr2=0x7f6924055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6924053950 0x7f6924055e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f693800d000 tx=0x7f6938009040).stop 2022-08-01T23:01:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 msgr2=0x7f69480b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f69480b0af0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f69400032f0 tx=0x7f6940016000).stop 2022-08-01T23:01:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 shutdown_connections 2022-08-01T23:01:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6924053950 0x7f6924055e10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69481405d0 0x7f69480b0af0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f69480ad750 0x7f6948153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 --2- 172.21.15.32:0/3651521506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69480614b0 0x7f6948152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 >> 172.21.15.32:0/3651521506 conn(0x7f6948066f70 msgr2=0x7f6948067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 shutdown_connections 2022-08-01T23:01:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.469+0000 7f694759e700 1 -- 172.21.15.32:0/3651521506 wait complete. 2022-08-01T23:01:35.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.593+0000 7fa114956700 1 Processor -- start 2022-08-01T23:01:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.593+0000 7fa114956700 1 -- start start 2022-08-01T23:01:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1100b2710 0x7fa11014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa1100a8c50 0x7fa11014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1100a7250 con 0x7fa1100b01a0 2022-08-01T23:01:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100a7480 con 0x7fa1100b2710 2022-08-01T23:01:35.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100a7720 con 0x7fa1100a8c50 2022-08-01T23:01:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1100b01a0 0x7fa1100a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60564/0 (socket says 172.21.15.32:60564) 2022-08-01T23:01:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10e59c700 1 -- 172.21.15.32:0/1521842991 learned_addr learned my addr 172.21.15.32:0/1521842991 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10ed9d700 1 --2- 172.21.15.32:0/1521842991 >> v2:172.21.15.32:3300/0 conn(0x7fa1100b2710 0x7fa11014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10f59e700 1 --2- 172.21.15.32:0/1521842991 >> v2:172.21.15.55:3300/0 conn(0x7fa1100a8c50 0x7fa11014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10dd9b700 1 -- 172.21.15.32:0/1521842991 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140018224 0 0) 0x7fa1100a7250 con 0x7fa1100b01a0 2022-08-01T23:01:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10dd9b700 1 -- 172.21.15.32:0/1521842991 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f4003260 con 0x7fa1100b01a0 2022-08-01T23:01:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10ed9d700 1 -- 172.21.15.32:0/1521842991 >> v2:172.21.15.55:3300/0 conn(0x7fa1100a8c50 msgr2=0x7fa11014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10ed9d700 1 --2- 172.21.15.32:0/1521842991 >> v2:172.21.15.55:3300/0 conn(0x7fa1100a8c50 0x7fa11014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10ed9d700 1 -- 172.21.15.32:0/1521842991 >> v1:172.21.15.32:6789/0 conn(0x7fa1100b01a0 legacy=0x7fa1100a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10dd9b700 1 -- 172.21.15.32:0/1521842991 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 411965558 0 0) 0x7fa0f4003260 con 0x7fa1100b01a0 2022-08-01T23:01:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10ed9d700 1 -- 172.21.15.32:0/1521842991 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100a79c0 con 0x7fa1100b2710 2022-08-01T23:01:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10ed9d700 1 --2- 172.21.15.32:0/1521842991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100b2710 0x7fa11014afd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fa100001800 tx=0x7fa104006df0).ready entity=mon.0 client_cookie=4a172d51c6e520a7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10dd9b700 1 -- 172.21.15.32:0/1521842991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa100001660 con 0x7fa1100b2710 2022-08-01T23:01:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10dd9b700 1 -- 172.21.15.32:0/1521842991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa104000e90 con 0x7fa1100b2710 2022-08-01T23:01:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa10dd9b700 1 -- 172.21.15.32:0/1521842991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa104012460 con 0x7fa1100b2710 2022-08-01T23:01:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- 172.21.15.32:0/1521842991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100b2710 msgr2=0x7fa11014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 --2- 172.21.15.32:0/1521842991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100b2710 0x7fa11014afd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fa100001800 tx=0x7fa104006df0).stop 2022-08-01T23:01:35.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- 172.21.15.32:0/1521842991 shutdown_connections 2022-08-01T23:01:35.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 --2- 172.21.15.32:0/1521842991 >> v2:172.21.15.55:3300/0 conn(0x7fa1100a8c50 0x7fa11014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 --2- 172.21.15.32:0/1521842991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100b2710 0x7fa11014afd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- 172.21.15.32:0/1521842991 >> 172.21.15.32:0/1521842991 conn(0x7fa110060310 msgr2=0x7fa1100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:35.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- 172.21.15.32:0/1521842991 shutdown_connections 2022-08-01T23:01:35.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.597+0000 7fa114956700 1 -- 172.21.15.32:0/1521842991 wait complete. 2022-08-01T23:01:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 Processor -- start 2022-08-01T23:01:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 -- start start 2022-08-01T23:01:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 0x7fa110156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1100b01a0 0x7fa1100aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1100b2710 0x7fa11013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101412e0 con 0x7fa1100a8c50 2022-08-01T23:01:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110146730 con 0x7fa1100b01a0 2022-08-01T23:01:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100a9450 con 0x7fa1100b2710 2022-08-01T23:01:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 0x7fa110156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 0x7fa110156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44796/0 (socket says 172.21.15.32:44796) 2022-08-01T23:01:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 -- 172.21.15.32:0/2123334843 learned_addr learned my addr 172.21.15.32:0/2123334843 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10f59e700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1100b2710 0x7fa11013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10e59c700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1100b01a0 0x7fa1100aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 -- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1100b2710 msgr2=0x7fa11013f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1100b2710 0x7fa11013f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 -- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1100b01a0 msgr2=0x7fa1100aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1100b01a0 0x7fa1100aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 -- 172.21.15.32:0/2123334843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10400b180 con 0x7fa1100a8c50 2022-08-01T23:01:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10f59e700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1100b2710 0x7fa11013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:35.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa10ed9d700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 0x7fa110156ef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fa100001800 tx=0x7fa104014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa10400d8a0 con 0x7fa1100a8c50 2022-08-01T23:01:35.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10400da00 con 0x7fa1100a8c50 2022-08-01T23:01:35.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1100a96f0 con 0x7fa1100a8c50 2022-08-01T23:01:35.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa10400dd60 con 0x7fa1100a8c50 2022-08-01T23:01:35.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.601+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11014d720 con 0x7fa1100a8c50 2022-08-01T23:01:35.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.605+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11014d2c0 con 0x7fa1100a8c50 2022-08-01T23:01:35.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.605+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa104025030 con 0x7fa1100a8c50 2022-08-01T23:01:35.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.605+0000 7fa0fffff700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa0ec053b30 0x7fa0ec055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.605+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa10406cc20 con 0x7fa1100a8c50 2022-08-01T23:01:35.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.605+0000 7fa10e59c700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa0ec053b30 0x7fa0ec055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.605+0000 7fa10e59c700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa0ec053b30 0x7fa0ec055ff0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fa10000d000 tx=0x7fa1000011a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.609+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1040438d0 con 0x7fa1100a8c50 2022-08-01T23:01:35.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.769+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1100574e0 con 0x7fa1100a8c50 2022-08-01T23:01:35.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.769+0000 7fa0fffff700 1 -- 172.21.15.32:0/2123334843 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fa104023020 con 0x7fa1100a8c50 2022-08-01T23:01:35.772 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:01:35.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa0ec053b30 msgr2=0x7fa0ec055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa0ec053b30 0x7fa0ec055ff0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fa10000d000 tx=0x7fa1000011a0).stop 2022-08-01T23:01:35.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 msgr2=0x7fa110156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 0x7fa110156ef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fa100001800 tx=0x7fa104014000).stop 2022-08-01T23:01:35.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 shutdown_connections 2022-08-01T23:01:35.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa0ec053b30 0x7fa0ec055ff0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa1100b2710 0x7fa11013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1100b01a0 0x7fa1100aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 --2- 172.21.15.32:0/2123334843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100a8c50 0x7fa110156ef0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 >> 172.21.15.32:0/2123334843 conn(0x7fa110060310 msgr2=0x7fa110148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:35.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 shutdown_connections 2022-08-01T23:01:35.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.773+0000 7fa114956700 1 -- 172.21.15.32:0/2123334843 wait complete. 2022-08-01T23:01:35.777 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:01:35.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:01:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.889+0000 7f485b016700 1 Processor -- start 2022-08-01T23:01:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.889+0000 7f485b016700 1 -- start start 2022-08-01T23:01:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.889+0000 7f485b016700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 0x7f48540af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.889+0000 7f485b016700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48540af610 con 0x7f485413fa40 2022-08-01T23:01:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.889+0000 7f485b016700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48540b0c40 con 0x7f48540ad670 2022-08-01T23:01:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48540b0ee0 con 0x7f48540b0570 2022-08-01T23:01:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485a815700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f485413fa40 0x7f485413fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60582/0 (socket says 172.21.15.32:60582) 2022-08-01T23:01:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485a815700 1 -- 172.21.15.32:0/3748589307 learned_addr learned my addr 172.21.15.32:0/3748589307 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 0x7f48540af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751482038 0 0) 0x7f484c001490 con 0x7f485413fa40 2022-08-01T23:01:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c003260 con 0x7f485413fa40 2022-08-01T23:01:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1664046144 0 0) 0x7f483c003260 con 0x7f485413fa40 2022-08-01T23:01:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 msgr2=0x7f48540af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 0x7f48540af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 >> v1:172.21.15.55:6789/0 conn(0x7f48540ad670 legacy=0x7f48540b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540ae9e0 con 0x7f485413fa40 2022-08-01T23:01:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f484c001b70 con 0x7f485413fa40 2022-08-01T23:01:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 >> v1:172.21.15.32:6789/0 conn(0x7f485413fa40 legacy=0x7f485413fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 0x7f483c003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483c003c70 0x7f483c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c0049f0 0x7f483c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540b0ee0 con 0x7f483c003c70 2022-08-01T23:01:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00b750 con 0x7f483c0049f0 2022-08-01T23:01:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00b910 con 0x7f48540b0570 2022-08-01T23:01:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483c003c70 0x7f483c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485a815700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c0049f0 0x7f483c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485a014700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 0x7f483c003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 -- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 msgr2=0x7f483c003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48540b0570 0x7f483c003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 -- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c0049f0 msgr2=0x7f483c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c0049f0 0x7f483c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 -- 172.21.15.32:0/3748589307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c003260 con 0x7f483c003c70 2022-08-01T23:01:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859813700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483c003c70 0x7f483c004500 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f484400c000 tx=0x7f4844006df0).ready entity=mon.0 client_cookie=14ecaaa627bc9d19 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484400e830 con 0x7f483c003c70 2022-08-01T23:01:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484400e990 con 0x7f483c003c70 2022-08-01T23:01:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f4859012700 1 -- 172.21.15.32:0/3748589307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484400ecf0 con 0x7f483c003c70 2022-08-01T23:01:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 -- 172.21.15.32:0/3748589307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483c003c70 msgr2=0x7f483c004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 --2- 172.21.15.32:0/3748589307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483c003c70 0x7f483c004500 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f484400c000 tx=0x7f4844006df0).stop 2022-08-01T23:01:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485a815700 1 -- 172.21.15.32:0/3748589307 reap_dead start 2022-08-01T23:01:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 -- 172.21.15.32:0/3748589307 shutdown_connections 2022-08-01T23:01:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 -- 172.21.15.32:0/3748589307 >> 172.21.15.32:0/3748589307 conn(0x7f4854061360 msgr2=0x7f4854062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 -- 172.21.15.32:0/3748589307 shutdown_connections 2022-08-01T23:01:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.893+0000 7f485b016700 1 -- 172.21.15.32:0/3748589307 wait complete. 2022-08-01T23:01:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 Processor -- start 2022-08-01T23:01:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 -- start start 2022-08-01T23:01:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c003c70 0x7f48540aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f485413f420 0x7f4854065350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 0x7f4854152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540b5fe0 con 0x7f4854065840 2022-08-01T23:01:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540b6960 con 0x7f483c003c70 2022-08-01T23:01:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48540b58e0 con 0x7f485413f420 2022-08-01T23:01:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 0x7f4854152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 0x7f4854152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44820/0 (socket says 172.21.15.32:44820) 2022-08-01T23:01:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 -- 172.21.15.32:0/1189313456 learned_addr learned my addr 172.21.15.32:0/1189313456 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a014700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c003c70 0x7f48540aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f4859813700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f485413f420 0x7f4854065350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 -- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f485413f420 msgr2=0x7f4854065350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f485413f420 0x7f4854065350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 -- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c003c70 msgr2=0x7f48540aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:35.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c003c70 0x7f48540aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:35.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 -- 172.21.15.32:0/1189313456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540b61b0 con 0x7f4854065840 2022-08-01T23:01:35.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f4859813700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f485413f420 0x7f4854065350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485a815700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 0x7f4854152730 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f484c000b60 tx=0x7f48540b0dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484c00f8a0 con 0x7f4854065840 2022-08-01T23:01:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c00fa00 con 0x7f4854065840 2022-08-01T23:01:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f484c00fd60 con 0x7f4854065840 2022-08-01T23:01:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48540b6450 con 0x7f4854065840 2022-08-01T23:01:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.897+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48540b5b40 con 0x7f4854065840 2022-08-01T23:01:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.901+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48540b5d00 con 0x7f4854065840 2022-08-01T23:01:35.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.901+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f484c01c360 con 0x7f4854065840 2022-08-01T23:01:35.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.901+0000 7f48437fe700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48340539e0 0x7f4834055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:35.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.901+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f484c06bb90 con 0x7f4854065840 2022-08-01T23:01:35.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.901+0000 7f485a014700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48340539e0 0x7f4834055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:35.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.901+0000 7f485a014700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48340539e0 0x7f4834055ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4850003000 tx=0x7f48500079e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:35.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:35.905+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f484c0427e0 con 0x7f4854065840 2022-08-01T23:01:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.033+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48540af610 con 0x7f4854065840 2022-08-01T23:01:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.033+0000 7f48437fe700 1 -- 172.21.15.32:0/1189313456 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f48540af610 con 0x7f4854065840 2022-08-01T23:01:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:36.036 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":76,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:01:34.459122+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:01:32.889134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T23:01:33.885837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:01:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48340539e0 msgr2=0x7f4834055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48340539e0 0x7f4834055ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4850003000 tx=0x7f48500079e0).stop 2022-08-01T23:01:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 msgr2=0x7f4854152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 0x7f4854152730 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f484c000b60 tx=0x7f48540b0dd0).stop 2022-08-01T23:01:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 shutdown_connections 2022-08-01T23:01:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48340539e0 0x7f4834055ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4854065840 0x7f4854152730 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f485413f420 0x7f4854065350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 --2- 172.21.15.32:0/1189313456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c003c70 0x7f48540aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 >> 172.21.15.32:0/1189313456 conn(0x7f4854061360 msgr2=0x7f4854142920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 shutdown_connections 2022-08-01T23:01:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.037+0000 7f485b016700 1 -- 172.21.15.32:0/1189313456 wait complete. 2022-08-01T23:01:36.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:01:36.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.165+0000 7fcc79137700 1 Processor -- start 2022-08-01T23:01:36.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.165+0000 7fcc79137700 1 -- start start 2022-08-01T23:01:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc74066780 0x7fcc74066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc74140e40 0x7fcc7413e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7405f8e0 con 0x7fcc740b1bc0 2022-08-01T23:01:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740b1630 con 0x7fcc74066780 2022-08-01T23:01:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc73fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc74066780 0x7fcc74066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7405fb10 con 0x7fcc74140e40 2022-08-01T23:01:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc737fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc740b1bc0 0x7fcc7405ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60600/0 (socket says 172.21.15.32:60600) 2022-08-01T23:01:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc737fe700 1 -- 172.21.15.32:0/1387652333 learned_addr learned my addr 172.21.15.32:0/1387652333 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc78936700 1 --2- 172.21.15.32:0/1387652333 >> v2:172.21.15.55:3300/0 conn(0x7fcc74140e40 0x7fcc7413e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc73fff700 1 -- 172.21.15.32:0/1387652333 >> v2:172.21.15.55:3300/0 conn(0x7fcc74140e40 msgr2=0x7fcc7413e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc73fff700 1 --2- 172.21.15.32:0/1387652333 >> v2:172.21.15.55:3300/0 conn(0x7fcc74140e40 0x7fcc7413e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc73fff700 1 -- 172.21.15.32:0/1387652333 >> v1:172.21.15.32:6789/0 conn(0x7fcc740b1bc0 legacy=0x7fcc7405ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc73fff700 1 -- 172.21.15.32:0/1387652333 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740b18d0 con 0x7fcc74066780 2022-08-01T23:01:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc72ffd700 1 -- 172.21.15.32:0/1387652333 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812994422 0 0) 0x7fcc7405f8e0 con 0x7fcc740b1bc0 2022-08-01T23:01:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc78936700 1 --2- 172.21.15.32:0/1387652333 >> v2:172.21.15.55:3300/0 conn(0x7fcc74140e40 0x7fcc7413e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc73fff700 1 --2- 172.21.15.32:0/1387652333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc74066b60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fcc64002f70 tx=0x7fcc64006df0).ready entity=mon.0 client_cookie=81e7b0b79c694c56 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc72ffd700 1 -- 172.21.15.32:0/1387652333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6400d5c0 con 0x7fcc74066780 2022-08-01T23:01:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc72ffd700 1 -- 172.21.15.32:0/1387652333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc64000e90 con 0x7fcc74066780 2022-08-01T23:01:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc72ffd700 1 -- 172.21.15.32:0/1387652333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6400d910 con 0x7fcc74066780 2022-08-01T23:01:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- 172.21.15.32:0/1387652333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 msgr2=0x7fcc74066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 --2- 172.21.15.32:0/1387652333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc74066b60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fcc64002f70 tx=0x7fcc64006df0).stop 2022-08-01T23:01:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- 172.21.15.32:0/1387652333 shutdown_connections 2022-08-01T23:01:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 --2- 172.21.15.32:0/1387652333 >> v2:172.21.15.55:3300/0 conn(0x7fcc74140e40 0x7fcc7413e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 --2- 172.21.15.32:0/1387652333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc74066b60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- 172.21.15.32:0/1387652333 >> 172.21.15.32:0/1387652333 conn(0x7fcc74063870 msgr2=0x7fcc74063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- 172.21.15.32:0/1387652333 shutdown_connections 2022-08-01T23:01:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- 172.21.15.32:0/1387652333 wait complete. 2022-08-01T23:01:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 Processor -- start 2022-08-01T23:01:36.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.169+0000 7fcc79137700 1 -- start start 2022-08-01T23:01:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc79137700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc7413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc79137700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc740b1bc0 0x7fcc74139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc79137700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc74140e40 0x7fcc74139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc79137700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740a90e0 con 0x7fcc74066780 2022-08-01T23:01:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc79137700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74156ad0 con 0x7fcc74140e40 2022-08-01T23:01:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc79137700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74065c30 con 0x7fcc740b1bc0 2022-08-01T23:01:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc7413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc7413a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44830/0 (socket says 172.21.15.32:44830) 2022-08-01T23:01:36.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 -- 172.21.15.32:0/4089535953 learned_addr learned my addr 172.21.15.32:0/4089535953 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc78936700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc74140e40 0x7fcc74139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc737fe700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc740b1bc0 0x7fcc74139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 -- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc740b1bc0 msgr2=0x7fcc74139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc740b1bc0 0x7fcc74139650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 -- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc74140e40 msgr2=0x7fcc74139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc74140e40 0x7fcc74139b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 -- 172.21.15.32:0/4089535953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6400b180 con 0x7fcc74066780 2022-08-01T23:01:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc73fff700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc7413a970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fcc64002f70 tx=0x7fcc64014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc64013e90 con 0x7fcc74066780 2022-08-01T23:01:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6401e400 con 0x7fcc74066780 2022-08-01T23:01:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.173+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6401e760 con 0x7fcc74066780 2022-08-01T23:01:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc740b0b00 con 0x7fcc74066780 2022-08-01T23:01:36.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc740a98f0 con 0x7fcc74066780 2022-08-01T23:01:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcc6401c070 con 0x7fcc74066780 2022-08-01T23:01:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc740b12b0 con 0x7fcc74066780 2022-08-01T23:01:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc717fa700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc54053aa0 0x7fcc54055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcc6406ca80 con 0x7fcc74066780 2022-08-01T23:01:36.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.177+0000 7fcc737fe700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc54053aa0 0x7fcc54055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.181+0000 7fcc737fe700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc54053aa0 0x7fcc54055f60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fcc6800b000 tx=0x7fcc68009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.181+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc64043870 con 0x7fcc74066780 2022-08-01T23:01:36.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.345+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fcc740574e0 con 0x7fcc74066780 2022-08-01T23:01:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.353+0000 7fcc717fa700 1 -- 172.21.15.32:0/4089535953 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7fcc64023020 con 0x7fcc74066780 2022-08-01T23:01:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc54053aa0 msgr2=0x7fcc54055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc54053aa0 0x7fcc54055f60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fcc6800b000 tx=0x7fcc68009040).stop 2022-08-01T23:01:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 msgr2=0x7fcc7413a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc7413a970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fcc64002f70 tx=0x7fcc64014000).stop 2022-08-01T23:01:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 shutdown_connections 2022-08-01T23:01:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc54053aa0 0x7fcc54055f60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc74140e40 0x7fcc74139b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc740b1bc0 0x7fcc74139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 --2- 172.21.15.32:0/4089535953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc74066780 0x7fcc7413a970 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 >> 172.21.15.32:0/4089535953 conn(0x7fcc74063870 msgr2=0x7fcc740b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 shutdown_connections 2022-08-01T23:01:36.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.357+0000 7fcc79137700 1 -- 172.21.15.32:0/4089535953 wait complete. 2022-08-01T23:01:36.361 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:01:36.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:01:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 Processor -- start 2022-08-01T23:01:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 -- start start 2022-08-01T23:01:36.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe3480614b0 0x7fe348061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480aee80 con 0x7fe3481405d0 2022-08-01T23:01:36.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3480b70b0 con 0x7fe3480614b0 2022-08-01T23:01:36.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d9e4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480af0b0 con 0x7fe3480ad750 2022-08-01T23:01:36.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d1e3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3481405d0 0x7fe3481466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60616/0 (socket says 172.21.15.32:60616) 2022-08-01T23:01:36.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.477+0000 7fe34d1e3700 1 -- 172.21.15.32:0/570352299 learned_addr learned my addr 172.21.15.32:0/570352299 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:36.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34c9e2700 1 --2- 172.21.15.32:0/570352299 >> v2:172.21.15.55:3300/0 conn(0x7fe3480614b0 0x7fe348061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe347fff700 1 --2- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe3477fe700 1 -- 172.21.15.32:0/570352299 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258762197 0 0) 0x7fe3480aee80 con 0x7fe3481405d0 2022-08-01T23:01:36.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe3477fe700 1 -- 172.21.15.32:0/570352299 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c003260 con 0x7fe3481405d0 2022-08-01T23:01:36.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34c9e2700 1 -- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 msgr2=0x7fe3480b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:36.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34c9e2700 1 --2- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34c9e2700 1 -- 172.21.15.32:0/570352299 >> v1:172.21.15.32:6789/0 conn(0x7fe3481405d0 legacy=0x7fe3481466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34c9e2700 1 -- 172.21.15.32:0/570352299 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480af350 con 0x7fe3480614b0 2022-08-01T23:01:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34c9e2700 1 --2- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348061890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fe334002f70 tx=0x7fe334006df0).ready entity=mon.1 client_cookie=3abf952b84b720a2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe3477fe700 1 -- 172.21.15.32:0/570352299 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe33400d4a0 con 0x7fe3480614b0 2022-08-01T23:01:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe3477fe700 1 -- 172.21.15.32:0/570352299 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334000e90 con 0x7fe3480614b0 2022-08-01T23:01:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe3477fe700 1 -- 172.21.15.32:0/570352299 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe33400d7f0 con 0x7fe3480614b0 2022-08-01T23:01:36.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 -- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 msgr2=0x7fe348061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348061890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fe334002f70 tx=0x7fe334006df0).stop 2022-08-01T23:01:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 -- 172.21.15.32:0/570352299 shutdown_connections 2022-08-01T23:01:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/570352299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348061890 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 -- 172.21.15.32:0/570352299 >> 172.21.15.32:0/570352299 conn(0x7fe348066f70 msgr2=0x7fe348067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 -- 172.21.15.32:0/570352299 shutdown_connections 2022-08-01T23:01:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.481+0000 7fe34d9e4700 1 -- 172.21.15.32:0/570352299 wait complete. 2022-08-01T23:01:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 Processor -- start 2022-08-01T23:01:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 -- start start 2022-08-01T23:01:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 0x7fe3480b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348152fa0 con 0x7fe3481405d0 2022-08-01T23:01:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480acb90 con 0x7fe3480614b0 2022-08-01T23:01:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d9e4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3481531d0 con 0x7fe3480ad750 2022-08-01T23:01:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d1e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 0x7fe3480b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d1e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 0x7fe3480b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44848/0 (socket says 172.21.15.32:44848) 2022-08-01T23:01:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34c9e2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe34d1e3700 1 -- 172.21.15.32:0/2302313829 learned_addr learned my addr 172.21.15.32:0/2302313829 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.485+0000 7fe347fff700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d1e3700 1 -- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 msgr2=0x7fe3480b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d1e3700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d1e3700 1 -- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 msgr2=0x7fe348140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d1e3700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d1e3700 1 -- 172.21.15.32:0/2302313829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33400b180 con 0x7fe3481405d0 2022-08-01T23:01:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34c9e2700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d1e3700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 0x7fe3480b1a10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fe340001490 tx=0x7fe340007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe340005ba0 con 0x7fe3481405d0 2022-08-01T23:01:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe340005d00 con 0x7fe3481405d0 2022-08-01T23:01:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3480ace30 con 0x7fe3481405d0 2022-08-01T23:01:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3400072c0 con 0x7fe3481405d0 2022-08-01T23:01:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3480b1010 con 0x7fe3481405d0 2022-08-01T23:01:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3480af5f0 con 0x7fe3481405d0 2022-08-01T23:01:36.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe34001a060 con 0x7fe3481405d0 2022-08-01T23:01:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe345ffb700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe328053a10 0x7fe328055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.489+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe34006bb70 con 0x7fe3481405d0 2022-08-01T23:01:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.493+0000 7fe34c9e2700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe328053a10 0x7fe328055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.493+0000 7fe34c9e2700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe328053a10 0x7fe328055ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fe33401a000 tx=0x7fe334014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.497+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe340042820 con 0x7fe3481405d0 2022-08-01T23:01:36.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.661+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe3480574e0 con 0x7fe3481405d0 2022-08-01T23:01:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.669+0000 7fe345ffb700 1 -- 172.21.15.32:0/2302313829 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe340018030 con 0x7fe3481405d0 2022-08-01T23:01:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe328053a10 msgr2=0x7fe328055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe328053a10 0x7fe328055ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fe33401a000 tx=0x7fe334014000).stop 2022-08-01T23:01:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 msgr2=0x7fe3480b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 0x7fe3480b1a10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fe340001490 tx=0x7fe340007e00).stop 2022-08-01T23:01:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 shutdown_connections 2022-08-01T23:01:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe328053a10 0x7fe328055ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3481405d0 0x7fe3480b1a10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3480ad750 0x7fe3480b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 --2- 172.21.15.32:0/2302313829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3480614b0 0x7fe348140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 >> 172.21.15.32:0/2302313829 conn(0x7fe348066f70 msgr2=0x7fe348144970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 shutdown_connections 2022-08-01T23:01:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.673+0000 7fe34d9e4700 1 -- 172.21.15.32:0/2302313829 wait complete. 2022-08-01T23:01:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:01:36.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:36.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 Processor -- start 2022-08-01T23:01:36.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 -- start start 2022-08-01T23:01:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29800b6510 0x7f298013d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2980064500 0x7f298013d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298013dc80 con 0x7f29800b0e60 2022-08-01T23:01:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2980066af0 con 0x7f29800b6510 2022-08-01T23:01:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2980066d90 con 0x7f2980064500 2022-08-01T23:01:36.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29800b6510 0x7f298013d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29800b6510 0x7f298013d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44852/0 (socket says 172.21.15.32:44852) 2022-08-01T23:01:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 -- 172.21.15.32:0/1007029658 learned_addr learned my addr 172.21.15.32:0/1007029658 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297ed9d700 1 --2- 172.21.15.32:0/1007029658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2980064500 0x7f298013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 -- 172.21.15.32:0/1007029658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2980064500 msgr2=0x7f298013d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 --2- 172.21.15.32:0/1007029658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2980064500 0x7f298013d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 -- 172.21.15.32:0/1007029658 >> v1:172.21.15.55:6789/0 conn(0x7f29800b0e60 legacy=0x7f29800609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 -- 172.21.15.32:0/1007029658 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980067030 con 0x7f29800b6510 2022-08-01T23:01:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297e59c700 1 --2- 172.21.15.32:0/1007029658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f298013d2a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f29680032f0 tx=0x7f2968009290).ready entity=mon.0 client_cookie=e6e855ea232352a1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297d59a700 1 -- 172.21.15.32:0/1007029658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f296800ba90 con 0x7f29800b6510 2022-08-01T23:01:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297d59a700 1 -- 172.21.15.32:0/1007029658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800bbf0 con 0x7f29800b6510 2022-08-01T23:01:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.793+0000 7f297d59a700 1 -- 172.21.15.32:0/1007029658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2968010470 con 0x7f29800b6510 2022-08-01T23:01:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- 172.21.15.32:0/1007029658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 msgr2=0x7f298013d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 --2- 172.21.15.32:0/1007029658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f298013d2a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f29680032f0 tx=0x7f2968009290).stop 2022-08-01T23:01:36.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- 172.21.15.32:0/1007029658 shutdown_connections 2022-08-01T23:01:36.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 --2- 172.21.15.32:0/1007029658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2980064500 0x7f298013d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 --2- 172.21.15.32:0/1007029658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f298013d2a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- 172.21.15.32:0/1007029658 >> 172.21.15.32:0/1007029658 conn(0x7f2980063800 msgr2=0x7f2980063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- 172.21.15.32:0/1007029658 shutdown_connections 2022-08-01T23:01:36.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- 172.21.15.32:0/1007029658 wait complete. 2022-08-01T23:01:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 Processor -- start 2022-08-01T23:01:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- start start 2022-08-01T23:01:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2980064500 0x7f298009ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29800b0e60 0x7f29800ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f29800aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800ad3c0 con 0x7f29800b6510 2022-08-01T23:01:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800aef60 con 0x7f2980064500 2022-08-01T23:01:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800af200 con 0x7f29800b0e60 2022-08-01T23:01:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f29800aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f29800aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44862/0 (socket says 172.21.15.32:44862) 2022-08-01T23:01:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2980064500 0x7f298009ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 -- 172.21.15.32:0/1464226245 learned_addr learned my addr 172.21.15.32:0/1464226245 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297dd9b700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29800b0e60 0x7f29800ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 -- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29800b0e60 msgr2=0x7f29800ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29800b0e60 0x7f29800ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 -- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2980064500 msgr2=0x7f298009ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2980064500 0x7f298009ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 -- 172.21.15.32:0/1464226245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968009530 con 0x7f29800b6510 2022-08-01T23:01:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.797+0000 7f297ed9d700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f29800aced0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f29800afdb0 tx=0x7f2978008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2978011780 con 0x7f29800b6510 2022-08-01T23:01:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29780118e0 con 0x7f29800b6510 2022-08-01T23:01:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29800b0750 con 0x7f29800b6510 2022-08-01T23:01:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2978011c40 con 0x7f29800b6510 2022-08-01T23:01:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29800abe30 con 0x7f29800b6510 2022-08-01T23:01:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29800574e0 con 0x7f29800b6510 2022-08-01T23:01:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2978012420 con 0x7f29800b6510 2022-08-01T23:01:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f29777fe700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2958053950 0x7f2958055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f297806bdd0 con 0x7f29800b6510 2022-08-01T23:01:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.801+0000 7f297e59c700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2958053950 0x7f2958055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.805+0000 7f297e59c700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2958053950 0x7f2958055e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f296801b000 tx=0x7f2968018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.809+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2978042b30 con 0x7f29800b6510 2022-08-01T23:01:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f298006ac80 con 0x7f29800b6510 2022-08-01T23:01:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f29777fe700 1 -- 172.21.15.32:0/1464226245 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f2978046150 con 0x7f29800b6510 2022-08-01T23:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:01:36.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2958053950 msgr2=0x7f2958055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2958053950 0x7f2958055e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f296801b000 tx=0x7f2968018000).stop 2022-08-01T23:01:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 msgr2=0x7f29800aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f29800aced0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f29800afdb0 tx=0x7f2978008040).stop 2022-08-01T23:01:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 shutdown_connections 2022-08-01T23:01:36.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2958053950 0x7f2958055e10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29800b6510 0x7f29800aced0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29800b0e60 0x7f29800ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 --2- 172.21.15.32:0/1464226245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2980064500 0x7f298009ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 >> 172.21.15.32:0/1464226245 conn(0x7f2980063800 msgr2=0x7f2980143300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 shutdown_connections 2022-08-01T23:01:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:36.969+0000 7f297f59e700 1 -- 172.21.15.32:0/1464226245 wait complete. 2022-08-01T23:01:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:01:37.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:01:34.459266+0000', 'modified': '2022-08-01T23:01:35.461964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6258}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6258': {'gid': 6258, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1927724642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1927724642}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1927724642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:01:37.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:01:37.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:01:37.404 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6258,"want_state":"up:active","state":"up:active","rank_uptime":2.9346883109999999,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":76,"uptime":7.2524406509999997} 2022-08-01T23:01:37.405 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6258, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 76, "rank_uptime": 2.934688311, "state": "up:active", "uptime": 7.252440651, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:01:37.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:01:37.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550f1d2700 1 Processor -- start 2022-08-01T23:01:37.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550f1d2700 1 -- start start 2022-08-01T23:01:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550f1d2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b01a0 0x7f55080a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550f1d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55080a7250 con 0x7f55080b2710 2022-08-01T23:01:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550f1d2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55080a7480 con 0x7f55080afd60 2022-08-01T23:01:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550f1d2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55080a7720 con 0x7f55080b01a0 2022-08-01T23:01:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550e1d0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55080b2710 0x7f550814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60642/0 (socket says 172.21.15.32:60642) 2022-08-01T23:01:37.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.509+0000 7f550e1d0700 1 -- 172.21.15.32:0/2026433286 learned_addr learned my addr 172.21.15.32:0/2026433286 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:37.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d9cf700 1 --2- 172.21.15.32:0/2026433286 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b01a0 0x7f55080a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743062874 0 0) 0x7f55080a7250 con 0x7f55080b2710 2022-08-01T23:01:37.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f0003260 con 0x7f55080b2710 2022-08-01T23:01:37.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115027428 0 0) 0x7f55080a7480 con 0x7f55080afd60 2022-08-01T23:01:37.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55080a7250 con 0x7f55080afd60 2022-08-01T23:01:37.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d9cf700 1 -- 172.21.15.32:0/2026433286 >> v1:172.21.15.55:6789/0 conn(0x7f55080afd60 legacy=0x7f5508151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d9cf700 1 -- 172.21.15.32:0/2026433286 >> v1:172.21.15.32:6789/0 conn(0x7f55080b2710 legacy=0x7f550814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665048194 0 0) 0x7f54f0003260 con 0x7f55080b2710 2022-08-01T23:01:37.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d9cf700 1 -- 172.21.15.32:0/2026433286 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55080a79c0 con 0x7f55080b01a0 2022-08-01T23:01:37.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d9cf700 1 --2- 172.21.15.32:0/2026433286 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b01a0 0x7f55080a8540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f54f8001110 tx=0x7f5504006df0).ready entity=mon.2 client_cookie=87f8fed3b8623aa7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:37.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f8001660 con 0x7f55080b01a0 2022-08-01T23:01:37.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504000e90 con 0x7f55080b01a0 2022-08-01T23:01:37.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550d1ce700 1 -- 172.21.15.32:0/2026433286 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5504012460 con 0x7f55080b01a0 2022-08-01T23:01:37.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 -- 172.21.15.32:0/2026433286 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b01a0 msgr2=0x7f55080a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 --2- 172.21.15.32:0/2026433286 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b01a0 0x7f55080a8540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f54f8001110 tx=0x7f5504006df0).stop 2022-08-01T23:01:37.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 -- 172.21.15.32:0/2026433286 shutdown_connections 2022-08-01T23:01:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 --2- 172.21.15.32:0/2026433286 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b01a0 0x7f55080a8540 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 -- 172.21.15.32:0/2026433286 >> 172.21.15.32:0/2026433286 conn(0x7f5508060310 msgr2=0x7f55080b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 -- 172.21.15.32:0/2026433286 shutdown_connections 2022-08-01T23:01:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 -- 172.21.15.32:0/2026433286 wait complete. 2022-08-01T23:01:37.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 Processor -- start 2022-08-01T23:01:37.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.513+0000 7f550f1d2700 1 -- start start 2022-08-01T23:01:37.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55080afd60 0x7f55080b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 0x7f55080b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b2710 0x7f55080b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55080a99a0 con 0x7f55080b01a0 2022-08-01T23:01:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55080aa980 con 0x7f55080afd60 2022-08-01T23:01:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55080b4910 con 0x7f55080b2710 2022-08-01T23:01:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 0x7f55080b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 0x7f55080b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44874/0 (socket says 172.21.15.32:44874) 2022-08-01T23:01:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 -- 172.21.15.32:0/947336616 learned_addr learned my addr 172.21.15.32:0/947336616 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550e1d0700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55080afd60 0x7f55080b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550e9d1700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b2710 0x7f55080b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 -- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b2710 msgr2=0x7f55080b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b2710 0x7f55080b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 -- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55080afd60 msgr2=0x7f55080b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55080afd60 0x7f55080b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 -- 172.21.15.32:0/947336616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550400b180 con 0x7f55080b01a0 2022-08-01T23:01:37.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550d9cf700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 0x7f55080b4c90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f54f8001110 tx=0x7f5504014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:37.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5504016030 con 0x7f55080b01a0 2022-08-01T23:01:37.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400dcb0 con 0x7f55080b01a0 2022-08-01T23:01:37.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55080a9bd0 con 0x7f55080b01a0 2022-08-01T23:01:37.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5504012460 con 0x7f55080b01a0 2022-08-01T23:01:37.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55080a9660 con 0x7f55080b01a0 2022-08-01T23:01:37.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.517+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55080574e0 con 0x7f55080b01a0 2022-08-01T23:01:37.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.521+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5504024020 con 0x7f55080b01a0 2022-08-01T23:01:37.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.521+0000 7f54f77fe700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54e8053980 0x7f54e8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.521+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5504017070 con 0x7f55080b01a0 2022-08-01T23:01:37.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.521+0000 7f550e1d0700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54e8053980 0x7f54e8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.521+0000 7f550e1d0700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54e8053980 0x7f54e8055e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f54f800b000 tx=0x7f54f8000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:37.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.525+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5504043d90 con 0x7f55080b01a0 2022-08-01T23:01:37.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.653+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5508069a70 con 0x7f55080b01a0 2022-08-01T23:01:37.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.653+0000 7f54f77fe700 1 -- 172.21.15.32:0/947336616 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f55040473b0 con 0x7f55080b01a0 2022-08-01T23:01:37.656 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:37.656 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":76,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:01:34.459122+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:01:32.889134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T23:01:33.885837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:01:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54e8053980 msgr2=0x7f54e8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54e8053980 0x7f54e8055e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f54f800b000 tx=0x7f54f8000bc0).stop 2022-08-01T23:01:37.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 msgr2=0x7f55080b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 0x7f55080b4c90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f54f8001110 tx=0x7f5504014000).stop 2022-08-01T23:01:37.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 shutdown_connections 2022-08-01T23:01:37.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f54e8053980 0x7f54e8055e40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f55080b2710 0x7f55080b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55080b01a0 0x7f55080b4c90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 --2- 172.21.15.32:0/947336616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55080afd60 0x7f55080b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 >> 172.21.15.32:0/947336616 conn(0x7f5508060310 msgr2=0x7f5508149a30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 shutdown_connections 2022-08-01T23:01:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.657+0000 7f550f1d2700 1 -- 172.21.15.32:0/947336616 wait complete. 2022-08-01T23:01:37.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:01:32.889134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:01:33.885837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:01:37.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:01:37.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e9db3700 1 Processor -- start 2022-08-01T23:01:37.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e9db3700 1 -- start start 2022-08-01T23:01:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e9db3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e4061890 0x7f02e41466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e9db3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e40aee80 con 0x7f02e40614b0 2022-08-01T23:01:37.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e9db3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e40b70b0 con 0x7f02e40ad750 2022-08-01T23:01:37.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e9db3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40af0b0 con 0x7f02e4061890 2022-08-01T23:01:37.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e4061890 0x7f02e41466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e4061890 0x7f02e41466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44884/0 (socket says 172.21.15.32:44884) 2022-08-01T23:01:37.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 -- 172.21.15.32:0/2337989839 learned_addr learned my addr 172.21.15.32:0/2337989839 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:37.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 -- 172.21.15.32:0/2337989839 >> v1:172.21.15.55:6789/0 conn(0x7f02e40ad750 legacy=0x7f02e41405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 -- 172.21.15.32:0/2337989839 >> v1:172.21.15.32:6789/0 conn(0x7f02e40614b0 legacy=0x7f02e40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 -- 172.21.15.32:0/2337989839 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40af350 con 0x7f02e4061890 2022-08-01T23:01:37.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02db7fe700 1 -- 172.21.15.32:0/2337989839 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735235929 0 0) 0x7f02e40aee80 con 0x7f02e40614b0 2022-08-01T23:01:37.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02e95b2700 1 --2- 172.21.15.32:0/2337989839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e4061890 0x7f02e41466b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f02dc0032f0 tx=0x7f02dc009290).ready entity=mon.0 client_cookie=64e67a3c1b2db916 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:37.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02db7fe700 1 -- 172.21.15.32:0/2337989839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02dc00b8a0 con 0x7f02e4061890 2022-08-01T23:01:37.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02db7fe700 1 -- 172.21.15.32:0/2337989839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc00ba00 con 0x7f02e4061890 2022-08-01T23:01:37.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.785+0000 7f02db7fe700 1 -- 172.21.15.32:0/2337989839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02dc00bb70 con 0x7f02e4061890 2022-08-01T23:01:37.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/2337989839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e4061890 msgr2=0x7f02e41466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 --2- 172.21.15.32:0/2337989839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e4061890 0x7f02e41466b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f02dc0032f0 tx=0x7f02dc009290).stop 2022-08-01T23:01:37.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/2337989839 shutdown_connections 2022-08-01T23:01:37.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 --2- 172.21.15.32:0/2337989839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e4061890 0x7f02e41466b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/2337989839 >> 172.21.15.32:0/2337989839 conn(0x7f02e4066f70 msgr2=0x7f02e4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:37.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/2337989839 shutdown_connections 2022-08-01T23:01:37.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/2337989839 wait complete. 2022-08-01T23:01:37.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 Processor -- start 2022-08-01T23:01:37.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- start start 2022-08-01T23:01:37.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 0x7f02e4144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02e4061890 0x7f02e4153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02e40ad750 0x7f02e41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4153550 con 0x7f02e40614b0 2022-08-01T23:01:37.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40ba3b0 con 0x7f02e40ad750 2022-08-01T23:01:37.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40b0c00 con 0x7f02e4061890 2022-08-01T23:01:37.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 0x7f02e4144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 0x7f02e4144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44886/0 (socket says 172.21.15.32:44886) 2022-08-01T23:01:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 -- 172.21.15.32:0/686105777 learned_addr learned my addr 172.21.15.32:0/686105777 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02dbfff700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02e4061890 0x7f02e4153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e95b2700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02e40ad750 0x7f02e41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 -- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02e4061890 msgr2=0x7f02e4153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02e4061890 0x7f02e4153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 -- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02e40ad750 msgr2=0x7f02e41560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02e40ad750 0x7f02e41560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 -- 172.21.15.32:0/686105777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc009530 con 0x7f02e40614b0 2022-08-01T23:01:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e95b2700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02e40ad750 0x7f02e41560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02dbfff700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02e4061890 0x7f02e4153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e8db1700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 0x7f02e4144720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f02e0000ed0 tx=0x7f02e0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02e000ce20 con 0x7f02e40614b0 2022-08-01T23:01:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e000f470 con 0x7f02e40614b0 2022-08-01T23:01:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e40b0ea0 con 0x7f02e40614b0 2022-08-01T23:01:37.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02e000f7d0 con 0x7f02e40614b0 2022-08-01T23:01:37.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.789+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4155c50 con 0x7f02e40614b0 2022-08-01T23:01:37.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.793+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e40af5f0 con 0x7f02e40614b0 2022-08-01T23:01:37.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.793+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f02e0019030 con 0x7f02e40614b0 2022-08-01T23:01:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.793+0000 7f02d9ffb700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c4053a10 0x7f02c4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.793+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f02e006b4b0 con 0x7f02e40614b0 2022-08-01T23:01:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.793+0000 7f02dbfff700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c4053a10 0x7f02c4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.793+0000 7f02dbfff700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c4053a10 0x7f02c4055ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f02d400b000 tx=0x7f02d4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.797+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02e00421b0 con 0x7f02e40614b0 2022-08-01T23:01:37.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.925+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f02e40574e0 con 0x7f02e40614b0 2022-08-01T23:01:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.925+0000 7f02d9ffb700 1 -- 172.21.15.32:0/686105777 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f02e00457d0 con 0x7f02e40614b0 2022-08-01T23:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:01:37.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c4053a10 msgr2=0x7f02c4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c4053a10 0x7f02c4055ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f02d400b000 tx=0x7f02d4009040).stop 2022-08-01T23:01:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 msgr2=0x7f02e4144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 0x7f02e4144720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f02e0000ed0 tx=0x7f02e0007ac0).stop 2022-08-01T23:01:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 shutdown_connections 2022-08-01T23:01:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02c4053a10 0x7f02c4055ed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02e40ad750 0x7f02e41560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02e4061890 0x7f02e4153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 --2- 172.21.15.32:0/686105777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e40614b0 0x7f02e4144720 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 >> 172.21.15.32:0/686105777 conn(0x7f02e4066f70 msgr2=0x7f02e4143b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 shutdown_connections 2022-08-01T23:01:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:37.929+0000 7f02e9db3700 1 -- 172.21.15.32:0/686105777 wait complete. 2022-08-01T23:01:37.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:01:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 Processor -- start 2022-08-01T23:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 -- start start 2022-08-01T23:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 0x7faed01403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faed00b4820 0x7faed00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed0143cf0 con 0x7faed00b4310 2022-08-01T23:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed01476a0 con 0x7faed00b4820 2022-08-01T23:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faed57ae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed0143f90 con 0x7faed0146ae0 2022-08-01T23:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faec67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faed00b4310 0x7faed01408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60668/0 (socket says 172.21.15.32:60668) 2022-08-01T23:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faec67fc700 1 -- 172.21.15.32:0/194354159 learned_addr learned my addr 172.21.15.32:0/194354159 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faecf7fe700 1 --2- 172.21.15.32:0/194354159 >> v2:172.21.15.55:3300/0 conn(0x7faed00b4820 0x7faed00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.049+0000 7faeceffd700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 0x7faed01403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216037821 0 0) 0x7faed0143cf0 con 0x7faed00b4310 2022-08-01T23:01:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb0003260 con 0x7faed00b4310 2022-08-01T23:01:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 882761505 0 0) 0x7faeb0003260 con 0x7faed00b4310 2022-08-01T23:01:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 msgr2=0x7faed01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 0x7faed01403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 >> v2:172.21.15.55:3300/0 conn(0x7faed00b4820 msgr2=0x7faed00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 --2- 172.21.15.32:0/194354159 >> v2:172.21.15.55:3300/0 conn(0x7faed00b4820 0x7faed00b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00b0e20 con 0x7faed00b4310 2022-08-01T23:01:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7faeb8001660 con 0x7faed00b4310 2022-08-01T23:01:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faecf7fe700 1 --2- 172.21.15.32:0/194354159 >> v2:172.21.15.55:3300/0 conn(0x7faed00b4820 0x7faed00b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 >> v1:172.21.15.32:6789/0 conn(0x7faed00b4310 legacy=0x7faed01408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb0003750 0x7faeb0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0004500 0x7faeb0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 0x7faeb0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed01476a0 con 0x7faeb0004500 2022-08-01T23:01:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed0143f90 con 0x7faeb0003750 2022-08-01T23:01:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb000b750 con 0x7faed0146ae0 2022-08-01T23:01:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb0003750 0x7faeb0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faec67fc700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0004500 0x7faeb0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faecf7fe700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 0x7faeb0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 -- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 msgr2=0x7faeb0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed0146ae0 0x7faeb0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 -- 172.21.15.32:0/194354159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0004500 msgr2=0x7faeb0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0004500 0x7faeb0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 -- 172.21.15.32:0/194354159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0003260 con 0x7faeb0003750 2022-08-01T23:01:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faec67fc700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0004500 0x7faeb0004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faeceffd700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb0003750 0x7faeb0003b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7faec000c000 tx=0x7faec0006df0).ready entity=mon.1 client_cookie=1a6810866289f71e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec000e830 con 0x7faeb0003750 2022-08-01T23:01:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec000e990 con 0x7faeb0003750 2022-08-01T23:01:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faece7fc700 1 -- 172.21.15.32:0/194354159 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec000ecf0 con 0x7faeb0003750 2022-08-01T23:01:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 -- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb0003750 msgr2=0x7faeb0003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 --2- 172.21.15.32:0/194354159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeb0003750 0x7faeb0003b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7faec000c000 tx=0x7faec0006df0).stop 2022-08-01T23:01:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faecf7fe700 1 -- 172.21.15.32:0/194354159 reap_dead start 2022-08-01T23:01:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 -- 172.21.15.32:0/194354159 shutdown_connections 2022-08-01T23:01:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 -- 172.21.15.32:0/194354159 >> 172.21.15.32:0/194354159 conn(0x7faed005c620 msgr2=0x7faed005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 -- 172.21.15.32:0/194354159 shutdown_connections 2022-08-01T23:01:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 -- 172.21.15.32:0/194354159 wait complete. 2022-08-01T23:01:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 Processor -- start 2022-08-01T23:01:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.053+0000 7faed57ae700 1 -- start start 2022-08-01T23:01:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 0x7faed00b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed013f600 0x7faed0152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed0152b20 0x7faed013de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00ae670 con 0x7faeb0003750 2022-08-01T23:01:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b75e0 con 0x7faed0152b20 2022-08-01T23:01:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed0156530 con 0x7faed013f600 2022-08-01T23:01:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 0x7faed00b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faecf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed0152b20 0x7faed013de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 0x7faed00b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44908/0 (socket says 172.21.15.32:44908) 2022-08-01T23:01:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faec67fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed013f600 0x7faed0152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 -- 172.21.15.32:0/851354153 learned_addr learned my addr 172.21.15.32:0/851354153 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 -- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed013f600 msgr2=0x7faed0152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed013f600 0x7faed0152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 -- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed0152b20 msgr2=0x7faed013de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed0152b20 0x7faed013de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 -- 172.21.15.32:0/851354153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00b7830 con 0x7faeb0003750 2022-08-01T23:01:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faec67fc700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed013f600 0x7faed0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeceffd700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 0x7faed00b2db0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7faec000c000 tx=0x7faec0019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec001b820 con 0x7faeb0003750 2022-08-01T23:01:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec0000e90 con 0x7faeb0003750 2022-08-01T23:01:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed0156790 con 0x7faeb0003750 2022-08-01T23:01:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec001bb70 con 0x7faeb0003750 2022-08-01T23:01:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00ad780 con 0x7faeb0003750 2022-08-01T23:01:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.057+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed0153310 con 0x7faeb0003750 2022-08-01T23:01:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.061+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faec001bcd0 con 0x7faeb0003750 2022-08-01T23:01:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.061+0000 7faeccff9700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faeb4053b00 0x7faeb4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.061+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faec006d650 con 0x7faeb0003750 2022-08-01T23:01:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.061+0000 7faec67fc700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faeb4053b00 0x7faeb4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.061+0000 7faec67fc700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faeb4053b00 0x7faeb4055fc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7faeb8005000 tx=0x7faed01431d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.065+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faec0044470 con 0x7faeb0003750 2022-08-01T23:01:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.189+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faed0156920 con 0x7faeb0003750 2022-08-01T23:01:38.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faeccff9700 1 -- 172.21.15.32:0/851354153 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7faec002a070 con 0x7faeb0003750 2022-08-01T23:01:38.192 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:38.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faeb4053b00 msgr2=0x7faeb4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faeb4053b00 0x7faeb4055fc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7faeb8005000 tx=0x7faed01431d0).stop 2022-08-01T23:01:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 msgr2=0x7faed00b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 0x7faed00b2db0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7faec000c000 tx=0x7faec0019000).stop 2022-08-01T23:01:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 shutdown_connections 2022-08-01T23:01:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faeb4053b00 0x7faeb4055fc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed0152b20 0x7faed013de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faed013f600 0x7faed0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 --2- 172.21.15.32:0/851354153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faeb0003750 0x7faed00b2db0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 >> 172.21.15.32:0/851354153 conn(0x7faed005c620 msgr2=0x7faed013fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 shutdown_connections 2022-08-01T23:01:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.193+0000 7faed57ae700 1 -- 172.21.15.32:0/851354153 wait complete. 2022-08-01T23:01:38.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:01:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 Processor -- start 2022-08-01T23:01:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- start start 2022-08-01T23:01:38.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a3c0b2050 0x7f3a3c0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0b05b0 con 0x7f3a3c0a7250 2022-08-01T23:01:38.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c0b2920 con 0x7f3a3c0b2050 2022-08-01T23:01:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0b07e0 con 0x7f3a3c0b0140 2022-08-01T23:01:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a3c0b2050 0x7f3a3c0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a3c0b2050 0x7f3a3c0b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44914/0 (socket says 172.21.15.32:44914) 2022-08-01T23:01:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 -- 172.21.15.32:0/3764703746 learned_addr learned my addr 172.21.15.32:0/3764703746 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3b7fe700 1 --2- 172.21.15.32:0/3764703746 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 -- 172.21.15.32:0/3764703746 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 msgr2=0x7f3a3c0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 --2- 172.21.15.32:0/3764703746 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 -- 172.21.15.32:0/3764703746 >> v1:172.21.15.55:6789/0 conn(0x7f3a3c0a7250 legacy=0x7f3a3c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 -- 172.21.15.32:0/3764703746 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0b0a80 con 0x7f3a3c0b2050 2022-08-01T23:01:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3bfff700 1 --2- 172.21.15.32:0/3764703746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c0b1460 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f3a2c002f70 tx=0x7f3a2c006eb0).ready entity=mon.0 client_cookie=a0cbb9173a9cf7c6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3affd700 1 -- 172.21.15.32:0/3764703746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a2c00d570 con 0x7f3a3c0b2050 2022-08-01T23:01:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3affd700 1 -- 172.21.15.32:0/3764703746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c000e90 con 0x7f3a3c0b2050 2022-08-01T23:01:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a3affd700 1 -- 172.21.15.32:0/3764703746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a2c00d900 con 0x7f3a3c0b2050 2022-08-01T23:01:38.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- 172.21.15.32:0/3764703746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 msgr2=0x7f3a3c0b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3764703746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c0b1460 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f3a2c002f70 tx=0x7f3a2c006eb0).stop 2022-08-01T23:01:38.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- 172.21.15.32:0/3764703746 shutdown_connections 2022-08-01T23:01:38.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3764703746 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3764703746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c0b1460 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.313+0000 7f3a4280b700 1 -- 172.21.15.32:0/3764703746 >> 172.21.15.32:0/3764703746 conn(0x7f3a3c05f840 msgr2=0x7f3a3c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 -- 172.21.15.32:0/3764703746 shutdown_connections 2022-08-01T23:01:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 -- 172.21.15.32:0/3764703746 wait complete. 2022-08-01T23:01:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 Processor -- start 2022-08-01T23:01:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 -- start start 2022-08-01T23:01:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c0a7250 0x7f3a3c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c157c90 con 0x7f3a3c0b2050 2022-08-01T23:01:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0b4130 con 0x7f3a3c0a7250 2022-08-01T23:01:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a4280b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1579f0 con 0x7f3a3c0b0140 2022-08-01T23:01:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c0a7250 0x7f3a3c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a40da8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c0a7250 0x7f3a3c0b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:54706/0 (socket says 172.21.15.32:54706) 2022-08-01T23:01:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a3bfff700 1 -- 172.21.15.32:0/3026073576 learned_addr learned my addr 172.21.15.32:0/3026073576 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.317+0000 7f3a3b7fe700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a40da8700 1 -- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 msgr2=0x7f3a3c0abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a40da8700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a40da8700 1 -- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c0a7250 msgr2=0x7f3a3c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a40da8700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c0a7250 0x7f3a3c0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a40da8700 1 -- 172.21.15.32:0/3026073576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c00b1e0 con 0x7f3a3c0b2050 2022-08-01T23:01:38.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a40da8700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c148eb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f3a3c06a280 tx=0x7f3a34007eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3400f470 con 0x7f3a3c0b2050 2022-08-01T23:01:38.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3400f5d0 con 0x7f3a3c0b2050 2022-08-01T23:01:38.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c147260 con 0x7f3a3c0b2050 2022-08-01T23:01:38.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3400f930 con 0x7f3a3c0b2050 2022-08-01T23:01:38.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0ac1d0 con 0x7f3a3c0b2050 2022-08-01T23:01:38.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a34010070 con 0x7f3a3c0b2050 2022-08-01T23:01:38.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c1496e0 con 0x7f3a3c0b2050 2022-08-01T23:01:38.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a397fa700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a1c053a10 0x7f3a1c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3a3406b4b0 con 0x7f3a3c0b2050 2022-08-01T23:01:38.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.321+0000 7f3a3bfff700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a1c053a10 0x7f3a1c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.325+0000 7f3a3bfff700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a1c053a10 0x7f3a1c055ed0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3a2c01c000 tx=0x7f3a2c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.332+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a34042270 con 0x7f3a3c0b2050 2022-08-01T23:01:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.456+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a3c0b0d20 con 0x7f3a3c0b2050 2022-08-01T23:01:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.456+0000 7f3a397fa700 1 -- 172.21.15.32:0/3026073576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3a34045890 con 0x7f3a3c0b2050 2022-08-01T23:01:38.457 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a1c053a10 msgr2=0x7f3a1c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a1c053a10 0x7f3a1c055ed0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3a2c01c000 tx=0x7f3a2c015000).stop 2022-08-01T23:01:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 msgr2=0x7f3a3c148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c148eb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f3a3c06a280 tx=0x7f3a34007eb0).stop 2022-08-01T23:01:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 shutdown_connections 2022-08-01T23:01:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a1c053a10 0x7f3a1c055ed0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a3c0b2050 0x7f3a3c148eb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a3c0b0140 0x7f3a3c0abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 --2- 172.21.15.32:0/3026073576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c0a7250 0x7f3a3c0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 >> 172.21.15.32:0/3026073576 conn(0x7f3a3c05f840 msgr2=0x7f3a3c13e030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 shutdown_connections 2022-08-01T23:01:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.460+0000 7f3a4280b700 1 -- 172.21.15.32:0/3026073576 wait complete. 2022-08-01T23:01:38.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:01:38.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 Processor -- start 2022-08-01T23:01:38.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 -- start start 2022-08-01T23:01:38.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 0x7f02b8144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f02b80655b0 0x7f02b8144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b81454e0 con 0x7f02b8060290 2022-08-01T23:01:38.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b8065e10 con 0x7f02b80655b0 2022-08-01T23:01:38.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bf294700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8060670 con 0x7f02b81445d0 2022-08-01T23:01:38.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bd831700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f02b80655b0 0x7f02b8144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 0x7f02b8144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 0x7f02b8144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:56668/0 (socket says 172.21.15.32:56668) 2022-08-01T23:01:38.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 -- 172.21.15.32:0/2090355221 learned_addr learned my addr 172.21.15.32:0/2090355221 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 -- 172.21.15.32:0/2090355221 >> v2:172.21.15.55:3300/0 conn(0x7f02b80655b0 msgr2=0x7f02b8144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 --2- 172.21.15.32:0/2090355221 >> v2:172.21.15.55:3300/0 conn(0x7f02b80655b0 0x7f02b8144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 -- 172.21.15.32:0/2090355221 >> v1:172.21.15.55:6789/0 conn(0x7f02b8060290 legacy=0x7f02b8064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 -- 172.21.15.32:0/2090355221 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80b2540 con 0x7f02b81445d0 2022-08-01T23:01:38.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.584+0000 7f02bc82f700 1 --2- 172.21.15.32:0/2090355221 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 0x7f02b8144b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f02b40032f0 tx=0x7f02b4009290).ready entity=mon.2 client_cookie=a1b7e81adaf15a3b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02abfff700 1 -- 172.21.15.32:0/2090355221 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b400b8a0 con 0x7f02b81445d0 2022-08-01T23:01:38.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02abfff700 1 -- 172.21.15.32:0/2090355221 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b400ba00 con 0x7f02b81445d0 2022-08-01T23:01:38.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02abfff700 1 -- 172.21.15.32:0/2090355221 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b400bb70 con 0x7f02b81445d0 2022-08-01T23:01:38.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 -- 172.21.15.32:0/2090355221 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 msgr2=0x7f02b8144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 --2- 172.21.15.32:0/2090355221 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 0x7f02b8144b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f02b40032f0 tx=0x7f02b4009290).stop 2022-08-01T23:01:38.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 -- 172.21.15.32:0/2090355221 shutdown_connections 2022-08-01T23:01:38.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 --2- 172.21.15.32:0/2090355221 >> v2:172.21.15.55:3300/0 conn(0x7f02b80655b0 0x7f02b8144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 --2- 172.21.15.32:0/2090355221 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b81445d0 0x7f02b8144b00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 -- 172.21.15.32:0/2090355221 >> 172.21.15.32:0/2090355221 conn(0x7f02b805d0f0 msgr2=0x7f02b805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 -- 172.21.15.32:0/2090355221 shutdown_connections 2022-08-01T23:01:38.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 -- 172.21.15.32:0/2090355221 wait complete. 2022-08-01T23:01:38.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 Processor -- start 2022-08-01T23:01:38.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 -- start start 2022-08-01T23:01:38.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02b8060290 0x7f02b813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.588+0000 7f02bf294700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b80655b0 0x7f02b80b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 0x7f02b80b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8151670 con 0x7f02b80ae8b0 2022-08-01T23:01:38.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8151140 con 0x7f02b8060290 2022-08-01T23:01:38.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b81513d0 con 0x7f02b80655b0 2022-08-01T23:01:38.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 0x7f02b80b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 0x7f02b80b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44938/0 (socket says 172.21.15.32:44938) 2022-08-01T23:01:38.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 -- 172.21.15.32:0/3096316764 learned_addr learned my addr 172.21.15.32:0/3096316764 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bc82f700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b80655b0 0x7f02b80b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd030700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02b8060290 0x7f02b813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 -- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b80655b0 msgr2=0x7f02b80b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b80655b0 0x7f02b80b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 -- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02b8060290 msgr2=0x7f02b813f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02b8060290 0x7f02b813f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 -- 172.21.15.32:0/3096316764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4009530 con 0x7f02b80ae8b0 2022-08-01T23:01:38.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd831700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 0x7f02b80b0880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f02b000c000 tx=0x7f02b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b0016070 con 0x7f02b80ae8b0 2022-08-01T23:01:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0011750 con 0x7f02b80ae8b0 2022-08-01T23:01:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b813bd70 con 0x7f02b80ae8b0 2022-08-01T23:01:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b0011ab0 con 0x7f02b80ae8b0 2022-08-01T23:01:38.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b80b0d70 con 0x7f02b80ae8b0 2022-08-01T23:01:38.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f02b0011c10 con 0x7f02b80ae8b0 2022-08-01T23:01:38.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b81400e0 con 0x7f02b80ae8b0 2022-08-01T23:01:38.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02aa7fc700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02a0053a70 0x7f02a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f02b006bfc0 con 0x7f02b80ae8b0 2022-08-01T23:01:38.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.592+0000 7f02bd030700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02a0053a70 0x7f02a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.596+0000 7f02bd030700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02a0053a70 0x7f02a0055f30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f02ac011000 tx=0x7f02ac007c20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.604+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02b0042d80 con 0x7f02b80ae8b0 2022-08-01T23:01:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.728+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02b813bf00 con 0x7f02b80ae8b0 2022-08-01T23:01:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.728+0000 7f02aa7fc700 1 -- 172.21.15.32:0/3096316764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7f02b00463a0 con 0x7f02b80ae8b0 2022-08-01T23:01:38.732 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02a0053a70 msgr2=0x7f02a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02a0053a70 0x7f02a0055f30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f02ac011000 tx=0x7f02ac007c20).stop 2022-08-01T23:01:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 msgr2=0x7f02b80b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 0x7f02b80b0880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f02b000c000 tx=0x7f02b0006df0).stop 2022-08-01T23:01:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 shutdown_connections 2022-08-01T23:01:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f02a0053a70 0x7f02a0055f30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02b80ae8b0 0x7f02b80b0880 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f02b80655b0 0x7f02b80b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 --2- 172.21.15.32:0/3096316764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02b8060290 0x7f02b813f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 >> 172.21.15.32:0/3096316764 conn(0x7f02b805d0f0 msgr2=0x7f02b8143e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 shutdown_connections 2022-08-01T23:01:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.732+0000 7f02bf294700 1 -- 172.21.15.32:0/3096316764 wait complete. 2022-08-01T23:01:38.745 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:01:38.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:38.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.844+0000 7f1cec290700 1 Processor -- start 2022-08-01T23:01:38.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- start start 2022-08-01T23:01:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ce40b1f40 0x7f1ce40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce41448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce40b04e0 con 0x7f1ce4148530 2022-08-01T23:01:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce40a9250 con 0x7f1ce40b1f40 2022-08-01T23:01:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce40b0780 con 0x7f1ce40a7250 2022-08-01T23:01:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ce40b1f40 0x7f1ce40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ce40b1f40 0x7f1ce40b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44942/0 (socket says 172.21.15.32:44942) 2022-08-01T23:01:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 -- 172.21.15.32:0/479225389 learned_addr learned my addr 172.21.15.32:0/479225389 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea82d700 1 --2- 172.21.15.32:0/479225389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce41448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 -- 172.21.15.32:0/479225389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 msgr2=0x7f1ce41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 --2- 172.21.15.32:0/479225389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce41448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 -- 172.21.15.32:0/479225389 >> v1:172.21.15.55:6789/0 conn(0x7f1ce4148530 legacy=0x7f1ce40b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 -- 172.21.15.32:0/479225389 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce40b0a20 con 0x7f1ce40b1f40 2022-08-01T23:01:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1ce902a700 1 -- 172.21.15.32:0/479225389 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245178366 0 0) 0x7f1ce40b04e0 con 0x7f1ce4148530 2022-08-01T23:01:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cea02c700 1 --2- 172.21.15.32:0/479225389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce40b1f40 0x7f1ce40b1160 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f1ce41478c0 tx=0x7f1ce0006eb0).ready entity=mon.0 client_cookie=f22e3c2f2a2d3d23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1ce902a700 1 -- 172.21.15.32:0/479225389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ce000d5c0 con 0x7f1ce40b1f40 2022-08-01T23:01:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1ce902a700 1 -- 172.21.15.32:0/479225389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce0000e90 con 0x7f1ce40b1f40 2022-08-01T23:01:38.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1ce902a700 1 -- 172.21.15.32:0/479225389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ce000d910 con 0x7f1ce40b1f40 2022-08-01T23:01:38.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- 172.21.15.32:0/479225389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce40b1f40 msgr2=0x7f1ce40b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 --2- 172.21.15.32:0/479225389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce40b1f40 0x7f1ce40b1160 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f1ce41478c0 tx=0x7f1ce0006eb0).stop 2022-08-01T23:01:38.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- 172.21.15.32:0/479225389 shutdown_connections 2022-08-01T23:01:38.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 --2- 172.21.15.32:0/479225389 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce41448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 --2- 172.21.15.32:0/479225389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce40b1f40 0x7f1ce40b1160 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- 172.21.15.32:0/479225389 >> 172.21.15.32:0/479225389 conn(0x7f1ce405f840 msgr2=0x7f1ce405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- 172.21.15.32:0/479225389 shutdown_connections 2022-08-01T23:01:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.848+0000 7f1cec290700 1 -- 172.21.15.32:0/479225389 wait complete. 2022-08-01T23:01:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 Processor -- start 2022-08-01T23:01:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 -- start start 2022-08-01T23:01:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce40ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce40b1f40 0x7f1ce4149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 0x7f1ce4148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce414a850 con 0x7f1ce4148530 2022-08-01T23:01:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce414b780 con 0x7f1ce40b1f40 2022-08-01T23:01:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce414ba20 con 0x7f1ce40a7250 2022-08-01T23:01:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 0x7f1ce4148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 0x7f1ce4148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44948/0 (socket says 172.21.15.32:44948) 2022-08-01T23:01:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 -- 172.21.15.32:0/3966656124 learned_addr learned my addr 172.21.15.32:0/3966656124 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea02c700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce40ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1ce982b700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce40b1f40 0x7f1ce4149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 -- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 msgr2=0x7f1ce40ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce40ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 -- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce40b1f40 msgr2=0x7f1ce4149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce40b1f40 0x7f1ce4149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 -- 172.21.15.32:0/3966656124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce000b1e0 con 0x7f1ce4148530 2022-08-01T23:01:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cea82d700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 0x7f1ce4148e20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f1cdc006e00 tx=0x7f1cdc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cdc00c670 con 0x7f1ce4148530 2022-08-01T23:01:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cdc00c7d0 con 0x7f1ce4148530 2022-08-01T23:01:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce40aee10 con 0x7f1ce4148530 2022-08-01T23:01:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cdc00cb30 con 0x7f1ce4148530 2022-08-01T23:01:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.852+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce414a450 con 0x7f1ce4148530 2022-08-01T23:01:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.856+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce40574e0 con 0x7f1ce4148530 2022-08-01T23:01:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.856+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1cdc00cc90 con 0x7f1ce4148530 2022-08-01T23:01:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.856+0000 7f1cd37fe700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1cc40539e0 0x7f1cc4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.856+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1cdc06c320 con 0x7f1ce4148530 2022-08-01T23:01:38.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.856+0000 7f1cea02c700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1cc40539e0 0x7f1cc4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:38.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.856+0000 7f1cea02c700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1cc40539e0 0x7f1cc4055ea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f1ce001c000 tx=0x7f1ce0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:38.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:38.868+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1cdc043040 con 0x7f1ce4148530 2022-08-01T23:01:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.028+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ce40447e0 con 0x7f1ce4148530 2022-08-01T23:01:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.028+0000 7f1cd37fe700 1 -- 172.21.15.32:0/3966656124 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f1cdc046660 con 0x7f1ce4148530 2022-08-01T23:01:39.031 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:39.031 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:01:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1cc40539e0 msgr2=0x7f1cc4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1cc40539e0 0x7f1cc4055ea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f1ce001c000 tx=0x7f1ce0015000).stop 2022-08-01T23:01:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 msgr2=0x7f1ce4148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 0x7f1ce4148e20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f1cdc006e00 tx=0x7f1cdc0091d0).stop 2022-08-01T23:01:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 shutdown_connections 2022-08-01T23:01:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1cc40539e0 0x7f1cc4055ea0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4148530 0x7f1ce4148e20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce40b1f40 0x7f1ce4149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 --2- 172.21.15.32:0/3966656124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1ce40a7250 0x7f1ce40ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 >> 172.21.15.32:0/3966656124 conn(0x7f1ce405f840 msgr2=0x7f1ce40b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 shutdown_connections 2022-08-01T23:01:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:39.032+0000 7f1cec290700 1 -- 172.21.15.32:0/3966656124 wait complete. 2022-08-01T23:01:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:01:39.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:01:34.459266+0000', 'modified': '2022-08-01T23:01:35.461964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6258}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6258': {'gid': 6258, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1927724642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1927724642}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1927724642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:01:39.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:01:39.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:01:39.464 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6258,"want_state":"up:active","state":"up:active","rank_uptime":4.9946566099999998,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":76,"uptime":9.3124080500000002} 2022-08-01T23:01:39.464 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6258, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 76, "rank_uptime": 4.99465661, "state": "up:active", "uptime": 9.31240805, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:01:39.465 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:01:39.465 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:01:39.466 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:01:39.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:39.467 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:39.475 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:01:39.493 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:01:39.493 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:01:39.550 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:01:39.551 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:01:39.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:39.552 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:01:39.557 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:01:39.558 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:39.601 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:01:39.618 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:01:39.619 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:01:39.630 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:01:39.630 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:01:39.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:39.632 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:01:39.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:01:39.678+0000 7f6e49c3a080 -1 init, newargv = 0x5638fd2f6ed0 newargc=9ceph-fuse[22973]: starting ceph client 2022-08-01T23:01:39.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr: 2022-08-01T23:01:39.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[22973]: starting fuse 2022-08-01T23:01:40.738 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:01:40.758 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:01:40.759 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:01:40.815 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:01:40.816 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:01:40.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:40.817 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:01:40.822 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:01:40.823 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:01:40.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:40.824 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:01:40.824 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:01:40.825 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:01:40.825 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:01:40.825 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:01:40.826 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:40.826 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:01:40.827 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:01:40.827 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:01:40.828 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:40.828 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:01:40.829 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:01:40.829 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:01:40.830 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:01:40.830 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:40.830 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:01:40.831 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:01:40.831 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:01:40.832 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:01:40.832 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:01:40.832 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:40.833 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:01:40.833 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:01:40.875 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.22973.asok 2022-08-01T23:01:40.879 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22973.asok 2022-08-01T23:01:40.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:40.880 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22973.asok status 2022-08-01T23:01:40.966 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:01:40.967 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:01:40.967 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:01:40.967 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:01:40.967 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:01:40.968 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:01:40.968 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:01:40.968 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "22973", 2022-08-01T23:01:40.968 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:01:40.968 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:01:40.969 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:01:40.969 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:01:40.969 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 5819, 2022-08-01T23:01:40.969 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:01:40.970 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:01:40.970 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:01:40.970 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 5819 2022-08-01T23:01:40.970 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:01:40.970 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:01:40.971 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:01:40.971 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:01:40.971 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 438018148 2022-08-01T23:01:40.971 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:01:40.972 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:01:40.972 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:01:40.972 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:01:40.972 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:01:40.972 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 438018148 2022-08-01T23:01:40.973 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:01:40.973 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.5819 172.21.15.130:0/438018148", 2022-08-01T23:01:40.973 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/438018148", 2022-08-01T23:01:40.973 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:01:40.974 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 33, 2022-08-01T23:01:40.974 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 76, 2022-08-01T23:01:40.974 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:01:40.974 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:01:40.974 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:01:40.975 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:01:40.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:40.976 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:40.977 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:40.983 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:01:40.985 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:01:40.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:40.985 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:41.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:01:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f6480c64700 1 Processor -- start 2022-08-01T23:01:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f6480c64700 1 -- start start 2022-08-01T23:01:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f6480c64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f647c14b950 0x7f647c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f6480c64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c0b60e0 con 0x7f647c0b6420 2022-08-01T23:01:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f6480c64700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c0b2100 con 0x7f647c0b41f0 2022-08-01T23:01:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f6480c64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c0b23a0 con 0x7f647c14b950 2022-08-01T23:01:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f647c0b6420 0x7f647c061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60728/0 (socket says 172.21.15.32:60728) 2022-08-01T23:01:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647b7fe700 1 -- 172.21.15.32:0/1854934445 learned_addr learned my addr 172.21.15.32:0/1854934445 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647bfff700 1 --2- 172.21.15.32:0/1854934445 >> v2:172.21.15.32:3300/0 conn(0x7f647c14b950 0x7f647c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726754363 0 0) 0x7f646c001110 con 0x7f647c0b6420 2022-08-01T23:01:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645c003260 con 0x7f647c0b6420 2022-08-01T23:01:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085948910 0 0) 0x7f647c0b2100 con 0x7f647c0b41f0 2022-08-01T23:01:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f646c001110 con 0x7f647c0b41f0 2022-08-01T23:01:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 932933000 0 0) 0x7f645c003260 con 0x7f647c0b6420 2022-08-01T23:01:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647bfff700 1 -- 172.21.15.32:0/1854934445 >> v1:172.21.15.55:6789/0 conn(0x7f647c0b41f0 legacy=0x7f647c061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647bfff700 1 -- 172.21.15.32:0/1854934445 >> v1:172.21.15.32:6789/0 conn(0x7f647c0b6420 legacy=0x7f647c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647bfff700 1 -- 172.21.15.32:0/1854934445 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c0b2640 con 0x7f647c14b950 2022-08-01T23:01:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647bfff700 1 --2- 172.21.15.32:0/1854934445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c14b950 0x7f647c14dd40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f646c0012a0 tx=0x7f64740091d0).ready entity=mon.0 client_cookie=ce4555009a8d37b7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.112+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c0017f0 con 0x7f647c14b950 2022-08-01T23:01:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647400f510 con 0x7f647c14b950 2022-08-01T23:01:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f647a7fc700 1 -- 172.21.15.32:0/1854934445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647400f680 con 0x7f647c14b950 2022-08-01T23:01:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 -- 172.21.15.32:0/1854934445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c14b950 msgr2=0x7f647c14dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 --2- 172.21.15.32:0/1854934445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c14b950 0x7f647c14dd40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f646c0012a0 tx=0x7f64740091d0).stop 2022-08-01T23:01:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 -- 172.21.15.32:0/1854934445 shutdown_connections 2022-08-01T23:01:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 --2- 172.21.15.32:0/1854934445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c14b950 0x7f647c14dd40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 -- 172.21.15.32:0/1854934445 >> 172.21.15.32:0/1854934445 conn(0x7f647c05c620 msgr2=0x7f647c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 -- 172.21.15.32:0/1854934445 shutdown_connections 2022-08-01T23:01:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 -- 172.21.15.32:0/1854934445 wait complete. 2022-08-01T23:01:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 Processor -- start 2022-08-01T23:01:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.116+0000 7f6480c64700 1 -- start start 2022-08-01T23:01:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 0x7f647c143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f647c0b6420 0x7f647c142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f647c14b950 0x7f647c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b00e0 con 0x7f647c0b41f0 2022-08-01T23:01:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b0890 con 0x7f647c0b6420 2022-08-01T23:01:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c141ce0 con 0x7f647c14b950 2022-08-01T23:01:41.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 0x7f647c143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 0x7f647c143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44960/0 (socket says 172.21.15.32:44960) 2022-08-01T23:01:41.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 -- 172.21.15.32:0/1402804187 learned_addr learned my addr 172.21.15.32:0/1402804187 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:41.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647affd700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f647c0b6420 0x7f647c142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647bfff700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f647c14b950 0x7f647c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 -- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f647c14b950 msgr2=0x7f647c0ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f647c14b950 0x7f647c0ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 -- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f647c0b6420 msgr2=0x7f647c142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f647c0b6420 0x7f647c142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 -- 172.21.15.32:0/1402804187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009470 con 0x7f647c0b41f0 2022-08-01T23:01:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647bfff700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f647c14b950 0x7f647c0ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f647b7fe700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 0x7f647c143e40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f647c0af6f0 tx=0x7f646c001c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c007170 con 0x7f647c0b41f0 2022-08-01T23:01:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c0072d0 con 0x7f647c0b41f0 2022-08-01T23:01:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c0afe40 con 0x7f647c0b41f0 2022-08-01T23:01:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c0049f0 con 0x7f647c0b41f0 2022-08-01T23:01:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.120+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c142630 con 0x7f647c0b41f0 2022-08-01T23:01:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.124+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c141b20 con 0x7f647c0b41f0 2022-08-01T23:01:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.124+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f646c010070 con 0x7f647c0b41f0 2022-08-01T23:01:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.124+0000 7f6478ff9700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64600539e0 0x7f6460055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.124+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f646c06be60 con 0x7f647c0b41f0 2022-08-01T23:01:41.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.124+0000 7f647affd700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64600539e0 0x7f6460055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.124+0000 7f647affd700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64600539e0 0x7f6460055ea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f646400d000 tx=0x7f6464007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.128+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f646c042b10 con 0x7f647c0b41f0 2022-08-01T23:01:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.256+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f647c0b60e0 con 0x7f647c0b41f0 2022-08-01T23:01:41.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6478ff9700 1 -- 172.21.15.32:0/1402804187 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f647c0b60e0 con 0x7f647c0b41f0 2022-08-01T23:01:41.261 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:41.261 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":76,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:01:34.459122+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:01:32.889134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T23:01:33.885837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:01:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64600539e0 msgr2=0x7f6460055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64600539e0 0x7f6460055ea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f646400d000 tx=0x7f6464007a20).stop 2022-08-01T23:01:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 msgr2=0x7f647c143e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 0x7f647c143e40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f647c0af6f0 tx=0x7f646c001c70).stop 2022-08-01T23:01:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 shutdown_connections 2022-08-01T23:01:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64600539e0 0x7f6460055ea0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f647c14b950 0x7f647c0ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f647c0b6420 0x7f647c142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 --2- 172.21.15.32:0/1402804187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c0b41f0 0x7f647c143e40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 >> 172.21.15.32:0/1402804187 conn(0x7f647c05c620 msgr2=0x7f647c149d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 shutdown_connections 2022-08-01T23:01:41.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.260+0000 7f6480c64700 1 -- 172.21.15.32:0/1402804187 wait complete. 2022-08-01T23:01:41.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:01:32.889134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:01:33.885837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:01:41.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:01:41.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.388+0000 7f9a529a9700 1 Processor -- start 2022-08-01T23:01:41.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.388+0000 7f9a529a9700 1 -- start start 2022-08-01T23:01:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a4c0afd60 0x7f9a4c0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0a7250 con 0x7f9a4c0b2710 2022-08-01T23:01:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c0a7480 con 0x7f9a4c0afd60 2022-08-01T23:01:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0a7720 con 0x7f9a4c0b01a0 2022-08-01T23:01:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a521a8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a4c0afd60 0x7f9a4c0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a519a7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a4c0b2710 0x7f9a4c14afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60742/0 (socket says 172.21.15.32:60742) 2022-08-01T23:01:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a519a7700 1 -- 172.21.15.32:0/45728446 learned_addr learned my addr 172.21.15.32:0/45728446 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a511a6700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a521a8700 1 -- 172.21.15.32:0/45728446 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 msgr2=0x7f9a4c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a521a8700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a521a8700 1 -- 172.21.15.32:0/45728446 >> v1:172.21.15.32:6789/0 conn(0x7f9a4c0b2710 legacy=0x7f9a4c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a509a5700 1 -- 172.21.15.32:0/45728446 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653666074 0 0) 0x7f9a4c0a7250 con 0x7f9a4c0b2710 2022-08-01T23:01:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a521a8700 1 -- 172.21.15.32:0/45728446 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c0a79c0 con 0x7f9a4c0afd60 2022-08-01T23:01:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a511a6700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a521a8700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c0a8a30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f9a44002f70 tx=0x7f9a44006df0).ready entity=mon.0 client_cookie=bd9a23622eac71a6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a509a5700 1 -- 172.21.15.32:0/45728446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4400d570 con 0x7f9a4c0afd60 2022-08-01T23:01:41.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a509a5700 1 -- 172.21.15.32:0/45728446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a44000e90 con 0x7f9a4c0afd60 2022-08-01T23:01:41.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a509a5700 1 -- 172.21.15.32:0/45728446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4400d8c0 con 0x7f9a4c0afd60 2022-08-01T23:01:41.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- 172.21.15.32:0/45728446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 msgr2=0x7f9a4c0a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c0a8a30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f9a44002f70 tx=0x7f9a44006df0).stop 2022-08-01T23:01:41.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- 172.21.15.32:0/45728446 shutdown_connections 2022-08-01T23:01:41.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c0a8a30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 --2- 172.21.15.32:0/45728446 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- 172.21.15.32:0/45728446 >> 172.21.15.32:0/45728446 conn(0x7f9a4c060310 msgr2=0x7f9a4c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- 172.21.15.32:0/45728446 shutdown_connections 2022-08-01T23:01:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 -- 172.21.15.32:0/45728446 wait complete. 2022-08-01T23:01:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.392+0000 7f9a529a9700 1 Processor -- start 2022-08-01T23:01:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 -- start start 2022-08-01T23:01:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c14a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c14cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c0b2710 0x7f9a4c14d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0b5670 con 0x7f9a4c0afd60 2022-08-01T23:01:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1574a0 con 0x7f9a4c0b2710 2022-08-01T23:01:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0aba70 con 0x7f9a4c0b01a0 2022-08-01T23:01:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c14a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c14a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44974/0 (socket says 172.21.15.32:44974) 2022-08-01T23:01:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 -- 172.21.15.32:0/848860713 learned_addr learned my addr 172.21.15.32:0/848860713 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a511a6700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c14cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a521a8700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c0b2710 0x7f9a4c14d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 -- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 msgr2=0x7f9a4c14cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c14cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 -- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c0b2710 msgr2=0x7f9a4c14d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c0b2710 0x7f9a4c14d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 -- 172.21.15.32:0/848860713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4400b180 con 0x7f9a4c0afd60 2022-08-01T23:01:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a519a7700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c14a460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f9a3c000ed0 tx=0x7f9a3c007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3c0070d0 con 0x7f9a4c0afd60 2022-08-01T23:01:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c007230 con 0x7f9a4c0afd60 2022-08-01T23:01:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a3c005610 con 0x7f9a4c0afd60 2022-08-01T23:01:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c0aae80 con 0x7f9a4c0afd60 2022-08-01T23:01:41.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c14a950 con 0x7f9a4c0afd60 2022-08-01T23:01:41.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.396+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a3c010070 con 0x7f9a4c0afd60 2022-08-01T23:01:41.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.400+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c0b3810 con 0x7f9a4c0afd60 2022-08-01T23:01:41.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.400+0000 7f9a3affd700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053b00 0x7f9a2c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.400+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9a3c06b650 con 0x7f9a4c0afd60 2022-08-01T23:01:41.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.400+0000 7f9a511a6700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053b00 0x7f9a2c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.400+0000 7f9a511a6700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053b00 0x7f9a2c055fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9a4800d000 tx=0x7f9a48008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.404+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a3c042410 con 0x7f9a4c0afd60 2022-08-01T23:01:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9a4c0574e0 con 0x7f9a4c0afd60 2022-08-01T23:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a3affd700 1 -- 172.21.15.32:0/848860713 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9a3c045a30 con 0x7f9a4c0afd60 2022-08-01T23:01:41.530 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:01:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053b00 msgr2=0x7f9a2c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053b00 0x7f9a2c055fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9a4800d000 tx=0x7f9a48008040).stop 2022-08-01T23:01:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 msgr2=0x7f9a4c14a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c14a460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f9a3c000ed0 tx=0x7f9a3c007ce0).stop 2022-08-01T23:01:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 shutdown_connections 2022-08-01T23:01:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053b00 0x7f9a2c055fc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a4c0b2710 0x7f9a4c14d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a4c0b01a0 0x7f9a4c14cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 --2- 172.21.15.32:0/848860713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4c0afd60 0x7f9a4c14a460 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.528+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 >> 172.21.15.32:0/848860713 conn(0x7f9a4c060310 msgr2=0x7f9a4c0af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.532+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 shutdown_connections 2022-08-01T23:01:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.532+0000 7f9a529a9700 1 -- 172.21.15.32:0/848860713 wait complete. 2022-08-01T23:01:41.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:01:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 Processor -- start 2022-08-01T23:01:41.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- start start 2022-08-01T23:01:41.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70180a9760 0x7f70180a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f70180acbe0 0x7f70180b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a8bb0 0x7f70180a9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7018139d50 con 0x7f70180a9760 2022-08-01T23:01:41.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7018140690 con 0x7f70180acbe0 2022-08-01T23:01:41.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180b02c0 con 0x7f70180a8bb0 2022-08-01T23:01:41.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70180a9760 0x7f70180a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70180a9760 0x7f70180a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44984/0 (socket says 172.21.15.32:44984) 2022-08-01T23:01:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f7017fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f70180acbe0 0x7f70180b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 -- 172.21.15.32:0/1905304010 learned_addr learned my addr 172.21.15.32:0/1905304010 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701d28a700 1 --2- 172.21.15.32:0/1905304010 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a8bb0 0x7f70180a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 -- 172.21.15.32:0/1905304010 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a8bb0 msgr2=0x7f70180a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 --2- 172.21.15.32:0/1905304010 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a8bb0 0x7f70180a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 -- 172.21.15.32:0/1905304010 >> v2:172.21.15.55:3300/0 conn(0x7f70180acbe0 msgr2=0x7f70180b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 --2- 172.21.15.32:0/1905304010 >> v2:172.21.15.55:3300/0 conn(0x7f70180acbe0 0x7f70180b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 -- 172.21.15.32:0/1905304010 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180b0560 con 0x7f70180a9760 2022-08-01T23:01:41.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701ca89700 1 --2- 172.21.15.32:0/1905304010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180a9760 0x7f70180a9b40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f7008002f70 tx=0x7f7008006eb0).ready entity=mon.0 client_cookie=e0fcefa277c1257 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f70177fe700 1 -- 172.21.15.32:0/1905304010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700800d570 con 0x7f70180a9760 2022-08-01T23:01:41.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f70177fe700 1 -- 172.21.15.32:0/1905304010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7008000e90 con 0x7f70180a9760 2022-08-01T23:01:41.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f70177fe700 1 -- 172.21.15.32:0/1905304010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700800d900 con 0x7f70180a9760 2022-08-01T23:01:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- 172.21.15.32:0/1905304010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180a9760 msgr2=0x7f70180a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- 172.21.15.32:0/1905304010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180a9760 0x7f70180a9b40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f7008002f70 tx=0x7f7008006eb0).stop 2022-08-01T23:01:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- 172.21.15.32:0/1905304010 shutdown_connections 2022-08-01T23:01:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- 172.21.15.32:0/1905304010 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a8bb0 0x7f70180a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- 172.21.15.32:0/1905304010 >> v2:172.21.15.55:3300/0 conn(0x7f70180acbe0 0x7f70180b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 --2- 172.21.15.32:0/1905304010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180a9760 0x7f70180a9b40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- 172.21.15.32:0/1905304010 >> 172.21.15.32:0/1905304010 conn(0x7f701805f840 msgr2=0x7f701805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- 172.21.15.32:0/1905304010 shutdown_connections 2022-08-01T23:01:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.648+0000 7f701eced700 1 -- 172.21.15.32:0/1905304010 wait complete. 2022-08-01T23:01:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 Processor -- start 2022-08-01T23:01:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 -- start start 2022-08-01T23:01:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70180a8bb0 0x7f7018144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a9760 0x7f70180ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 0x7f7018133920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018141880 con 0x7f70180acbe0 2022-08-01T23:01:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018144240 con 0x7f70180a8bb0 2022-08-01T23:01:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701eced700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7018142740 con 0x7f70180a9760 2022-08-01T23:01:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 0x7f7018133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 0x7f7018133920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44992/0 (socket says 172.21.15.32:44992) 2022-08-01T23:01:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 -- 172.21.15.32:0/1668732885 learned_addr learned my addr 172.21.15.32:0/1668732885 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f7017fff700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a9760 0x7f70180ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701ca89700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70180a8bb0 0x7f7018144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 -- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a9760 msgr2=0x7f70180ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a9760 0x7f70180ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 -- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70180a8bb0 msgr2=0x7f7018144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70180a8bb0 0x7f7018144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 -- 172.21.15.32:0/1668732885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700800b1e0 con 0x7f70180acbe0 2022-08-01T23:01:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.652+0000 7f701d28a700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 0x7f7018133920 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f7010006e00 tx=0x7f701000b470).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701001a470 con 0x7f70180acbe0 2022-08-01T23:01:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701001a5d0 con 0x7f70180acbe0 2022-08-01T23:01:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7018142d30 con 0x7f70180acbe0 2022-08-01T23:01:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701001a930 con 0x7f70180acbe0 2022-08-01T23:01:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7018133fb0 con 0x7f70180acbe0 2022-08-01T23:01:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70181419f0 con 0x7f70180acbe0 2022-08-01T23:01:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7010021060 con 0x7f70180acbe0 2022-08-01T23:01:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f7015ffb700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6ff8053950 0x7f6ff8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f701006f300 con 0x7f70180acbe0 2022-08-01T23:01:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.656+0000 7f701ca89700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6ff8053950 0x7f6ff8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.660+0000 7f701ca89700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6ff8053950 0x7f6ff8055e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f700801c000 tx=0x7f7008015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.664+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70181419f0 con 0x7f70180acbe0 2022-08-01T23:01:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.792+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70181440a0 con 0x7f70180acbe0 2022-08-01T23:01:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.792+0000 7f7015ffb700 1 -- 172.21.15.32:0/1668732885 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7f70181440a0 con 0x7f70180acbe0 2022-08-01T23:01:41.794 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.792+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6ff8053950 msgr2=0x7f6ff8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.792+0000 7f701eced700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6ff8053950 0x7f6ff8055e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f700801c000 tx=0x7f7008015000).stop 2022-08-01T23:01:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.792+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 msgr2=0x7f7018133920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.792+0000 7f701eced700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 0x7f7018133920 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f7010006e00 tx=0x7f701000b470).stop 2022-08-01T23:01:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 shutdown_connections 2022-08-01T23:01:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6ff8053950 0x7f6ff8055e10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70180acbe0 0x7f7018133920 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70180a9760 0x7f70180ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 --2- 172.21.15.32:0/1668732885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70180a8bb0 0x7f7018144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 >> 172.21.15.32:0/1668732885 conn(0x7f701805f840 msgr2=0x7f70180ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 shutdown_connections 2022-08-01T23:01:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.796+0000 7f701eced700 1 -- 172.21.15.32:0/1668732885 wait complete. 2022-08-01T23:01:41.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:01:41.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.912+0000 7f6e04921700 1 Processor -- start 2022-08-01T23:01:41.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.912+0000 7f6e04921700 1 -- start start 2022-08-01T23:01:41.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e00060330 0x7f6e000b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 0x7f6e000a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e000aa630 con 0x7f6e000af550 2022-08-01T23:01:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e000b15e0 con 0x7f6e00060330 2022-08-01T23:01:41.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000af930 con 0x7f6e0005c6f0 2022-08-01T23:01:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 0x7f6e000a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6df6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e00060330 0x7f6e000b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 0x7f6e000a8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:56738/0 (socket says 172.21.15.32:56738) 2022-08-01T23:01:41.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 -- 172.21.15.32:0/4163569457 learned_addr learned my addr 172.21.15.32:0/4163569457 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 -- 172.21.15.32:0/4163569457 >> v2:172.21.15.55:3300/0 conn(0x7f6e00060330 msgr2=0x7f6e000b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 --2- 172.21.15.32:0/4163569457 >> v2:172.21.15.55:3300/0 conn(0x7f6e00060330 0x7f6e000b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 -- 172.21.15.32:0/4163569457 >> v1:172.21.15.55:6789/0 conn(0x7f6e000af550 legacy=0x7f6e001445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 -- 172.21.15.32:0/4163569457 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0005fd10 con 0x7f6e0005c6f0 2022-08-01T23:01:41.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfffff700 1 --2- 172.21.15.32:0/4163569457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 0x7f6e000a8d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6df80032f0 tx=0x7f6df8009290).ready entity=mon.2 client_cookie=352582b08eb8d255 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfeffd700 1 -- 172.21.15.32:0/4163569457 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df800b8a0 con 0x7f6e0005c6f0 2022-08-01T23:01:41.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfeffd700 1 -- 172.21.15.32:0/4163569457 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800ba00 con 0x7f6e0005c6f0 2022-08-01T23:01:41.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6dfeffd700 1 -- 172.21.15.32:0/4163569457 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df800bb70 con 0x7f6e0005c6f0 2022-08-01T23:01:41.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- 172.21.15.32:0/4163569457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 msgr2=0x7f6e000a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 --2- 172.21.15.32:0/4163569457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 0x7f6e000a8d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6df80032f0 tx=0x7f6df8009290).stop 2022-08-01T23:01:41.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- 172.21.15.32:0/4163569457 shutdown_connections 2022-08-01T23:01:41.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 --2- 172.21.15.32:0/4163569457 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e0005c6f0 0x7f6e000a8d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 --2- 172.21.15.32:0/4163569457 >> v2:172.21.15.55:3300/0 conn(0x7f6e00060330 0x7f6e000b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- 172.21.15.32:0/4163569457 >> 172.21.15.32:0/4163569457 conn(0x7f6e000612d0 msgr2=0x7f6e00062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- 172.21.15.32:0/4163569457 shutdown_connections 2022-08-01T23:01:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.916+0000 7f6e04921700 1 -- 172.21.15.32:0/4163569457 wait complete. 2022-08-01T23:01:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 Processor -- start 2022-08-01T23:01:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 -- start start 2022-08-01T23:01:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e0005c6f0 0x7f6e00141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 0x7f6e00141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 0x7f6e00142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001444c0 con 0x7f6e000af550 2022-08-01T23:01:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000b49f0 con 0x7f6e0005c6f0 2022-08-01T23:01:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6e04921700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001457c0 con 0x7f6e00060330 2022-08-01T23:01:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6dfffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 0x7f6e00142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6df6ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 0x7f6e00141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6dfffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 0x7f6e00142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45006/0 (socket says 172.21.15.32:45006) 2022-08-01T23:01:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6dfffff700 1 -- 172.21.15.32:0/4183726332 learned_addr learned my addr 172.21.15.32:0/4183726332 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6dff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e0005c6f0 0x7f6e00141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.920+0000 7f6df6ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 0x7f6e00141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:56742/0 (socket says 172.21.15.32:56742) 2022-08-01T23:01:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6df6ffd700 1 -- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e0005c6f0 msgr2=0x7f6e00141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6df6ffd700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e0005c6f0 0x7f6e00141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6df6ffd700 1 -- 172.21.15.32:0/4183726332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 msgr2=0x7f6e00142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6df6ffd700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 0x7f6e00142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6df6ffd700 1 -- 172.21.15.32:0/4183726332 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009530 con 0x7f6e00060330 2022-08-01T23:01:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6dfffff700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 0x7f6e00142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6df6ffd700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 0x7f6e00141900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f6df000c000 tx=0x7f6df0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df0010780 con 0x7f6e00060330 2022-08-01T23:01:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df00108e0 con 0x7f6e00060330 2022-08-01T23:01:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0005f840 con 0x7f6e00060330 2022-08-01T23:01:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df0010c40 con 0x7f6e00060330 2022-08-01T23:01:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e000b5160 con 0x7f6e00060330 2022-08-01T23:01:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e00142bd0 con 0x7f6e00060330 2022-08-01T23:01:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6df00324d0 con 0x7f6e00060330 2022-08-01T23:01:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.924+0000 7f6dfd7fa700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dd8053b30 0x7f6dd8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.928+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6df006c380 con 0x7f6e00060330 2022-08-01T23:01:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.928+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dd8053b30 0x7f6dd8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.928+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dd8053b30 0x7f6dd8055ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f6de800d000 tx=0x7f6de8007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:41.932+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6df0043140 con 0x7f6e00060330 2022-08-01T23:01:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e0005fa70 con 0x7f6e00060330 2022-08-01T23:01:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6dfd7fa700 1 -- 172.21.15.32:0/4183726332 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6df0046760 con 0x7f6e00060330 2022-08-01T23:01:42.062 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dd8053b30 msgr2=0x7f6dd8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dd8053b30 0x7f6dd8055ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f6de800d000 tx=0x7f6de8007b70).stop 2022-08-01T23:01:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 msgr2=0x7f6e00141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 0x7f6e00141900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f6df000c000 tx=0x7f6df0006df0).stop 2022-08-01T23:01:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 shutdown_connections 2022-08-01T23:01:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6dd8053b30 0x7f6dd8055ff0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e000af550 0x7f6e00142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e00060330 0x7f6e00141900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 --2- 172.21.15.32:0/4183726332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e0005c6f0 0x7f6e00141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.060+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 >> 172.21.15.32:0/4183726332 conn(0x7f6e000612d0 msgr2=0x7f6e000b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.064+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 shutdown_connections 2022-08-01T23:01:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.064+0000 7f6e04921700 1 -- 172.21.15.32:0/4183726332 wait complete. 2022-08-01T23:01:42.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:01:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.184+0000 7f78aa12f700 1 Processor -- start 2022-08-01T23:01:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.184+0000 7f78aa12f700 1 -- start start 2022-08-01T23:01:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f78a40b2710 0x7f78a41401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a4067a10 con 0x7f78a40b0270 2022-08-01T23:01:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a40ad740 con 0x7f78a40b2710 2022-08-01T23:01:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4067cb0 con 0x7f78a40ac880 2022-08-01T23:01:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a41406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:56750/0 (socket says 172.21.15.32:56750) 2022-08-01T23:01:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 -- 172.21.15.32:0/2150559171 learned_addr learned my addr 172.21.15.32:0/2150559171 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a912d700 1 --2- 172.21.15.32:0/2150559171 >> v2:172.21.15.55:3300/0 conn(0x7f78a40b2710 0x7f78a41401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 -- 172.21.15.32:0/2150559171 >> v2:172.21.15.55:3300/0 conn(0x7f78a40b2710 msgr2=0x7f78a41401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 --2- 172.21.15.32:0/2150559171 >> v2:172.21.15.55:3300/0 conn(0x7f78a40b2710 0x7f78a41401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 -- 172.21.15.32:0/2150559171 >> v1:172.21.15.55:6789/0 conn(0x7f78a40b0270 legacy=0x7f78a4143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f789bfff700 1 -- 172.21.15.32:0/2150559171 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799052889 0 0) 0x7f78a4067a10 con 0x7f78a40b0270 2022-08-01T23:01:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 -- 172.21.15.32:0/2150559171 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4147160 con 0x7f78a40ac880 2022-08-01T23:01:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78a992e700 1 --2- 172.21.15.32:0/2150559171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a41406d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f789c002f70 tx=0x7f789c006eb0).ready entity=mon.2 client_cookie=edc7460eb47771ba server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f789bfff700 1 -- 172.21.15.32:0/2150559171 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00d570 con 0x7f78a40ac880 2022-08-01T23:01:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f789bfff700 1 -- 172.21.15.32:0/2150559171 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c000e90 con 0x7f78a40ac880 2022-08-01T23:01:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f789bfff700 1 -- 172.21.15.32:0/2150559171 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00d8c0 con 0x7f78a40ac880 2022-08-01T23:01:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- 172.21.15.32:0/2150559171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 msgr2=0x7f78a41406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 --2- 172.21.15.32:0/2150559171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a41406d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f789c002f70 tx=0x7f789c006eb0).stop 2022-08-01T23:01:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- 172.21.15.32:0/2150559171 shutdown_connections 2022-08-01T23:01:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 --2- 172.21.15.32:0/2150559171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a41406d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 --2- 172.21.15.32:0/2150559171 >> v2:172.21.15.55:3300/0 conn(0x7f78a40b2710 0x7f78a41401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- 172.21.15.32:0/2150559171 >> 172.21.15.32:0/2150559171 conn(0x7f78a4060310 msgr2=0x7f78a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- 172.21.15.32:0/2150559171 shutdown_connections 2022-08-01T23:01:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.188+0000 7f78aa12f700 1 -- 172.21.15.32:0/2150559171 wait complete. 2022-08-01T23:01:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 Processor -- start 2022-08-01T23:01:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 -- start start 2022-08-01T23:01:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a40ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78a40b0270 0x7f78a413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 0x7f78a413eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4152400 con 0x7f78a40b2710 2022-08-01T23:01:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40a0790 con 0x7f78a40b0270 2022-08-01T23:01:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40a0a30 con 0x7f78a40ac880 2022-08-01T23:01:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a992e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 0x7f78a413eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a992e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 0x7f78a413eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45020/0 (socket says 172.21.15.32:45020) 2022-08-01T23:01:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a992e700 1 -- 172.21.15.32:0/4228091921 learned_addr learned my addr 172.21.15.32:0/4228091921 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a40ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a892c700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78a40b0270 0x7f78a413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 -- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78a40b0270 msgr2=0x7f78a413e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78a40b0270 0x7f78a413e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 -- 172.21.15.32:0/4228091921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 msgr2=0x7f78a413eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 0x7f78a413eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 -- 172.21.15.32:0/4228091921 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c00b1e0 con 0x7f78a40ac880 2022-08-01T23:01:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a992e700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 0x7f78a413eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:01:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78a912d700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a40ab720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f789400c000 tx=0x7f7894006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7894011750 con 0x7f78a40ac880 2022-08-01T23:01:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a40a0cd0 con 0x7f78a40ac880 2022-08-01T23:01:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78940118b0 con 0x7f78a40ac880 2022-08-01T23:01:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7894011c10 con 0x7f78a40ac880 2022-08-01T23:01:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.192+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a413f000 con 0x7f78a40ac880 2022-08-01T23:01:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.196+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a40574e0 con 0x7f78a40ac880 2022-08-01T23:01:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.196+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7894011d70 con 0x7f78a40ac880 2022-08-01T23:01:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.196+0000 7f789a7fc700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f788c053aa0 0x7f788c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.196+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f789406c320 con 0x7f78a40ac880 2022-08-01T23:01:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.196+0000 7f78a892c700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f788c053aa0 0x7f788c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.196+0000 7f78a892c700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f788c053aa0 0x7f788c055f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f78a000d000 tx=0x7f78a0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.204+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7894042fe0 con 0x7f78a40ac880 2022-08-01T23:01:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.328+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78a40b1bb0 con 0x7f78a40ac880 2022-08-01T23:01:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.328+0000 7f789a7fc700 1 -- 172.21.15.32:0/4228091921 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7894046600 con 0x7f78a40ac880 2022-08-01T23:01:42.331 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:01:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f788c053aa0 msgr2=0x7f788c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f788c053aa0 0x7f788c055f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f78a000d000 tx=0x7f78a0009040).stop 2022-08-01T23:01:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 msgr2=0x7f78a40ab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a40ab720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f789400c000 tx=0x7f7894006df0).stop 2022-08-01T23:01:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 shutdown_connections 2022-08-01T23:01:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f788c053aa0 0x7f788c055f60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a40b2710 0x7f78a413eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78a40b0270 0x7f78a413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 --2- 172.21.15.32:0/4228091921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f78a40ac880 0x7f78a40ab720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 >> 172.21.15.32:0/4228091921 conn(0x7f78a4060310 msgr2=0x7f78a413fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 shutdown_connections 2022-08-01T23:01:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.332+0000 7f78aa12f700 1 -- 172.21.15.32:0/4228091921 wait complete. 2022-08-01T23:01:42.344 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:01:42.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.444+0000 7f723958c700 1 Processor -- start 2022-08-01T23:01:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.444+0000 7f723958c700 1 -- start start 2022-08-01T23:01:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72340b4550 0x7f72340b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7234144240 con 0x7f72340b6af0 2022-08-01T23:01:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7234143d00 con 0x7f72340b6710 2022-08-01T23:01:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7234143fa0 con 0x7f72340b4550 2022-08-01T23:01:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72337fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72340b6af0 0x7f723412bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60802/0 (socket says 172.21.15.32:60802) 2022-08-01T23:01:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72337fe700 1 -- 172.21.15.32:0/429674974 learned_addr learned my addr 172.21.15.32:0/429674974 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72327fc700 1 --2- 172.21.15.32:0/429674974 >> v2:172.21.15.55:3300/0 conn(0x7f72340b4550 0x7f72340b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f7231ffb700 1 -- 172.21.15.32:0/429674974 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594409365 0 0) 0x7f7234144240 con 0x7f72340b6af0 2022-08-01T23:01:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f7231ffb700 1 -- 172.21.15.32:0/429674974 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7214003260 con 0x7f72340b6af0 2022-08-01T23:01:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72327fc700 1 -- 172.21.15.32:0/429674974 >> v1:172.21.15.55:6789/0 conn(0x7f72340b6710 legacy=0x7f7234061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72327fc700 1 -- 172.21.15.32:0/429674974 >> v1:172.21.15.32:6789/0 conn(0x7f72340b6af0 legacy=0x7f723412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72327fc700 1 -- 172.21.15.32:0/429674974 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7234061780 con 0x7f72340b4550 2022-08-01T23:01:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f72327fc700 1 --2- 172.21.15.32:0/429674974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b4550 0x7f72340b5d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f72341432b0 tx=0x7f721c0091d0).ready entity=mon.1 client_cookie=2406ed36f5645f27 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f7231ffb700 1 -- 172.21.15.32:0/429674974 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721c00b6d0 con 0x7f72340b4550 2022-08-01T23:01:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f7231ffb700 1 -- 172.21.15.32:0/429674974 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c00b830 con 0x7f72340b4550 2022-08-01T23:01:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f7231ffb700 1 -- 172.21.15.32:0/429674974 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721c00b9a0 con 0x7f72340b4550 2022-08-01T23:01:42.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- 172.21.15.32:0/429674974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b4550 msgr2=0x7f72340b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 --2- 172.21.15.32:0/429674974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b4550 0x7f72340b5d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f72341432b0 tx=0x7f721c0091d0).stop 2022-08-01T23:01:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- 172.21.15.32:0/429674974 shutdown_connections 2022-08-01T23:01:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 --2- 172.21.15.32:0/429674974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b4550 0x7f72340b5d80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- 172.21.15.32:0/429674974 >> 172.21.15.32:0/429674974 conn(0x7f723405c620 msgr2=0x7f723405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- 172.21.15.32:0/429674974 shutdown_connections 2022-08-01T23:01:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.448+0000 7f723958c700 1 -- 172.21.15.32:0/429674974 wait complete. 2022-08-01T23:01:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 Processor -- start 2022-08-01T23:01:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 -- start start 2022-08-01T23:01:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340b4550 0x7f7234144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 0x7f7234129b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b6af0 0x7f723412a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234126d40 con 0x7f72340b6710 2022-08-01T23:01:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b5df0 con 0x7f72340b6af0 2022-08-01T23:01:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b28c0 con 0x7f72340b4550 2022-08-01T23:01:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 0x7f7234129b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 0x7f7234129b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45032/0 (socket says 172.21.15.32:45032) 2022-08-01T23:01:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 -- 172.21.15.32:0/1411330294 learned_addr learned my addr 172.21.15.32:0/1411330294 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f7232ffd700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340b4550 0x7f7234144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72337fe700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b6af0 0x7f723412a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 -- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340b4550 msgr2=0x7f7234144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340b4550 0x7f7234144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 -- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b6af0 msgr2=0x7f723412a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b6af0 0x7f723412a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 -- 172.21.15.32:0/1411330294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009470 con 0x7f72340b6710 2022-08-01T23:01:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f7232ffd700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340b4550 0x7f7234144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f72327fc700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 0x7f7234129b40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f72341432b0 tx=0x7f721c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721c0107a0 con 0x7f72340b6710 2022-08-01T23:01:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c010900 con 0x7f72340b6710 2022-08-01T23:01:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721c010c60 con 0x7f72340b6710 2022-08-01T23:01:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340b2b60 con 0x7f72340b6710 2022-08-01T23:01:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.452+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72340b32a0 con 0x7f72340b6710 2022-08-01T23:01:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.456+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f721c027020 con 0x7f72340b6710 2022-08-01T23:01:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.456+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7234061a20 con 0x7f72340b6710 2022-08-01T23:01:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.456+0000 7f722bfff700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7218053a10 0x7f7218055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.456+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f721c06d680 con 0x7f72340b6710 2022-08-01T23:01:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.456+0000 7f7232ffd700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7218053a10 0x7f7218055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.456+0000 7f7232ffd700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7218053a10 0x7f7218055ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f722400b000 tx=0x7f7224009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.464+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f721c044340 con 0x7f72340b6710 2022-08-01T23:01:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72340574e0 con 0x7f72340b6710 2022-08-01T23:01:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f722bfff700 1 -- 172.21.15.32:0/1411330294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f721c0159d0 con 0x7f72340b6710 2022-08-01T23:01:42.630 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:42.630 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:01:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7218053a10 msgr2=0x7f7218055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7218053a10 0x7f7218055ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f722400b000 tx=0x7f7224009040).stop 2022-08-01T23:01:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 msgr2=0x7f7234129b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 0x7f7234129b40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f72341432b0 tx=0x7f721c016000).stop 2022-08-01T23:01:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 shutdown_connections 2022-08-01T23:01:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7218053a10 0x7f7218055ed0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72340b6af0 0x7f723412a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340b6710 0x7f7234129b40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 --2- 172.21.15.32:0/1411330294 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340b4550 0x7f7234144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 >> 172.21.15.32:0/1411330294 conn(0x7f723405c620 msgr2=0x7f72341488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 shutdown_connections 2022-08-01T23:01:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:42.628+0000 7f723958c700 1 -- 172.21.15.32:0/1411330294 wait complete. 2022-08-01T23:01:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:01:42.945 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:01:34.459266+0000', 'modified': '2022-08-01T23:01:35.461964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6258}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6258': {'gid': 6258, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1927724642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1927724642}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1927724642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:01:42.946 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:01:42.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:01:43.062 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6258,"want_state":"up:active","state":"up:active","rank_uptime":8.5930828209999994,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":76,"uptime":12.910835999} 2022-08-01T23:01:43.063 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6258, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 76, "rank_uptime": 8.593082821, "state": "up:active", "uptime": 12.910835999, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:01:43.063 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:01:43.064 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:01:43.064 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:01:43.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:43.065 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:43.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:01:43.131 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:01:43.131 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:01:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:01:43.143 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:01:43.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:43.144 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:01:43.149 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:01:43.150 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:43.152 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:01:43.169 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:01:43.170 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:01:43.229 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:01:43.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:01:43.226+0000 7f10e78c3080 -1 init, newargv = 0x5557a9f66900 newargc=9 2022-08-01T23:01:43.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22576]: starting ceph client 2022-08-01T23:01:43.231 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:01:43.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:43.232 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:01:43.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22576]: starting fuse 2022-08-01T23:01:44.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:01:44.358 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:01:44.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:01:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:01:44.373 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:01:44.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:44.374 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:01:44.424 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:01:44.425 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:01:44.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:44.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:01:44.426 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:01:44.427 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:01:44.427 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:01:44.427 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:01:44.428 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:01:44.428 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:01:44.429 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:01:44.429 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:01:44.429 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:01:44.430 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:01:44.430 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:01:44.431 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:01:44.431 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:01:44.431 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:01:44.432 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:01:44.432 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:01:44.433 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:01:44.433 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:01:44.433 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:01:44.434 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:01:44.434 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:01:44.435 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:01:44.478 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.22576.asok 2022-08-01T23:01:44.483 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22576.asok 2022-08-01T23:01:44.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:44.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22576.asok status 2022-08-01T23:01:44.577 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:01:44.577 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:01:44.578 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:01:44.578 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:01:44.578 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:01:44.578 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:01:44.579 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:01:44.579 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "22576", 2022-08-01T23:01:44.579 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:01:44.579 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:01:44.579 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:01:44.580 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:01:44.580 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6403, 2022-08-01T23:01:44.580 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:01:44.580 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:01:44.581 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:01:44.581 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6403 2022-08-01T23:01:44.581 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:01:44.581 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:01:44.581 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:01:44.582 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:01:44.582 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3566809591 2022-08-01T23:01:44.582 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:01:44.582 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:01:44.582 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:01:44.583 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:01:44.583 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:01:44.583 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3566809591 2022-08-01T23:01:44.583 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:01:44.584 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6403 172.21.15.157:0/3566809591", 2022-08-01T23:01:44.584 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/3566809591", 2022-08-01T23:01:44.584 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:01:44.584 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 33, 2022-08-01T23:01:44.584 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 76, 2022-08-01T23:01:44.585 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:01:44.585 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:01:44.585 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:01:44.585 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:01:44.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:44.587 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:44.588 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:44.594 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:01:44.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:01:44.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:44.596 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:44.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:44.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 Processor -- start 2022-08-01T23:01:44.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- start start 2022-08-01T23:01:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc70c0b1160 0x7fc70c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c0b04e0 con 0x7fc70c0b1f40 2022-08-01T23:01:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c0b0780 con 0x7fc70c148530 2022-08-01T23:01:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c0a9250 con 0x7fc70c0b1160 2022-08-01T23:01:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc70c0b1f40 0x7fc70c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60814/0 (socket says 172.21.15.32:60814) 2022-08-01T23:01:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70b7fe700 1 -- 172.21.15.32:0/1048387760 learned_addr learned my addr 172.21.15.32:0/1048387760 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:01:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70bfff700 1 --2- 172.21.15.32:0/1048387760 >> v2:172.21.15.55:3300/0 conn(0x7fc70c0b1160 0x7fc70c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:44.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70a7fc700 1 -- 172.21.15.32:0/1048387760 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147172776 0 0) 0x7fc70c0b04e0 con 0x7fc70c0b1f40 2022-08-01T23:01:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70a7fc700 1 -- 172.21.15.32:0/1048387760 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f0003260 con 0x7fc70c0b1f40 2022-08-01T23:01:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70bfff700 1 -- 172.21.15.32:0/1048387760 >> v1:172.21.15.55:6789/0 conn(0x7fc70c148530 legacy=0x7fc70c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70bfff700 1 -- 172.21.15.32:0/1048387760 >> v1:172.21.15.32:6789/0 conn(0x7fc70c0b1f40 legacy=0x7fc70c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70bfff700 1 -- 172.21.15.32:0/1048387760 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0b0a20 con 0x7fc70c0b1160 2022-08-01T23:01:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70bfff700 1 --2- 172.21.15.32:0/1048387760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c1448b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fc70c1478c0 tx=0x7fc7040091d0).ready entity=mon.1 client_cookie=698def37a472410f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70a7fc700 1 -- 172.21.15.32:0/1048387760 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70400b680 con 0x7fc70c0b1160 2022-08-01T23:01:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70a7fc700 1 -- 172.21.15.32:0/1048387760 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400b7e0 con 0x7fc70c0b1160 2022-08-01T23:01:44.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc70a7fc700 1 -- 172.21.15.32:0/1048387760 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70400b950 con 0x7fc70c0b1160 2022-08-01T23:01:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- 172.21.15.32:0/1048387760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 msgr2=0x7fc70c1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 --2- 172.21.15.32:0/1048387760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c1448b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fc70c1478c0 tx=0x7fc7040091d0).stop 2022-08-01T23:01:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- 172.21.15.32:0/1048387760 shutdown_connections 2022-08-01T23:01:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 --2- 172.21.15.32:0/1048387760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c1448b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- 172.21.15.32:0/1048387760 >> 172.21.15.32:0/1048387760 conn(0x7fc70c05f840 msgr2=0x7fc70c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- 172.21.15.32:0/1048387760 shutdown_connections 2022-08-01T23:01:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.768+0000 7fc711efb700 1 -- 172.21.15.32:0/1048387760 wait complete. 2022-08-01T23:01:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 Processor -- start 2022-08-01T23:01:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 -- start start 2022-08-01T23:01:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc70c0b1f40 0x7fc70c143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 0x7fc70c144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c144510 con 0x7fc70c148530 2022-08-01T23:01:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c14a200 con 0x7fc70c0b1160 2022-08-01T23:01:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0aeeb0 con 0x7fc70c0b1f40 2022-08-01T23:01:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 0x7fc70c144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 0x7fc70c144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45048/0 (socket says 172.21.15.32:45048) 2022-08-01T23:01:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 -- 172.21.15.32:0/1340192895 learned_addr learned my addr 172.21.15.32:0/1340192895 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70b7fe700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70affd700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc70c0b1f40 0x7fc70c143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 -- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc70c0b1f40 msgr2=0x7fc70c143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc70c0b1f40 0x7fc70c143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 -- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 msgr2=0x7fc70c0adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c0adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 -- 172.21.15.32:0/1340192895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc704009470 con 0x7fc70c148530 2022-08-01T23:01:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70affd700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc70c0b1f40 0x7fc70c143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc70bfff700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 0x7fc70c144020 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fc70c0af040 tx=0x7fc704016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70400b680 con 0x7fc70c148530 2022-08-01T23:01:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704021630 con 0x7fc70c148530 2022-08-01T23:01:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c0aec10 con 0x7fc70c148530 2022-08-01T23:01:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc704021990 con 0x7fc70c148530 2022-08-01T23:01:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.772+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c0afaf0 con 0x7fc70c148530 2022-08-01T23:01:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.776+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c0b0cc0 con 0x7fc70c148530 2022-08-01T23:01:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.776+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc704027020 con 0x7fc70c148530 2022-08-01T23:01:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.776+0000 7fc708ff9700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.776+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc70406d440 con 0x7fc70c148530 2022-08-01T23:01:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.776+0000 7fc70b7fe700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.776+0000 7fc70b7fe700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fc6f400d000 tx=0x7fc6f4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.784+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc704044240 con 0x7fc70c148530 2022-08-01T23:01:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc70c0574e0 con 0x7fc70c148530 2022-08-01T23:01:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc708ff9700 1 -- 172.21.15.32:0/1340192895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fc704015090 con 0x7fc70c148530 2022-08-01T23:01:44.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:44.950 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:01:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 msgr2=0x7fc6e4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fc6f400d000 tx=0x7fc6f4009040).stop 2022-08-01T23:01:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 msgr2=0x7fc70c144020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 0x7fc70c144020 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fc70c0af040 tx=0x7fc704016000).stop 2022-08-01T23:01:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 shutdown_connections 2022-08-01T23:01:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc70c148530 0x7fc70c144020 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc70c0b1f40 0x7fc70c143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 --2- 172.21.15.32:0/1340192895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc70c0b1160 0x7fc70c0adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 >> 172.21.15.32:0/1340192895 conn(0x7fc70c05f840 msgr2=0x7fc70c144890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 shutdown_connections 2022-08-01T23:01:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:44.948+0000 7fc711efb700 1 -- 172.21.15.32:0/1340192895 wait complete. 2022-08-01T23:01:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:01:44.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:01:45.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.064+0000 7f0cdbc30700 1 Processor -- start 2022-08-01T23:01:45.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.064+0000 7f0cdbc30700 1 -- start start 2022-08-01T23:01:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:45.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cd4148530 0x7f0cd40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40b04e0 con 0x7f0cd40a7250 2022-08-01T23:01:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40a9250 con 0x7f0cd4148530 2022-08-01T23:01:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b0780 con 0x7f0cd40b1f40 2022-08-01T23:01:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cd4148530 0x7f0cd40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cd4148530 0x7f0cd40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45052/0 (socket says 172.21.15.32:45052) 2022-08-01T23:01:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 -- 172.21.15.32:0/2001485679 learned_addr learned my addr 172.21.15.32:0/2001485679 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd99cc700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 -- 172.21.15.32:0/2001485679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 msgr2=0x7f0cd40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:01:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 -- 172.21.15.32:0/2001485679 >> v1:172.21.15.32:6789/0 conn(0x7f0cd40a7250 legacy=0x7f0cd41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 -- 172.21.15.32:0/2001485679 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40b0a20 con 0x7f0cd4148530 2022-08-01T23:01:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd99cc700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:01:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd91cb700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd40b2320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f0cd0002f70 tx=0x7f0cd0006eb0).ready entity=mon.0 client_cookie=3a0605a7daa35961 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd89ca700 1 -- 172.21.15.32:0/2001485679 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd000d5c0 con 0x7f0cd4148530 2022-08-01T23:01:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd89ca700 1 -- 172.21.15.32:0/2001485679 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0000e90 con 0x7f0cd4148530 2022-08-01T23:01:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cd89ca700 1 -- 172.21.15.32:0/2001485679 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cd000d950 con 0x7f0cd4148530 2022-08-01T23:01:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- 172.21.15.32:0/2001485679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 msgr2=0x7f0cd40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd40b2320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f0cd0002f70 tx=0x7f0cd0006eb0).stop 2022-08-01T23:01:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- 172.21.15.32:0/2001485679 shutdown_connections 2022-08-01T23:01:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd40b2320 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/2001485679 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- 172.21.15.32:0/2001485679 >> 172.21.15.32:0/2001485679 conn(0x7f0cd405f840 msgr2=0x7f0cd405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- 172.21.15.32:0/2001485679 shutdown_connections 2022-08-01T23:01:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.068+0000 7f0cdbc30700 1 -- 172.21.15.32:0/2001485679 wait complete. 2022-08-01T23:01:45.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 Processor -- start 2022-08-01T23:01:45.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 -- start start 2022-08-01T23:01:45.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cd40a7250 0x7f0cd4157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:45.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:45.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd4142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:45.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4157960 con 0x7f0cd4148530 2022-08-01T23:01:45.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40aabf0 con 0x7f0cd40a7250 2022-08-01T23:01:45.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40aae90 con 0x7f0cd40b1f40 2022-08-01T23:01:45.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45062/0 (socket says 172.21.15.32:45062) 2022-08-01T23:01:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 -- 172.21.15.32:0/81349496 learned_addr learned my addr 172.21.15.32:0/81349496 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:01:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cd99cc700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cd40a7250 0x7f0cd4157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:45.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cd91cb700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 -- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 msgr2=0x7f0cd40ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 -- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cd40a7250 msgr2=0x7f0cd4157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cd40a7250 0x7f0cd4157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 -- 172.21.15.32:0/81349496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd000b1e0 con 0x7f0cd4148530 2022-08-01T23:01:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cd99cc700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cd40a7250 0x7f0cd4157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:01:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cda1cd700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd4142680 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f0cd4061530 tx=0x7f0ccc007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ccc00f470 con 0x7f0cd4148530 2022-08-01T23:01:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00f5d0 con 0x7f0cd4148530 2022-08-01T23:01:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd40ab130 con 0x7f0cd4148530 2022-08-01T23:01:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ccc00f930 con 0x7f0cd4148530 2022-08-01T23:01:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.072+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd40ada50 con 0x7f0cd4148530 2022-08-01T23:01:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.076+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd40aa9b0 con 0x7f0cd4148530 2022-08-01T23:01:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.076+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ccc019030 con 0x7f0cd4148530 2022-08-01T23:01:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.076+0000 7f0cc6ffd700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0cb4053a10 0x7f0cb4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:01:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.076+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0ccc06b6c0 con 0x7f0cd4148530 2022-08-01T23:01:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.076+0000 7f0cd99cc700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0cb4053a10 0x7f0cb4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:01:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.076+0000 7f0cd99cc700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0cb4053a10 0x7f0cb4055ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f0cc800b000 tx=0x7f0cc8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:01:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.080+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0cd40aa9b0 con 0x7f0cd4148530 2022-08-01T23:01:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.244+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cd40aa9b0 con 0x7f0cd4148530 2022-08-01T23:01:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.244+0000 7f0cc6ffd700 1 -- 172.21.15.32:0/81349496 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f0cd40aa9b0 con 0x7f0cd4148530 2022-08-01T23:01:45.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:01:45.248 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:01:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0cb4053a10 msgr2=0x7f0cb4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0cb4053a10 0x7f0cb4055ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f0cc800b000 tx=0x7f0cc8008040).stop 2022-08-01T23:01:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 msgr2=0x7f0cd4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:01:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd4142680 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f0cd4061530 tx=0x7f0ccc007f00).stop 2022-08-01T23:01:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 shutdown_connections 2022-08-01T23:01:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0cb4053a10 0x7f0cb4055ed0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4148530 0x7f0cd4142680 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cd40b1f40 0x7f0cd40ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 --2- 172.21.15.32:0/81349496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cd40a7250 0x7f0cd4157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:01:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 >> 172.21.15.32:0/81349496 conn(0x7f0cd405f840 msgr2=0x7f0cd40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:01:45.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 shutdown_connections 2022-08-01T23:01:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:01:45.248+0000 7f0cdbc30700 1 -- 172.21.15.32:0/81349496 wait complete. 2022-08-01T23:01:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:01:45.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2022-08-01T23:01:45.364 INFO:teuthology.orchestra.run.smithi032.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2022-08-01T23:01:45.364 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2022-08-01T23:01:45.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:45.365 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:45.365 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:01:45.374 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:01:45.375 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:01:45.376 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T23:01:45.376 DEBUG:teuthology.orchestra.run.smithi130:> import time 2022-08-01T23:01:45.377 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:01:45.377 DEBUG:teuthology.orchestra.run.smithi130:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2022-08-01T23:01:45.377 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content'"'"') 2022-08-01T23:01:45.378 DEBUG:teuthology.orchestra.run.smithi130:> f.flush() 2022-08-01T23:01:45.378 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content2'"'"') 2022-08-01T23:01:45.378 DEBUG:teuthology.orchestra.run.smithi130:> while True: 2022-08-01T23:01:45.379 DEBUG:teuthology.orchestra.run.smithi130:> time.sleep(1) 2022-08-01T23:01:45.379 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:01:45.417 DEBUG:teuthology.orchestra.run.smithi130:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-08-01T23:01:45.435 INFO:teuthology.orchestra.run.smithi130.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2022-08-01T23:01:45.436 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-08-01T23:01:46.438 DEBUG:teuthology.orchestra.run.smithi130:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-08-01T23:01:46.455 INFO:teuthology.orchestra.run.smithi130.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2022-08-01T23:01:46.457 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2022-08-01T23:01:46.457 DEBUG:teuthology.orchestra.run.smithi157:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2022-08-01T23:01:46.498 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2022-08-01T23:01:46.499 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2022-08-01T23:01:46.500 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:01:46.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:01:46.516 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:01:46.516 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:01:46.525 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:01:46.526 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:01:46.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T23:01:46.527 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:01:46.527 DEBUG:teuthology.orchestra.run.smithi157:> import time 2022-08-01T23:01:46.527 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:01:46.528 DEBUG:teuthology.orchestra.run.smithi157:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2022-08-01T23:01:46.528 DEBUG:teuthology.orchestra.run.smithi157:> try: 2022-08-01T23:01:46.528 DEBUG:teuthology.orchestra.run.smithi157:> while True: 2022-08-01T23:01:46.529 DEBUG:teuthology.orchestra.run.smithi157:> os.write(fd, b'"'"'content'"'"') 2022-08-01T23:01:46.529 DEBUG:teuthology.orchestra.run.smithi157:> time.sleep(1) 2022-08-01T23:01:46.530 DEBUG:teuthology.orchestra.run.smithi157:> if not False: 2022-08-01T23:01:46.530 DEBUG:teuthology.orchestra.run.smithi157:> break 2022-08-01T23:01:46.530 DEBUG:teuthology.orchestra.run.smithi157:> except IOError as e: 2022-08-01T23:01:46.531 DEBUG:teuthology.orchestra.run.smithi157:> pass 2022-08-01T23:01:46.531 DEBUG:teuthology.orchestra.run.smithi157:> os.close(fd) 2022-08-01T23:01:46.532 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:01:54.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:54.217 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:54.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:01:54.222 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:24.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:24.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:24.380 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:24.383 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:26.298 INFO:tasks.cephfs.test_misc:cap_waiter waited 39.72932243347168s 2022-08-01T23:02:26.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2022-08-01T23:02:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 Processor -- start 2022-08-01T23:02:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 -- start start 2022-08-01T23:02:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0aa040 0x7fe5bc0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5bc0af200 0x7fe5bc144610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc0aa420 con 0x7fe5bc0b32c0 2022-08-01T23:02:26.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc0ae490 con 0x7fe5bc0af200 2022-08-01T23:02:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5c0e9d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc148300 con 0x7fe5bc0aa040 2022-08-01T23:02:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5bad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5bc0af200 0x7fe5bc144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5bad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5bc0af200 0x7fe5bc144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45072/0 (socket says 172.21.15.32:45072) 2022-08-01T23:02:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5b1d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe5bc0b32c0 0x7fe5bc14b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:53518/0 (socket says 172.21.15.32:53518) 2022-08-01T23:02:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5bad9d700 1 -- 172.21.15.32:0/1489091660 learned_addr learned my addr 172.21.15.32:0/1489091660 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5ba59c700 1 --2- 172.21.15.32:0/1489091660 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0aa040 0x7fe5bc0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5b9d9b700 1 -- 172.21.15.32:0/1489091660 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157153004 0 0) 0x7fe5bc0aa420 con 0x7fe5bc0b32c0 2022-08-01T23:02:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5b9d9b700 1 -- 172.21.15.32:0/1489091660 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59c003260 con 0x7fe5bc0b32c0 2022-08-01T23:02:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5bad9d700 1 -- 172.21.15.32:0/1489091660 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0aa040 msgr2=0x7fe5bc0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5bad9d700 1 --2- 172.21.15.32:0/1489091660 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0aa040 0x7fe5bc0b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.410+0000 7fe5bad9d700 1 -- 172.21.15.32:0/1489091660 >> v1:172.21.15.55:6789/0 conn(0x7fe5bc0b32c0 legacy=0x7fe5bc14b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5bad9d700 1 -- 172.21.15.32:0/1489091660 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc1485a0 con 0x7fe5bc0af200 2022-08-01T23:02:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5bad9d700 1 --2- 172.21.15.32:0/1489091660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc144610 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fe5b40032f0 tx=0x7fe5b4009290).ready entity=mon.0 client_cookie=bb000dcf151a486 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:26.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5b9d9b700 1 -- 172.21.15.32:0/1489091660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5b400b6c0 con 0x7fe5bc0af200 2022-08-01T23:02:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5b9d9b700 1 -- 172.21.15.32:0/1489091660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b400b820 con 0x7fe5bc0af200 2022-08-01T23:02:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5b9d9b700 1 -- 172.21.15.32:0/1489091660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5b400b990 con 0x7fe5bc0af200 2022-08-01T23:02:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/1489091660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 msgr2=0x7fe5bc144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/1489091660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc144610 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fe5b40032f0 tx=0x7fe5b4009290).stop 2022-08-01T23:02:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/1489091660 shutdown_connections 2022-08-01T23:02:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/1489091660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc144610 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/1489091660 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0aa040 0x7fe5bc0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/1489091660 >> 172.21.15.32:0/1489091660 conn(0x7fe5bc060950 msgr2=0x7fe5bc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/1489091660 shutdown_connections 2022-08-01T23:02:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.414+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/1489091660 wait complete. 2022-08-01T23:02:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 Processor -- start 2022-08-01T23:02:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 -- start start 2022-08-01T23:02:26.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0aa040 0x7fe5bc149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0b32c0 0x7fe5bc156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc143de0 con 0x7fe5bc0af200 2022-08-01T23:02:26.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0b1680 con 0x7fe5bc0aa040 2022-08-01T23:02:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc143b40 con 0x7fe5bc0b32c0 2022-08-01T23:02:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc1569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45076/0 (socket says 172.21.15.32:45076) 2022-08-01T23:02:26.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 -- 172.21.15.32:0/3206106865 learned_addr learned my addr 172.21.15.32:0/3206106865 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5ba59c700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0aa040 0x7fe5bc149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5bad9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0b32c0 0x7fe5bc156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 -- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0b32c0 msgr2=0x7fe5bc156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0b32c0 0x7fe5bc156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 -- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0aa040 msgr2=0x7fe5bc149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0aa040 0x7fe5bc149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 -- 172.21.15.32:0/3206106865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b4009530 con 0x7fe5bc0af200 2022-08-01T23:02:26.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b1d9b700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc1569d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fe5a4001110 tx=0x7fe5a4007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:26.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a40070d0 con 0x7fe5bc0af200 2022-08-01T23:02:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a4007230 con 0x7fe5bc0af200 2022-08-01T23:02:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc0b18e0 con 0x7fe5bc0af200 2022-08-01T23:02:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.418+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a4005740 con 0x7fe5bc0af200 2022-08-01T23:02:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.422+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc146fe0 con 0x7fe5bc0af200 2022-08-01T23:02:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.422+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc14afb0 con 0x7fe5bc0af200 2022-08-01T23:02:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.422+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe5a401a060 con 0x7fe5bc0af200 2022-08-01T23:02:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.422+0000 7fe5b3fff700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5a0053980 0x7fe5a0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.422+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fe5a406b710 con 0x7fe5bc0af200 2022-08-01T23:02:26.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.422+0000 7fe5ba59c700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5a0053980 0x7fe5a0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.426+0000 7fe5ba59c700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5a0053980 0x7fe5a0055e40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fe5ac00b000 tx=0x7fe5ac006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.430+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5a40424b0 con 0x7fe5bc0af200 2022-08-01T23:02:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7fe5bc0574e0 con 0x7fe5bc0af200 2022-08-01T23:02:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5b3fff700 1 -- 172.21.15.32:0/3206106865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v77) v1 ==== 98+0+80 (secure 0 0 0) 0x7fe5a4018070 con 0x7fe5bc0af200 2022-08-01T23:02:26.575 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5a0053980 msgr2=0x7fe5a0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5a0053980 0x7fe5a0055e40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fe5ac00b000 tx=0x7fe5ac006df0).stop 2022-08-01T23:02:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 msgr2=0x7fe5bc1569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc1569d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fe5a4001110 tx=0x7fe5a4007aa0).stop 2022-08-01T23:02:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 shutdown_connections 2022-08-01T23:02:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5a0053980 0x7fe5a0055e40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5bc0b32c0 0x7fe5bc156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5bc0af200 0x7fe5bc1569d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 --2- 172.21.15.32:0/3206106865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0aa040 0x7fe5bc149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 >> 172.21.15.32:0/3206106865 conn(0x7fe5bc060950 msgr2=0x7fe5bc14bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 shutdown_connections 2022-08-01T23:02:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.574+0000 7fe5c0e9d700 1 -- 172.21.15.32:0/3206106865 wait complete. 2022-08-01T23:02:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:listed 1 entries 2022-08-01T23:02:26.588 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr":"172.21.15.130:0/438018148","until":"2022-08-02T00:02:20.149979+0000"}] 2022-08-01T23:02:26.604 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-08-01T23:02:26.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:26.605 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:26.605 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:26.611 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T23:02:26.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:02:26.613 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T23:02:26.614 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:02:26.614 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:02:26.615 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:26.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:26.631 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:26.632 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:26.639 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:02:26.640 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:02:26.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:02:26.641 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:26.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 Processor -- start 2022-08-01T23:02:26.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 -- start start 2022-08-01T23:02:26.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39300b1a80 0x7f39300b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300a9da0 0x7f3930147e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300b3570 con 0x7f39300b3190 2022-08-01T23:02:26.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300b2980 con 0x7f39300b1a80 2022-08-01T23:02:26.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3935398700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300b0e00 con 0x7f39300a9da0 2022-08-01T23:02:26.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39300b1a80 0x7f39300b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39300b1a80 0x7f39300b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45086/0 (socket says 172.21.15.32:45086) 2022-08-01T23:02:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f39300b3190 0x7f39300656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:53530/0 (socket says 172.21.15.32:53530) 2022-08-01T23:02:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 -- 172.21.15.32:0/3657790036 learned_addr learned my addr 172.21.15.32:0/3657790036 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f3934b97700 1 --2- 172.21.15.32:0/3657790036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300a9da0 0x7f3930147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 -- 172.21.15.32:0/3657790036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300a9da0 msgr2=0x7f3930147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:26.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 --2- 172.21.15.32:0/3657790036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300a9da0 0x7f3930147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 -- 172.21.15.32:0/3657790036 >> v1:172.21.15.55:6789/0 conn(0x7f39300b3190 legacy=0x7f39300656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 -- 172.21.15.32:0/3657790036 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300b10a0 con 0x7f39300b1a80 2022-08-01T23:02:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392effd700 1 -- 172.21.15.32:0/3657790036 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473725775 0 0) 0x7f39300b3570 con 0x7f39300b3190 2022-08-01T23:02:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392f7fe700 1 --2- 172.21.15.32:0/3657790036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f39300b2490 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f3918002f70 tx=0x7f3918006eb0).ready entity=mon.0 client_cookie=a9bc068747b3b351 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392effd700 1 -- 172.21.15.32:0/3657790036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f391800d570 con 0x7f39300b1a80 2022-08-01T23:02:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392effd700 1 -- 172.21.15.32:0/3657790036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918000e90 con 0x7f39300b1a80 2022-08-01T23:02:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.802+0000 7f392effd700 1 -- 172.21.15.32:0/3657790036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f391800d8c0 con 0x7f39300b1a80 2022-08-01T23:02:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- 172.21.15.32:0/3657790036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 msgr2=0x7f39300b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 --2- 172.21.15.32:0/3657790036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f39300b2490 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f3918002f70 tx=0x7f3918006eb0).stop 2022-08-01T23:02:26.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- 172.21.15.32:0/3657790036 shutdown_connections 2022-08-01T23:02:26.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 --2- 172.21.15.32:0/3657790036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f39300b2490 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 --2- 172.21.15.32:0/3657790036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300a9da0 0x7f3930147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- 172.21.15.32:0/3657790036 >> 172.21.15.32:0/3657790036 conn(0x7f3930063870 msgr2=0x7f3930063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:26.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- 172.21.15.32:0/3657790036 shutdown_connections 2022-08-01T23:02:26.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- 172.21.15.32:0/3657790036 wait complete. 2022-08-01T23:02:26.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 Processor -- start 2022-08-01T23:02:26.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- start start 2022-08-01T23:02:26.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39300a9da0 0x7f39300b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f393014cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300b3190 0x7f3930159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300b5dd0 con 0x7f39300b1a80 2022-08-01T23:02:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930157110 con 0x7f39300a9da0 2022-08-01T23:02:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f3935398700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39301573a0 con 0x7f39300b3190 2022-08-01T23:02:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f392f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f393014cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f392ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39300a9da0 0x7f39300b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f392f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f393014cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45090/0 (socket says 172.21.15.32:45090) 2022-08-01T23:02:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.806+0000 7f392f7fe700 1 -- 172.21.15.32:0/1302015736 learned_addr learned my addr 172.21.15.32:0/1302015736 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f3934b97700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300b3190 0x7f3930159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392f7fe700 1 -- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300b3190 msgr2=0x7f3930159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392f7fe700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300b3190 0x7f3930159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392f7fe700 1 -- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39300a9da0 msgr2=0x7f39300b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392f7fe700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39300a9da0 0x7f39300b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392f7fe700 1 -- 172.21.15.32:0/1302015736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391800b1e0 con 0x7f39300b1a80 2022-08-01T23:02:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392f7fe700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f393014cdb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f3918002f70 tx=0x7f3918015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3918011860 con 0x7f39300b1a80 2022-08-01T23:02:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39180119c0 con 0x7f39300b1a80 2022-08-01T23:02:26.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3930157640 con 0x7f39300b1a80 2022-08-01T23:02:26.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3918011d20 con 0x7f39300b1a80 2022-08-01T23:02:26.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39300b5440 con 0x7f39300b1a80 2022-08-01T23:02:26.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39300574e0 con 0x7f39300b1a80 2022-08-01T23:02:26.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3918025030 con 0x7f39300b1a80 2022-08-01T23:02:26.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392d7fa700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3914053b00 0x7f3914055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:26.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.810+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f391806d1e0 con 0x7f39300b1a80 2022-08-01T23:02:26.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.814+0000 7f392ffff700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3914053b00 0x7f3914055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:26.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.814+0000 7f392ffff700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3914053b00 0x7f3914055fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f392000d000 tx=0x7f3920007d80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:26.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.818+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3918043f80 con 0x7f39300b1a80 2022-08-01T23:02:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.942+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3930069a70 con 0x7f39300b1a80 2022-08-01T23:02:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f392d7fa700 1 -- 172.21.15.32:0/1302015736 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16335 (secure 0 0 0) 0x7f3918023020 con 0x7f39300b1a80 2022-08-01T23:02:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:26.947 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":77,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:20.401055+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:01:32.889134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T23:01:33.885837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.130:0/438018148":"2022-08-02T00:02:20.149979+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3914053b00 msgr2=0x7f3914055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3914053b00 0x7f3914055fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f392000d000 tx=0x7f3920007d80).stop 2022-08-01T23:02:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 msgr2=0x7f393014cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f393014cdb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f3918002f70 tx=0x7f3918015000).stop 2022-08-01T23:02:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 shutdown_connections 2022-08-01T23:02:26.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3914053b00 0x7f3914055fc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39300b3190 0x7f3930159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39300b1a80 0x7f393014cdb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 --2- 172.21.15.32:0/1302015736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39300a9da0 0x7f39300b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 >> 172.21.15.32:0/1302015736 conn(0x7f3930063870 msgr2=0x7f39300b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 shutdown_connections 2022-08-01T23:02:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:26.946+0000 7f3935398700 1 -- 172.21.15.32:0/1302015736 wait complete. 2022-08-01T23:02:26.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:01:32.889134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:01:33.885837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:02:26.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:02:27.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.070+0000 7f6b381cc700 1 Processor -- start 2022-08-01T23:02:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.070+0000 7f6b381cc700 1 -- start start 2022-08-01T23:02:27.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.070+0000 7f6b381cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b30146ae0 0x7f6b301403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4310 0x7f6b301408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b30143cf0 con 0x7f6b300b4820 2022-08-01T23:02:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b301476a0 con 0x7f6b30146ae0 2022-08-01T23:02:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30143f90 con 0x7f6b300b4310 2022-08-01T23:02:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b30146ae0 0x7f6b301403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35767700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4310 0x7f6b301408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b30146ae0 0x7f6b301403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45096/0 (socket says 172.21.15.32:45096) 2022-08-01T23:02:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 -- 172.21.15.32:0/3166476978 learned_addr learned my addr 172.21.15.32:0/3166476978 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 -- 172.21.15.32:0/3166476978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4310 msgr2=0x7f6b301408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 --2- 172.21.15.32:0/3166476978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4310 0x7f6b301408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 -- 172.21.15.32:0/3166476978 >> v1:172.21.15.55:6789/0 conn(0x7f6b300b4820 legacy=0x7f6b300b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b34f66700 1 -- 172.21.15.32:0/3166476978 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336513324 0 0) 0x7f6b30143cf0 con 0x7f6b300b4820 2022-08-01T23:02:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 -- 172.21.15.32:0/3166476978 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300b0e20 con 0x7f6b30146ae0 2022-08-01T23:02:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b35f68700 1 --2- 172.21.15.32:0/3166476978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b301403c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f6b24002f70 tx=0x7f6b24006eb0).ready entity=mon.0 client_cookie=54a422bd753a60b2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b34f66700 1 -- 172.21.15.32:0/3166476978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2400d570 con 0x7f6b30146ae0 2022-08-01T23:02:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b34f66700 1 -- 172.21.15.32:0/3166476978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24000e90 con 0x7f6b30146ae0 2022-08-01T23:02:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b34f66700 1 -- 172.21.15.32:0/3166476978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2400d8c0 con 0x7f6b30146ae0 2022-08-01T23:02:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- 172.21.15.32:0/3166476978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 msgr2=0x7f6b301403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3166476978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b301403c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f6b24002f70 tx=0x7f6b24006eb0).stop 2022-08-01T23:02:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- 172.21.15.32:0/3166476978 shutdown_connections 2022-08-01T23:02:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3166476978 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4310 0x7f6b301408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3166476978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b301403c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- 172.21.15.32:0/3166476978 >> 172.21.15.32:0/3166476978 conn(0x7f6b3005c620 msgr2=0x7f6b3005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- 172.21.15.32:0/3166476978 shutdown_connections 2022-08-01T23:02:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.074+0000 7f6b381cc700 1 -- 172.21.15.32:0/3166476978 wait complete. 2022-08-01T23:02:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 Processor -- start 2022-08-01T23:02:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 -- start start 2022-08-01T23:02:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b300b4310 0x7f6b30141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4820 0x7f6b30152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b300b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30152280 con 0x7f6b30146ae0 2022-08-01T23:02:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30141560 con 0x7f6b300b4310 2022-08-01T23:02:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30143040 con 0x7f6b300b4820 2022-08-01T23:02:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b300b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b300b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45106/0 (socket says 172.21.15.32:45106) 2022-08-01T23:02:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 -- 172.21.15.32:0/3974488234 learned_addr learned my addr 172.21.15.32:0/3974488234 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b35767700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4820 0x7f6b30152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b35f68700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b300b4310 0x7f6b30141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 -- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4820 msgr2=0x7f6b30152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4820 0x7f6b30152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 -- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b300b4310 msgr2=0x7f6b30141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b300b4310 0x7f6b30141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 -- 172.21.15.32:0/3974488234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2400b1e0 con 0x7f6b30146ae0 2022-08-01T23:02:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b35f68700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b300b4310 0x7f6b30141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b35767700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4820 0x7f6b30152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b36769700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b300b0210 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f6b28001250 tx=0x7f6b28007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2800e4a0 con 0x7f6b30146ae0 2022-08-01T23:02:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b30152470 con 0x7f6b30146ae0 2022-08-01T23:02:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2800e600 con 0x7f6b30146ae0 2022-08-01T23:02:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2800e960 con 0x7f6b30146ae0 2022-08-01T23:02:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.078+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b301423a0 con 0x7f6b30146ae0 2022-08-01T23:02:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.082+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b30152910 con 0x7f6b30146ae0 2022-08-01T23:02:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.082+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b28010070 con 0x7f6b30146ae0 2022-08-01T23:02:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.082+0000 7f6b22ffd700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b10053a10 0x7f6b10055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.082+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f6b2806b510 con 0x7f6b30146ae0 2022-08-01T23:02:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.082+0000 7f6b35f68700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b10053a10 0x7f6b10055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.082+0000 7f6b35f68700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b10053a10 0x7f6b10055ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f6b2401c000 tx=0x7f6b24015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.090+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b30152910 con 0x7f6b30146ae0 2022-08-01T23:02:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.218+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6b30152910 con 0x7f6b30146ae0 2022-08-01T23:02:27.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.218+0000 7f6b22ffd700 1 -- 172.21.15.32:0/3974488234 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v77) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6b30152910 con 0x7f6b30146ae0 2022-08-01T23:02:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:02:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b10053a10 msgr2=0x7f6b10055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b10053a10 0x7f6b10055ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f6b2401c000 tx=0x7f6b24015000).stop 2022-08-01T23:02:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 msgr2=0x7f6b300b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b300b0210 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f6b28001250 tx=0x7f6b28007dc0).stop 2022-08-01T23:02:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 shutdown_connections 2022-08-01T23:02:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b10053a10 0x7f6b10055ed0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b30146ae0 0x7f6b300b0210 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b300b4820 0x7f6b30152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 --2- 172.21.15.32:0/3974488234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b300b4310 0x7f6b30141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 >> 172.21.15.32:0/3974488234 conn(0x7f6b3005c620 msgr2=0x7f6b30144ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 shutdown_connections 2022-08-01T23:02:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.222+0000 7f6b381cc700 1 -- 172.21.15.32:0/3974488234 wait complete. 2022-08-01T23:02:27.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:02:27.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f3319700 1 Processor -- start 2022-08-01T23:02:27.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f3319700 1 -- start start 2022-08-01T23:02:27.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f3319700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9ec0ad750 0x7fd9ec061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f3319700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec0aee80 con 0x7fd9ec1405d0 2022-08-01T23:02:27.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f3319700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec0b70b0 con 0x7fd9ec0614b0 2022-08-01T23:02:27.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f3319700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0af0b0 con 0x7fd9ec0ad750 2022-08-01T23:02:27.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f2b18700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9ec1405d0 0x7fd9ec1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60888/0 (socket says 172.21.15.32:60888) 2022-08-01T23:02:27.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f2b18700 1 -- 172.21.15.32:0/1245038063 learned_addr learned my addr 172.21.15.32:0/1245038063 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:27.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/1245038063 >> v2:172.21.15.55:3300/0 conn(0x7fd9ec0ad750 0x7fd9ec061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575047202 0 0) 0x7fd9ec0aee80 con 0x7fd9ec1405d0 2022-08-01T23:02:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d4003260 con 0x7fd9ec1405d0 2022-08-01T23:02:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428940205 0 0) 0x7fd9ec0b70b0 con 0x7fd9ec0614b0 2022-08-01T23:02:27.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ec0aee80 con 0x7fd9ec0614b0 2022-08-01T23:02:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1201701913 0 0) 0x7fd9d4003260 con 0x7fd9ec1405d0 2022-08-01T23:02:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 >> v2:172.21.15.55:3300/0 conn(0x7fd9ec0ad750 msgr2=0x7fd9ec061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 --2- 172.21.15.32:0/1245038063 >> v2:172.21.15.55:3300/0 conn(0x7fd9ec0ad750 0x7fd9ec061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 >> v1:172.21.15.55:6789/0 conn(0x7fd9ec0614b0 legacy=0x7fd9ec0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0af350 con 0x7fd9ec1405d0 2022-08-01T23:02:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.346+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fd9e40019e0 con 0x7fd9ec1405d0 2022-08-01T23:02:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 >> v1:172.21.15.32:6789/0 conn(0x7fd9ec1405d0 legacy=0x7fd9ec1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d40038e0 0x7fd9d4003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9d4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4004e70 0x7fd9d400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0af0b0 con 0x7fd9d4004500 2022-08-01T23:02:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d400bcf0 con 0x7fd9d40038e0 2022-08-01T23:02:27.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d400beb0 con 0x7fd9d4004e70 2022-08-01T23:02:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9d4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f2317700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d40038e0 0x7fd9d4003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 -- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4004e70 msgr2=0x7fd9d400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9d4004e70 0x7fd9d400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T23:02:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 -- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d40038e0 msgr2=0x7fd9d4003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d40038e0 0x7fd9d4003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 -- 172.21.15.32:0/1245038063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4003260 con 0x7fd9d4004500 2022-08-01T23:02:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f2b18700 1 -- 172.21.15.32:0/1245038063 reap_dead start 2022-08-01T23:02:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f2317700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9d40038e0 0x7fd9d4003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:27.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9d4004980 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7fd9e800c000 tx=0x7fd9e8006df0).ready entity=mon.0 client_cookie=74ff4199f05c84c5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e800e800 con 0x7fd9d4004500 2022-08-01T23:02:27.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e800e960 con 0x7fd9d4004500 2022-08-01T23:02:27.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f1315700 1 -- 172.21.15.32:0/1245038063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e800ecc0 con 0x7fd9d4004500 2022-08-01T23:02:27.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 -- 172.21.15.32:0/1245038063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 msgr2=0x7fd9d4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9d4004980 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7fd9e800c000 tx=0x7fd9e8006df0).stop 2022-08-01T23:02:27.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 -- 172.21.15.32:0/1245038063 shutdown_connections 2022-08-01T23:02:27.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 --2- 172.21.15.32:0/1245038063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9d4004980 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 -- 172.21.15.32:0/1245038063 >> 172.21.15.32:0/1245038063 conn(0x7fd9ec066f70 msgr2=0x7fd9ec067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 -- 172.21.15.32:0/1245038063 shutdown_connections 2022-08-01T23:02:27.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 -- 172.21.15.32:0/1245038063 wait complete. 2022-08-01T23:02:27.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 Processor -- start 2022-08-01T23:02:27.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.350+0000 7fd9f3319700 1 -- start start 2022-08-01T23:02:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9ec0b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9ec13ea10 0x7fd9ec13ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9ec155dd0 0x7fd9ec156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b2e10 con 0x7fd9d4004500 2022-08-01T23:02:27.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b2b70 con 0x7fd9ec13ea10 2022-08-01T23:02:27.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b3190 con 0x7fd9ec155dd0 2022-08-01T23:02:27.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9ec0b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9ec0b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45122/0 (socket says 172.21.15.32:45122) 2022-08-01T23:02:27.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 -- 172.21.15.32:0/4113131028 learned_addr learned my addr 172.21.15.32:0/4113131028 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:27.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9ec13ea10 0x7fd9ec13ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2b18700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9ec155dd0 0x7fd9ec156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 -- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9ec155dd0 msgr2=0x7fd9ec156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9ec155dd0 0x7fd9ec156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 -- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9ec13ea10 msgr2=0x7fd9ec13ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9ec13ea10 0x7fd9ec13ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 -- 172.21.15.32:0/4113131028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0b3430 con 0x7fd9d4004500 2022-08-01T23:02:27.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f2317700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9ec0b7f70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fd9dc00b000 tx=0x7fd9dc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9dc014620 con 0x7fd9d4004500 2022-08-01T23:02:27.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc014780 con 0x7fd9d4004500 2022-08-01T23:02:27.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec0b36d0 con 0x7fd9d4004500 2022-08-01T23:02:27.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9dc014ae0 con 0x7fd9d4004500 2022-08-01T23:02:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec0b1bb0 con 0x7fd9d4004500 2022-08-01T23:02:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.354+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec0b2910 con 0x7fd9d4004500 2022-08-01T23:02:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.358+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd9dc014c40 con 0x7fd9d4004500 2022-08-01T23:02:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.358+0000 7fd9db7fe700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9cc053aa0 0x7fd9cc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.358+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fd9dc06c430 con 0x7fd9d4004500 2022-08-01T23:02:27.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.358+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9cc053aa0 0x7fd9cc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.358+0000 7fd9f1b16700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9cc053aa0 0x7fd9cc055f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fd9e800c000 tx=0x7fd9e8019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.362+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9dc0430c0 con 0x7fd9d4004500 2022-08-01T23:02:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.486+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd9ec0af5f0 con 0x7fd9d4004500 2022-08-01T23:02:27.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.486+0000 7fd9db7fe700 1 -- 172.21.15.32:0/4113131028 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd9dc0466e0 con 0x7fd9d4004500 2022-08-01T23:02:27.488 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:27.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9cc053aa0 msgr2=0x7fd9cc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9cc053aa0 0x7fd9cc055f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fd9e800c000 tx=0x7fd9e8019000).stop 2022-08-01T23:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 msgr2=0x7fd9ec0b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9ec0b7f70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fd9dc00b000 tx=0x7fd9dc009040).stop 2022-08-01T23:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 shutdown_connections 2022-08-01T23:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd9cc053aa0 0x7fd9cc055f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd9ec155dd0 0x7fd9ec156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9ec13ea10 0x7fd9ec13ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 --2- 172.21.15.32:0/4113131028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9d4004500 0x7fd9ec0b7f70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 >> 172.21.15.32:0/4113131028 conn(0x7fd9ec066f70 msgr2=0x7fd9ec13fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 shutdown_connections 2022-08-01T23:02:27.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.490+0000 7fd9f3319700 1 -- 172.21.15.32:0/4113131028 wait complete. 2022-08-01T23:02:27.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:02:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.610+0000 7f96ff59e700 1 Processor -- start 2022-08-01T23:02:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.610+0000 7f96ff59e700 1 -- start start 2022-08-01T23:02:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97001489f0 0x7f970014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9700067030 con 0x7f97000b31d0 2022-08-01T23:02:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97000b35b0 con 0x7f97000a9da0 2022-08-01T23:02:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9700064a70 con 0x7f97001489f0 2022-08-01T23:02:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97001489f0 0x7f970014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fe59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97000b31d0 0x7f9700066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60904/0 (socket says 172.21.15.32:60904) 2022-08-01T23:02:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fe59c700 1 -- 172.21.15.32:0/1813893041 learned_addr learned my addr 172.21.15.32:0/1813893041 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fed9d700 1 -- 172.21.15.32:0/1813893041 >> v1:172.21.15.55:6789/0 conn(0x7f97000a9da0 legacy=0x7f9700060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fed9d700 1 -- 172.21.15.32:0/1813893041 >> v1:172.21.15.32:6789/0 conn(0x7f97000b31d0 legacy=0x7f9700066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fed9d700 1 -- 172.21.15.32:0/1813893041 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700066af0 con 0x7f97001489f0 2022-08-01T23:02:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fed9d700 1 --2- 172.21.15.32:0/1813893041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97001489f0 0x7f970014ade0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f96f80032f0 tx=0x7f96f800b470).ready entity=mon.0 client_cookie=815e4d3fa6cc2319 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fd59a700 1 -- 172.21.15.32:0/1813893041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96f800de70 con 0x7f97001489f0 2022-08-01T23:02:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fd59a700 1 -- 172.21.15.32:0/1813893041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8010470 con 0x7f97001489f0 2022-08-01T23:02:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96fd59a700 1 -- 172.21.15.32:0/1813893041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96f80107d0 con 0x7f97001489f0 2022-08-01T23:02:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- 172.21.15.32:0/1813893041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97001489f0 msgr2=0x7f970014ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 --2- 172.21.15.32:0/1813893041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97001489f0 0x7f970014ade0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f96f80032f0 tx=0x7f96f800b470).stop 2022-08-01T23:02:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- 172.21.15.32:0/1813893041 shutdown_connections 2022-08-01T23:02:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 --2- 172.21.15.32:0/1813893041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97001489f0 0x7f970014ade0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- 172.21.15.32:0/1813893041 >> 172.21.15.32:0/1813893041 conn(0x7f9700063870 msgr2=0x7f9700063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- 172.21.15.32:0/1813893041 shutdown_connections 2022-08-01T23:02:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.614+0000 7f96ff59e700 1 -- 172.21.15.32:0/1813893041 wait complete. 2022-08-01T23:02:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 Processor -- start 2022-08-01T23:02:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 -- start start 2022-08-01T23:02:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97000a9da0 0x7f97000af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 0x7f970013bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97001489f0 0x7f9700153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97001402b0 con 0x7f97000b31d0 2022-08-01T23:02:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000afe50 con 0x7f97001489f0 2022-08-01T23:02:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97001526a0 con 0x7f97000a9da0 2022-08-01T23:02:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 0x7f970013bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 0x7f970013bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45138/0 (socket says 172.21.15.32:45138) 2022-08-01T23:02:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 -- 172.21.15.32:0/297430099 learned_addr learned my addr 172.21.15.32:0/297430099 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fe59c700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97000a9da0 0x7f97000af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fed9d700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97001489f0 0x7f9700153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 -- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97000a9da0 msgr2=0x7f97000af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97000a9da0 0x7f97000af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 -- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97001489f0 msgr2=0x7f9700153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97001489f0 0x7f9700153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 -- 172.21.15.32:0/297430099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f800b710 con 0x7f97000b31d0 2022-08-01T23:02:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96fdd9b700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 0x7f970013bb90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f97000634e0 tx=0x7f96e8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e8012400 con 0x7f97000b31d0 2022-08-01T23:02:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8012560 con 0x7f97000b31d0 2022-08-01T23:02:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9700152940 con 0x7f97000b31d0 2022-08-01T23:02:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e80128c0 con 0x7f97000b31d0 2022-08-01T23:02:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.618+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970013c640 con 0x7f97000b31d0 2022-08-01T23:02:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.622+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9700066d90 con 0x7f97000b31d0 2022-08-01T23:02:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.622+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96e8019030 con 0x7f97000b31d0 2022-08-01T23:02:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.622+0000 7f96f77fe700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96e4053b30 0x7f96e4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.622+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f96e806b470 con 0x7f97000b31d0 2022-08-01T23:02:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.622+0000 7f96fe59c700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96e4053b30 0x7f96e4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.622+0000 7f96fe59c700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96e4053b30 0x7f96e4055ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f96f000b000 tx=0x7f96f0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.630+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96e8042100 con 0x7f97000b31d0 2022-08-01T23:02:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97000574e0 con 0x7f97000b31d0 2022-08-01T23:02:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96f77fe700 1 -- 172.21.15.32:0/297430099 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+10 (secure 0 0 0) 0x7f96e8045720 con 0x7f97000b31d0 2022-08-01T23:02:27.752 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96e4053b30 msgr2=0x7f96e4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96e4053b30 0x7f96e4055ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f96f000b000 tx=0x7f96f0009040).stop 2022-08-01T23:02:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 msgr2=0x7f970013bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 0x7f970013bb90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f97000634e0 tx=0x7f96e8007c20).stop 2022-08-01T23:02:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 shutdown_connections 2022-08-01T23:02:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96e4053b30 0x7f96e4055ff0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97001489f0 0x7f9700153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97000b31d0 0x7f970013bb90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 --2- 172.21.15.32:0/297430099 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97000a9da0 0x7f97000af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 >> 172.21.15.32:0/297430099 conn(0x7f9700063870 msgr2=0x7f97000b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 shutdown_connections 2022-08-01T23:02:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.750+0000 7f96ff59e700 1 -- 172.21.15.32:0/297430099 wait complete. 2022-08-01T23:02:27.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:02:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 Processor -- start 2022-08-01T23:02:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 -- start start 2022-08-01T23:02:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40614b0 0x7f93d4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f93d41405d0 0x7f93d4140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d40aee80 con 0x7f93d40ad750 2022-08-01T23:02:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d40b70b0 con 0x7f93d41405d0 2022-08-01T23:02:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d9f90700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d40af0b0 con 0x7f93d40614b0 2022-08-01T23:02:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93d40ad750 0x7f93d40b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60918/0 (socket says 172.21.15.32:60918) 2022-08-01T23:02:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.874+0000 7f93d3fff700 1 -- 172.21.15.32:0/1059028732 learned_addr learned my addr 172.21.15.32:0/1059028732 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 --2- 172.21.15.32:0/1059028732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40614b0 0x7f93d4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d978f700 1 --2- 172.21.15.32:0/1059028732 >> v2:172.21.15.55:3300/0 conn(0x7f93d41405d0 0x7f93d4140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d37fe700 1 -- 172.21.15.32:0/1059028732 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715507804 0 0) 0x7f93d40aee80 con 0x7f93d40ad750 2022-08-01T23:02:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d37fe700 1 -- 172.21.15.32:0/1059028732 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c0003260 con 0x7f93d40ad750 2022-08-01T23:02:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1059028732 >> v2:172.21.15.55:3300/0 conn(0x7f93d41405d0 msgr2=0x7f93d4140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 --2- 172.21.15.32:0/1059028732 >> v2:172.21.15.55:3300/0 conn(0x7f93d41405d0 0x7f93d4140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1059028732 >> v1:172.21.15.32:6789/0 conn(0x7f93d40ad750 legacy=0x7f93d40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1059028732 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d40af350 con 0x7f93d40614b0 2022-08-01T23:02:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 --2- 172.21.15.32:0/1059028732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40614b0 0x7f93d4061890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f93c8002f70 tx=0x7f93c8006df0).ready entity=mon.2 client_cookie=c8d91c3828362eb2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d37fe700 1 -- 172.21.15.32:0/1059028732 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93c800d4a0 con 0x7f93d40614b0 2022-08-01T23:02:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d37fe700 1 -- 172.21.15.32:0/1059028732 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8000e90 con 0x7f93d40614b0 2022-08-01T23:02:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d37fe700 1 -- 172.21.15.32:0/1059028732 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93c800d7f0 con 0x7f93d40614b0 2022-08-01T23:02:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- 172.21.15.32:0/1059028732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40614b0 msgr2=0x7f93d4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1059028732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40614b0 0x7f93d4061890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f93c8002f70 tx=0x7f93c8006df0).stop 2022-08-01T23:02:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- 172.21.15.32:0/1059028732 shutdown_connections 2022-08-01T23:02:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1059028732 >> v2:172.21.15.55:3300/0 conn(0x7f93d41405d0 0x7f93d4140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1059028732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40614b0 0x7f93d4061890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- 172.21.15.32:0/1059028732 >> 172.21.15.32:0/1059028732 conn(0x7f93d4066f70 msgr2=0x7f93d4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:27.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- 172.21.15.32:0/1059028732 shutdown_connections 2022-08-01T23:02:27.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- 172.21.15.32:0/1059028732 wait complete. 2022-08-01T23:02:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 Processor -- start 2022-08-01T23:02:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- start start 2022-08-01T23:02:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 0x7f93d413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40ad750 0x7f93d413ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93d41405d0 0x7f93d413b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d413c8a0 con 0x7f93d40614b0 2022-08-01T23:02:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d413ca90 con 0x7f93d41405d0 2022-08-01T23:02:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d9f90700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d4145250 con 0x7f93d40ad750 2022-08-01T23:02:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 0x7f93d413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 0x7f93d413f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45150/0 (socket says 172.21.15.32:45150) 2022-08-01T23:02:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.878+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1456386876 learned_addr learned my addr 172.21.15.32:0/1456386876 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d978f700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93d41405d0 0x7f93d413b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d3fff700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40ad750 0x7f93d413ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40ad750 msgr2=0x7f93d413ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d8f8e700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40ad750 0x7f93d413ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93d41405d0 msgr2=0x7f93d413b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d8f8e700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93d41405d0 0x7f93d413b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d8f8e700 1 -- 172.21.15.32:0/1456386876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c800b180 con 0x7f93d40614b0 2022-08-01T23:02:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d978f700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93d41405d0 0x7f93d413b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d8f8e700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 0x7f93d413f2a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f93c8002f70 tx=0x7f93c8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93c8016070 con 0x7f93d40614b0 2022-08-01T23:02:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8013a40 con 0x7f93d40614b0 2022-08-01T23:02:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d4145e50 con 0x7f93d40614b0 2022-08-01T23:02:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93c8013da0 con 0x7f93d40614b0 2022-08-01T23:02:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d41462f0 con 0x7f93d40614b0 2022-08-01T23:02:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d40af5f0 con 0x7f93d40614b0 2022-08-01T23:02:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f93c8025030 con 0x7f93d40614b0 2022-08-01T23:02:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d1ffb700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93b4053950 0x7f93b4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f93c806d060 con 0x7f93d40614b0 2022-08-01T23:02:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.882+0000 7f93d3fff700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93b4053950 0x7f93b4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.886+0000 7f93d3fff700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93b4053950 0x7f93b4055e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f93c400b000 tx=0x7f93c4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:27.890+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93c8043e00 con 0x7f93d40614b0 2022-08-01T23:02:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.014+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93d40b4050 con 0x7f93d40614b0 2022-08-01T23:02:28.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d1ffb700 1 -- 172.21.15.32:0/1456386876 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+10 (secure 0 0 0) 0x7f93c80230a0 con 0x7f93d40614b0 2022-08-01T23:02:28.018 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:28.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93b4053950 msgr2=0x7f93b4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93b4053950 0x7f93b4055e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f93c400b000 tx=0x7f93c4009040).stop 2022-08-01T23:02:28.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 msgr2=0x7f93d413f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 0x7f93d413f2a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f93c8002f70 tx=0x7f93c8014000).stop 2022-08-01T23:02:28.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 shutdown_connections 2022-08-01T23:02:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93b4053950 0x7f93b4055e10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93d41405d0 0x7f93d413b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f93d40ad750 0x7f93d413ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 --2- 172.21.15.32:0/1456386876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d40614b0 0x7f93d413f2a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 >> 172.21.15.32:0/1456386876 conn(0x7f93d4066f70 msgr2=0x7f93d41423a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 shutdown_connections 2022-08-01T23:02:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.018+0000 7f93d9f90700 1 -- 172.21.15.32:0/1456386876 wait complete. 2022-08-01T23:02:28.031 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:02:28.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 Processor -- start 2022-08-01T23:02:28.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 -- start start 2022-08-01T23:02:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcab00b41f0 0x7fcab0061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcab0142ff0 0x7fcab0143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab00b60e0 con 0x7fcab00b6420 2022-08-01T23:02:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab00b2100 con 0x7fcab0142ff0 2022-08-01T23:02:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab63c6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab00b23a0 con 0x7fcab00b41f0 2022-08-01T23:02:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab53c4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcab00b6420 0x7fcab0061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60932/0 (socket says 172.21.15.32:60932) 2022-08-01T23:02:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab53c4700 1 -- 172.21.15.32:0/3723895170 learned_addr learned my addr 172.21.15.32:0/3723895170 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.138+0000 7fcab5bc5700 1 --2- 172.21.15.32:0/3723895170 >> v2:172.21.15.32:3300/0 conn(0x7fcab0142ff0 0x7fcab0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab4bc3700 1 --2- 172.21.15.32:0/3723895170 >> v2:172.21.15.55:3300/0 conn(0x7fcab00b41f0 0x7fcab0061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcaa3fff700 1 -- 172.21.15.32:0/3723895170 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711318312 0 0) 0x7fcaac001110 con 0x7fcab00b6420 2022-08-01T23:02:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcaa3fff700 1 -- 172.21.15.32:0/3723895170 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca94003260 con 0x7fcab00b6420 2022-08-01T23:02:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab5bc5700 1 -- 172.21.15.32:0/3723895170 >> v2:172.21.15.55:3300/0 conn(0x7fcab00b41f0 msgr2=0x7fcab0061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab5bc5700 1 --2- 172.21.15.32:0/3723895170 >> v2:172.21.15.55:3300/0 conn(0x7fcab00b41f0 0x7fcab0061d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab5bc5700 1 -- 172.21.15.32:0/3723895170 >> v1:172.21.15.32:6789/0 conn(0x7fcab00b6420 legacy=0x7fcab0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab5bc5700 1 -- 172.21.15.32:0/3723895170 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00b2640 con 0x7fcab0142ff0 2022-08-01T23:02:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab5bc5700 1 --2- 172.21.15.32:0/3723895170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab0142ff0 0x7fcab0143480 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7fcaa80032f0 tx=0x7fcaa80091d0).ready entity=mon.0 client_cookie=d737d03bc5b45298 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcaa3fff700 1 -- 172.21.15.32:0/3723895170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaa800b600 con 0x7fcab0142ff0 2022-08-01T23:02:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcaa3fff700 1 -- 172.21.15.32:0/3723895170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa800b760 con 0x7fcab0142ff0 2022-08-01T23:02:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcaa3fff700 1 -- 172.21.15.32:0/3723895170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaa800b8d0 con 0x7fcab0142ff0 2022-08-01T23:02:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 -- 172.21.15.32:0/3723895170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab0142ff0 msgr2=0x7fcab0143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3723895170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab0142ff0 0x7fcab0143480 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7fcaa80032f0 tx=0x7fcaa80091d0).stop 2022-08-01T23:02:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 -- 172.21.15.32:0/3723895170 shutdown_connections 2022-08-01T23:02:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3723895170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab0142ff0 0x7fcab0143480 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3723895170 >> v2:172.21.15.55:3300/0 conn(0x7fcab00b41f0 0x7fcab0061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 -- 172.21.15.32:0/3723895170 >> 172.21.15.32:0/3723895170 conn(0x7fcab005c620 msgr2=0x7fcab005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 -- 172.21.15.32:0/3723895170 shutdown_connections 2022-08-01T23:02:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 -- 172.21.15.32:0/3723895170 wait complete. 2022-08-01T23:02:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 Processor -- start 2022-08-01T23:02:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.142+0000 7fcab63c6700 1 -- start start 2022-08-01T23:02:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 0x7fcab0146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcab00b6420 0x7fcab012aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcab0142ff0 0x7fcab012b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00b0a60 con 0x7fcab00b41f0 2022-08-01T23:02:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0140410 con 0x7fcab00b6420 2022-08-01T23:02:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab013fa30 con 0x7fcab0142ff0 2022-08-01T23:02:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 0x7fcab0146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 0x7fcab0146c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45164/0 (socket says 172.21.15.32:45164) 2022-08-01T23:02:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 -- 172.21.15.32:0/3737079848 learned_addr learned my addr 172.21.15.32:0/3737079848 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab5bc5700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcab0142ff0 0x7fcab012b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab4bc3700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcab00b6420 0x7fcab012aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 -- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcab0142ff0 msgr2=0x7fcab012b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcab0142ff0 0x7fcab012b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 -- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcab00b6420 msgr2=0x7fcab012aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcab00b6420 0x7fcab012aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 -- 172.21.15.32:0/3737079848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa8009470 con 0x7fcab00b41f0 2022-08-01T23:02:28.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab5bc5700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcab0142ff0 0x7fcab012b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:28.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab4bc3700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcab00b6420 0x7fcab012aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:28.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab53c4700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 0x7fcab0146c50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fcaac0012a0 tx=0x7fcaac007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:28.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaac00ce70 con 0x7fcab00b41f0 2022-08-01T23:02:28.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac0072e0 con 0x7fcab00b41f0 2022-08-01T23:02:28.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab013fcd0 con 0x7fcab00b41f0 2022-08-01T23:02:28.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcaac005800 con 0x7fcab00b41f0 2022-08-01T23:02:28.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab012b8b0 con 0x7fcab00b41f0 2022-08-01T23:02:28.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab00b28e0 con 0x7fcab00b41f0 2022-08-01T23:02:28.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.146+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcaac01a060 con 0x7fcab00b41f0 2022-08-01T23:02:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.150+0000 7fcaa27fc700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca98053b00 0x7fca98055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.150+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fcaac06b710 con 0x7fcab00b41f0 2022-08-01T23:02:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.150+0000 7fcab4bc3700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca98053b00 0x7fca98055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.150+0000 7fcab4bc3700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca98053b00 0x7fca98055fc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fcaa400b000 tx=0x7fcaa4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:28.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.154+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcaac0424b0 con 0x7fcab00b41f0 2022-08-01T23:02:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.317+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab00b60e0 con 0x7fcab00b41f0 2022-08-01T23:02:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.317+0000 7fcaa27fc700 1 -- 172.21.15.32:0/3737079848 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fcab00b60e0 con 0x7fcab00b41f0 2022-08-01T23:02:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:28.321 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:02:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca98053b00 msgr2=0x7fca98055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca98053b00 0x7fca98055fc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fcaa400b000 tx=0x7fcaa4008040).stop 2022-08-01T23:02:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 msgr2=0x7fcab0146c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 0x7fcab0146c50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fcaac0012a0 tx=0x7fcaac007920).stop 2022-08-01T23:02:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 shutdown_connections 2022-08-01T23:02:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fca98053b00 0x7fca98055fc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcab0142ff0 0x7fcab012b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcab00b6420 0x7fcab012aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 --2- 172.21.15.32:0/3737079848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab00b41f0 0x7fcab0146c50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 >> 172.21.15.32:0/3737079848 conn(0x7fcab005c620 msgr2=0x7fcab00b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 shutdown_connections 2022-08-01T23:02:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:28.321+0000 7fcab63c6700 1 -- 172.21.15.32:0/3737079848 wait complete. 2022-08-01T23:02:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:02:28.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:01:34.459266+0000', 'modified': '2022-08-01T23:01:35.461964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6258}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6258': {'gid': 6258, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1927724642', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1927724642}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1927724642}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:02:28.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:02:28.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:02:28.760 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6258,"want_state":"up:active","state":"up:active","rank_uptime":54.288004235000002,"mdsmap_epoch":33,"osdmap_epoch":77,"osdmap_epoch_barrier":77,"uptime":58.605756997} 2022-08-01T23:02:28.760 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6258, "mdsmap_epoch": 33, "osdmap_epoch": 77, "osdmap_epoch_barrier": 77, "rank_uptime": 54.288004235, "state": "up:active", "uptime": 58.605756997, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:02:28.761 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:02:28.761 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:02:28.762 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:02:28.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:28.763 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:28.771 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:02:28.786 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:28.786 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:28.842 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:28.843 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:28.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:28.844 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:02:28.849 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:02:28.850 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:28.852 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:02:28.871 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:28.871 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:28.880 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:28.881 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:28.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:28.883 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:02:28.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:02:28.926+0000 7f584402d080 -1 init, newargv = 0x55b569f33870 newargc=9ceph-fuse[ 2022-08-01T23:02:28.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:23058]: starting ceph client 2022-08-01T23:02:28.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23058]: starting fuse 2022-08-01T23:02:30.032 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:02:30.050 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:30.051 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:30.061 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:30.062 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:30.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.063 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:02:30.113 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:02:30.114 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:02:30.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.115 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:02:30.115 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:02:30.116 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:02:30.116 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:02:30.116 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:02:30.117 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:30.117 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:02:30.118 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:02:30.118 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:02:30.118 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:30.119 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:02:30.119 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:02:30.120 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:02:30.120 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:02:30.120 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:30.121 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:02:30.121 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:02:30.122 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:02:30.122 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:02:30.122 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:02:30.123 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:30.123 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:02:30.124 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:02:30.166 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23058.asok 2022-08-01T23:02:30.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23058.asok 2022-08-01T23:02:30.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.172 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23058.asok status 2022-08-01T23:02:30.309 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:02:30.309 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:02:30.309 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:02:30.309 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:02:30.310 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:02:30.310 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:02:30.310 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:02:30.310 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23058", 2022-08-01T23:02:30.311 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:02:30.311 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:30.311 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:02:30.311 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:02:30.312 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 6540, 2022-08-01T23:02:30.312 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:02:30.312 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:02:30.312 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:02:30.312 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 6540 2022-08-01T23:02:30.313 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:30.313 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:02:30.313 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:02:30.313 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:02:30.314 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 823985288 2022-08-01T23:02:30.314 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:02:30.314 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:30.314 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:02:30.315 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:02:30.315 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:02:30.315 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 823985288 2022-08-01T23:02:30.315 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:30.316 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.6540 172.21.15.130:0/823985288", 2022-08-01T23:02:30.316 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/823985288", 2022-08-01T23:02:30.316 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:02:30.316 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 33, 2022-08-01T23:02:30.316 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 77, 2022-08-01T23:02:30.317 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:02:30.317 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:02:30.317 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:02:30.317 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:02:30.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.319 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:30.320 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:30.326 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:02:30.328 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:02:30.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.329 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:30.344 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:02:30.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:30.366 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:30.387 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:30.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:30.428 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:02:30.429 INFO:tasks.cephfs.mount:Terminating background process 2022-08-01T23:02:30.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.430 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:30.430 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:30.438 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:02:30.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:02:30.440 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:02:30.440 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:02:30.441 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:30.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23058]: fuse finished with error 0 and tester_r 0 2022-08-01T23:02:30.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.500 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:30.500 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:30.506 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:02:30.507 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:02:30.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:02:30.624 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:30.632 INFO:tasks.cephfs.mount:Terminating background process 2022-08-01T23:02:30.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.633 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:30.634 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:30.642 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:02:30.643 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:02:30.644 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:02:30.644 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:02:30.644 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:30.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22576]: fuse finished with error 0 and tester_r 0 2022-08-01T23:02:30.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:30.662 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:30.663 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:02:30.713 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:02:30.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:02:30.833 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:30.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:02:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.945+0000 7fb42d2eb700 1 Processor -- start 2022-08-01T23:02:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.945+0000 7fb42d2eb700 1 -- start start 2022-08-01T23:02:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4280b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4280b6af0 0x7fb42812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb428144240 con 0x7fb4280b6710 2022-08-01T23:02:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb428143d00 con 0x7fb4280b6af0 2022-08-01T23:02:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb428143fa0 con 0x7fb4280b4550 2022-08-01T23:02:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4280b6af0 0x7fb42812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4280b6af0 0x7fb42812bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45176/0 (socket says 172.21.15.32:45176) 2022-08-01T23:02:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 -- 172.21.15.32:0/1157622307 learned_addr learned my addr 172.21.15.32:0/1157622307 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:30.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb41e7fc700 1 --2- 172.21.15.32:0/1157622307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4280b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 -- 172.21.15.32:0/1157622307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 msgr2=0x7fb4280b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 --2- 172.21.15.32:0/1157622307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4280b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 -- 172.21.15.32:0/1157622307 >> v1:172.21.15.55:6789/0 conn(0x7fb4280b6710 legacy=0x7fb428061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4267fc700 1 -- 172.21.15.32:0/1157622307 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373156268 0 0) 0x7fb428144240 con 0x7fb4280b6710 2022-08-01T23:02:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 -- 172.21.15.32:0/1157622307 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb428061780 con 0x7fb4280b6af0 2022-08-01T23:02:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4277fe700 1 --2- 172.21.15.32:0/1157622307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb42812bb90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7fb4200032f0 tx=0x7fb420009290).ready entity=mon.0 client_cookie=c6e86dc3ef5e632d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4267fc700 1 -- 172.21.15.32:0/1157622307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb42000b8a0 con 0x7fb4280b6af0 2022-08-01T23:02:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4267fc700 1 -- 172.21.15.32:0/1157622307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42000ba00 con 0x7fb4280b6af0 2022-08-01T23:02:30.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb4267fc700 1 -- 172.21.15.32:0/1157622307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb42000bb70 con 0x7fb4280b6af0 2022-08-01T23:02:30.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1157622307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 msgr2=0x7fb42812bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:30.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1157622307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb42812bb90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7fb4200032f0 tx=0x7fb420009290).stop 2022-08-01T23:02:30.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1157622307 shutdown_connections 2022-08-01T23:02:30.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1157622307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb42812bb90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:30.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1157622307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4280b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:30.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1157622307 >> 172.21.15.32:0/1157622307 conn(0x7fb42805c620 msgr2=0x7fb42805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:30.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1157622307 shutdown_connections 2022-08-01T23:02:30.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.949+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1157622307 wait complete. 2022-08-01T23:02:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.953+0000 7fb42d2eb700 1 Processor -- start 2022-08-01T23:02:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.953+0000 7fb42d2eb700 1 -- start start 2022-08-01T23:02:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4281447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4280b6710 0x7fb428155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb4281563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb428126a80 con 0x7fb4280b6af0 2022-08-01T23:02:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb428155a60 con 0x7fb4280b6710 2022-08-01T23:02:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb428144cd0 con 0x7fb4280b4550 2022-08-01T23:02:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb4281563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb4281563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45182/0 (socket says 172.21.15.32:45182) 2022-08-01T23:02:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb41e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4280b6710 0x7fb428155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb426ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4281447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 -- 172.21.15.32:0/1465481845 learned_addr learned my addr 172.21.15.32:0/1465481845 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 -- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 msgr2=0x7fb4281447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4281447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 -- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4280b6710 msgr2=0x7fb428155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4280b6710 0x7fb428155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 -- 172.21.15.32:0/1465481845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420009530 con 0x7fb4280b6af0 2022-08-01T23:02:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb4277fe700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb4281563f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fb4200032f0 tx=0x7fb420016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb42000b8a0 con 0x7fb4280b6af0 2022-08-01T23:02:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420021630 con 0x7fb4280b6af0 2022-08-01T23:02:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4281263f0 con 0x7fb4280b6af0 2022-08-01T23:02:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb420021990 con 0x7fb4280b6af0 2022-08-01T23:02:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb428127de0 con 0x7fb4280b6af0 2022-08-01T23:02:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.957+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb428155c20 con 0x7fb4280b6af0 2022-08-01T23:02:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.961+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb420027020 con 0x7fb4280b6af0 2022-08-01T23:02:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.961+0000 7fb424ff9700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb40c053b00 0x7fb40c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.961+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fb42006d4e0 con 0x7fb4280b6af0 2022-08-01T23:02:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.961+0000 7fb426ffd700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb40c053b00 0x7fb40c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.961+0000 7fb426ffd700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb40c053b00 0x7fb40c055fc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb41800d000 tx=0x7fb418007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:30.969+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb420044170 con 0x7fb4280b6af0 2022-08-01T23:02:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.097+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb428061a20 con 0x7fb4280b6af0 2022-08-01T23:02:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb424ff9700 1 -- 172.21.15.32:0/1465481845 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v77) v1 ==== 81+0+34116 (secure 0 0 0) 0x7fb420015090 con 0x7fb4280b6af0 2022-08-01T23:02:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:31.104 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:02:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 77, 2022-08-01T23:02:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:02:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:02:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:02:20.401055+0000", 2022-08-01T23:02:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:02:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:02:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:02:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:02:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:02:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:02:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:02:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:02:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:02:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:02:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:02:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:02:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:02:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:02:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 14, 2022-08-01T23:02:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:02:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:02:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:02:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:02:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:02:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:02:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:02:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:02:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:02:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:02:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:02:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:02:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:02:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:02:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:31.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:31.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:31.119 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:31.119 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:31.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:31.120 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:31.120 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:31.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:31.120 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:31.121 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:31.121 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:31.121 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:31.121 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:31.122 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:31.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:31.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.122 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:31.123 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:31.123 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:31.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:31.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:31.123 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:31.124 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:31.124 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:31.124 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:31.124 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:31.124 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:31.125 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:02:31.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:02:31.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.125 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:31.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:02:31.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.127 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:02:31.127 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:02:31.127 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:02:31.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:31.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:31.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:31.128 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:31.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:31.128 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:31.128 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:31.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:31.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:31.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:31.129 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:31.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:31.130 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:31.130 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:31.130 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:31.131 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:31.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:31.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:31.131 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:31.131 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:02:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:31.134 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:31.134 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:31.134 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:31.134 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:31.134 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:31.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:31.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:31.135 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:31.135 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:31.136 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:31.136 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:31.136 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:31.136 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:31.136 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:31.137 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:31.137 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:31.137 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:31.137 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:31.138 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:31.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:31.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.138 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:31.138 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:31.139 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:31.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:31.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:31.139 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:31.140 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:31.140 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:31.140 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:31.140 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:31.140 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:31.141 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:02:31.141 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:31.141 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:02:31.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.142 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 13, 2022-08-01T23:02:31.142 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:02:31.143 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:01:32.889134+0000", 2022-08-01T23:02:31.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:31.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:31.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:31.143 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:31.144 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:31.144 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:31.144 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:31.144 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:31.144 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:31.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:31.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:31.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:31.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:31.146 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:31.146 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:31.146 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:31.146 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:31.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:31.147 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:31.147 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:31.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.147 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "76", 2022-08-01T23:02:31.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:31.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:31.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:31.148 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:31.149 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:31.149 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:31.149 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:31.149 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:31.149 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:31.150 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:31.150 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:31.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:31.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:31.151 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:31.151 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:31.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:31.151 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:31.151 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:31.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:31.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:31.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:31.152 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:31.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:31.153 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:31.153 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:31.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:31.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.154 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:31.154 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:31.154 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:31.154 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:31.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:31.155 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:31.155 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:31.155 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:31.155 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:31.156 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:31.156 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:31.156 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:02:31.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:02:31.156 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:02:31.157 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:02:31.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.157 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:31.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:02:31.158 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:02:31.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.159 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 14, 2022-08-01T23:02:31.159 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:02:31.159 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:01:33.885837+0000", 2022-08-01T23:02:31.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:31.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:31.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:31.160 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:31.160 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:31.160 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:31.161 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:31.161 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:31.161 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:31.161 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:31.161 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:31.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:31.162 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:31.162 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:31.162 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:31.163 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:31.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:31.163 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:31.163 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:31.164 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:31.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "76", 2022-08-01T23:02:31.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:31.164 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:31.165 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:31.165 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:31.165 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:31.165 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:31.165 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:31.166 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:31.166 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:31.166 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:31.166 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:31.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:31.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:31.167 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:31.167 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:31.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:31.168 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:31.168 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:31.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:31.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:31.169 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:31.169 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:31.169 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:31.169 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:31.170 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:31.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:31.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.170 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:31.170 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:31.171 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:31.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:31.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:31.171 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:31.171 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:31.172 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:31.172 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:31.172 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:31.172 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:31.172 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:02:31.173 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:31.173 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:02:31.173 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:02:31.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.174 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:31.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:02:31.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:02:31.175 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:02:31.175 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.176 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.176 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.176 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.176 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.177 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.177 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:31.177 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.177 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.178 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:02:31.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:02:31.180 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.181 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.181 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:02:31.182 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:02:31.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.184 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.185 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:02:31.185 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:02:31.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.187 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.188 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:02:31.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.189 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:02:31.189 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:31.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.190 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:02:31.190 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:02:31.191 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:02:31.191 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:02:31.191 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.191 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.192 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:02:31.193 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:02:31.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.193 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.193 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.194 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.194 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.194 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:02:31.195 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.195 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.195 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:02:31.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.197 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:02:31.197 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.198 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:02:31.199 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.200 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:02:31.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.201 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.201 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:02:31.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:02:31.203 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.204 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:02:31.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:02:31.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:31.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.207 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.207 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:02:31.208 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:02:31.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:02:31.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:02:31.208 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.209 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.209 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.209 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.210 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:02:31.210 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:02:31.210 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.210 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.211 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.211 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.211 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.211 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.212 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.212 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:31.212 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.212 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.212 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:02:31.214 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:02:31.215 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.216 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:02:31.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:02:31.218 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.219 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.219 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.220 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:02:31.220 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:02:31.221 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.222 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.222 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:02:31.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:02:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:31.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.225 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.225 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:02:31.225 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:02:31.226 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:02:31.226 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:02:31.226 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.226 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.227 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:02:31.228 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:02:31.228 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.228 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.228 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.229 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.229 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.229 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:31.230 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.230 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.230 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.230 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.231 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:02:31.231 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:02:31.232 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.233 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.233 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:02:31.234 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.235 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:02:31.236 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.237 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.237 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.237 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:02:31.238 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.239 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:02:31.239 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.240 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.240 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.241 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:02:31.241 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.242 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:02:31.242 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:31.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.243 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:02:31.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:02:31.243 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:02:31.244 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:02:31.244 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.244 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.244 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.245 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:02:31.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:02:31.246 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.246 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.246 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.246 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.246 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.247 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.247 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.247 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 73, 2022-08-01T23:02:31.247 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.248 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.248 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:02:31.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:02:31.250 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.251 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.252 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:02:31.252 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:02:31.253 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.254 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.254 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.254 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.255 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:02:31.255 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.256 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:02:31.256 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.257 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.257 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.258 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:02:31.258 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.259 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:02:31.259 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:31.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.260 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:02:31.260 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:02:31.261 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:02:31.261 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:02:31.261 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.261 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.261 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.262 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.262 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:02:31.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:02:31.263 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.263 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.263 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.264 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.264 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.264 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:31.265 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.265 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.265 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.265 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:02:31.266 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:02:31.267 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.268 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.269 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:02:31.269 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:02:31.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.271 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.271 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.272 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:02:31.272 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.273 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:02:31.273 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.274 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.274 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.275 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:02:31.275 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:02:31.276 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:31.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.277 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:02:31.277 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:02:31.278 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:02:31.278 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:02:31.278 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.278 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.279 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.280 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:02:31.280 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:02:31.280 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.280 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.281 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.281 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.281 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.281 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.281 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.282 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:02:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:02:31.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:02:31.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:02:31.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.289 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.290 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:02:31.290 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.291 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:02:31.291 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.291 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.292 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.292 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.293 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:02:31.293 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:02:31.294 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:31.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.295 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:02:31.295 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:02:31.295 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:02:31.296 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:02:31.296 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.296 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.296 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.297 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:02:31.297 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:02:31.298 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:31.298 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:31.298 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:31.298 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:31.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:31.299 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:31.299 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:31.299 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 73, 2022-08-01T23:02:31.299 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:31.300 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:31.300 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:31.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:02:31.301 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:02:31.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.303 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:31.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:02:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:02:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.306 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:31.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.307 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:02:31.307 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.308 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:02:31.308 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.309 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:31.309 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:31.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:31.310 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:02:31.310 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:31.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:02:31.311 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:31.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.312 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:02:31.312 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:02:31.313 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:02:31.313 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:02:31.313 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:31.313 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:31.313 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:02:31.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:02:31.315 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.315 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.316 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.316 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.316 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.316 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:02:31.316 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.317 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:02:31.317 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.318 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.318 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.318 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.318 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.318 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:02:31.319 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.319 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:02:31.320 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.320 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.320 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.320 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.321 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.321 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:02:31.321 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.322 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:02:31.322 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.322 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.322 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.323 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.323 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:02:31.323 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.324 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:02:31.324 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.324 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.325 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.325 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.325 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.325 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:02:31.325 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.326 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:02:31.326 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.327 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.327 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.327 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.327 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.328 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:02:31.328 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.328 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:02:31.329 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.329 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.329 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.329 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.330 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.330 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:02:31.330 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:31.331 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:02:31.331 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:31.331 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:31.331 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:31.332 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:31.332 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:31.332 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:02:31.332 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:31.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.333 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:31.333 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:02:31.333 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:02:31.333 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:02:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:02:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:02:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.130:0/438018148": "2022-08-02T00:02:20.149979+0000" 2022-08-01T23:02:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:02:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:02:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:02:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:02:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:02:31.336 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:02:31.336 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:02:31.336 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:02:31.336 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:02:31.336 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:02:31.337 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:02:31.337 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:02:31.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:02:31.337 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:02:31.338 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:02:31.338 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:02:31.338 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:02:31.338 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:02:31.339 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:02:31.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:31.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:31.339 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:02:31.339 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:02:31.340 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:02:31.340 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:02:31.340 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:02:31.340 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:02:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb40c053b00 msgr2=0x7fb40c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb40c053b00 0x7fb40c055fc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb41800d000 tx=0x7fb418007b90).stop 2022-08-01T23:02:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 msgr2=0x7fb4281563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb4281563f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fb4200032f0 tx=0x7fb420016000).stop 2022-08-01T23:02:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 shutdown_connections 2022-08-01T23:02:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb40c053b00 0x7fb40c055fc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4280b6af0 0x7fb4281563f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4280b6710 0x7fb428155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 --2- 172.21.15.32:0/1465481845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4280b4550 0x7fb4281447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 >> 172.21.15.32:0/1465481845 conn(0x7fb42805c620 msgr2=0x7fb428061de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 shutdown_connections 2022-08-01T23:02:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.101+0000 7fb42d2eb700 1 -- 172.21.15.32:0/1465481845 wait complete. 2022-08-01T23:02:31.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:31.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 Processor -- start 2022-08-01T23:02:31.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 -- start start 2022-08-01T23:02:31.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280b2710 0x7f432814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43280b01a0 0x7f432814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280a7250 con 0x7f43280afd60 2022-08-01T23:02:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43280a7480 con 0x7f43280b01a0 2022-08-01T23:02:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432e00a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280a7720 con 0x7f43280b2710 2022-08-01T23:02:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432c807700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43280b01a0 0x7f432814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432d809700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43280afd60 0x7f43280a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60964/0 (socket says 172.21.15.32:60964) 2022-08-01T23:02:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432d809700 1 -- 172.21.15.32:0/1832760597 learned_addr learned my addr 172.21.15.32:0/1832760597 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432d008700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280b2710 0x7f432814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f431ffff700 1 -- 172.21.15.32:0/1832760597 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244451376 0 0) 0x7f43280a7250 con 0x7f43280afd60 2022-08-01T23:02:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f431ffff700 1 -- 172.21.15.32:0/1832760597 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4314003260 con 0x7f43280afd60 2022-08-01T23:02:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432c807700 1 -- 172.21.15.32:0/1832760597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280b2710 msgr2=0x7f432814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432c807700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280b2710 0x7f432814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432c807700 1 -- 172.21.15.32:0/1832760597 >> v1:172.21.15.32:6789/0 conn(0x7f43280afd60 legacy=0x7f43280a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432c807700 1 -- 172.21.15.32:0/1832760597 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280a79c0 con 0x7f43280b01a0 2022-08-01T23:02:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432d008700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280b2710 0x7f432814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f432c807700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f432814b4c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f43200032f0 tx=0x7f43200091d0).ready entity=mon.0 client_cookie=84f8161f630a95e5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f431ffff700 1 -- 172.21.15.32:0/1832760597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432000b600 con 0x7f43280b01a0 2022-08-01T23:02:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f431ffff700 1 -- 172.21.15.32:0/1832760597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000b760 con 0x7f43280b01a0 2022-08-01T23:02:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.461+0000 7f431ffff700 1 -- 172.21.15.32:0/1832760597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432000b8d0 con 0x7f43280b01a0 2022-08-01T23:02:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 -- 172.21.15.32:0/1832760597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 msgr2=0x7f432814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f432814b4c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f43200032f0 tx=0x7f43200091d0).stop 2022-08-01T23:02:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 -- 172.21.15.32:0/1832760597 shutdown_connections 2022-08-01T23:02:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f432814b4c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 --2- 172.21.15.32:0/1832760597 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280b2710 0x7f432814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 -- 172.21.15.32:0/1832760597 >> 172.21.15.32:0/1832760597 conn(0x7f4328060310 msgr2=0x7f43280b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 -- 172.21.15.32:0/1832760597 shutdown_connections 2022-08-01T23:02:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 -- 172.21.15.32:0/1832760597 wait complete. 2022-08-01T23:02:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 Processor -- start 2022-08-01T23:02:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.465+0000 7f432e00a700 1 -- start start 2022-08-01T23:02:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280afd60 0x7f4328143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f4328141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43280b2710 0x7f4328141c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432814d200 con 0x7f43280b01a0 2022-08-01T23:02:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328157570 con 0x7f43280b2710 2022-08-01T23:02:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4328142100 con 0x7f43280afd60 2022-08-01T23:02:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f4328141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f4328141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45194/0 (socket says 172.21.15.32:45194) 2022-08-01T23:02:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 -- 172.21.15.32:0/4061612721 learned_addr learned my addr 172.21.15.32:0/4061612721 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432d809700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43280b2710 0x7f4328141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432d008700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280afd60 0x7f4328143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 -- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280afd60 msgr2=0x7f4328143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280afd60 0x7f4328143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 -- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43280b2710 msgr2=0x7f4328141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43280b2710 0x7f4328141c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 -- 172.21.15.32:0/4061612721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432400b180 con 0x7f43280b01a0 2022-08-01T23:02:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432d809700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43280b2710 0x7f4328141c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432c807700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f4328141720 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f43200032f0 tx=0x7f4320016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432000b8d0 con 0x7f43280b01a0 2022-08-01T23:02:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432001f3f0 con 0x7f43280b01a0 2022-08-01T23:02:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432000b600 con 0x7f43280b01a0 2022-08-01T23:02:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4320009470 con 0x7f43280b01a0 2022-08-01T23:02:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4328157030 con 0x7f43280b01a0 2022-08-01T23:02:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.469+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43280574e0 con 0x7f43280b01a0 2022-08-01T23:02:31.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.473+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4320026030 con 0x7f43280b01a0 2022-08-01T23:02:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.473+0000 7f431e7fc700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4308053b00 0x7f4308055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.473+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f432006cd20 con 0x7f43280b01a0 2022-08-01T23:02:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.473+0000 7f432d008700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4308053b00 0x7f4308055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.473+0000 7f432d008700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4308053b00 0x7f4308055fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f432400b000 tx=0x7f4324006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:31.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.481+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43200439b0 con 0x7f43280b01a0 2022-08-01T23:02:31.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4328069a70 con 0x7f43280b01a0 2022-08-01T23:02:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f431e7fc700 1 -- 172.21.15.32:0/4061612721 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f4320024020 con 0x7f43280b01a0 2022-08-01T23:02:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:31.652 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:01:35.461964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6258":{"gid":6258,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1927724642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1927724642},{"type":"v1","addr":"172.21.15.32:6835","nonce":1927724642}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:02:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4308053b00 msgr2=0x7f4308055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4308053b00 0x7f4308055fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f432400b000 tx=0x7f4324006df0).stop 2022-08-01T23:02:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 msgr2=0x7f4328141720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f4328141720 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f43200032f0 tx=0x7f4320016000).stop 2022-08-01T23:02:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 shutdown_connections 2022-08-01T23:02:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4308053b00 0x7f4308055fc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43280b2710 0x7f4328141c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43280b01a0 0x7f4328141720 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 --2- 172.21.15.32:0/4061612721 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43280afd60 0x7f4328143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 >> 172.21.15.32:0/4061612721 conn(0x7f4328060310 msgr2=0x7f4328144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 shutdown_connections 2022-08-01T23:02:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.649+0000 7f432e00a700 1 -- 172.21.15.32:0/4061612721 wait complete. 2022-08-01T23:02:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2022-08-01T23:02:31.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:02:31.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 Processor -- start 2022-08-01T23:02:31.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 -- start start 2022-08-01T23:02:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc0b5630 0x7f8ccc0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8ccc0b07a0 0x7f8ccc147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ccc067730 0x7f8ccc0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc145470 con 0x7f8ccc0b07a0 2022-08-01T23:02:31.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0ae180 con 0x7f8ccc067730 2022-08-01T23:02:31.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd1f7f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0b5a10 con 0x7f8ccc0b5630 2022-08-01T23:02:31.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8ccc0b07a0 0x7f8ccc147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8ccc0b07a0 0x7f8ccc147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45200/0 (socket says 172.21.15.32:45200) 2022-08-01T23:02:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 -- 172.21.15.32:0/3669037521 learned_addr learned my addr 172.21.15.32:0/3669037521 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd177e700 1 --2- 172.21.15.32:0/3669037521 >> v2:172.21.15.55:3300/0 conn(0x7f8ccc067730 0x7f8ccc0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd0f7d700 1 --2- 172.21.15.32:0/3669037521 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc0b5630 0x7f8ccc0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 -- 172.21.15.32:0/3669037521 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc0b5630 msgr2=0x7f8ccc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 --2- 172.21.15.32:0/3669037521 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc0b5630 0x7f8ccc0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 -- 172.21.15.32:0/3669037521 >> v2:172.21.15.55:3300/0 conn(0x7f8ccc067730 msgr2=0x7f8ccc0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 --2- 172.21.15.32:0/3669037521 >> v2:172.21.15.55:3300/0 conn(0x7f8ccc067730 0x7f8ccc0acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 -- 172.21.15.32:0/3669037521 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0ac990 con 0x7f8ccc0b07a0 2022-08-01T23:02:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8cd177e700 1 --2- 172.21.15.32:0/3669037521 >> v2:172.21.15.55:3300/0 conn(0x7f8ccc067730 0x7f8ccc0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccbfff700 1 --2- 172.21.15.32:0/3669037521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc147730 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f8cbc0032f0 tx=0x7f8cbc009290).ready entity=mon.0 client_cookie=677021cec7b82389 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccb7fe700 1 -- 172.21.15.32:0/3669037521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cbc00b870 con 0x7f8ccc0b07a0 2022-08-01T23:02:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccb7fe700 1 -- 172.21.15.32:0/3669037521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc00b9d0 con 0x7f8ccc0b07a0 2022-08-01T23:02:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.769+0000 7f8ccb7fe700 1 -- 172.21.15.32:0/3669037521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cbc00bb40 con 0x7f8ccc0b07a0 2022-08-01T23:02:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/3669037521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b07a0 msgr2=0x7f8ccc147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/3669037521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc147730 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f8cbc0032f0 tx=0x7f8cbc009290).stop 2022-08-01T23:02:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/3669037521 shutdown_connections 2022-08-01T23:02:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/3669037521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc147730 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/3669037521 >> v2:172.21.15.55:3300/0 conn(0x7f8ccc067730 0x7f8ccc0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/3669037521 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc0b5630 0x7f8ccc0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/3669037521 >> 172.21.15.32:0/3669037521 conn(0x7f8ccc0626f0 msgr2=0x7f8ccc062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/3669037521 shutdown_connections 2022-08-01T23:02:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/3669037521 wait complete. 2022-08-01T23:02:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 Processor -- start 2022-08-01T23:02:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.773+0000 7f8cd1f7f700 1 -- start start 2022-08-01T23:02:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc067730 0x7f8ccc14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc0634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 0x7f8ccc15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc064e80 con 0x7f8ccc0b5630 2022-08-01T23:02:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc066f40 con 0x7f8ccc0b07a0 2022-08-01T23:02:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0639a0 con 0x7f8ccc067730 2022-08-01T23:02:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 0x7f8ccc15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 0x7f8ccc15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45210/0 (socket says 172.21.15.32:45210) 2022-08-01T23:02:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 -- 172.21.15.32:0/4057052841 learned_addr learned my addr 172.21.15.32:0/4057052841 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd0f7d700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc067730 0x7f8ccc14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8ccbfff700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc0634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 -- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc067730 msgr2=0x7f8ccc14d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc067730 0x7f8ccc14d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 -- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ccc0b07a0 msgr2=0x7f8ccc0634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc0634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 -- 172.21.15.32:0/4057052841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc009530 con 0x7f8ccc0b5630 2022-08-01T23:02:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd177e700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 0x7f8ccc15a080 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f8ccc063b30 tx=0x7f8cc4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cc4011780 con 0x7f8ccc0b5630 2022-08-01T23:02:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc40118e0 con 0x7f8ccc0b5630 2022-08-01T23:02:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc066ca0 con 0x7f8ccc0b5630 2022-08-01T23:02:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cc4011c40 con 0x7f8ccc0b5630 2022-08-01T23:02:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc15a570 con 0x7f8ccc0b5630 2022-08-01T23:02:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.777+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc0b1680 con 0x7f8ccc0b5630 2022-08-01T23:02:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.781+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8cc401c460 con 0x7f8ccc0b5630 2022-08-01T23:02:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.781+0000 7f8cc9ffb700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cac053a70 0x7f8cac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.781+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f8cc406c2b0 con 0x7f8ccc0b5630 2022-08-01T23:02:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.781+0000 7f8cd0f7d700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cac053a70 0x7f8cac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.781+0000 7f8cd0f7d700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cac053a70 0x7f8cac055f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f8cc000c000 tx=0x7f8cc0006f90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.789+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8cc4042ee0 con 0x7f8ccc0b5630 2022-08-01T23:02:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:31.949+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8ccc0574e0 con 0x7f8ccc0b5630 2022-08-01T23:02:32.403 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:02:32.401+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:02:32.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.409+0000 7f8cc9ffb700 1 -- 172.21.15.32:0/4057052841 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8cc4046500 con 0x7f8ccc0b5630 2022-08-01T23:02:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cac053a70 msgr2=0x7f8cac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cac053a70 0x7f8cac055f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f8cc000c000 tx=0x7f8cc0006f90).stop 2022-08-01T23:02:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 msgr2=0x7f8ccc15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 0x7f8ccc15a080 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f8ccc063b30 tx=0x7f8cc4008040).stop 2022-08-01T23:02:32.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 shutdown_connections 2022-08-01T23:02:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8cac053a70 0x7f8cac055f30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ccc0b5630 0x7f8ccc15a080 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ccc0b07a0 0x7f8ccc0634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 --2- 172.21.15.32:0/4057052841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ccc067730 0x7f8ccc14d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 >> 172.21.15.32:0/4057052841 conn(0x7f8ccc0626f0 msgr2=0x7f8ccc0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 shutdown_connections 2022-08-01T23:02:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.413+0000 7f8cd1f7f700 1 -- 172.21.15.32:0/4057052841 wait complete. 2022-08-01T23:02:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:02:32.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:32.445 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:02:32.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e86f700 1 Processor -- start 2022-08-01T23:02:32.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e86f700 1 -- start start 2022-08-01T23:02:32.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e86f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88680ad4b0 0x7f88680af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e86f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680aff30 con 0x7f88680ad9e0 2022-08-01T23:02:32.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e86f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886805c7a0 con 0x7f88680af450 2022-08-01T23:02:32.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e86f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8868140990 con 0x7f88680ad4b0 2022-08-01T23:02:32.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886d06c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88680ad4b0 0x7f88680af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e06e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88680ad9e0 0x7f88680a8e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32776/0 (socket says 172.21.15.32:32776) 2022-08-01T23:02:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886e06e700 1 -- 172.21.15.32:0/2654363126 learned_addr learned my addr 172.21.15.32:0/2654363126 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.529+0000 7f886c86b700 1 -- 172.21.15.32:0/2654363126 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717651729 0 0) 0x7f88680aff30 con 0x7f88680ad9e0 2022-08-01T23:02:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886c86b700 1 -- 172.21.15.32:0/2654363126 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8850003260 con 0x7f88680ad9e0 2022-08-01T23:02:32.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886c86b700 1 -- 172.21.15.32:0/2654363126 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181411929 0 0) 0x7f886805c7a0 con 0x7f88680af450 2022-08-01T23:02:32.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886d06c700 1 -- 172.21.15.32:0/2654363126 >> v1:172.21.15.55:6789/0 conn(0x7f88680af450 legacy=0x7f88681432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886d06c700 1 -- 172.21.15.32:0/2654363126 >> v1:172.21.15.32:6789/0 conn(0x7f88680ad9e0 legacy=0x7f88680a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886d06c700 1 -- 172.21.15.32:0/2654363126 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88681439f0 con 0x7f88680ad4b0 2022-08-01T23:02:32.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886d06c700 1 --2- 172.21.15.32:0/2654363126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad4b0 0x7f88680af830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f88680ad900 tx=0x7f8858006df0).ready entity=mon.0 client_cookie=4b978c3a6691cc8c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:32.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886c86b700 1 -- 172.21.15.32:0/2654363126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885800d610 con 0x7f88680ad4b0 2022-08-01T23:02:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886c86b700 1 -- 172.21.15.32:0/2654363126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8858000e90 con 0x7f88680ad4b0 2022-08-01T23:02:32.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886c86b700 1 -- 172.21.15.32:0/2654363126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885800d960 con 0x7f88680ad4b0 2022-08-01T23:02:32.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 -- 172.21.15.32:0/2654363126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad4b0 msgr2=0x7f88680af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 --2- 172.21.15.32:0/2654363126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad4b0 0x7f88680af830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f88680ad900 tx=0x7f8858006df0).stop 2022-08-01T23:02:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 -- 172.21.15.32:0/2654363126 shutdown_connections 2022-08-01T23:02:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 --2- 172.21.15.32:0/2654363126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad4b0 0x7f88680af830 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 -- 172.21.15.32:0/2654363126 >> 172.21.15.32:0/2654363126 conn(0x7f88680612d0 msgr2=0x7f8868062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 -- 172.21.15.32:0/2654363126 shutdown_connections 2022-08-01T23:02:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.533+0000 7f886e86f700 1 -- 172.21.15.32:0/2654363126 wait complete. 2022-08-01T23:02:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 Processor -- start 2022-08-01T23:02:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 -- start start 2022-08-01T23:02:32.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88680ad4b0 0x7f88680b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 0x7f8868063860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88680af450 0x7f886813beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868150c10 con 0x7f88680ad9e0 2022-08-01T23:02:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868150e40 con 0x7f88680af450 2022-08-01T23:02:32.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886805f970 con 0x7f88680ad4b0 2022-08-01T23:02:32.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 0x7f8868063860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 0x7f8868063860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45240/0 (socket says 172.21.15.32:45240) 2022-08-01T23:02:32.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 -- 172.21.15.32:0/1140627825 learned_addr learned my addr 172.21.15.32:0/1140627825 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:32.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d86d700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88680ad4b0 0x7f88680b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e06e700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88680af450 0x7f886813beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 -- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88680ad4b0 msgr2=0x7f88680b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88680ad4b0 0x7f88680b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 -- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88680af450 msgr2=0x7f886813beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88680af450 0x7f886813beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 -- 172.21.15.32:0/1140627825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885800b180 con 0x7f88680ad9e0 2022-08-01T23:02:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886d06c700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 0x7f8868063860 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f88680ad900 tx=0x7f8858014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:32.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8858016070 con 0x7f88680ad9e0 2022-08-01T23:02:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885800de90 con 0x7f88680ad9e0 2022-08-01T23:02:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8868064250 con 0x7f88680ad9e0 2022-08-01T23:02:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8858002f70 con 0x7f88680ad9e0 2022-08-01T23:02:32.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.537+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88680b5820 con 0x7f88680ad9e0 2022-08-01T23:02:32.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.541+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88680574e0 con 0x7f88680ad9e0 2022-08-01T23:02:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.541+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8858025030 con 0x7f88680ad9e0 2022-08-01T23:02:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.541+0000 7f8856ffd700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8848053950 0x7f8848055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.541+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f885806d2a0 con 0x7f88680ad9e0 2022-08-01T23:02:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.541+0000 7f886d86d700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8848053950 0x7f8848055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.541+0000 7f886d86d700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8848053950 0x7f8848055e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f886400b000 tx=0x7f8864009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:32.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.549+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8858043ed0 con 0x7f88680ad9e0 2022-08-01T23:02:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.709+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8868061b20 con 0x7f88680ad9e0 2022-08-01T23:02:32.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f8856ffd700 1 -- 172.21.15.32:0/1140627825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f8858023020 con 0x7f88680ad9e0 2022-08-01T23:02:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:32.716 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:01:34.459266+0000","modified":"2022-08-01T23:02:32.403525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T23:02:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8848053950 msgr2=0x7f8848055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8848053950 0x7f8848055e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f886400b000 tx=0x7f8864009040).stop 2022-08-01T23:02:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 msgr2=0x7f8868063860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 0x7f8868063860 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f88680ad900 tx=0x7f8858014000).stop 2022-08-01T23:02:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 shutdown_connections 2022-08-01T23:02:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8848053950 0x7f8848055e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88680af450 0x7f886813beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88680ad9e0 0x7f8868063860 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 --2- 172.21.15.32:0/1140627825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f88680ad4b0 0x7f88680b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 >> 172.21.15.32:0/1140627825 conn(0x7f88680612d0 msgr2=0x7f88681432d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 shutdown_connections 2022-08-01T23:02:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.713+0000 7f886e86f700 1 -- 172.21.15.32:0/1140627825 wait complete. 2022-08-01T23:02:32.721 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2022-08-01T23:02:32.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:02:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 Processor -- start 2022-08-01T23:02:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 -- start start 2022-08-01T23:02:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5ec0b2710 0x7fd5ec14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd5ec0b01a0 0x7fd5ec14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec0a7250 con 0x7fd5ec0afd60 2022-08-01T23:02:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec0a7480 con 0x7fd5ec0b2710 2022-08-01T23:02:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f33e3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec0a7720 con 0x7fd5ec0b01a0 2022-08-01T23:02:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5ec0b2710 0x7fd5ec14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5ec0b2710 0x7fd5ec14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45246/0 (socket says 172.21.15.32:45246) 2022-08-01T23:02:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 -- 172.21.15.32:0/2464762193 learned_addr learned my addr 172.21.15.32:0/2464762193 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f1be0700 1 --2- 172.21.15.32:0/2464762193 >> v2:172.21.15.55:3300/0 conn(0x7fd5ec0b01a0 0x7fd5ec14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 -- 172.21.15.32:0/2464762193 >> v2:172.21.15.55:3300/0 conn(0x7fd5ec0b01a0 msgr2=0x7fd5ec14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 --2- 172.21.15.32:0/2464762193 >> v2:172.21.15.55:3300/0 conn(0x7fd5ec0b01a0 0x7fd5ec14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 -- 172.21.15.32:0/2464762193 >> v1:172.21.15.55:6789/0 conn(0x7fd5ec0afd60 legacy=0x7fd5ec0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 -- 172.21.15.32:0/2464762193 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0a79c0 con 0x7fd5ec0b2710 2022-08-01T23:02:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f23e1700 1 --2- 172.21.15.32:0/2464762193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec14afd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fd5d40032f0 tx=0x7fd5d4009290).ready entity=mon.0 client_cookie=61f70557b924fe86 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.881+0000 7fd5f13df700 1 -- 172.21.15.32:0/2464762193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d400b870 con 0x7fd5ec0b2710 2022-08-01T23:02:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f13df700 1 -- 172.21.15.32:0/2464762193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d400b9d0 con 0x7fd5ec0b2710 2022-08-01T23:02:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f13df700 1 -- 172.21.15.32:0/2464762193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d400bb40 con 0x7fd5ec0b2710 2022-08-01T23:02:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- 172.21.15.32:0/2464762193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0b2710 msgr2=0x7fd5ec14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/2464762193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec14afd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fd5d40032f0 tx=0x7fd5d4009290).stop 2022-08-01T23:02:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- 172.21.15.32:0/2464762193 shutdown_connections 2022-08-01T23:02:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/2464762193 >> v2:172.21.15.55:3300/0 conn(0x7fd5ec0b01a0 0x7fd5ec14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/2464762193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec14afd0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- 172.21.15.32:0/2464762193 >> 172.21.15.32:0/2464762193 conn(0x7fd5ec060310 msgr2=0x7fd5ec0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- 172.21.15.32:0/2464762193 shutdown_connections 2022-08-01T23:02:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- 172.21.15.32:0/2464762193 wait complete. 2022-08-01T23:02:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 Processor -- start 2022-08-01T23:02:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- start start 2022-08-01T23:02:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 0x7fd5ec066000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5ec0b01a0 0x7fd5ec0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0abcd0 con 0x7fd5ec0afd60 2022-08-01T23:02:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec157140 con 0x7fd5ec0b2710 2022-08-01T23:02:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f33e3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0aae90 con 0x7fd5ec0b01a0 2022-08-01T23:02:32.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f23e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 0x7fd5ec066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f23e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 0x7fd5ec066000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45254/0 (socket says 172.21.15.32:45254) 2022-08-01T23:02:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f23e1700 1 -- 172.21.15.32:0/1378301908 learned_addr learned my addr 172.21.15.32:0/1378301908 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f1be0700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5ec0b01a0 0x7fd5ec0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.885+0000 7fd5f2be2700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f23e1700 1 -- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5ec0b01a0 msgr2=0x7fd5ec0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f23e1700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5ec0b01a0 0x7fd5ec0664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f23e1700 1 -- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5ec0b2710 msgr2=0x7fd5ec153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:32.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f23e1700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f23e1700 1 -- 172.21.15.32:0/1378301908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4009530 con 0x7fd5ec0afd60 2022-08-01T23:02:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f2be2700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f23e1700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 0x7fd5ec066000 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fd5ec1572d0 tx=0x7fd5d4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d400b870 con 0x7fd5ec0afd60 2022-08-01T23:02:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4021630 con 0x7fd5ec0afd60 2022-08-01T23:02:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec0abf00 con 0x7fd5ec0afd60 2022-08-01T23:02:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d4021990 con 0x7fd5ec0afd60 2022-08-01T23:02:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec0ab930 con 0x7fd5ec0afd60 2022-08-01T23:02:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec0574e0 con 0x7fd5ec0afd60 2022-08-01T23:02:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.889+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd5d4027020 con 0x7fd5ec0afd60 2022-08-01T23:02:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.893+0000 7fd5eb7fe700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd5d0053b00 0x7fd5d0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.893+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fd5d406d4d0 con 0x7fd5ec0afd60 2022-08-01T23:02:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.893+0000 7fd5f1be0700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd5d0053b00 0x7fd5d0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:32.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.893+0000 7fd5f1be0700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd5d0053b00 0x7fd5d0055fc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fd5e400c000 tx=0x7fd5e4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:32.893+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5d40441b0 con 0x7fd5ec0afd60 2022-08-01T23:02:33.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.061+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5ec1575b0 con 0x7fd5ec0afd60 2022-08-01T23:02:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.965+0000 7fd5eb7fe700 1 -- 172.21.15.32:0/1378301908 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd5d40477d0 con 0x7fd5ec0afd60 2022-08-01T23:02:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd5d0053b00 msgr2=0x7fd5d0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd5d0053b00 0x7fd5d0055fc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fd5e400c000 tx=0x7fd5e4008040).stop 2022-08-01T23:02:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 msgr2=0x7fd5ec066000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 0x7fd5ec066000 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fd5ec1572d0 tx=0x7fd5d4016000).stop 2022-08-01T23:02:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 shutdown_connections 2022-08-01T23:02:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd5d0053b00 0x7fd5d0055fc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5ec0b2710 0x7fd5ec153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd5ec0b01a0 0x7fd5ec0664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 --2- 172.21.15.32:0/1378301908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5ec0afd60 0x7fd5ec066000 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 >> 172.21.15.32:0/1378301908 conn(0x7fd5ec060310 msgr2=0x7fd5ec05be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 shutdown_connections 2022-08-01T23:02:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:33.969+0000 7fd5f33e3700 1 -- 172.21.15.32:0/1378301908 wait complete. 2022-08-01T23:02:33.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:02:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 Processor -- start 2022-08-01T23:02:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 -- start start 2022-08-01T23:02:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a000b6710 0x7f9a000b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9a000b4550 0x7f9a000b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a00062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a00144240 con 0x7f9a000b6710 2022-08-01T23:02:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000b6fe0 con 0x7f9a000b4550 2022-08-01T23:02:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.101+0000 7f9a06bad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00143d00 con 0x7f9a00061ec0 2022-08-01T23:02:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a000b6710 0x7f9a000b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a000b6710 0x7f9a000b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45260/0 (socket says 172.21.15.32:45260) 2022-08-01T23:02:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 -- 172.21.15.32:0/3594905096 learned_addr learned my addr 172.21.15.32:0/3594905096 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f99fffff700 1 --2- 172.21.15.32:0/3594905096 >> v2:172.21.15.55:3300/0 conn(0x7f9a000b4550 0x7f9a000b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a0514a700 1 --2- 172.21.15.32:0/3594905096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a00062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 -- 172.21.15.32:0/3594905096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 msgr2=0x7f9a00062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 --2- 172.21.15.32:0/3594905096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a00062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 -- 172.21.15.32:0/3594905096 >> v2:172.21.15.55:3300/0 conn(0x7f9a000b4550 msgr2=0x7f9a000b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 --2- 172.21.15.32:0/3594905096 >> v2:172.21.15.55:3300/0 conn(0x7f9a000b4550 0x7f9a000b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 -- 172.21.15.32:0/3594905096 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00143fa0 con 0x7f9a000b6710 2022-08-01T23:02:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f99fffff700 1 --2- 172.21.15.32:0/3594905096 >> v2:172.21.15.55:3300/0 conn(0x7f9a000b4550 0x7f9a000b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a04949700 1 --2- 172.21.15.32:0/3594905096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a000b5d80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f9a001432b0 tx=0x7f99f4006eb0).ready entity=mon.0 client_cookie=7286222abe439f26 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f99ff7fe700 1 -- 172.21.15.32:0/3594905096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99f400d570 con 0x7f9a000b6710 2022-08-01T23:02:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f99ff7fe700 1 -- 172.21.15.32:0/3594905096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4000e90 con 0x7f9a000b6710 2022-08-01T23:02:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f99ff7fe700 1 -- 172.21.15.32:0/3594905096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99f400d900 con 0x7f9a000b6710 2022-08-01T23:02:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 -- 172.21.15.32:0/3594905096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 msgr2=0x7f9a000b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 --2- 172.21.15.32:0/3594905096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a000b5d80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f9a001432b0 tx=0x7f99f4006eb0).stop 2022-08-01T23:02:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 -- 172.21.15.32:0/3594905096 shutdown_connections 2022-08-01T23:02:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 --2- 172.21.15.32:0/3594905096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a00062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 --2- 172.21.15.32:0/3594905096 >> v2:172.21.15.55:3300/0 conn(0x7f9a000b4550 0x7f9a000b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 --2- 172.21.15.32:0/3594905096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a000b5d80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 -- 172.21.15.32:0/3594905096 >> 172.21.15.32:0/3594905096 conn(0x7f9a0005c620 msgr2=0x7f9a0005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 -- 172.21.15.32:0/3594905096 shutdown_connections 2022-08-01T23:02:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.105+0000 7f9a06bad700 1 -- 172.21.15.32:0/3594905096 wait complete. 2022-08-01T23:02:34.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 Processor -- start 2022-08-01T23:02:34.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 -- start start 2022-08-01T23:02:34.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a000b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a000b4550 0x7f9a00125800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a00125cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000afbc0 con 0x7f9a000b6710 2022-08-01T23:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000afe60 con 0x7f9a000b4550 2022-08-01T23:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000b0100 con 0x7f9a00061ec0 2022-08-01T23:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a00125cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a00125cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45270/0 (socket says 172.21.15.32:45270) 2022-08-01T23:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 -- 172.21.15.32:0/4272562658 learned_addr learned my addr 172.21.15.32:0/4272562658 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a04949700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a000b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f99fffff700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a000b4550 0x7f9a00125800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 -- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 msgr2=0x7f9a000b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:34.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a000b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 -- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a000b4550 msgr2=0x7f9a00125800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:34.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a000b4550 0x7f9a00125800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 -- 172.21.15.32:0/4272562658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f400b1e0 con 0x7f9a000b6710 2022-08-01T23:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f99fffff700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a000b4550 0x7f9a00125800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a0514a700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a00125cf0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f99f8006e00 tx=0x7f99f80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99f800c670 con 0x7f9a000b6710 2022-08-01T23:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800c7d0 con 0x7f9a000b6710 2022-08-01T23:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99f800cb30 con 0x7f9a000b6710 2022-08-01T23:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a000b03a0 con 0x7f9a000b6710 2022-08-01T23:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.109+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a00126bb0 con 0x7f9a000b6710 2022-08-01T23:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.113+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a000b2bf0 con 0x7f9a000b6710 2022-08-01T23:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.113+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f99f800cc90 con 0x7f9a000b6710 2022-08-01T23:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.113+0000 7f99fdffb700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99e00539e0 0x7f99e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.113+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f99f806c380 con 0x7f9a000b6710 2022-08-01T23:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.113+0000 7f9a04949700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99e00539e0 0x7f99e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.113+0000 7f9a04949700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99e00539e0 0x7f99e0055ea0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f99f401c000 tx=0x7f99f4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.117+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f99f80430c0 con 0x7f9a000b6710 2022-08-01T23:02:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.245+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a00061780 con 0x7f9a000b6710 2022-08-01T23:02:34.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.977+0000 7f99fdffb700 1 -- 172.21.15.32:0/4272562658 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 185+0+0 (secure 0 0 0) 0x7f99f80466e0 con 0x7f9a000b6710 2022-08-01T23:02:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99e00539e0 msgr2=0x7f99e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99e00539e0 0x7f99e0055ea0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f99f401c000 tx=0x7f99f4015000).stop 2022-08-01T23:02:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 msgr2=0x7f9a00125cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a00125cf0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f99f8006e00 tx=0x7f99f80091d0).stop 2022-08-01T23:02:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 shutdown_connections 2022-08-01T23:02:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99e00539e0 0x7f99e0055ea0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b6710 0x7f9a00125cf0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a000b4550 0x7f9a00125800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 --2- 172.21.15.32:0/4272562658 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a00061ec0 0x7f9a000b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 >> 172.21.15.32:0/4272562658 conn(0x7f9a0005c620 msgr2=0x7f9a00128f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 shutdown_connections 2022-08-01T23:02:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:34.981+0000 7f9a06bad700 1 -- 172.21.15.32:0/4272562658 wait complete. 2022-08-01T23:02:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:02:35.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:02:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.109+0000 7ffa7ed53700 1 Processor -- start 2022-08-01T23:02:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.109+0000 7ffa7ed53700 1 -- start start 2022-08-01T23:02:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7ed53700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffa780b6420 0x7ffa78061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7ed53700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa780b41f0 0x7ffa78061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7ed53700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780b60e0 con 0x7ffa78061fe0 2022-08-01T23:02:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7ed53700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa780b2100 con 0x7ffa780b6420 2022-08-01T23:02:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7ed53700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b23a0 con 0x7ffa780b41f0 2022-08-01T23:02:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7e552700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa78061fe0 0x7ffa781465c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32820/0 (socket says 172.21.15.32:32820) 2022-08-01T23:02:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7e552700 1 -- 172.21.15.32:0/3105528196 learned_addr learned my addr 172.21.15.32:0/3105528196 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/3105528196 >> v2:172.21.15.55:3300/0 conn(0x7ffa780b6420 0x7ffa78061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7d550700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa780b41f0 0x7ffa78061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702005736 0 0) 0x7ffa70001490 con 0x7ffa78061fe0 2022-08-01T23:02:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa60003260 con 0x7ffa78061fe0 2022-08-01T23:02:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854373283 0 0) 0x7ffa60003260 con 0x7ffa78061fe0 2022-08-01T23:02:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa780b41f0 msgr2=0x7ffa78061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa780b41f0 0x7ffa78061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 >> v2:172.21.15.55:3300/0 conn(0x7ffa780b6420 msgr2=0x7ffa78061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 --2- 172.21.15.32:0/3105528196 >> v2:172.21.15.55:3300/0 conn(0x7ffa780b6420 0x7ffa78061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780b2640 con 0x7ffa78061fe0 2022-08-01T23:02:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7ffa70001b70 con 0x7ffa78061fe0 2022-08-01T23:02:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 >> v1:172.21.15.32:6789/0 conn(0x7ffa78061fe0 legacy=0x7ffa781465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa60003750 0x7ffa60003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa60004500 0x7ffa60004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa60004e70 0x7ffa6000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b2100 con 0x7ffa60003750 2022-08-01T23:02:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b23a0 con 0x7ffa60004500 2022-08-01T23:02:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6000bc40 con 0x7ffa60004e70 2022-08-01T23:02:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7d550700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa780b41f0 0x7ffa78061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa60003750 0x7ffa60003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7e552700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa60004e70 0x7ffa6000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7d550700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa60004500 0x7ffa60004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 -- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa60004e70 msgr2=0x7ffa6000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa60004e70 0x7ffa6000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 -- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa60004500 msgr2=0x7ffa60004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa60004500 0x7ffa60004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 -- 172.21.15.32:0/3105528196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60003260 con 0x7ffa60003750 2022-08-01T23:02:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7d550700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa60004500 0x7ffa60004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa60003750 0x7ffa60003b70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ffa60004470 tx=0x7ffa6c006df0).ready entity=mon.0 client_cookie=1035580f7b7bb542 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa6c00e830 con 0x7ffa60003750 2022-08-01T23:02:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c00be90 con 0x7ffa60003750 2022-08-01T23:02:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.113+0000 7ffa7cd4f700 1 -- 172.21.15.32:0/3105528196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa6c00eb80 con 0x7ffa60003750 2022-08-01T23:02:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 -- 172.21.15.32:0/3105528196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa60003750 msgr2=0x7ffa60003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/3105528196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa60003750 0x7ffa60003b70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ffa60004470 tx=0x7ffa6c006df0).stop 2022-08-01T23:02:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7e552700 1 -- 172.21.15.32:0/3105528196 reap_dead start 2022-08-01T23:02:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 -- 172.21.15.32:0/3105528196 shutdown_connections 2022-08-01T23:02:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 -- 172.21.15.32:0/3105528196 >> 172.21.15.32:0/3105528196 conn(0x7ffa7805c620 msgr2=0x7ffa7805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 -- 172.21.15.32:0/3105528196 shutdown_connections 2022-08-01T23:02:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 -- 172.21.15.32:0/3105528196 wait complete. 2022-08-01T23:02:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 Processor -- start 2022-08-01T23:02:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.117+0000 7ffa7ed53700 1 -- start start 2022-08-01T23:02:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa781563b0 0x7ffa78145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa78156790 0x7ffa780ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 0x7ffa78128190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b0aa0 con 0x7ffa78127ce0 2022-08-01T23:02:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780adb10 con 0x7ffa781563b0 2022-08-01T23:02:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781570d0 con 0x7ffa78156790 2022-08-01T23:02:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 0x7ffa78128190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 0x7ffa78128190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45290/0 (socket says 172.21.15.32:45290) 2022-08-01T23:02:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 -- 172.21.15.32:0/1316685911 learned_addr learned my addr 172.21.15.32:0/1316685911 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa781563b0 0x7ffa78145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7d550700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa78156790 0x7ffa780ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 -- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa78156790 msgr2=0x7ffa780ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa78156790 0x7ffa780ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 -- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa781563b0 msgr2=0x7ffa78145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa781563b0 0x7ffa78145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 -- 172.21.15.32:0/1316685911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78156c70 con 0x7ffa78127ce0 2022-08-01T23:02:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7e552700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 0x7ffa78128190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ffa70000b60 tx=0x7ffa7805d2e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa700116b0 con 0x7ffa78127ce0 2022-08-01T23:02:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa70011810 con 0x7ffa78127ce0 2022-08-01T23:02:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7812ab10 con 0x7ffa78127ce0 2022-08-01T23:02:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa70011b70 con 0x7ffa78127ce0 2022-08-01T23:02:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.121+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa780af5d0 con 0x7ffa78127ce0 2022-08-01T23:02:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.125+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa780b28e0 con 0x7ffa78127ce0 2022-08-01T23:02:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.125+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa70011cd0 con 0x7ffa78127ce0 2022-08-01T23:02:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.125+0000 7ffa6affd700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa58053aa0 0x7ffa58055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.125+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6111+0+0 (secure 0 0 0) 0x7ffa7006b7f0 con 0x7ffa78127ce0 2022-08-01T23:02:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.125+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa58053aa0 0x7ffa58055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.125+0000 7ffa7dd51700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa58053aa0 0x7ffa58055f60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ffa6c01d000 tx=0x7ffa6c019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.133+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffa70042610 con 0x7ffa78127ce0 2022-08-01T23:02:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.261+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa780b60e0 con 0x7ffa78127ce0 2022-08-01T23:02:35.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.981+0000 7ffa6affd700 1 -- 172.21.15.32:0/1316685911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 173+0+0 (secure 0 0 0) 0x7ffa780b60e0 con 0x7ffa78127ce0 2022-08-01T23:02:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa58053aa0 msgr2=0x7ffa58055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa58053aa0 0x7ffa58055f60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ffa6c01d000 tx=0x7ffa6c019000).stop 2022-08-01T23:02:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 msgr2=0x7ffa78128190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 0x7ffa78128190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ffa70000b60 tx=0x7ffa7805d2e0).stop 2022-08-01T23:02:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 shutdown_connections 2022-08-01T23:02:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa58053aa0 0x7ffa58055f60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa78127ce0 0x7ffa78128190 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa78156790 0x7ffa780ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 --2- 172.21.15.32:0/1316685911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa781563b0 0x7ffa78145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 >> 172.21.15.32:0/1316685911 conn(0x7ffa7805c620 msgr2=0x7ffa780b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 shutdown_connections 2022-08-01T23:02:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:35.985+0000 7ffa7ed53700 1 -- 172.21.15.32:0/1316685911 wait complete. 2022-08-01T23:02:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:02:36.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:02:36.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.109+0000 7f28218a5700 1 Processor -- start 2022-08-01T23:02:36.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.109+0000 7f28218a5700 1 -- start start 2022-08-01T23:02:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f281c0b2710 0x7f281c0abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f281c0b13f0 0x7f281c0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0adf10 con 0x7f281c0b1920 2022-08-01T23:02:36.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c0ae1b0 con 0x7f281c0b13f0 2022-08-01T23:02:36.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c0a9520 con 0x7f281c0b2710 2022-08-01T23:02:36.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f281c0b13f0 0x7f281c0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28210a4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f281c0b1920 0x7f281c0afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32840/0 (socket says 172.21.15.32:32840) 2022-08-01T23:02:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28210a4700 1 -- 172.21.15.32:0/1411288279 learned_addr learned my addr 172.21.15.32:0/1411288279 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28208a3700 1 --2- 172.21.15.32:0/1411288279 >> v2:172.21.15.55:3300/0 conn(0x7f281c0b2710 0x7f281c0abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281b7fe700 1 -- 172.21.15.32:0/1411288279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951580724 0 0) 0x7f281c0adf10 con 0x7f281c0b1920 2022-08-01T23:02:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281b7fe700 1 -- 172.21.15.32:0/1411288279 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800003260 con 0x7f281c0b1920 2022-08-01T23:02:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281bfff700 1 -- 172.21.15.32:0/1411288279 >> v2:172.21.15.55:3300/0 conn(0x7f281c0b2710 msgr2=0x7f281c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281bfff700 1 --2- 172.21.15.32:0/1411288279 >> v2:172.21.15.55:3300/0 conn(0x7f281c0b2710 0x7f281c0abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281b7fe700 1 -- 172.21.15.32:0/1411288279 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1813660023 0 0) 0x7f2800003260 con 0x7f281c0b1920 2022-08-01T23:02:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281bfff700 1 -- 172.21.15.32:0/1411288279 >> v1:172.21.15.32:6789/0 conn(0x7f281c0b1920 legacy=0x7f281c0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281bfff700 1 -- 172.21.15.32:0/1411288279 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0a99c0 con 0x7f281c0b13f0 2022-08-01T23:02:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28208a3700 1 --2- 172.21.15.32:0/1411288279 >> v2:172.21.15.55:3300/0 conn(0x7f281c0b2710 0x7f281c0abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281bfff700 1 --2- 172.21.15.32:0/1411288279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b13f0 0x7f281c0ac190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f2814001b80 tx=0x7f2808006df0).ready entity=mon.0 client_cookie=21ba7123a7ca84a8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281b7fe700 1 -- 172.21.15.32:0/1411288279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28140019e0 con 0x7f281c0b13f0 2022-08-01T23:02:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281b7fe700 1 -- 172.21.15.32:0/1411288279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808000e90 con 0x7f281c0b13f0 2022-08-01T23:02:36.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f281b7fe700 1 -- 172.21.15.32:0/1411288279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2808012460 con 0x7f281c0b13f0 2022-08-01T23:02:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- 172.21.15.32:0/1411288279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b13f0 msgr2=0x7f281c0ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 --2- 172.21.15.32:0/1411288279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b13f0 0x7f281c0ac190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f2814001b80 tx=0x7f2808006df0).stop 2022-08-01T23:02:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- 172.21.15.32:0/1411288279 shutdown_connections 2022-08-01T23:02:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 --2- 172.21.15.32:0/1411288279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b13f0 0x7f281c0ac190 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 --2- 172.21.15.32:0/1411288279 >> v2:172.21.15.55:3300/0 conn(0x7f281c0b2710 0x7f281c0abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- 172.21.15.32:0/1411288279 >> 172.21.15.32:0/1411288279 conn(0x7f281c060310 msgr2=0x7f281c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:36.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- 172.21.15.32:0/1411288279 shutdown_connections 2022-08-01T23:02:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.113+0000 7f28218a5700 1 -- 172.21.15.32:0/1411288279 wait complete. 2022-08-01T23:02:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 Processor -- start 2022-08-01T23:02:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 -- start start 2022-08-01T23:02:36.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f281c0b13f0 0x7f281c147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 0x7f281c0a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f281c0b2710 0x7f281c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c09fdc0 con 0x7f281c0b1920 2022-08-01T23:02:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0abed0 con 0x7f281c0b13f0 2022-08-01T23:02:36.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0abc30 con 0x7f281c0b2710 2022-08-01T23:02:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 0x7f281c0a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 0x7f281c0a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45302/0 (socket says 172.21.15.32:45302) 2022-08-01T23:02:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 -- 172.21.15.32:0/2545615167 learned_addr learned my addr 172.21.15.32:0/2545615167 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28208a3700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f281c0b13f0 0x7f281c147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28210a4700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f281c0b2710 0x7f281c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 -- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f281c0b2710 msgr2=0x7f281c152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f281c0b2710 0x7f281c152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 -- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f281c0b13f0 msgr2=0x7f281c147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f281c0b13f0 0x7f281c147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 -- 172.21.15.32:0/2545615167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281000b180 con 0x7f281c0b1920 2022-08-01T23:02:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28210a4700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f281c0b2710 0x7f281c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f281bfff700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 0x7f281c0a04c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f2814001b80 tx=0x7f2808014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f280800d920 con 0x7f281c0b1920 2022-08-01T23:02:36.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800da80 con 0x7f281c0b1920 2022-08-01T23:02:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280800b180 con 0x7f281c0b1920 2022-08-01T23:02:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f280800dde0 con 0x7f281c0b1920 2022-08-01T23:02:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.117+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c0a26f0 con 0x7f281c0b1920 2022-08-01T23:02:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.121+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c0574e0 con 0x7f281c0b1920 2022-08-01T23:02:36.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.121+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2808025030 con 0x7f281c0b1920 2022-08-01T23:02:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.121+0000 7f2819ffb700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f27fc053b00 0x7f27fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.121+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f280806ca90 con 0x7f281c0b1920 2022-08-01T23:02:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.121+0000 7f28208a3700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f27fc053b00 0x7f27fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.121+0000 7f28208a3700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f27fc053b00 0x7f27fc055fc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f281000b000 tx=0x7f2810006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:36.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.129+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28080439f0 con 0x7f281c0b1920 2022-08-01T23:02:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.257+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f281c0a0220 con 0x7f281c0b1920 2022-08-01T23:02:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f2819ffb700 1 -- 172.21.15.32:0/2545615167 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2808023020 con 0x7f281c0b1920 2022-08-01T23:02:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f27fc053b00 msgr2=0x7f27fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f27fc053b00 0x7f27fc055fc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f281000b000 tx=0x7f2810006df0).stop 2022-08-01T23:02:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 msgr2=0x7f281c0a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 0x7f281c0a04c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f2814001b80 tx=0x7f2808014000).stop 2022-08-01T23:02:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 shutdown_connections 2022-08-01T23:02:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f27fc053b00 0x7f27fc055fc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f281c0b2710 0x7f281c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0b1920 0x7f281c0a04c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 --2- 172.21.15.32:0/2545615167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f281c0b13f0 0x7f281c147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 >> 172.21.15.32:0/2545615167 conn(0x7f281c060310 msgr2=0x7f281c05c430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 shutdown_connections 2022-08-01T23:02:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.265+0000 7f28218a5700 1 -- 172.21.15.32:0/2545615167 wait complete. 2022-08-01T23:02:36.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-08-01T23:02:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.377+0000 7fbff0d47700 1 Processor -- start 2022-08-01T23:02:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.377+0000 7fbff0d47700 1 -- start start 2022-08-01T23:02:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfec140e80 0x7fbfec0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec0a8e00 0x7fbfec0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec0af220 con 0x7fbfec0b1410 2022-08-01T23:02:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec1404a0 con 0x7fbfec140e80 2022-08-01T23:02:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec140740 con 0x7fbfec0a8e00 2022-08-01T23:02:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfec140e80 0x7fbfec0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfec140e80 0x7fbfec0ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45310/0 (socket says 172.21.15.32:45310) 2022-08-01T23:02:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/1480869307 learned_addr learned my addr 172.21.15.32:0/1480869307 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfea59c700 1 --1- 172.21.15.32:0/1480869307 >> v1:172.21.15.32:6789/0 conn(0x7fbfec0b1410 0x7fbfec0b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32850/0 (socket says 172.21.15.32:32850) 2022-08-01T23:02:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfead9d700 1 --2- 172.21.15.32:0/1480869307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec0a8e00 0x7fbfec0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/1480869307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec0a8e00 msgr2=0x7fbfec0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 --2- 172.21.15.32:0/1480869307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec0a8e00 0x7fbfec0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/1480869307 >> v1:172.21.15.32:6789/0 conn(0x7fbfec0b1410 legacy=0x7fbfec0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/1480869307 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec1409e0 con 0x7fbfec140e80 2022-08-01T23:02:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe9d9b700 1 --2- 172.21.15.32:0/1480869307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec140e80 0x7fbfec0ae840 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fbfd4002f70 tx=0x7fbfd4006eb0).ready entity=mon.0 client_cookie=ebbbd732c36ed009 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe959a700 1 -- 172.21.15.32:0/1480869307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd400d5c0 con 0x7fbfec140e80 2022-08-01T23:02:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe959a700 1 -- 172.21.15.32:0/1480869307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4000e90 con 0x7fbfec140e80 2022-08-01T23:02:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbfe959a700 1 -- 172.21.15.32:0/1480869307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd400d950 con 0x7fbfec140e80 2022-08-01T23:02:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- 172.21.15.32:0/1480869307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec140e80 msgr2=0x7fbfec0ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 --2- 172.21.15.32:0/1480869307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec140e80 0x7fbfec0ae840 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fbfd4002f70 tx=0x7fbfd4006eb0).stop 2022-08-01T23:02:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- 172.21.15.32:0/1480869307 shutdown_connections 2022-08-01T23:02:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 --2- 172.21.15.32:0/1480869307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec0a8e00 0x7fbfec0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 --2- 172.21.15.32:0/1480869307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec140e80 0x7fbfec0ae840 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- 172.21.15.32:0/1480869307 >> 172.21.15.32:0/1480869307 conn(0x7fbfec05f840 msgr2=0x7fbfec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- 172.21.15.32:0/1480869307 shutdown_connections 2022-08-01T23:02:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.381+0000 7fbff0d47700 1 -- 172.21.15.32:0/1480869307 wait complete. 2022-08-01T23:02:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 Processor -- start 2022-08-01T23:02:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 -- start start 2022-08-01T23:02:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfec0a8e00 0x7fbfec13a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 0x7fbfec1323f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec140e80 0x7fbfec13b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1445e0 con 0x7fbfec0b1410 2022-08-01T23:02:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec139a30 con 0x7fbfec0a8e00 2022-08-01T23:02:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0b4e00 con 0x7fbfec140e80 2022-08-01T23:02:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 0x7fbfec1323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 0x7fbfec1323f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45318/0 (socket says 172.21.15.32:45318) 2022-08-01T23:02:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/571222205 learned_addr learned my addr 172.21.15.32:0/571222205 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfea59c700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfec0a8e00 0x7fbfec13a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfead9d700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec140e80 0x7fbfec13b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec140e80 msgr2=0x7fbfec13b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec140e80 0x7fbfec13b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfec0a8e00 msgr2=0x7fbfec13a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfec0a8e00 0x7fbfec13a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 -- 172.21.15.32:0/571222205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd400b1e0 con 0x7fbfec0b1410 2022-08-01T23:02:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe9d9b700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 0x7fbfec1323f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fbfd4002f70 tx=0x7fbfd4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd40117e0 con 0x7fbfec0b1410 2022-08-01T23:02:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4011940 con 0x7fbfec0b1410 2022-08-01T23:02:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec139c60 con 0x7fbfec0b1410 2022-08-01T23:02:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfd4011ca0 con 0x7fbfec0b1410 2022-08-01T23:02:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.385+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec0b5390 con 0x7fbfec0b1410 2022-08-01T23:02:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.389+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec139df0 con 0x7fbfec0b1410 2022-08-01T23:02:36.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.389+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbfd4025030 con 0x7fbfec0b1410 2022-08-01T23:02:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.389+0000 7fbfe37fe700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfd0053b00 0x7fbfd0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.389+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fbfd406d0a0 con 0x7fbfec0b1410 2022-08-01T23:02:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.389+0000 7fbfea59c700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfd0053b00 0x7fbfd0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.389+0000 7fbfea59c700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfd0053b00 0x7fbfd0055fc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fbfdc00d000 tx=0x7fbfdc007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.397+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfd4044000 con 0x7fbfec0b1410 2022-08-01T23:02:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.565+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fbfec140190 con 0x7fbfec0b1410 2022-08-01T23:02:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.984+0000 7fbfe37fe700 1 -- 172.21.15.32:0/571222205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v290) v1 ==== 139+0+0 (secure 0 0 0) 0x7fbfd4023020 con 0x7fbfec0b1410 2022-08-01T23:02:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfd0053b00 msgr2=0x7fbfd0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfd0053b00 0x7fbfd0055fc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fbfdc00d000 tx=0x7fbfdc007b80).stop 2022-08-01T23:02:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 msgr2=0x7fbfec1323f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 0x7fbfec1323f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fbfd4002f70 tx=0x7fbfd4015000).stop 2022-08-01T23:02:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 shutdown_connections 2022-08-01T23:02:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfd0053b00 0x7fbfd0055fc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfec140e80 0x7fbfec13b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfec0b1410 0x7fbfec1323f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 --2- 172.21.15.32:0/571222205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfec0a8e00 0x7fbfec13a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 >> 172.21.15.32:0/571222205 conn(0x7fbfec05f840 msgr2=0x7fbfec0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 shutdown_connections 2022-08-01T23:02:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:36.988+0000 7fbff0d47700 1 -- 172.21.15.32:0/571222205 wait complete. 2022-08-01T23:02:37.009 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:02:37.010 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:02:37.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-08-01T23:02:37.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 Processor -- start 2022-08-01T23:02:37.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 -- start start 2022-08-01T23:02:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef380aec40 0x7fef380af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef38140110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef380b0190 con 0x7fef380b0610 2022-08-01T23:02:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef380ae0d0 con 0x7fef380aec40 2022-08-01T23:02:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3d4b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef380ad230 con 0x7fef38147060 2022-08-01T23:02:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef380aec40 0x7fef380af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef380aec40 0x7fef380af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45322/0 (socket says 172.21.15.32:45322) 2022-08-01T23:02:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 -- 172.21.15.32:0/1658296453 learned_addr learned my addr 172.21.15.32:0/1658296453 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3ccb6700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef38140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 -- 172.21.15.32:0/1658296453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 msgr2=0x7fef38140110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef38140110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 -- 172.21.15.32:0/1658296453 >> v1:172.21.15.55:6789/0 conn(0x7fef380b0610 legacy=0x7fef38146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef37fff700 1 -- 172.21.15.32:0/1658296453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef381474e0 con 0x7fef380aec40 2022-08-01T23:02:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.120+0000 7fef3ccb6700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef38140110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef37fff700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380aec40 0x7fef380af020 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fef280032f0 tx=0x7fef28009290).ready entity=mon.0 client_cookie=d18bbcdd140d1650 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef36ffd700 1 -- 172.21.15.32:0/1658296453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2800b8a0 con 0x7fef380aec40 2022-08-01T23:02:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef36ffd700 1 -- 172.21.15.32:0/1658296453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2800ba00 con 0x7fef380aec40 2022-08-01T23:02:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef36ffd700 1 -- 172.21.15.32:0/1658296453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2800bb70 con 0x7fef380aec40 2022-08-01T23:02:37.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 -- 172.21.15.32:0/1658296453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380aec40 msgr2=0x7fef380af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380aec40 0x7fef380af020 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fef280032f0 tx=0x7fef28009290).stop 2022-08-01T23:02:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 -- 172.21.15.32:0/1658296453 shutdown_connections 2022-08-01T23:02:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef38140110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/1658296453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380aec40 0x7fef380af020 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 -- 172.21.15.32:0/1658296453 >> 172.21.15.32:0/1658296453 conn(0x7fef380613c0 msgr2=0x7fef38062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 -- 172.21.15.32:0/1658296453 shutdown_connections 2022-08-01T23:02:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.124+0000 7fef3d4b7700 1 -- 172.21.15.32:0/1658296453 wait complete. 2022-08-01T23:02:37.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 Processor -- start 2022-08-01T23:02:37.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 -- start start 2022-08-01T23:02:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef380aec40 0x7fef3813cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 0x7fef38065300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef380657f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38065ce0 con 0x7fef380b0610 2022-08-01T23:02:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38153560 con 0x7fef380aec40 2022-08-01T23:02:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef38142c50 con 0x7fef38147060 2022-08-01T23:02:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 0x7fef38065300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 0x7fef38065300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45330/0 (socket says 172.21.15.32:45330) 2022-08-01T23:02:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 -- 172.21.15.32:0/279846854 learned_addr learned my addr 172.21.15.32:0/279846854 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3ccb6700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef380657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef37fff700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef380aec40 0x7fef3813cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 -- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 msgr2=0x7fef380657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef380657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 -- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef380aec40 msgr2=0x7fef3813cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef380aec40 0x7fef3813cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 -- 172.21.15.32:0/279846854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef28009530 con 0x7fef380b0610 2022-08-01T23:02:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef377fe700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 0x7fef38065300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fef38064b70 tx=0x7fef2c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2c012400 con 0x7fef380b0610 2022-08-01T23:02:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c012560 con 0x7fef380b0610 2022-08-01T23:02:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef38153020 con 0x7fef380b0610 2022-08-01T23:02:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef2c0128c0 con 0x7fef380b0610 2022-08-01T23:02:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.128+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef381427b0 con 0x7fef380b0610 2022-08-01T23:02:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.132+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef380b0b20 con 0x7fef380b0610 2022-08-01T23:02:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.132+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef2c019030 con 0x7fef380b0610 2022-08-01T23:02:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.132+0000 7fef357fa700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef18053950 0x7fef18055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.132+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fef2c06b0d0 con 0x7fef380b0610 2022-08-01T23:02:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.132+0000 7fef37fff700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef18053950 0x7fef18055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.132+0000 7fef37fff700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef18053950 0x7fef18055e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fef2801a000 tx=0x7fef28016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.140+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef2c042140 con 0x7fef380b0610 2022-08-01T23:02:37.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.308+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7fef380574e0 con 0x7fef380b0610 2022-08-01T23:02:37.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.984+0000 7fef357fa700 1 -- 172.21.15.32:0/279846854 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v291) v1 ==== 129+0+0 (secure 0 0 0) 0x7fef2c045760 con 0x7fef380b0610 2022-08-01T23:02:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef18053950 msgr2=0x7fef18055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef18053950 0x7fef18055e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fef2801a000 tx=0x7fef28016000).stop 2022-08-01T23:02:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 msgr2=0x7fef38065300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 0x7fef38065300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fef38064b70 tx=0x7fef2c007c20).stop 2022-08-01T23:02:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 shutdown_connections 2022-08-01T23:02:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef18053950 0x7fef18055e10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fef38147060 0x7fef380657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef380b0610 0x7fef38065300 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 --2- 172.21.15.32:0/279846854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef380aec40 0x7fef3813cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 >> 172.21.15.32:0/279846854 conn(0x7fef380613c0 msgr2=0x7fef38148140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 shutdown_connections 2022-08-01T23:02:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:37.992+0000 7fef3d4b7700 1 -- 172.21.15.32:0/279846854 wait complete. 2022-08-01T23:02:38.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:02:38.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 Processor -- start 2022-08-01T23:02:38.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 -- start start 2022-08-01T23:02:38.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44f005ff50 0x7f44f0061240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f0060330 0x7f44f01476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f00b2b70 con 0x7f44f00b38c0 2022-08-01T23:02:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f00b28d0 con 0x7f44f005ff50 2022-08-01T23:02:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44f5023700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f0060b00 con 0x7f44f0060330 2022-08-01T23:02:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44f005ff50 0x7f44f0061240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44f005ff50 0x7f44f0061240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45338/0 (socket says 172.21.15.32:45338) 2022-08-01T23:02:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 -- 172.21.15.32:0/2951179460 learned_addr learned my addr 172.21.15.32:0/2951179460 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44ef59e700 1 --2- 172.21.15.32:0/2951179460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f0060330 0x7f44f01476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 -- 172.21.15.32:0/2951179460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f0060330 msgr2=0x7f44f01476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 --2- 172.21.15.32:0/2951179460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f0060330 0x7f44f01476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 -- 172.21.15.32:0/2951179460 >> v1:172.21.15.55:6789/0 conn(0x7f44f00b38c0 legacy=0x7f44f00aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 -- 172.21.15.32:0/2951179460 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f0060da0 con 0x7f44f005ff50 2022-08-01T23:02:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44eed9d700 1 --2- 172.21.15.32:0/2951179460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f005ff50 0x7f44f0061240 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f44e0002f70 tx=0x7f44e0006eb0).ready entity=mon.0 client_cookie=940cd952deeeae66 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44ee59c700 1 -- 172.21.15.32:0/2951179460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e000d570 con 0x7f44f005ff50 2022-08-01T23:02:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44ee59c700 1 -- 172.21.15.32:0/2951179460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0000e90 con 0x7f44f005ff50 2022-08-01T23:02:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.120+0000 7f44ee59c700 1 -- 172.21.15.32:0/2951179460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e000d900 con 0x7f44f005ff50 2022-08-01T23:02:38.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 -- 172.21.15.32:0/2951179460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f005ff50 msgr2=0x7f44f0061240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 --2- 172.21.15.32:0/2951179460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f005ff50 0x7f44f0061240 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f44e0002f70 tx=0x7f44e0006eb0).stop 2022-08-01T23:02:38.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 -- 172.21.15.32:0/2951179460 shutdown_connections 2022-08-01T23:02:38.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 --2- 172.21.15.32:0/2951179460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f0060330 0x7f44f01476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 --2- 172.21.15.32:0/2951179460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f005ff50 0x7f44f0061240 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 -- 172.21.15.32:0/2951179460 >> 172.21.15.32:0/2951179460 conn(0x7f44f005d0f0 msgr2=0x7f44f005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:38.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 -- 172.21.15.32:0/2951179460 shutdown_connections 2022-08-01T23:02:38.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 -- 172.21.15.32:0/2951179460 wait complete. 2022-08-01T23:02:38.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 Processor -- start 2022-08-01T23:02:38.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.124+0000 7f44f5023700 1 -- start start 2022-08-01T23:02:38.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f005ff50 0x7f44f00665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44f0060330 0x7f44f0156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 0x7f44f01571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f0156910 con 0x7f44f00b38c0 2022-08-01T23:02:38.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00b0da0 con 0x7f44f0060330 2022-08-01T23:02:38.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00b1040 con 0x7f44f005ff50 2022-08-01T23:02:38.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 0x7f44f01571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 0x7f44f01571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45348/0 (socket says 172.21.15.32:45348) 2022-08-01T23:02:38.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 -- 172.21.15.32:0/991811480 learned_addr learned my addr 172.21.15.32:0/991811480 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:38.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44eed9d700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f005ff50 0x7f44f00665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44e659c700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44f0060330 0x7f44f0156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 -- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f005ff50 msgr2=0x7f44f00665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f005ff50 0x7f44f00665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 -- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44f0060330 msgr2=0x7f44f0156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44f0060330 0x7f44f0156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 -- 172.21.15.32:0/991811480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e000b1e0 con 0x7f44f00b38c0 2022-08-01T23:02:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ef59e700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 0x7f44f01571f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f44e8006e00 tx=0x7f44e80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e800c640 con 0x7f44f00b38c0 2022-08-01T23:02:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e800c7a0 con 0x7f44f00b38c0 2022-08-01T23:02:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f00b12e0 con 0x7f44f00b38c0 2022-08-01T23:02:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44e800cb00 con 0x7f44f00b38c0 2022-08-01T23:02:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f01576e0 con 0x7f44f00b38c0 2022-08-01T23:02:38.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f00574e0 con 0x7f44f00b38c0 2022-08-01T23:02:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.128+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f44e8022070 con 0x7f44f00b38c0 2022-08-01T23:02:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.132+0000 7f44ecd99700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44d4053a70 0x7f44d4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.132+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f44e806c460 con 0x7f44f00b38c0 2022-08-01T23:02:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.132+0000 7f44eed9d700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44d4053a70 0x7f44d4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.132+0000 7f44eed9d700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44d4053a70 0x7f44d4055f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f44e001c000 tx=0x7f44e0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.136+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44e80433d0 con 0x7f44f00b38c0 2022-08-01T23:02:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.260+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f44f0069a70 con 0x7f44f00b38c0 2022-08-01T23:02:38.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44ecd99700 1 -- 172.21.15.32:0/991811480 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f44e80469f0 con 0x7f44f00b38c0 2022-08-01T23:02:38.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44d4053a70 msgr2=0x7f44d4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44d4053a70 0x7f44d4055f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f44e001c000 tx=0x7f44e0015000).stop 2022-08-01T23:02:38.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 msgr2=0x7f44f01571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 0x7f44f01571f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f44e8006e00 tx=0x7f44e80091d0).stop 2022-08-01T23:02:38.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 shutdown_connections 2022-08-01T23:02:38.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f44d4053a70 0x7f44d4055f30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f00b38c0 0x7f44f01571f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44f0060330 0x7f44f0156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 --2- 172.21.15.32:0/991811480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f44f005ff50 0x7f44f00665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 >> 172.21.15.32:0/991811480 conn(0x7f44f005d0f0 msgr2=0x7f44f0065230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:38.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.268+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 shutdown_connections 2022-08-01T23:02:38.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.272+0000 7f44f5023700 1 -- 172.21.15.32:0/991811480 wait complete. 2022-08-01T23:02:38.289 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:02:38.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:38.308 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:38.328 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:38.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:38.370 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:02:38.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:38.371 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:38.372 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:38.397 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:02:38.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:02:38.399 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:02:38.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:38.400 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:38.400 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:38.407 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:02:38.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:02:38.409 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:02:38.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:02:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 Processor -- start 2022-08-01T23:02:38.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 -- start start 2022-08-01T23:02:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2760139c70 0x7f2760142f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2760143430 0x7f27600ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27600ac600 con 0x7f27600b1de0 2022-08-01T23:02:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27600b31a0 con 0x7f2760139c70 2022-08-01T23:02:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.516+0000 7f27665e6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600af4b0 con 0x7f2760143430 2022-08-01T23:02:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2760139c70 0x7f2760142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f2764b83700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f27600b1de0 0x7f27600a8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32898/0 (socket says 172.21.15.32:32898) 2022-08-01T23:02:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f2764b83700 1 -- 172.21.15.32:0/3815855509 learned_addr learned my addr 172.21.15.32:0/3815855509 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275f7fe700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2760143430 0x7f27600ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275effd700 1 -- 172.21.15.32:0/3815855509 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549766363 0 0) 0x7f27600ac600 con 0x7f27600b1de0 2022-08-01T23:02:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275effd700 1 -- 172.21.15.32:0/3815855509 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f274c003260 con 0x7f27600b1de0 2022-08-01T23:02:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275ffff700 1 -- 172.21.15.32:0/3815855509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2760143430 msgr2=0x7f27600ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275ffff700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2760143430 0x7f27600ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275ffff700 1 -- 172.21.15.32:0/3815855509 >> v1:172.21.15.32:6789/0 conn(0x7f27600b1de0 legacy=0x7f27600a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275ffff700 1 -- 172.21.15.32:0/3815855509 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27601438c0 con 0x7f2760139c70 2022-08-01T23:02:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275f7fe700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2760143430 0x7f27600ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275ffff700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2760139c70 0x7f2760142f40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f27600af2b0 tx=0x7f2750006df0).ready entity=mon.0 client_cookie=19b5bf60ef785110 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275effd700 1 -- 172.21.15.32:0/3815855509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f275000d570 con 0x7f2760139c70 2022-08-01T23:02:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275effd700 1 -- 172.21.15.32:0/3815855509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750000e90 con 0x7f2760139c70 2022-08-01T23:02:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f275effd700 1 -- 172.21.15.32:0/3815855509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f275000d8c0 con 0x7f2760139c70 2022-08-01T23:02:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 -- 172.21.15.32:0/3815855509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2760139c70 msgr2=0x7f2760142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2760139c70 0x7f2760142f40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f27600af2b0 tx=0x7f2750006df0).stop 2022-08-01T23:02:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 -- 172.21.15.32:0/3815855509 shutdown_connections 2022-08-01T23:02:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2760143430 0x7f27600ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 --2- 172.21.15.32:0/3815855509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2760139c70 0x7f2760142f40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 -- 172.21.15.32:0/3815855509 >> 172.21.15.32:0/3815855509 conn(0x7f276005f840 msgr2=0x7f276005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 -- 172.21.15.32:0/3815855509 shutdown_connections 2022-08-01T23:02:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.520+0000 7f27665e6700 1 -- 172.21.15.32:0/3815855509 wait complete. 2022-08-01T23:02:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 Processor -- start 2022-08-01T23:02:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 -- start start 2022-08-01T23:02:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f27600b1de0 0x7f27600b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 0x7f276013a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f276013a730 0x7f2760152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760152910 con 0x7f276013c350 2022-08-01T23:02:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276013c7d0 con 0x7f276013a730 2022-08-01T23:02:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600b48d0 con 0x7f27600b1de0 2022-08-01T23:02:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 0x7f276013a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 0x7f276013a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45360/0 (socket says 172.21.15.32:45360) 2022-08-01T23:02:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 -- 172.21.15.32:0/2782355218 learned_addr learned my addr 172.21.15.32:0/2782355218 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275ffff700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f27600b1de0 0x7f27600b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f2764b83700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f276013a730 0x7f2760152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 -- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f27600b1de0 msgr2=0x7f27600b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f27600b1de0 0x7f27600b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 -- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f276013a730 msgr2=0x7f2760152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f276013a730 0x7f2760152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 -- 172.21.15.32:0/2782355218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275000b180 con 0x7f276013c350 2022-08-01T23:02:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f2764b83700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f276013a730 0x7f2760152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275f7fe700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 0x7f276013a240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f2754006940 tx=0x7f2754008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2754011780 con 0x7f276013c350 2022-08-01T23:02:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27540118e0 con 0x7f276013c350 2022-08-01T23:02:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276013beb0 con 0x7f276013c350 2022-08-01T23:02:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2754011c40 con 0x7f276013c350 2022-08-01T23:02:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.524+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276013b8a0 con 0x7f276013c350 2022-08-01T23:02:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.528+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276013c040 con 0x7f276013c350 2022-08-01T23:02:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.528+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2754012420 con 0x7f276013c350 2022-08-01T23:02:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.528+0000 7f275d7fa700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2740053980 0x7f2740055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.528+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f275406bb60 con 0x7f276013c350 2022-08-01T23:02:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.528+0000 7f275ffff700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2740053980 0x7f2740055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.528+0000 7f275ffff700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2740053980 0x7f2740055e40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f275001a000 tx=0x7f2750014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:38.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.536+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2754042a60 con 0x7f276013c350 2022-08-01T23:02:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2760143e00 con 0x7f276013c350 2022-08-01T23:02:38.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f275d7fa700 1 -- 172.21.15.32:0/2782355218 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v80) v1 ==== 81+0+29292 (secure 0 0 0) 0x7f2754046080 con 0x7f276013c350 2022-08-01T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:38.669 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 80, 2022-08-01T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:02:35.975656+0000", 2022-08-01T23:02:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:02:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:02:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:02:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:02:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:02:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:02:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:02:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:02:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:02:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:02:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 14, 2022-08-01T23:02:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:02:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:02:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:02:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:02:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:02:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:02:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:02:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:02:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:02:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:02:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:02:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:02:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:02:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:02:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:02:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:02:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:02:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:02:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:02:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:02:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:02:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:02:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:02:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:02:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:02:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:02:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:02:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:02:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.710 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.711 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:02:38.711 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:02:38.712 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.713 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.713 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:02:38.714 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:02:38.715 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.716 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:02:38.717 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.718 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:02:38.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:38.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.718 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.719 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:02:38.719 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:02:38.719 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:02:38.719 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:02:38.720 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.720 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.720 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.721 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:02:38.721 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:02:38.721 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.721 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.722 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.722 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.723 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.723 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 63, 2022-08-01T23:02:38.723 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.723 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.723 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:02:38.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.725 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:02:38.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.726 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.726 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.727 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:02:38.727 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:02:38.728 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.729 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:02:38.730 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:02:38.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.732 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.733 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:02:38.733 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:02:38.735 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:38.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.735 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:02:38.736 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:02:38.736 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:02:38.736 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:02:38.736 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.736 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.737 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.737 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:02:38.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:02:38.738 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.738 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.738 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.739 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.739 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.739 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:38.740 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.740 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.740 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.740 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.741 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:02:38.741 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.742 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:02:38.742 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.742 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.743 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:02:38.744 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.745 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:02:38.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.746 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.747 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:02:38.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:02:38.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.749 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:02:38.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:02:38.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:38.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.752 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:02:38.752 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:02:38.752 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:02:38.752 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:02:38.753 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.753 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.753 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.753 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.754 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:02:38.754 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:02:38.754 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.755 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.755 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.755 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.755 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.756 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.756 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.756 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:38.756 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.756 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.757 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.757 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.758 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:02:38.758 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.759 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:02:38.759 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.759 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.760 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.760 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.761 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:02:38.761 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:02:38.762 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.763 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:02:38.764 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.765 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:02:38.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:02:38.767 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:02:38.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:38.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.769 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:02:38.769 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:02:38.770 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:02:38.770 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:02:38.770 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.770 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.770 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.771 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:02:38.771 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:02:38.772 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.772 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.772 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.772 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.773 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.773 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.773 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.773 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 73, 2022-08-01T23:02:38.773 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.773 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.774 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:02:38.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.776 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:02:38.776 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.777 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:02:38.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.778 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:02:38.779 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.779 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.780 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.780 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.781 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:02:38.781 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:02:38.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.784 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.784 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:02:38.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:02:38.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:38.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.788 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:02:38.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:02:38.789 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:02:38.789 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:02:38.789 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.789 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.790 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.790 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:02:38.791 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:02:38.791 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.792 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.792 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.792 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.793 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.793 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.793 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.794 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:38.794 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.794 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.795 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:02:38.797 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:02:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.800 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:02:38.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:02:38.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:02:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:02:38.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.808 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:02:38.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:02:38.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:38.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.811 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:02:38.811 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:02:38.812 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:02:38.812 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:02:38.812 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.812 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.813 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.813 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:02:38.814 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:02:38.814 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.814 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.814 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.814 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.815 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.815 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 74, 2022-08-01T23:02:38.816 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.816 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.816 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.817 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:02:38.817 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:02:38.818 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:02:38.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:02:38.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:02:38.823 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:02:38.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.825 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:02:38.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:02:38.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:38.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.828 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:02:38.829 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:02:38.829 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:02:38.829 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:02:38.829 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.830 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.830 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.831 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:02:38.831 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:02:38.831 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:38.831 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:38.832 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:38.832 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:38.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:38.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:38.832 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:38.833 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 73, 2022-08-01T23:02:38.833 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:38.833 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:38.833 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:38.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:02:38.834 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:02:38.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:38.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:02:38.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:02:38.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.843 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:38.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:02:38.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:02:38.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.846 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:38.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:38.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:38.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:02:38.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:38.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:02:38.848 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:38.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.849 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:02:38.849 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:02:38.849 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:02:38.849 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:02:38.850 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:38.850 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:38.850 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:38.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:38.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.851 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:38.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:02:38.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:02:38.852 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.852 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.852 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.852 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.852 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:02:38.853 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.854 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:02:38.854 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.854 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.854 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.854 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.855 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.855 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:02:38.855 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.856 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:02:38.856 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.856 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.856 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.857 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.857 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.857 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:02:38.857 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.858 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:02:38.858 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.858 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.859 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.859 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.859 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.859 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:02:38.860 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:02:38.860 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.861 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.861 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.861 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.861 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:02:38.862 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.863 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:02:38.864 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.864 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.864 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.865 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.865 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:02:38.866 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:02:38.867 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.867 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.867 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.867 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.868 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.868 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:02:38.868 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:38.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:02:38.869 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:38.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:38.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:38.870 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:38.871 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:38.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:02:38.871 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:38.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.872 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:38.872 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:02:38.872 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:02:38.872 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:02:38.872 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:02:38.873 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:02:38.873 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/1927724642": "2022-08-02T23:02:31.955270+0000", 2022-08-01T23:02:38.873 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/1927724642": "2022-08-02T23:02:31.955270+0000", 2022-08-01T23:02:38.874 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.130:0/438018148": "2022-08-02T00:02:20.149979+0000" 2022-08-01T23:02:38.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.875 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:02:38.875 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:02:38.875 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:02:38.876 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:02:38.876 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:02:38.877 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:02:38.877 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:02:38.878 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:02:38.878 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:02:38.878 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:02:38.879 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:02:38.879 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:02:38.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.880 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:02:38.881 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:02:38.881 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:02:38.881 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:02:38.882 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:02:38.882 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:02:38.882 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:02:38.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:38.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:38.883 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:02:38.884 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:02:38.884 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:02:38.885 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:02:38.885 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:02:38.885 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:02:38.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2740053980 msgr2=0x7f2740055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2740053980 0x7f2740055e40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f275001a000 tx=0x7f2750014000).stop 2022-08-01T23:02:38.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 msgr2=0x7f276013a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:38.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 0x7f276013a240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f2754006940 tx=0x7f2754008040).stop 2022-08-01T23:02:38.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 shutdown_connections 2022-08-01T23:02:38.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2740053980 0x7f2740055e40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f276013a730 0x7f2760152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276013c350 0x7f276013a240 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 --2- 172.21.15.32:0/2782355218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f27600b1de0 0x7f27600b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:38.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.664+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 >> 172.21.15.32:0/2782355218 conn(0x7f276005f840 msgr2=0x7f27600b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:38.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.668+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 shutdown_connections 2022-08-01T23:02:38.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.668+0000 7f27665e6700 1 -- 172.21.15.32:0/2782355218 wait complete. 2022-08-01T23:02:38.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:38.996+0000 7f1257582700 1 Processor -- start 2022-08-01T23:02:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- start start 2022-08-01T23:02:39.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12500b01a0 0x7f12500a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12500afd60 0x7f12500a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12500a7250 con 0x7f12500b2710 2022-08-01T23:02:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12500a7480 con 0x7f12500b01a0 2022-08-01T23:02:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12500a7720 con 0x7f12500afd60 2022-08-01T23:02:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256580700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f12500b2710 0x7f125014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32908/0 (socket says 172.21.15.32:32908) 2022-08-01T23:02:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256580700 1 -- 172.21.15.32:0/217499227 learned_addr learned my addr 172.21.15.32:0/217499227 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1255d7f700 1 --2- 172.21.15.32:0/217499227 >> v2:172.21.15.32:3300/0 conn(0x7f12500b01a0 0x7f12500a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 --2- 172.21.15.32:0/217499227 >> v2:172.21.15.55:3300/0 conn(0x7f12500afd60 0x7f12500a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385966368 0 0) 0x7f12500a7250 con 0x7f12500b2710 2022-08-01T23:02:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f123c003260 con 0x7f12500b2710 2022-08-01T23:02:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835426036 0 0) 0x7f123c003260 con 0x7f12500b2710 2022-08-01T23:02:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 >> v2:172.21.15.55:3300/0 conn(0x7f12500afd60 msgr2=0x7f12500a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 --2- 172.21.15.32:0/217499227 >> v2:172.21.15.55:3300/0 conn(0x7f12500afd60 0x7f12500a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 >> v2:172.21.15.32:3300/0 conn(0x7f12500b01a0 msgr2=0x7f12500a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 --2- 172.21.15.32:0/217499227 >> v2:172.21.15.32:3300/0 conn(0x7f12500b01a0 0x7f12500a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500a79c0 con 0x7f12500b2710 2022-08-01T23:02:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f12400019e0 con 0x7f12500b2710 2022-08-01T23:02:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 >> v1:172.21.15.32:6789/0 conn(0x7f12500b2710 legacy=0x7f125014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c003750 0x7f123c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f123c004500 0x7f123c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c004e70 0x7f123c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500a7480 con 0x7f123c004e70 2022-08-01T23:02:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500a7720 con 0x7f123c004500 2022-08-01T23:02:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c00bc40 con 0x7f123c003750 2022-08-01T23:02:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c004e70 0x7f123c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1255d7f700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f123c004500 0x7f123c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256580700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c003750 0x7f123c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 -- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c003750 msgr2=0x7f123c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c003750 0x7f123c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 -- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f123c004500 msgr2=0x7f123c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f123c004500 0x7f123c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 -- 172.21.15.32:0/217499227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c003260 con 0x7f123c004e70 2022-08-01T23:02:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1255d7f700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f123c004500 0x7f123c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256580700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c003750 0x7f123c003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:02:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 -- 172.21.15.32:0/217499227 reap_dead start 2022-08-01T23:02:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1256d81700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c004e70 0x7f123c00b750 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f124800c000 tx=0x7f1248006df0).ready entity=mon.0 client_cookie=d82bc41ae203ef08 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f124800e7a0 con 0x7f123c004e70 2022-08-01T23:02:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124800e900 con 0x7f123c004e70 2022-08-01T23:02:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f125557e700 1 -- 172.21.15.32:0/217499227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f124800ec60 con 0x7f123c004e70 2022-08-01T23:02:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- 172.21.15.32:0/217499227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c004e70 msgr2=0x7f123c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c004e70 0x7f123c00b750 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f124800c000 tx=0x7f1248006df0).stop 2022-08-01T23:02:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- 172.21.15.32:0/217499227 shutdown_connections 2022-08-01T23:02:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 --2- 172.21.15.32:0/217499227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c004e70 0x7f123c00b750 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.000+0000 7f1257582700 1 -- 172.21.15.32:0/217499227 >> 172.21.15.32:0/217499227 conn(0x7f1250060310 msgr2=0x7f12500b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- 172.21.15.32:0/217499227 shutdown_connections 2022-08-01T23:02:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- 172.21.15.32:0/217499227 wait complete. 2022-08-01T23:02:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 Processor -- start 2022-08-01T23:02:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- start start 2022-08-01T23:02:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c004e70 0x7f12500679d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12500aa4d0 0x7f12500b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 0x7f12500b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500b2f60 con 0x7f12500b49d0 2022-08-01T23:02:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500ace50 con 0x7f12500aa4d0 2022-08-01T23:02:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12500b3830 con 0x7f123c004e70 2022-08-01T23:02:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 0x7f12500b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 0x7f12500b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45382/0 (socket says 172.21.15.32:45382) 2022-08-01T23:02:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 -- 172.21.15.32:0/3637961374 learned_addr learned my addr 172.21.15.32:0/3637961374 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1255d7f700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12500aa4d0 0x7f12500b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256580700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c004e70 0x7f12500679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 -- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c004e70 msgr2=0x7f12500679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c004e70 0x7f12500679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 -- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12500aa4d0 msgr2=0x7f12500b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12500aa4d0 0x7f12500b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 -- 172.21.15.32:0/3637961374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500b3190 con 0x7f12500b49d0 2022-08-01T23:02:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1256d81700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 0x7f12500b4e50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f124800c000 tx=0x7f1248018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1248023070 con 0x7f12500b49d0 2022-08-01T23:02:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1248000e90 con 0x7f12500b49d0 2022-08-01T23:02:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12500b3430 con 0x7f12500b49d0 2022-08-01T23:02:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f124800be60 con 0x7f12500b49d0 2022-08-01T23:02:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.004+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125014bf50 con 0x7f12500b49d0 2022-08-01T23:02:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.008+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12500574e0 con 0x7f12500b49d0 2022-08-01T23:02:39.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.008+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f124801f430 con 0x7f12500b49d0 2022-08-01T23:02:39.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.008+0000 7f124f7fe700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12300539e0 0x7f1230055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.008+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f124806d280 con 0x7f12500b49d0 2022-08-01T23:02:39.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.008+0000 7f1256580700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12300539e0 0x7f1230055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.008+0000 7f1256580700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12300539e0 0x7f1230055ea0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f124000c000 tx=0x7f1240009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.016+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1248044290 con 0x7f12500b49d0 2022-08-01T23:02:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.176+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1250069a70 con 0x7f12500b49d0 2022-08-01T23:02:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.176+0000 7f124f7fe700 1 -- 172.21.15.32:0/3637961374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f124800b3c0 con 0x7f12500b49d0 2022-08-01T23:02:39.181 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:39.182 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6567,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1091686254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1091686254},{"type":"v1","addr":"172.21.15.32:6835","nonce":1091686254}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[]} 2022-08-01T23:02:39.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12300539e0 msgr2=0x7f1230055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12300539e0 0x7f1230055ea0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f124000c000 tx=0x7f1240009040).stop 2022-08-01T23:02:39.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 msgr2=0x7f12500b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 0x7f12500b4e50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f124800c000 tx=0x7f1248018000).stop 2022-08-01T23:02:39.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 shutdown_connections 2022-08-01T23:02:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12300539e0 0x7f1230055ea0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12500b49d0 0x7f12500b4e50 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12500aa4d0 0x7f12500b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 --2- 172.21.15.32:0/3637961374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f123c004e70 0x7f12500679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 >> 172.21.15.32:0/3637961374 conn(0x7f1250060310 msgr2=0x7f1250144da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 shutdown_connections 2022-08-01T23:02:39.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.180+0000 7f1257582700 1 -- 172.21.15.32:0/3637961374 wait complete. 2022-08-01T23:02:39.187 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2022-08-01T23:02:39.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 Processor -- start 2022-08-01T23:02:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- start start 2022-08-01T23:02:39.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07dc0b1f40 0x7f07dc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07dc148530 0x7f07dc0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc0b04e0 con 0x7f07dc0a7250 2022-08-01T23:02:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc0a9250 con 0x7f07dc148530 2022-08-01T23:02:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc0b0780 con 0x7f07dc0b1f40 2022-08-01T23:02:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07dc148530 0x7f07dc0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07dc148530 0x7f07dc0b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45386/0 (socket says 172.21.15.32:45386) 2022-08-01T23:02:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 -- 172.21.15.32:0/2343939383 learned_addr learned my addr 172.21.15.32:0/2343939383 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e226f700 1 --2- 172.21.15.32:0/2343939383 >> v2:172.21.15.55:3300/0 conn(0x7f07dc0b1f40 0x7f07dc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 -- 172.21.15.32:0/2343939383 >> v2:172.21.15.55:3300/0 conn(0x7f07dc0b1f40 msgr2=0x7f07dc0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 --2- 172.21.15.32:0/2343939383 >> v2:172.21.15.55:3300/0 conn(0x7f07dc0b1f40 0x7f07dc0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 -- 172.21.15.32:0/2343939383 >> v1:172.21.15.32:6789/0 conn(0x7f07dc0a7250 legacy=0x7f07dc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 -- 172.21.15.32:0/2343939383 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0b0a20 con 0x7f07dc148530 2022-08-01T23:02:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e126d700 1 -- 172.21.15.32:0/2343939383 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901418360 0 0) 0x7f07dc0b04e0 con 0x7f07dc0a7250 2022-08-01T23:02:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e226f700 1 --2- 172.21.15.32:0/2343939383 >> v2:172.21.15.55:3300/0 conn(0x7f07dc0b1f40 0x7f07dc0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:39.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e1a6e700 1 --2- 172.21.15.32:0/2343939383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc148530 0x7f07dc0b2320 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f07dc1478c0 tx=0x7f07cc006eb0).ready entity=mon.0 client_cookie=78b474d73bf8e4b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e126d700 1 -- 172.21.15.32:0/2343939383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00d610 con 0x7f07dc148530 2022-08-01T23:02:39.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e126d700 1 -- 172.21.15.32:0/2343939383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc000e90 con 0x7f07dc148530 2022-08-01T23:02:39.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e126d700 1 -- 172.21.15.32:0/2343939383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00d960 con 0x7f07dc148530 2022-08-01T23:02:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- 172.21.15.32:0/2343939383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc148530 msgr2=0x7f07dc0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 --2- 172.21.15.32:0/2343939383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc148530 0x7f07dc0b2320 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f07dc1478c0 tx=0x7f07cc006eb0).stop 2022-08-01T23:02:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- 172.21.15.32:0/2343939383 shutdown_connections 2022-08-01T23:02:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 --2- 172.21.15.32:0/2343939383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc148530 0x7f07dc0b2320 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 --2- 172.21.15.32:0/2343939383 >> v2:172.21.15.55:3300/0 conn(0x7f07dc0b1f40 0x7f07dc0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- 172.21.15.32:0/2343939383 >> 172.21.15.32:0/2343939383 conn(0x7f07dc05f840 msgr2=0x7f07dc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- 172.21.15.32:0/2343939383 shutdown_connections 2022-08-01T23:02:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.296+0000 7f07e44d3700 1 -- 172.21.15.32:0/2343939383 wait complete. 2022-08-01T23:02:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 Processor -- start 2022-08-01T23:02:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 -- start start 2022-08-01T23:02:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 0x7f07dc0abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07dc0b1f40 0x7f07dc14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07dc148530 0x7f07dc14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc14a7d0 con 0x7f07dc0a7250 2022-08-01T23:02:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc149df0 con 0x7f07dc148530 2022-08-01T23:02:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e44d3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc14a090 con 0x7f07dc0b1f40 2022-08-01T23:02:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 0x7f07dc0abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 0x7f07dc0abe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45392/0 (socket says 172.21.15.32:45392) 2022-08-01T23:02:39.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 -- 172.21.15.32:0/3072032100 learned_addr learned my addr 172.21.15.32:0/3072032100 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:39.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e1a6e700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07dc0b1f40 0x7f07dc14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e2a70700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07dc148530 0x7f07dc14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 -- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07dc0b1f40 msgr2=0x7f07dc14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07dc0b1f40 0x7f07dc14aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 -- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07dc148530 msgr2=0x7f07dc14b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07dc148530 0x7f07dc14b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 -- 172.21.15.32:0/3072032100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc00b1e0 con 0x7f07dc0a7250 2022-08-01T23:02:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e2a70700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07dc148530 0x7f07dc14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:02:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07e226f700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 0x7f07dc0abe20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f07d8006940 tx=0x7f07d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.300+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d8011780 con 0x7f07dc0a7250 2022-08-01T23:02:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d80118e0 con 0x7f07dc0a7250 2022-08-01T23:02:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc14a330 con 0x7f07dc0a7250 2022-08-01T23:02:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d8011c40 con 0x7f07dc0a7250 2022-08-01T23:02:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc149870 con 0x7f07dc0a7250 2022-08-01T23:02:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc0a9770 con 0x7f07dc0a7250 2022-08-01T23:02:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07d8012420 con 0x7f07dc0a7250 2022-08-01T23:02:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07cb7fe700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07bc053b30 0x7f07bc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f07d806bd70 con 0x7f07dc0a7250 2022-08-01T23:02:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.304+0000 7f07e1a6e700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07bc053b30 0x7f07bc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.308+0000 7f07e1a6e700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07bc053b30 0x7f07bc055ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f07cc01a000 tx=0x7f07cc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.312+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07d8042ce0 con 0x7f07dc0a7250 2022-08-01T23:02:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.472+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07dc14a4c0 con 0x7f07dc0a7250 2022-08-01T23:02:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.476+0000 7f07cb7fe700 1 -- 172.21.15.32:0/3072032100 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f07d8046300 con 0x7f07dc0a7250 2022-08-01T23:02:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:39.480 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/2535209700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2535209700},{"type":"v1","addr":"172.21.15.55:6835","nonce":2535209700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/806127818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":806127818},{"type":"v1","addr":"172.21.15.32:6833","nonce":806127818}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32},{"gid":6567,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1091686254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1091686254},{"type":"v1","addr":"172.21.15.32:6835","nonce":1091686254}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[]} 2022-08-01T23:02:39.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07bc053b30 msgr2=0x7f07bc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07bc053b30 0x7f07bc055ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f07cc01a000 tx=0x7f07cc015000).stop 2022-08-01T23:02:39.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 msgr2=0x7f07dc0abe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 0x7f07dc0abe20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f07d8006940 tx=0x7f07d8008040).stop 2022-08-01T23:02:39.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 shutdown_connections 2022-08-01T23:02:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f07bc053b30 0x7f07bc055ff0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07dc148530 0x7f07dc14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07dc0b1f40 0x7f07dc14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 --2- 172.21.15.32:0/3072032100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc0a7250 0x7f07dc0abe20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 >> 172.21.15.32:0/3072032100 conn(0x7f07dc05f840 msgr2=0x7f07dc0b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 shutdown_connections 2022-08-01T23:02:39.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.480+0000 7f07e44d3700 1 -- 172.21.15.32:0/3072032100 wait complete. 2022-08-01T23:02:39.487 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2022-08-01T23:02:39.498 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:02:39.498 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:02:39.499 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:02:39.499 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:02:39.499 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:02:39.499 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:02:39.500 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:02:39.500 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:02:39.501 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:02:39.501 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:02:39.501 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:02:39.501 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:02:39.511 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:02:39.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:02:39.513 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:02:39.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:02:39.518 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:02:39.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:02:39.521 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:02:39.521 DEBUG:teuthology.parallel:result is None 2022-08-01T23:02:39.556 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:02:39.556 DEBUG:teuthology.parallel:result is None 2022-08-01T23:02:39.557 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:02:39.558 DEBUG:teuthology.parallel:result is None 2022-08-01T23:02:39.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:02:39.597 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:02:39.623 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:02:39.635 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:02:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 Processor -- start 2022-08-01T23:02:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 -- start start 2022-08-01T23:02:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b180ada00 0x7f5b180b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b180b15c0 0x7f5b18067190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b180acfa0 con 0x7f5b180acbc0 2022-08-01T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b180b52f0 con 0x7f5b180b15c0 2022-08-01T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1e582700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b180ae070 con 0x7f5b180ada00 2022-08-01T23:02:39.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b180b15c0 0x7f5b18067190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b180b15c0 0x7f5b18067190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45434/0 (socket says 172.21.15.32:45434) 2022-08-01T23:02:39.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 -- 172.21.15.32:0/1557097599 learned_addr learned my addr 172.21.15.32:0/1557097599 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:39.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1cd7f700 1 --2- 172.21.15.32:0/1557097599 >> v2:172.21.15.55:3300/0 conn(0x7f5b180ada00 0x7f5b180b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 -- 172.21.15.32:0/1557097599 >> v2:172.21.15.55:3300/0 conn(0x7f5b180ada00 msgr2=0x7f5b180b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 --2- 172.21.15.32:0/1557097599 >> v2:172.21.15.55:3300/0 conn(0x7f5b180ada00 0x7f5b180b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 -- 172.21.15.32:0/1557097599 >> v1:172.21.15.32:6789/0 conn(0x7f5b180acbc0 legacy=0x7f5b18143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 -- 172.21.15.32:0/1557097599 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b180ae310 con 0x7f5b180b15c0 2022-08-01T23:02:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1cd7f700 1 --2- 172.21.15.32:0/1557097599 >> v2:172.21.15.55:3300/0 conn(0x7f5b180ada00 0x7f5b180b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.656+0000 7f5b1dd81700 1 --2- 172.21.15.32:0/1557097599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180b15c0 0x7f5b18067190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f5b180afa40 tx=0x7f5b10009290).ready entity=mon.0 client_cookie=9b493e1addf62fa2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b07fff700 1 -- 172.21.15.32:0/1557097599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1000b830 con 0x7f5b180b15c0 2022-08-01T23:02:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b07fff700 1 -- 172.21.15.32:0/1557097599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000b990 con 0x7f5b180b15c0 2022-08-01T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b07fff700 1 -- 172.21.15.32:0/1557097599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1000bb40 con 0x7f5b180b15c0 2022-08-01T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- 172.21.15.32:0/1557097599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180b15c0 msgr2=0x7f5b18067190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 --2- 172.21.15.32:0/1557097599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180b15c0 0x7f5b18067190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f5b180afa40 tx=0x7f5b10009290).stop 2022-08-01T23:02:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- 172.21.15.32:0/1557097599 shutdown_connections 2022-08-01T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 --2- 172.21.15.32:0/1557097599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180b15c0 0x7f5b18067190 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 --2- 172.21.15.32:0/1557097599 >> v2:172.21.15.55:3300/0 conn(0x7f5b180ada00 0x7f5b180b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- 172.21.15.32:0/1557097599 >> 172.21.15.32:0/1557097599 conn(0x7f5b180626f0 msgr2=0x7f5b18062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- 172.21.15.32:0/1557097599 shutdown_connections 2022-08-01T23:02:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- 172.21.15.32:0/1557097599 wait complete. 2022-08-01T23:02:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 Processor -- start 2022-08-01T23:02:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- start start 2022-08-01T23:02:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 0x7f5b18145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b180ada00 0x7f5b1813f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b180b15c0 0x7f5b18146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1813db50 con 0x7f5b180acbc0 2022-08-01T23:02:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18141fb0 con 0x7f5b180b15c0 2022-08-01T23:02:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1e582700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18144b20 con 0x7f5b180ada00 2022-08-01T23:02:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1d580700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 0x7f5b18145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1d580700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 0x7f5b18145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45436/0 (socket says 172.21.15.32:45436) 2022-08-01T23:02:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.660+0000 7f5b1d580700 1 -- 172.21.15.32:0/3158907665 learned_addr learned my addr 172.21.15.32:0/3158907665 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1cd7f700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b180ada00 0x7f5b1813f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1dd81700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b180b15c0 0x7f5b18146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1d580700 1 -- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b180ada00 msgr2=0x7f5b1813f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1d580700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b180ada00 0x7f5b1813f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1d580700 1 -- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b180b15c0 msgr2=0x7f5b18146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1d580700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b180b15c0 0x7f5b18146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1d580700 1 -- 172.21.15.32:0/3158907665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009530 con 0x7f5b180acbc0 2022-08-01T23:02:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1d580700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 0x7f5b18145260 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f5b14002f90 tx=0x7f5b14006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b14009e20 con 0x7f5b180acbc0 2022-08-01T23:02:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14013460 con 0x7f5b180acbc0 2022-08-01T23:02:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b18144dc0 con 0x7f5b180acbc0 2022-08-01T23:02:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b140137c0 con 0x7f5b180acbc0 2022-08-01T23:02:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b18141380 con 0x7f5b180acbc0 2022-08-01T23:02:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18144f50 con 0x7f5b180acbc0 2022-08-01T23:02:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b1401a060 con 0x7f5b180acbc0 2022-08-01T23:02:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b067fc700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af8053b00 0x7f5af8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f5b1406b610 con 0x7f5b180acbc0 2022-08-01T23:02:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.664+0000 7f5b1cd7f700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af8053b00 0x7f5af8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:39.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.668+0000 7f5b1cd7f700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af8053b00 0x7f5af8055fc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f5b0800c000 tx=0x7f5b08008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:39.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.672+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b14042680 con 0x7f5b180acbc0 2022-08-01T23:02:39.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:39.808+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5b180ae5b0 con 0x7f5b180acbc0 2022-08-01T23:02:40.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.664+0000 7f5b067fc700 1 -- 172.21.15.32:0/3158907665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5b14045ca0 con 0x7f5b180acbc0 2022-08-01T23:02:40.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af8053b00 msgr2=0x7f5af8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:40.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af8053b00 0x7f5af8055fc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f5b0800c000 tx=0x7f5b08008040).stop 2022-08-01T23:02:40.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 msgr2=0x7f5b18145260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:40.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 0x7f5b18145260 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f5b14002f90 tx=0x7f5b14006df0).stop 2022-08-01T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 shutdown_connections 2022-08-01T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af8053b00 0x7f5af8055fc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b180b15c0 0x7f5b18146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b180ada00 0x7f5b1813f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 --2- 172.21.15.32:0/3158907665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b180acbc0 0x7f5b18145260 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 >> 172.21.15.32:0/3158907665 conn(0x7f5b180626f0 msgr2=0x7f5b181433b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 shutdown_connections 2022-08-01T23:02:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.668+0000 7f5b1e582700 1 -- 172.21.15.32:0/3158907665 wait complete. 2022-08-01T23:02:40.678 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:02:40.691 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:02:40.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.792+0000 7fb5f636f700 1 Processor -- start 2022-08-01T23:02:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.792+0000 7fb5f636f700 1 -- start start 2022-08-01T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5f013e8d0 0x7fb5f0141210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:40.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f00b1180 0x7fb5f0141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f00aac20 con 0x7fb5f00b28b0 2022-08-01T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f0140e20 con 0x7fb5f00b1180 2022-08-01T23:02:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f00b2c90 con 0x7fb5f013e8d0 2022-08-01T23:02:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f00b1180 0x7fb5f0141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5f00b1180 0x7fb5f0141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45452/0 (socket says 172.21.15.32:45452) 2022-08-01T23:02:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3046188521 learned_addr learned my addr 172.21.15.32:0/3046188521 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f4b6c700 1 --2- 172.21.15.32:0/3046188521 >> v2:172.21.15.55:3300/0 conn(0x7fb5f013e8d0 0x7fb5f0141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3046188521 >> v2:172.21.15.55:3300/0 conn(0x7fb5f013e8d0 msgr2=0x7fb5f0141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 --2- 172.21.15.32:0/3046188521 >> v2:172.21.15.55:3300/0 conn(0x7fb5f013e8d0 0x7fb5f0141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3046188521 >> v1:172.21.15.55:6789/0 conn(0x7fb5f00b28b0 legacy=0x7fb5f013e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3046188521 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00aa440 con 0x7fb5f00b1180 2022-08-01T23:02:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f5b6e700 1 --2- 172.21.15.32:0/3046188521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b1180 0x7fb5f0141700 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fb5e80032f0 tx=0x7fb5e8009290).ready entity=mon.0 client_cookie=74f5aaaf855ce572 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5dffff700 1 -- 172.21.15.32:0/3046188521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e800b870 con 0x7fb5f00b1180 2022-08-01T23:02:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5dffff700 1 -- 172.21.15.32:0/3046188521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e800b9d0 con 0x7fb5f00b1180 2022-08-01T23:02:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5dffff700 1 -- 172.21.15.32:0/3046188521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e800bb40 con 0x7fb5f00b1180 2022-08-01T23:02:40.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- 172.21.15.32:0/3046188521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b1180 msgr2=0x7fb5f0141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:40.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3046188521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b1180 0x7fb5f0141700 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fb5e80032f0 tx=0x7fb5e8009290).stop 2022-08-01T23:02:40.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- 172.21.15.32:0/3046188521 shutdown_connections 2022-08-01T23:02:40.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3046188521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b1180 0x7fb5f0141700 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3046188521 >> v2:172.21.15.55:3300/0 conn(0x7fb5f013e8d0 0x7fb5f0141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- 172.21.15.32:0/3046188521 >> 172.21.15.32:0/3046188521 conn(0x7fb5f0060310 msgr2=0x7fb5f00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.796+0000 7fb5f636f700 1 -- 172.21.15.32:0/3046188521 shutdown_connections 2022-08-01T23:02:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.800+0000 7fb5f636f700 1 -- 172.21.15.32:0/3046188521 wait complete. 2022-08-01T23:02:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.800+0000 7fb5f636f700 1 Processor -- start 2022-08-01T23:02:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.800+0000 7fb5f636f700 1 -- start start 2022-08-01T23:02:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5f00b1180 0x7fb5f0063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 0x7fb5f00acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5f01310f0 0x7fb5f01315a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f013df40 con 0x7fb5f00b28b0 2022-08-01T23:02:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00ae660 con 0x7fb5f01310f0 2022-08-01T23:02:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b4be0 con 0x7fb5f00b1180 2022-08-01T23:02:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f4b6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 0x7fb5f00acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f4b6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 0x7fb5f00acad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45456/0 (socket says 172.21.15.32:45456) 2022-08-01T23:02:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f4b6c700 1 -- 172.21.15.32:0/3313431784 learned_addr learned my addr 172.21.15.32:0/3313431784 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5f01310f0 0x7fb5f01315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f536d700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5f00b1180 0x7fb5f0063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5f00b1180 msgr2=0x7fb5f0063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5f00b1180 0x7fb5f0063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3313431784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 msgr2=0x7fb5f00acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 0x7fb5f00acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 -- 172.21.15.32:0/3313431784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8009530 con 0x7fb5f01310f0 2022-08-01T23:02:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f4b6c700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 0x7fb5f00acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f5b6e700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5f01310f0 0x7fb5f01315a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fb5e80032f0 tx=0x7fb5e8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e800b870 con 0x7fb5f01310f0 2022-08-01T23:02:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8021480 con 0x7fb5f01310f0 2022-08-01T23:02:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f00ae8c0 con 0x7fb5f01310f0 2022-08-01T23:02:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5e80217e0 con 0x7fb5f01310f0 2022-08-01T23:02:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.804+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f0133b50 con 0x7fb5f01310f0 2022-08-01T23:02:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.808+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f00b3de0 con 0x7fb5f01310f0 2022-08-01T23:02:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.808+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5e800f070 con 0x7fb5f01310f0 2022-08-01T23:02:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.808+0000 7fb5de7fc700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5d00539e0 0x7fb5d0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.808+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb5e806cf90 con 0x7fb5f01310f0 2022-08-01T23:02:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.808+0000 7fb5f536d700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5d00539e0 0x7fb5d0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.808+0000 7fb5f536d700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5d00539e0 0x7fb5d0055ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fb5ec00d000 tx=0x7fb5ec000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:40.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.816+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5e80440d0 con 0x7fb5f01310f0 2022-08-01T23:02:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb5f00aea50 con 0x7fb5f01310f0 2022-08-01T23:02:40.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5de7fc700 1 -- 172.21.15.32:0/3313431784 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fb5e8015030 con 0x7fb5f01310f0 2022-08-01T23:02:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:40.993 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:02:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:02:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:02:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:02:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:02:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:02:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:02:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:02:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:02:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:02:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:02:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:02:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:02:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:02:41.003 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:02:41.003 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.004 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:02:41.005 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:02:41.005 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.005 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.005 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.006 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.007 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:02:41.007 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:02:41.007 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.007 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.008 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.009 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:02:41.009 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:02:41.009 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.009 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.009 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.010 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.011 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:02:41.011 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:02:41.011 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.011 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.011 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.012 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.013 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:02:41.013 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:02:41.013 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.013 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.013 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.014 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.015 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:02:41.015 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:02:41.015 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.015 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.016 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.016 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.017 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:02:41.017 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:02:41.017 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.017 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:02:41.018 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:02:41.018 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:02:41.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.019 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:02:41.019 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:02:41.019 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.019 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:02:41.020 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:02:41.020 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:02:41.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.021 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:02:41.021 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:02:41.021 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.021 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:02:41.022 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:02:41.022 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:02:41.022 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.023 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:02:41.023 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:02:41.023 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.024 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:02:41.024 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:02:41.024 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:41.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:41.025 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:02:41.025 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:02:41.025 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:02:41.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:02:41.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:02:41.026 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:02:41.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:41.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:41.027 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:02:41.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5d00539e0 msgr2=0x7fb5d0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5d00539e0 0x7fb5d0055ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fb5ec00d000 tx=0x7fb5ec000bc0).stop 2022-08-01T23:02:41.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5f01310f0 msgr2=0x7fb5f01315a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5f01310f0 0x7fb5f01315a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fb5e80032f0 tx=0x7fb5e8016000).stop 2022-08-01T23:02:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 shutdown_connections 2022-08-01T23:02:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5d00539e0 0x7fb5d0055ea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5f01310f0 0x7fb5f01315a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5f00b28b0 0x7fb5f00acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 --2- 172.21.15.32:0/3313431784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5f00b1180 0x7fb5f0063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 >> 172.21.15.32:0/3313431784 conn(0x7fb5f0060310 msgr2=0x7fb5f0137850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 shutdown_connections 2022-08-01T23:02:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:40.988+0000 7fb5f636f700 1 -- 172.21.15.32:0/3313431784 wait complete. 2022-08-01T23:02:41.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.132+0000 7f6d6d5b4700 1 Processor -- start 2022-08-01T23:02:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.132+0000 7f6d6d5b4700 1 -- start start 2022-08-01T23:02:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.132+0000 7f6d6d5b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d680b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.132+0000 7f6d6d5b4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d680b2510 con 0x7f6d680b31f0 2022-08-01T23:02:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.132+0000 7f6d6d5b4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d681443e0 con 0x7f6d680b7500 2022-08-01T23:02:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.132+0000 7f6d6d5b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680b3700 con 0x7f6d680aadb0 2022-08-01T23:02:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d677fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d680b31f0 0x7f6d68064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33004/0 (socket says 172.21.15.32:33004) 2022-08-01T23:02:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d677fe700 1 -- 172.21.15.32:0/927641979 learned_addr learned my addr 172.21.15.32:0/927641979 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6cdb3700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d680b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068270779 0 0) 0x7f6d680b2510 con 0x7f6d680b31f0 2022-08-01T23:02:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d54003260 con 0x7f6d680b31f0 2022-08-01T23:02:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4138182008 0 0) 0x7f6d54003260 con 0x7f6d680b31f0 2022-08-01T23:02:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 msgr2=0x7f6d680b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d680b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 >> v1:172.21.15.55:6789/0 conn(0x7f6d680b7500 legacy=0x7f6d6813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680b2740 con 0x7f6d680b31f0 2022-08-01T23:02:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f6d58001680 con 0x7f6d680b31f0 2022-08-01T23:02:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 >> v1:172.21.15.32:6789/0 conn(0x7f6d680b31f0 legacy=0x7f6d68064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d54003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d54004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540049f0 0x7f6d54004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680b3700 con 0x7f6d54003780 2022-08-01T23:02:41.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5400b750 con 0x7f6d540049f0 2022-08-01T23:02:41.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d54003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5400b8e0 con 0x7f6d680aadb0 2022-08-01T23:02:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6cdb3700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540049f0 0x7f6d54004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d677fe700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d54004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 -- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 msgr2=0x7f6d54004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d54004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 -- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540049f0 msgr2=0x7f6d54004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540049f0 0x7f6d54004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 -- 172.21.15.32:0/927641979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d54003260 con 0x7f6d54003780 2022-08-01T23:02:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6cdb3700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540049f0 0x7f6d54004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:02:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d677fe700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d680aadb0 0x7f6d54004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:02:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d67fff700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d54003b60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f6d5c001160 tx=0x7f6d5c007a50).ready entity=mon.0 client_cookie=d683858acf5e27b5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d5c007d60 con 0x7f6d54003780 2022-08-01T23:02:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c007030 con 0x7f6d54003780 2022-08-01T23:02:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d66ffd700 1 -- 172.21.15.32:0/927641979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d5c007390 con 0x7f6d54003780 2022-08-01T23:02:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/927641979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 msgr2=0x7f6d54003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/927641979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d54003b60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f6d5c001160 tx=0x7f6d5c007a50).stop 2022-08-01T23:02:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6cdb3700 1 -- 172.21.15.32:0/927641979 reap_dead start 2022-08-01T23:02:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/927641979 shutdown_connections 2022-08-01T23:02:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/927641979 >> 172.21.15.32:0/927641979 conn(0x7f6d6805f840 msgr2=0x7f6d6805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/927641979 shutdown_connections 2022-08-01T23:02:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.136+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/927641979 wait complete. 2022-08-01T23:02:41.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 Processor -- start 2022-08-01T23:02:41.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 -- start start 2022-08-01T23:02:41.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d681445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d68143b60 0x7f6d680ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d680ada20 0x7f6d680adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680acc50 con 0x7f6d54003780 2022-08-01T23:02:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d68062170 con 0x7f6d680ada20 2022-08-01T23:02:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680ae390 con 0x7f6d68143b60 2022-08-01T23:02:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d681445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d681445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45474/0 (socket says 172.21.15.32:45474) 2022-08-01T23:02:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 -- 172.21.15.32:0/241108093 learned_addr learned my addr 172.21.15.32:0/241108093 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6cdb3700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d680ada20 0x7f6d680adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d677fe700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d68143b60 0x7f6d680ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 -- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d68143b60 msgr2=0x7f6d680ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d68143b60 0x7f6d680ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 -- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d680ada20 msgr2=0x7f6d680adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d680ada20 0x7f6d680adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 -- 172.21.15.32:0/241108093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d68061c30 con 0x7f6d54003780 2022-08-01T23:02:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d677fe700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d68143b60 0x7f6d680ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:02:41.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d67fff700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d681445e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f6d680aa470 tx=0x7f6d5c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d5c01e620 con 0x7f6d54003780 2022-08-01T23:02:41.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c01e780 con 0x7f6d54003780 2022-08-01T23:02:41.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d68061ed0 con 0x7f6d54003780 2022-08-01T23:02:41.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d5c01eae0 con 0x7f6d54003780 2022-08-01T23:02:41.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.140+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d680ab9b0 con 0x7f6d54003780 2022-08-01T23:02:41.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.144+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d680b2920 con 0x7f6d54003780 2022-08-01T23:02:41.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.144+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6d5c01b040 con 0x7f6d54003780 2022-08-01T23:02:41.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.144+0000 7f6d657fa700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d48053950 0x7f6d48055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.144+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6d5c06ced0 con 0x7f6d54003780 2022-08-01T23:02:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.144+0000 7f6d677fe700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d48053950 0x7f6d48055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.144+0000 7f6d677fe700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d48053950 0x7f6d48055e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f6d58009000 tx=0x7f6d58009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.148+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d5c043fd0 con 0x7f6d54003780 2022-08-01T23:02:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.276+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d680574e0 con 0x7f6d54003780 2022-08-01T23:02:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.276+0000 7f6d657fa700 1 -- 172.21.15.32:0/241108093 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f6d5c0475f0 con 0x7f6d54003780 2022-08-01T23:02:41.281 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:41.281 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:40.660125+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.276+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d48053950 msgr2=0x7f6d48055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.276+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d48053950 0x7f6d48055e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f6d58009000 tx=0x7f6d58009040).stop 2022-08-01T23:02:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.276+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 msgr2=0x7f6d681445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.276+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d681445e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f6d680aa470 tx=0x7f6d5c016000).stop 2022-08-01T23:02:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 shutdown_connections 2022-08-01T23:02:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6d48053950 0x7f6d48055e10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d680ada20 0x7f6d680adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d68143b60 0x7f6d680ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 --2- 172.21.15.32:0/241108093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d54003780 0x7f6d681445e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 >> 172.21.15.32:0/241108093 conn(0x7f6d6805f840 msgr2=0x7f6d68140ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 shutdown_connections 2022-08-01T23:02:41.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.280+0000 7f6d6d5b4700 1 -- 172.21.15.32:0/241108093 wait complete. 2022-08-01T23:02:41.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:02:41.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:02:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 Processor -- start 2022-08-01T23:02:41.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 -- start start 2022-08-01T23:02:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b2710 0x7fe27814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2780a8c50 0x7fe27814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780a7250 con 0x7fe2780b01a0 2022-08-01T23:02:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2780a7480 con 0x7fe2780a8c50 2022-08-01T23:02:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27f068700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780a7720 con 0x7fe2780b2710 2022-08-01T23:02:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2780a8c50 0x7fe27814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2780a8c50 0x7fe27814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45484/0 (socket says 172.21.15.32:45484) 2022-08-01T23:02:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 -- 172.21.15.32:0/3147237166 learned_addr learned my addr 172.21.15.32:0/3147237166 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e066700 1 --2- 172.21.15.32:0/3147237166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b2710 0x7fe27814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 -- 172.21.15.32:0/3147237166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b2710 msgr2=0x7fe27814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 --2- 172.21.15.32:0/3147237166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b2710 0x7fe27814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 -- 172.21.15.32:0/3147237166 >> v1:172.21.15.55:6789/0 conn(0x7fe2780b01a0 legacy=0x7fe2780a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 -- 172.21.15.32:0/3147237166 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780a79c0 con 0x7fe2780a8c50 2022-08-01T23:02:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.388+0000 7fe27e867700 1 --2- 172.21.15.32:0/3147237166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe27814b4c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe270002f70 tx=0x7fe270006eb0).ready entity=mon.0 client_cookie=e7582e52ff359c6e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27d064700 1 -- 172.21.15.32:0/3147237166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27000d5c0 con 0x7fe2780a8c50 2022-08-01T23:02:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27d064700 1 -- 172.21.15.32:0/3147237166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270000e90 con 0x7fe2780a8c50 2022-08-01T23:02:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27d064700 1 -- 172.21.15.32:0/3147237166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27000d950 con 0x7fe2780a8c50 2022-08-01T23:02:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- 172.21.15.32:0/3147237166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 msgr2=0x7fe27814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 --2- 172.21.15.32:0/3147237166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe27814b4c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe270002f70 tx=0x7fe270006eb0).stop 2022-08-01T23:02:41.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- 172.21.15.32:0/3147237166 shutdown_connections 2022-08-01T23:02:41.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 --2- 172.21.15.32:0/3147237166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe27814b4c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 --2- 172.21.15.32:0/3147237166 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b2710 0x7fe27814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- 172.21.15.32:0/3147237166 >> 172.21.15.32:0/3147237166 conn(0x7fe278060310 msgr2=0x7fe2780b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- 172.21.15.32:0/3147237166 shutdown_connections 2022-08-01T23:02:41.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- 172.21.15.32:0/3147237166 wait complete. 2022-08-01T23:02:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 Processor -- start 2022-08-01T23:02:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- start start 2022-08-01T23:02:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe2780af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b01a0 0x7fe27813e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2780b2710 0x7fe27813eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b54b0 con 0x7fe2780a8c50 2022-08-01T23:02:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b5680 con 0x7fe2780b2710 2022-08-01T23:02:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.392+0000 7fe27f068700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b5920 con 0x7fe2780b01a0 2022-08-01T23:02:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe2780af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe2780af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45488/0 (socket says 172.21.15.32:45488) 2022-08-01T23:02:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 -- 172.21.15.32:0/1903634375 learned_addr learned my addr 172.21.15.32:0/1903634375 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e867700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2780b2710 0x7fe27813eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27d865700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b01a0 0x7fe27813e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 -- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b01a0 msgr2=0x7fe27813e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b01a0 0x7fe27813e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 -- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2780b2710 msgr2=0x7fe27813eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2780b2710 0x7fe27813eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 -- 172.21.15.32:0/1903634375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27000b1e0 con 0x7fe2780a8c50 2022-08-01T23:02:41.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e867700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2780b2710 0x7fe27813eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:41.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27e066700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe2780af130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fe26800a000 tx=0x7fe268008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe268011780 con 0x7fe2780a8c50 2022-08-01T23:02:41.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2680118e0 con 0x7fe2780a8c50 2022-08-01T23:02:41.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2780b5bc0 con 0x7fe2780a8c50 2022-08-01T23:02:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe268011c40 con 0x7fe2780a8c50 2022-08-01T23:02:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2780b5010 con 0x7fe2780a8c50 2022-08-01T23:02:41.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2780574e0 con 0x7fe2780a8c50 2022-08-01T23:02:41.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.396+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe26801c460 con 0x7fe2780a8c50 2022-08-01T23:02:41.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.400+0000 7fe2777fe700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe258053b30 0x7fe258055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.400+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe26806c010 con 0x7fe2780a8c50 2022-08-01T23:02:41.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.400+0000 7fe27d865700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe258053b30 0x7fe258055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.400+0000 7fe27d865700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe258053b30 0x7fe258055ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fe26c00d000 tx=0x7fe26c000f40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.408+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2680430b0 con 0x7fe2780a8c50 2022-08-01T23:02:41.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.532+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe278149c20 con 0x7fe2780a8c50 2022-08-01T23:02:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.532+0000 7fe2777fe700 1 -- 172.21.15.32:0/1903634375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v83) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe2680466d0 con 0x7fe2780a8c50 2022-08-01T23:02:41.538 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:02:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe258053b30 msgr2=0x7fe258055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe258053b30 0x7fe258055ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fe26c00d000 tx=0x7fe26c000f40).stop 2022-08-01T23:02:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 msgr2=0x7fe2780af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe2780af130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fe26800a000 tx=0x7fe268008040).stop 2022-08-01T23:02:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 shutdown_connections 2022-08-01T23:02:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe258053b30 0x7fe258055ff0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2780b2710 0x7fe27813eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2780b01a0 0x7fe27813e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 --2- 172.21.15.32:0/1903634375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a8c50 0x7fe2780af130 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 >> 172.21.15.32:0/1903634375 conn(0x7fe278060310 msgr2=0x7fe27814a050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 shutdown_connections 2022-08-01T23:02:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.536+0000 7fe27f068700 1 -- 172.21.15.32:0/1903634375 wait complete. 2022-08-01T23:02:41.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:02:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 Processor -- start 2022-08-01T23:02:41.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 -- start start 2022-08-01T23:02:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6e4064b10 0x7fe6e4144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e4141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e4064ef0 con 0x7fe6e4141860 2022-08-01T23:02:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e413ebf0 con 0x7fe6e4064b10 2022-08-01T23:02:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4066690 con 0x7fe6e40b36e0 2022-08-01T23:02:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e1d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e4141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6e4064b10 0x7fe6e4144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e1d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e4141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:57234/0 (socket says 172.21.15.32:57234) 2022-08-01T23:02:41.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e2d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe6e4141860 0x7fe6e4066ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:53948/0 (socket says 172.21.15.32:53948) 2022-08-01T23:02:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.656+0000 7fe6e1d9b700 1 -- 172.21.15.32:0/561073465 learned_addr learned my addr 172.21.15.32:0/561073465 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e159a700 1 -- 172.21.15.32:0/561073465 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181171914 0 0) 0x7fe6e4064ef0 con 0x7fe6e4141860 2022-08-01T23:02:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e159a700 1 -- 172.21.15.32:0/561073465 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6c4003260 con 0x7fe6e4141860 2022-08-01T23:02:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e1d9b700 1 -- 172.21.15.32:0/561073465 >> v2:172.21.15.55:3300/0 conn(0x7fe6e4064b10 msgr2=0x7fe6e4144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e1d9b700 1 --2- 172.21.15.32:0/561073465 >> v2:172.21.15.55:3300/0 conn(0x7fe6e4064b10 0x7fe6e4144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e1d9b700 1 -- 172.21.15.32:0/561073465 >> v1:172.21.15.55:6789/0 conn(0x7fe6e4141860 legacy=0x7fe6e4066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e1d9b700 1 -- 172.21.15.32:0/561073465 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40b4850 con 0x7fe6e40b36e0 2022-08-01T23:02:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e1d9b700 1 --2- 172.21.15.32:0/561073465 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e4141370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe6dc0032f0 tx=0x7fe6dc009290).ready entity=mon.2 client_cookie=9c725f973ffc9450 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e159a700 1 -- 172.21.15.32:0/561073465 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6dc00b6c0 con 0x7fe6e40b36e0 2022-08-01T23:02:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e159a700 1 -- 172.21.15.32:0/561073465 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc00b820 con 0x7fe6e40b36e0 2022-08-01T23:02:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e159a700 1 -- 172.21.15.32:0/561073465 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6dc00b990 con 0x7fe6e40b36e0 2022-08-01T23:02:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 -- 172.21.15.32:0/561073465 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 msgr2=0x7fe6e4141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 --2- 172.21.15.32:0/561073465 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e4141370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe6dc0032f0 tx=0x7fe6dc009290).stop 2022-08-01T23:02:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 -- 172.21.15.32:0/561073465 shutdown_connections 2022-08-01T23:02:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 --2- 172.21.15.32:0/561073465 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e4141370 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 --2- 172.21.15.32:0/561073465 >> v2:172.21.15.55:3300/0 conn(0x7fe6e4064b10 0x7fe6e4144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 -- 172.21.15.32:0/561073465 >> 172.21.15.32:0/561073465 conn(0x7fe6e405f840 msgr2=0x7fe6e405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 -- 172.21.15.32:0/561073465 shutdown_connections 2022-08-01T23:02:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 -- 172.21.15.32:0/561073465 wait complete. 2022-08-01T23:02:41.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 Processor -- start 2022-08-01T23:02:41.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.660+0000 7fe6e359e700 1 -- start start 2022-08-01T23:02:41.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 0x7fe6e413aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e41502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6e4141860 0x7fe6e41507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4150c90 con 0x7fe6e4064b10 2022-08-01T23:02:41.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4153990 con 0x7fe6e4141860 2022-08-01T23:02:41.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4153c30 con 0x7fe6e40b36e0 2022-08-01T23:02:41.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 0x7fe6e413aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 0x7fe6e413aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45502/0 (socket says 172.21.15.32:45502) 2022-08-01T23:02:41.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 -- 172.21.15.32:0/1016918368 learned_addr learned my addr 172.21.15.32:0/1016918368 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:41.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e1d9b700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e41502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e2d9d700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6e4141860 0x7fe6e41507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 -- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 msgr2=0x7fe6e41502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e41502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 -- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6e4141860 msgr2=0x7fe6e41507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6e4141860 0x7fe6e41507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 -- 172.21.15.32:0/1016918368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc009530 con 0x7fe6e4064b10 2022-08-01T23:02:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e2d9d700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6e4141860 0x7fe6e41507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:02:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e259c700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 0x7fe6e413aa00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fe6d400c000 tx=0x7fe6d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6d4010780 con 0x7fe6e4064b10 2022-08-01T23:02:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d40108e0 con 0x7fe6e4064b10 2022-08-01T23:02:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e4153ed0 con 0x7fe6e4064b10 2022-08-01T23:02:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6d4010c40 con 0x7fe6e4064b10 2022-08-01T23:02:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e405cf00 con 0x7fe6e4064b10 2022-08-01T23:02:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e40b4af0 con 0x7fe6e4064b10 2022-08-01T23:02:41.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.664+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe6d401c460 con 0x7fe6e4064b10 2022-08-01T23:02:41.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.668+0000 7fe6db7fe700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe6c8053980 0x7fe6c8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.668+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe6d406c190 con 0x7fe6e4064b10 2022-08-01T23:02:41.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.668+0000 7fe6e1d9b700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe6c8053980 0x7fe6c8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.668+0000 7fe6e1d9b700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe6c8053980 0x7fe6c8055e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fe6dc019000 tx=0x7fe6dc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.672+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe6d4043230 con 0x7fe6e4064b10 2022-08-01T23:02:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.796+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe6e41547b0 con 0x7fe6e4064b10 2022-08-01T23:02:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6db7fe700 1 -- 172.21.15.32:0/1016918368 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6d4046850 con 0x7fe6e4064b10 2022-08-01T23:02:41.804 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:41.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe6c8053980 msgr2=0x7fe6c8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe6c8053980 0x7fe6c8055e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fe6dc019000 tx=0x7fe6dc016000).stop 2022-08-01T23:02:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 msgr2=0x7fe6e413aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 0x7fe6e413aa00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fe6d400c000 tx=0x7fe6d4006df0).stop 2022-08-01T23:02:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 shutdown_connections 2022-08-01T23:02:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe6c8053980 0x7fe6c8055e40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6e4141860 0x7fe6e41507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe6e40b36e0 0x7fe6e41502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 --2- 172.21.15.32:0/1016918368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6e4064b10 0x7fe6e413aa00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 >> 172.21.15.32:0/1016918368 conn(0x7fe6e405f840 msgr2=0x7fe6e40b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 shutdown_connections 2022-08-01T23:02:41.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.800+0000 7fe6e359e700 1 -- 172.21.15.32:0/1016918368 wait complete. 2022-08-01T23:02:41.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:02:41.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.912+0000 7f0cc6d63700 1 Processor -- start 2022-08-01T23:02:41.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.912+0000 7f0cc6d63700 1 -- start start 2022-08-01T23:02:41.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc6d63700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 0x7f0cc014b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc6d63700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc014bb40 con 0x7f0cc00672f0 2022-08-01T23:02:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc6d63700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc00aadb0 con 0x7f0cc0066f10 2022-08-01T23:02:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc6d63700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00b50a0 con 0x7f0cc014a780 2022-08-01T23:02:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc5300700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0cc00672f0 0x7f0cc0147c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33056/0 (socket says 172.21.15.32:33056) 2022-08-01T23:02:41.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc5300700 1 -- 172.21.15.32:0/2096713430 learned_addr learned my addr 172.21.15.32:0/2096713430 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:41.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 0x7f0cc014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956665926 0 0) 0x7f0cc014bb40 con 0x7f0cc00672f0 2022-08-01T23:02:41.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ca4003260 con 0x7f0cc00672f0 2022-08-01T23:02:41.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763766765 0 0) 0x7f0cc00aadb0 con 0x7f0cc0066f10 2022-08-01T23:02:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc014bb40 con 0x7f0cc0066f10 2022-08-01T23:02:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 930548596 0 0) 0x7f0ca4003260 con 0x7f0cc00672f0 2022-08-01T23:02:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 msgr2=0x7f0cc014b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 0x7f0cc014b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 >> v1:172.21.15.55:6789/0 conn(0x7f0cc0066f10 legacy=0x7f0cc014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00ab050 con 0x7f0cc00672f0 2022-08-01T23:02:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f0cb0001660 con 0x7f0cc00672f0 2022-08-01T23:02:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 >> v1:172.21.15.32:6789/0 conn(0x7f0cc00672f0 legacy=0x7f0cc0147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 0x7f0ca4003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca4003e20 0x7f0ca4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca40049f0 0x7f0ca4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca400b800 con 0x7f0ca4003e20 2022-08-01T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca400b940 con 0x7f0ca40049f0 2022-08-01T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca400bb00 con 0x7f0cc014a780 2022-08-01T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca4003e20 0x7f0ca4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc4aff700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 0x7f0ca4003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc5300700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca40049f0 0x7f0ca4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 -- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 msgr2=0x7f0ca4003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0cc014a780 0x7f0ca4003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 -- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca40049f0 msgr2=0x7f0ca4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca40049f0 0x7f0ca4004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 -- 172.21.15.32:0/2096713430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4003260 con 0x7f0ca4003e20 2022-08-01T23:02:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbffff700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca4003e20 0x7f0ca4004500 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f0ca800c000 tx=0x7f0ca8006df0).ready entity=mon.0 client_cookie=918e4766e1acb99f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca800e830 con 0x7f0ca4003e20 2022-08-01T23:02:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca800e990 con 0x7f0ca4003e20 2022-08-01T23:02:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/2096713430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca800ecf0 con 0x7f0ca4003e20 2022-08-01T23:02:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc6d63700 1 -- 172.21.15.32:0/2096713430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca4003e20 msgr2=0x7f0ca4004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/2096713430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca4003e20 0x7f0ca4004500 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f0ca800c000 tx=0x7f0ca8006df0).stop 2022-08-01T23:02:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.916+0000 7f0cc5300700 1 -- 172.21.15.32:0/2096713430 reap_dead start 2022-08-01T23:02:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- 172.21.15.32:0/2096713430 shutdown_connections 2022-08-01T23:02:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- 172.21.15.32:0/2096713430 >> 172.21.15.32:0/2096713430 conn(0x7f0cc005c620 msgr2=0x7f0cc005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- 172.21.15.32:0/2096713430 shutdown_connections 2022-08-01T23:02:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- 172.21.15.32:0/2096713430 wait complete. 2022-08-01T23:02:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 Processor -- start 2022-08-01T23:02:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- start start 2022-08-01T23:02:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0ca4003e20 0x7f0cc00b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cc00ad390 0x7f0cc00aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 0x7f0cc0159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00ad910 con 0x7f0cc00af0f0 2022-08-01T23:02:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00aca80 con 0x7f0cc00ad390 2022-08-01T23:02:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc6d63700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00acd20 con 0x7f0ca4003e20 2022-08-01T23:02:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 0x7f0cc0159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 0x7f0cc0159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45526/0 (socket says 172.21.15.32:45526) 2022-08-01T23:02:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 -- 172.21.15.32:0/579283773 learned_addr learned my addr 172.21.15.32:0/579283773 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cbffff700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cc00ad390 0x7f0cc00aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc4aff700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0ca4003e20 0x7f0cc00b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 -- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0ca4003e20 msgr2=0x7f0cc00b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0ca4003e20 0x7f0cc00b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 -- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cc00ad390 msgr2=0x7f0cc00aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:41.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cc00ad390 0x7f0cc00aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:41.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 -- 172.21.15.32:0/579283773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00acfc0 con 0x7f0cc00af0f0 2022-08-01T23:02:41.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.920+0000 7f0cc5300700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 0x7f0cc0159e40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f0cb0000b60 tx=0x7f0cc0066af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cb0012540 con 0x7f0cc00af0f0 2022-08-01T23:02:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb00126a0 con 0x7f0cc00af0f0 2022-08-01T23:02:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc00b16e0 con 0x7f0cc00af0f0 2022-08-01T23:02:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cb0012a00 con 0x7f0cc00af0f0 2022-08-01T23:02:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc00ae2b0 con 0x7f0cc00af0f0 2022-08-01T23:02:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc00af8e0 con 0x7f0cc00af0f0 2022-08-01T23:02:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0cb0012b60 con 0x7f0cc00af0f0 2022-08-01T23:02:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cbdffb700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c98053a10 0x7f0c98055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0cb006b5f0 con 0x7f0cc00af0f0 2022-08-01T23:02:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.924+0000 7f0cc4aff700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c98053a10 0x7f0c98055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.928+0000 7f0cc4aff700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c98053a10 0x7f0c98055ed0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f0cb8003000 tx=0x7f0cb8007d80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:41.932+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0cb0042650 con 0x7f0cc00af0f0 2022-08-01T23:02:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.060+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0cc00ab2f0 con 0x7f0cc00af0f0 2022-08-01T23:02:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.060+0000 7f0cbdffb700 1 -- 172.21.15.32:0/579283773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0cb0019020 con 0x7f0cc00af0f0 2022-08-01T23:02:42.067 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:42.067 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:02:42.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:02:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:02:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:02:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:02:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:42.069 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:02:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c98053a10 msgr2=0x7f0c98055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c98053a10 0x7f0c98055ed0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f0cb8003000 tx=0x7f0cb8007d80).stop 2022-08-01T23:02:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 msgr2=0x7f0cc0159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 0x7f0cc0159e40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f0cb0000b60 tx=0x7f0cc0066af0).stop 2022-08-01T23:02:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 shutdown_connections 2022-08-01T23:02:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0c98053a10 0x7f0c98055ed0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc00af0f0 0x7f0cc0159e40 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cc00ad390 0x7f0cc00aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 --2- 172.21.15.32:0/579283773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0ca4003e20 0x7f0cc00b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 >> 172.21.15.32:0/579283773 conn(0x7f0cc005c620 msgr2=0x7f0cc005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 shutdown_connections 2022-08-01T23:02:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.064+0000 7f0cc6d63700 1 -- 172.21.15.32:0/579283773 wait complete. 2022-08-01T23:02:42.079 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:02:42.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:02:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9cd8a2700 1 Processor -- start 2022-08-01T23:02:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9cd8a2700 1 -- start start 2022-08-01T23:02:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9cd8a2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c80aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9cd8a2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c8061b20 con 0x7fa9c813f920 2022-08-01T23:02:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9cd8a2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c8061dc0 con 0x7fa9c80aa050 2022-08-01T23:02:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9cd8a2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8060d20 con 0x7fa9c8060560 2022-08-01T23:02:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9c77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9c813f920 0x7fa9c814a1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33076/0 (socket says 172.21.15.32:33076) 2022-08-01T23:02:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9c77fe700 1 -- 172.21.15.32:0/2421293841 learned_addr learned my addr 172.21.15.32:0/2421293841 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9c67fc700 1 --2- 172.21.15.32:0/2421293841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c80aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.180+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141841569 0 0) 0x7fa9c8061b20 con 0x7fa9c813f920 2022-08-01T23:02:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ac003260 con 0x7fa9c813f920 2022-08-01T23:02:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216920990 0 0) 0x7fa9c8061dc0 con 0x7fa9c80aa050 2022-08-01T23:02:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c8061b20 con 0x7fa9c80aa050 2022-08-01T23:02:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186721831 0 0) 0x7fa9ac003260 con 0x7fa9c813f920 2022-08-01T23:02:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c67fc700 1 -- 172.21.15.32:0/2421293841 >> v1:172.21.15.55:6789/0 conn(0x7fa9c80aa050 legacy=0x7fa9c813ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c67fc700 1 -- 172.21.15.32:0/2421293841 >> v1:172.21.15.32:6789/0 conn(0x7fa9c813f920 legacy=0x7fa9c814a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c67fc700 1 -- 172.21.15.32:0/2421293841 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c8060fc0 con 0x7fa9c8060560 2022-08-01T23:02:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c67fc700 1 --2- 172.21.15.32:0/2421293841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c80aa430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa9c0001490 tx=0x7fa9bc006e80).ready entity=mon.2 client_cookie=590e8dbcb0be1195 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9bc014070 con 0x7fa9c8060560 2022-08-01T23:02:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc000e90 con 0x7fa9c8060560 2022-08-01T23:02:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9c5ffb700 1 -- 172.21.15.32:0/2421293841 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c00019e0 con 0x7fa9c8060560 2022-08-01T23:02:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2421293841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 msgr2=0x7fa9c80aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2421293841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c80aa430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa9c0001490 tx=0x7fa9bc006e80).stop 2022-08-01T23:02:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2421293841 shutdown_connections 2022-08-01T23:02:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2421293841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c80aa430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2421293841 >> 172.21.15.32:0/2421293841 conn(0x7fa9c80b2b00 msgr2=0x7fa9c80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2421293841 shutdown_connections 2022-08-01T23:02:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.184+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2421293841 wait complete. 2022-08-01T23:02:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 Processor -- start 2022-08-01T23:02:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 -- start start 2022-08-01T23:02:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c8156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9c80aa050 0x7fa9c8141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 0x7fa9c8142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8139210 con 0x7fa9c813f920 2022-08-01T23:02:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c81437c0 con 0x7fa9c80aa050 2022-08-01T23:02:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9cd8a2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8156da0 con 0x7fa9c8060560 2022-08-01T23:02:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 0x7fa9c8142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 0x7fa9c8142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45540/0 (socket says 172.21.15.32:45540) 2022-08-01T23:02:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 -- 172.21.15.32:0/2221289239 learned_addr learned my addr 172.21.15.32:0/2221289239 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c67fc700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9c80aa050 0x7fa9c8141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c6ffd700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c8156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 -- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 msgr2=0x7fa9c8156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c8156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 -- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9c80aa050 msgr2=0x7fa9c8141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9c80aa050 0x7fa9c8141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 -- 172.21.15.32:0/2221289239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc00c310 con 0x7fa9c813f920 2022-08-01T23:02:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c6ffd700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9c77fe700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 0x7fa9c8142210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fa9c80b0280 tx=0x7fa9c0001c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c0004ab0 con 0x7fa9c813f920 2022-08-01T23:02:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.188+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c0004c10 con 0x7fa9c813f920 2022-08-01T23:02:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c8067120 con 0x7fa9c813f920 2022-08-01T23:02:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c0007170 con 0x7fa9c813f920 2022-08-01T23:02:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c8156710 con 0x7fa9c813f920 2022-08-01T23:02:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c8156ab0 con 0x7fa9c813f920 2022-08-01T23:02:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9c001b070 con 0x7fa9c813f920 2022-08-01T23:02:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9b7fff700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9a4053a70 0x7fa9a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa9c006b650 con 0x7fa9c813f920 2022-08-01T23:02:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9c6ffd700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9a4053a70 0x7fa9a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.192+0000 7fa9c6ffd700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9a4053a70 0x7fa9a4055f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fa9b800b000 tx=0x7fa9b8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.200+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9c0042750 con 0x7fa9c813f920 2022-08-01T23:02:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:42.328+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa9c8061260 con 0x7fa9c813f920 2022-08-01T23:02:43.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.304+0000 7fa9b7fff700 1 -- 172.21.15.32:0/2221289239 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa9c0045d70 con 0x7fa9c813f920 2022-08-01T23:02:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.308+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9a4053a70 msgr2=0x7fa9a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.308+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9a4053a70 0x7fa9a4055f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fa9b800b000 tx=0x7fa9b8009040).stop 2022-08-01T23:02:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.308+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 msgr2=0x7fa9c8142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.308+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 0x7fa9c8142210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fa9c80b0280 tx=0x7fa9c0001c50).stop 2022-08-01T23:02:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 shutdown_connections 2022-08-01T23:02:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9a4053a70 0x7fa9a4055f30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c813f920 0x7fa9c8142210 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9c80aa050 0x7fa9c8141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 --2- 172.21.15.32:0/2221289239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9c8060560 0x7fa9c8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 >> 172.21.15.32:0/2221289239 conn(0x7fa9c80b2b00 msgr2=0x7fa9c813e340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 shutdown_connections 2022-08-01T23:02:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.312+0000 7fa9cd8a2700 1 -- 172.21.15.32:0/2221289239 wait complete. 2022-08-01T23:02:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:02:43.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:02:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.440+0000 7f3d67018700 1 Processor -- start 2022-08-01T23:02:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.440+0000 7f3d67018700 1 -- start start 2022-08-01T23:02:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 0x7f3d60064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d600b2510 con 0x7f3d600b7500 2022-08-01T23:02:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d601443e0 con 0x7f3d600aadb0 2022-08-01T23:02:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600b3700 con 0x7f3d600b31f0 2022-08-01T23:02:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d66817700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 0x7f3d60064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d66817700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 0x7f3d60064d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:55334/0 (socket says 172.21.15.32:55334) 2022-08-01T23:02:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:57284/0 (socket says 172.21.15.32:57284) 2022-08-01T23:02:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d66817700 1 -- 172.21.15.32:0/884107937 learned_addr learned my addr 172.21.15.32:0/884107937 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 -- 172.21.15.32:0/884107937 >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 msgr2=0x7f3d60064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 --2- 172.21.15.32:0/884107937 >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 0x7f3d60064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 -- 172.21.15.32:0/884107937 >> v1:172.21.15.55:6789/0 conn(0x7f3d600b7500 legacy=0x7f3d6013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 -- 172.21.15.32:0/884107937 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600b2740 con 0x7f3d600b31f0 2022-08-01T23:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d66817700 1 --2- 172.21.15.32:0/884107937 >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 0x7f3d60064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65815700 1 --2- 172.21.15.32:0/884107937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600b78e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3d54002f70 tx=0x7f3d54006df0).ready entity=mon.2 client_cookie=65413a5da5ba67a2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65014700 1 -- 172.21.15.32:0/884107937 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5400d570 con 0x7f3d600b31f0 2022-08-01T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65014700 1 -- 172.21.15.32:0/884107937 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d54000e90 con 0x7f3d600b31f0 2022-08-01T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d65014700 1 -- 172.21.15.32:0/884107937 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5400d8c0 con 0x7f3d600b31f0 2022-08-01T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- 172.21.15.32:0/884107937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 msgr2=0x7f3d600b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 --2- 172.21.15.32:0/884107937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600b78e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3d54002f70 tx=0x7f3d54006df0).stop 2022-08-01T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- 172.21.15.32:0/884107937 shutdown_connections 2022-08-01T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 --2- 172.21.15.32:0/884107937 >> v2:172.21.15.55:3300/0 conn(0x7f3d600aadb0 0x7f3d60064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 --2- 172.21.15.32:0/884107937 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600b78e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- 172.21.15.32:0/884107937 >> 172.21.15.32:0/884107937 conn(0x7f3d6005f840 msgr2=0x7f3d6005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:43.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- 172.21.15.32:0/884107937 shutdown_connections 2022-08-01T23:02:43.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.444+0000 7f3d67018700 1 -- 172.21.15.32:0/884107937 wait complete. 2022-08-01T23:02:43.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.448+0000 7f3d67018700 1 Processor -- start 2022-08-01T23:02:43.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.448+0000 7f3d67018700 1 -- start start 2022-08-01T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d600aadb0 0x7f3d600af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 0x7f3d60141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600ac670 con 0x7f3d600b7500 2022-08-01T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600af9f0 con 0x7f3d600aadb0 2022-08-01T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600ab640 con 0x7f3d600b31f0 2022-08-01T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 0x7f3d60141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d65815700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66016700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d600aadb0 0x7f3d600af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 0x7f3d60141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45554/0 (socket says 172.21.15.32:45554) 2022-08-01T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 -- 172.21.15.32:0/2784052657 learned_addr learned my addr 172.21.15.32:0/2784052657 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d65815700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:57290/0 (socket says 172.21.15.32:57290) 2022-08-01T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 -- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 msgr2=0x7f3d600af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 -- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d600aadb0 msgr2=0x7f3d600af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d600aadb0 0x7f3d600af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 -- 172.21.15.32:0/2784052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d58009530 con 0x7f3d600b7500 2022-08-01T23:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d66817700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 0x7f3d60141f60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f3d58006ec0 tx=0x7f3d58009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5800c850 con 0x7f3d600b7500 2022-08-01T23:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5800c9b0 con 0x7f3d600b7500 2022-08-01T23:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d5800cd10 con 0x7f3d600b7500 2022-08-01T23:02:43.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5400b180 con 0x7f3d600b7500 2022-08-01T23:02:43.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d600ab8a0 con 0x7f3d600b7500 2022-08-01T23:02:43.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3d58022070 con 0x7f3d600b7500 2022-08-01T23:02:43.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d60141c80 con 0x7f3d600b7500 2022-08-01T23:02:43.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d537fe700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3d40053b30 0x7f3d40055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.452+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f3d5806c700 con 0x7f3d600b7500 2022-08-01T23:02:43.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.456+0000 7f3d66016700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3d40053b30 0x7f3d40055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.456+0000 7f3d66016700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3d40053b30 0x7f3d40055ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f3d5c00d000 tx=0x7f3d5c007be0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:43.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.464+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d580435f0 con 0x7f3d600b7500 2022-08-01T23:02:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.592+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f3d600574e0 con 0x7f3d600b7500 2022-08-01T23:02:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.656+0000 7f3d537fe700 1 -- 172.21.15.32:0/2784052657 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v85) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3d58016020 con 0x7f3d600b7500 2022-08-01T23:02:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3d40053b30 msgr2=0x7f3d40055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3d40053b30 0x7f3d40055ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f3d5c00d000 tx=0x7f3d5c007be0).stop 2022-08-01T23:02:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 msgr2=0x7f3d60141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 0x7f3d60141f60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f3d58006ec0 tx=0x7f3d58009290).stop 2022-08-01T23:02:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 shutdown_connections 2022-08-01T23:02:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3d40053b30 0x7f3d40055ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d600b7500 0x7f3d60141f60 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3d600b31f0 0x7f3d600af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 --2- 172.21.15.32:0/2784052657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d600aadb0 0x7f3d600af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 >> 172.21.15.32:0/2784052657 conn(0x7f3d6005f840 msgr2=0x7f3d6013fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 shutdown_connections 2022-08-01T23:02:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.660+0000 7f3d67018700 1 -- 172.21.15.32:0/2784052657 wait complete. 2022-08-01T23:02:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:02:43.678 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:02:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 Processor -- start 2022-08-01T23:02:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 -- start start 2022-08-01T23:02:43.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f368c0b7840 0x7f368c060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c143400 0x7f368c143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0671e0 con 0x7f368c0b2930 2022-08-01T23:02:43.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c065540 con 0x7f368c0b7840 2022-08-01T23:02:43.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f369239c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b7c20 con 0x7f368c143400 2022-08-01T23:02:43.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f3690b99700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f368c0b2930 0x7f368c0610d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33102/0 (socket says 172.21.15.32:33102) 2022-08-01T23:02:43.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.780+0000 7f3690b99700 1 -- 172.21.15.32:0/4132104192 learned_addr learned my addr 172.21.15.32:0/4132104192 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:43.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f3691b9b700 1 --2- 172.21.15.32:0/4132104192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c143400 0x7f368c143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369139a700 1 --2- 172.21.15.32:0/4132104192 >> v2:172.21.15.55:3300/0 conn(0x7f368c0b7840 0x7f368c060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f367bfff700 1 -- 172.21.15.32:0/4132104192 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290962847 0 0) 0x7f368c0671e0 con 0x7f368c0b2930 2022-08-01T23:02:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f367bfff700 1 -- 172.21.15.32:0/4132104192 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3674003260 con 0x7f368c0b2930 2022-08-01T23:02:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f3691b9b700 1 -- 172.21.15.32:0/4132104192 >> v2:172.21.15.55:3300/0 conn(0x7f368c0b7840 msgr2=0x7f368c060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f3691b9b700 1 --2- 172.21.15.32:0/4132104192 >> v2:172.21.15.55:3300/0 conn(0x7f368c0b7840 0x7f368c060be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f3691b9b700 1 -- 172.21.15.32:0/4132104192 >> v1:172.21.15.32:6789/0 conn(0x7f368c0b2930 legacy=0x7f368c0610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f3691b9b700 1 -- 172.21.15.32:0/4132104192 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0617e0 con 0x7f368c143400 2022-08-01T23:02:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369139a700 1 --2- 172.21.15.32:0/4132104192 >> v2:172.21.15.55:3300/0 conn(0x7f368c0b7840 0x7f368c060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f3691b9b700 1 --2- 172.21.15.32:0/4132104192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c143400 0x7f368c143860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f36840032f0 tx=0x7f36840091d0).ready entity=mon.2 client_cookie=b89ce50ba6922407 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f367bfff700 1 -- 172.21.15.32:0/4132104192 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368400b600 con 0x7f368c143400 2022-08-01T23:02:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f367bfff700 1 -- 172.21.15.32:0/4132104192 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368400b760 con 0x7f368c143400 2022-08-01T23:02:43.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f367bfff700 1 -- 172.21.15.32:0/4132104192 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368400b8d0 con 0x7f368c143400 2022-08-01T23:02:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 -- 172.21.15.32:0/4132104192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c143400 msgr2=0x7f368c143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 --2- 172.21.15.32:0/4132104192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c143400 0x7f368c143860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f36840032f0 tx=0x7f36840091d0).stop 2022-08-01T23:02:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 -- 172.21.15.32:0/4132104192 shutdown_connections 2022-08-01T23:02:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 --2- 172.21.15.32:0/4132104192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c143400 0x7f368c143860 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 --2- 172.21.15.32:0/4132104192 >> v2:172.21.15.55:3300/0 conn(0x7f368c0b7840 0x7f368c060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 -- 172.21.15.32:0/4132104192 >> 172.21.15.32:0/4132104192 conn(0x7f368c05f840 msgr2=0x7f368c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:43.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 -- 172.21.15.32:0/4132104192 shutdown_connections 2022-08-01T23:02:43.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.784+0000 7f369239c700 1 -- 172.21.15.32:0/4132104192 wait complete. 2022-08-01T23:02:43.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 Processor -- start 2022-08-01T23:02:43.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 -- start start 2022-08-01T23:02:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c0b2930 0x7f368c0b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 0x7f368c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f368c143400 0x7f368c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b0b70 con 0x7f368c0b7840 2022-08-01T23:02:43.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0ac190 con 0x7f368c143400 2022-08-01T23:02:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0ac430 con 0x7f368c0b2930 2022-08-01T23:02:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3690b99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 0x7f368c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3690b99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 0x7f368c0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45566/0 (socket says 172.21.15.32:45566) 2022-08-01T23:02:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369139a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c0b2930 0x7f368c0b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3690b99700 1 -- 172.21.15.32:0/2780486379 learned_addr learned my addr 172.21.15.32:0/2780486379 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f368c143400 0x7f368c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 -- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c0b2930 msgr2=0x7f368c0b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c0b2930 0x7f368c0b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 -- 172.21.15.32:0/2780486379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 msgr2=0x7f368c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 0x7f368c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 -- 172.21.15.32:0/2780486379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368800b120 con 0x7f368c143400 2022-08-01T23:02:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3690b99700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 0x7f368c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f3691b9b700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f368c143400 0x7f368c0b14c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f36840032f0 tx=0x7f3684016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368400b8d0 con 0x7f368c143400 2022-08-01T23:02:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368401f3f0 con 0x7f368c143400 2022-08-01T23:02:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3684009470 con 0x7f368c143400 2022-08-01T23:02:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.788+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f368400b600 con 0x7f368c143400 2022-08-01T23:02:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.792+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0b07b0 con 0x7f368c143400 2022-08-01T23:02:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.792+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c0574e0 con 0x7f368c143400 2022-08-01T23:02:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.792+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3684026030 con 0x7f368c143400 2022-08-01T23:02:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.792+0000 7f367a7fc700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f366c053aa0 0x7f366c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.792+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f368406cd60 con 0x7f368c143400 2022-08-01T23:02:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.792+0000 7f369139a700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f366c053aa0 0x7f366c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:43.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.796+0000 7f369139a700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f366c053aa0 0x7f366c055f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f3688008000 tx=0x7f3688008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:43.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.800+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3684043a90 con 0x7f368c143400 2022-08-01T23:02:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.964+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f368c060330 con 0x7f368c143400 2022-08-01T23:02:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.976+0000 7f367a7fc700 1 -- 172.21.15.32:0/2780486379 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v38) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3684024070 con 0x7f368c143400 2022-08-01T23:02:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f366c053aa0 msgr2=0x7f366c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f366c053aa0 0x7f366c055f60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f3688008000 tx=0x7f3688008040).stop 2022-08-01T23:02:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f368c143400 msgr2=0x7f368c0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f368c143400 0x7f368c0b14c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f36840032f0 tx=0x7f3684016000).stop 2022-08-01T23:02:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 shutdown_connections 2022-08-01T23:02:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f366c053aa0 0x7f366c055f60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f368c143400 0x7f368c0b14c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368c0b7840 0x7f368c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 --2- 172.21.15.32:0/2780486379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f368c0b2930 0x7f368c0b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 >> 172.21.15.32:0/2780486379 conn(0x7f368c05f840 msgr2=0x7f368c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:43.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.984+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 shutdown_connections 2022-08-01T23:02:43.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:43.988+0000 7f369239c700 1 -- 172.21.15.32:0/2780486379 wait complete. 2022-08-01T23:02:43.999 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 15 and data pool 16 2022-08-01T23:02:44.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4d414700 1 Processor -- start 2022-08-01T23:02:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4d414700 1 -- start start 2022-08-01T23:02:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4d414700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 0x7f4a48061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4d414700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480aee80 con 0x7f4a480ad750 2022-08-01T23:02:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4d414700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480b70b0 con 0x7f4a481405d0 2022-08-01T23:02:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4d414700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480af0b0 con 0x7f4a480614b0 2022-08-01T23:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a477fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4a480ad750 0x7f4a480b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33128/0 (socket says 172.21.15.32:33128) 2022-08-01T23:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a477fe700 1 -- 172.21.15.32:0/3807353096 learned_addr learned my addr 172.21.15.32:0/3807353096 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 0x7f4a48061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166189683 0 0) 0x7f4a480aee80 con 0x7f4a480ad750 2022-08-01T23:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a2c003260 con 0x7f4a480ad750 2022-08-01T23:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1810284196 0 0) 0x7f4a2c003260 con 0x7f4a480ad750 2022-08-01T23:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 msgr2=0x7f4a48061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 0x7f4a48061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 >> v1:172.21.15.55:6789/0 conn(0x7f4a481405d0 legacy=0x7f4a481466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480af350 con 0x7f4a480ad750 2022-08-01T23:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f4a38001660 con 0x7f4a480ad750 2022-08-01T23:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 >> v1:172.21.15.32:6789/0 conn(0x7f4a480ad750 legacy=0x7f4a480b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a2c003780 0x7f4a2c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a2c004500 0x7f4a2c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 0x7f4a2c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480af0b0 con 0x7f4a2c003780 2022-08-01T23:02:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c00b780 con 0x7f4a2c004500 2022-08-01T23:02:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c00b940 con 0x7f4a480614b0 2022-08-01T23:02:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a2c003780 0x7f4a2c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a4cc13700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 0x7f4a2c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a477fe700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a2c004500 0x7f4a2c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 -- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 msgr2=0x7f4a2c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a480614b0 0x7f4a2c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 -- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a2c004500 msgr2=0x7f4a2c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a2c004500 0x7f4a2c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 -- 172.21.15.32:0/3807353096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c003260 con 0x7f4a2c003780 2022-08-01T23:02:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a47fff700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a2c003780 0x7f4a2c003ba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f4a3000c000 tx=0x7f4a30006df0).ready entity=mon.0 client_cookie=34920aa9edaccc5d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3000e830 con 0x7f4a2c003780 2022-08-01T23:02:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.096+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000e990 con 0x7f4a2c003780 2022-08-01T23:02:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a46ffd700 1 -- 172.21.15.32:0/3807353096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a3000ecf0 con 0x7f4a2c003780 2022-08-01T23:02:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- 172.21.15.32:0/3807353096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a2c003780 msgr2=0x7f4a2c003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 --2- 172.21.15.32:0/3807353096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a2c003780 0x7f4a2c003ba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f4a3000c000 tx=0x7f4a30006df0).stop 2022-08-01T23:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 -- 172.21.15.32:0/3807353096 reap_dead start 2022-08-01T23:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- 172.21.15.32:0/3807353096 shutdown_connections 2022-08-01T23:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- 172.21.15.32:0/3807353096 >> 172.21.15.32:0/3807353096 conn(0x7f4a48066f70 msgr2=0x7f4a48067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- 172.21.15.32:0/3807353096 shutdown_connections 2022-08-01T23:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- 172.21.15.32:0/3807353096 wait complete. 2022-08-01T23:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 Processor -- start 2022-08-01T23:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- start start 2022-08-01T23:02:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a2c003780 0x7f4a480a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a480b36b0 0x7f4a480a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 0x7f4a4814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480a2380 con 0x7f4a4814faa0 2022-08-01T23:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48153520 con 0x7f4a480b36b0 2022-08-01T23:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481537c0 con 0x7f4a2c003780 2022-08-01T23:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 0x7f4a4814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 0x7f4a4814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45598/0 (socket says 172.21.15.32:45598) 2022-08-01T23:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a47fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a2c003780 0x7f4a480a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 -- 172.21.15.32:0/2699996946 learned_addr learned my addr 172.21.15.32:0/2699996946 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a477fe700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a480b36b0 0x7f4a480a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 -- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a2c003780 msgr2=0x7f4a480a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a2c003780 0x7f4a480a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 -- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a480b36b0 msgr2=0x7f4a480a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a480b36b0 0x7f4a480a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 -- 172.21.15.32:0/2699996946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480b0af0 con 0x7f4a4814faa0 2022-08-01T23:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a477fe700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a480b36b0 0x7f4a480a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4cc13700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 0x7f4a4814ff50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f4a40001210 tx=0x7f4a40009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a400032c0 con 0x7f4a4814faa0 2022-08-01T23:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40003420 con 0x7f4a4814faa0 2022-08-01T23:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a40006a80 con 0x7f4a4814faa0 2022-08-01T23:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.100+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a480b0d90 con 0x7f4a4814faa0 2022-08-01T23:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.104+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a48150440 con 0x7f4a4814faa0 2022-08-01T23:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.104+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a480b0f20 con 0x7f4a4814faa0 2022-08-01T23:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.104+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4a4001e070 con 0x7f4a4814faa0 2022-08-01T23:02:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.104+0000 7f4a457fa700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4a20053a70 0x7f4a20055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.104+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a4006b6b0 con 0x7f4a4814faa0 2022-08-01T23:02:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.104+0000 7f4a47fff700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4a20053a70 0x7f4a20055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.108+0000 7f4a47fff700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4a20053a70 0x7f4a20055f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f4a3001a000 tx=0x7f4a30019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.112+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a400423d0 con 0x7f4a4814faa0 2022-08-01T23:02:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.240+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a480574e0 con 0x7f4a4814faa0 2022-08-01T23:02:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.244+0000 7f4a457fa700 1 -- 172.21.15.32:0/2699996946 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f4a400459f0 con 0x7f4a4814faa0 2022-08-01T23:02:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:44.249 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:43.969160+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:02:42.386170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T23:02:43.649622+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4a20053a70 msgr2=0x7f4a20055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4a20053a70 0x7f4a20055f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f4a3001a000 tx=0x7f4a30019000).stop 2022-08-01T23:02:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 msgr2=0x7f4a4814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 0x7f4a4814ff50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f4a40001210 tx=0x7f4a40009040).stop 2022-08-01T23:02:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 shutdown_connections 2022-08-01T23:02:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4a20053a70 0x7f4a20055f30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a4814faa0 0x7f4a4814ff50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a480b36b0 0x7f4a480a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 --2- 172.21.15.32:0/2699996946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4a2c003780 0x7f4a480a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 >> 172.21.15.32:0/2699996946 conn(0x7f4a48066f70 msgr2=0x7f4a48146bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 shutdown_connections 2022-08-01T23:02:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.248+0000 7f4a4d414700 1 -- 172.21.15.32:0/2699996946 wait complete. 2022-08-01T23:02:44.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8abcb700 1 Processor -- start 2022-08-01T23:02:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8abcb700 1 -- start start 2022-08-01T23:02:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8abcb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e840a8c50 0x7f1e84152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8abcb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e840a7250 con 0x7f1e840b2710 2022-08-01T23:02:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8abcb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e840a7480 con 0x7f1e840b01a0 2022-08-01T23:02:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8abcb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e840a7720 con 0x7f1e840a8c50 2022-08-01T23:02:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8a3ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e840a8c50 0x7f1e84152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8a3ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e840a8c50 0x7f1e84152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45606/0 (socket says 172.21.15.32:45606) 2022-08-01T23:02:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/3442299702 learned_addr learned my addr 172.21.15.32:0/3442299702 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e89bc9700 1 --1- 172.21.15.32:0/3442299702 >> v1:172.21.15.32:6789/0 conn(0x7f1e840b2710 0x7f1e8414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33146/0 (socket says 172.21.15.32:33146) 2022-08-01T23:02:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/3442299702 >> v1:172.21.15.55:6789/0 conn(0x7f1e840b01a0 legacy=0x7f1e840a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e88bc7700 1 -- 172.21.15.32:0/3442299702 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264061275 0 0) 0x7f1e840a7250 con 0x7f1e840b2710 2022-08-01T23:02:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/3442299702 >> v1:172.21.15.32:6789/0 conn(0x7f1e840b2710 legacy=0x7f1e8414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.368+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/3442299702 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840a79c0 con 0x7f1e840a8c50 2022-08-01T23:02:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8a3ca700 1 --2- 172.21.15.32:0/3442299702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840a8c50 0x7f1e84152d70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f1e78001110 tx=0x7f1e7c009290).ready entity=mon.0 client_cookie=bd383fa87ad86f7d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e88bc7700 1 -- 172.21.15.32:0/3442299702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c00b7e0 con 0x7f1e840a8c50 2022-08-01T23:02:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e88bc7700 1 -- 172.21.15.32:0/3442299702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c00b940 con 0x7f1e840a8c50 2022-08-01T23:02:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e88bc7700 1 -- 172.21.15.32:0/3442299702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c00bab0 con 0x7f1e840a8c50 2022-08-01T23:02:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 -- 172.21.15.32:0/3442299702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840a8c50 msgr2=0x7f1e84152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/3442299702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840a8c50 0x7f1e84152d70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f1e78001110 tx=0x7f1e7c009290).stop 2022-08-01T23:02:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 -- 172.21.15.32:0/3442299702 shutdown_connections 2022-08-01T23:02:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/3442299702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840a8c50 0x7f1e84152d70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 -- 172.21.15.32:0/3442299702 >> 172.21.15.32:0/3442299702 conn(0x7f1e84060310 msgr2=0x7f1e840b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 -- 172.21.15.32:0/3442299702 shutdown_connections 2022-08-01T23:02:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.372+0000 7f1e8abcb700 1 -- 172.21.15.32:0/3442299702 wait complete. 2022-08-01T23:02:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 Processor -- start 2022-08-01T23:02:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 -- start start 2022-08-01T23:02:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e840a8c50 0x7f1e840640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e840b01a0 0x7f1e84066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 0x7f1e84066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e841574c0 con 0x7f1e840b2710 2022-08-01T23:02:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84149640 con 0x7f1e840a8c50 2022-08-01T23:02:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84157110 con 0x7f1e840b01a0 2022-08-01T23:02:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 0x7f1e84066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 0x7f1e84066fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45612/0 (socket says 172.21.15.32:45612) 2022-08-01T23:02:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/712732031 learned_addr learned my addr 172.21.15.32:0/712732031 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e89bc9700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e840a8c50 0x7f1e840640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e893c8700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e840b01a0 0x7f1e84066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e840b01a0 msgr2=0x7f1e84066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e840b01a0 0x7f1e84066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e840a8c50 msgr2=0x7f1e840640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e840a8c50 0x7f1e840640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 -- 172.21.15.32:0/712732031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c009530 con 0x7f1e840b2710 2022-08-01T23:02:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8a3ca700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 0x7f1e84066fb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f1e78001110 tx=0x7f1e7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c00b7e0 con 0x7f1e840b2710 2022-08-01T23:02:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c0032d0 con 0x7f1e840b2710 2022-08-01T23:02:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e7c011ea0 con 0x7f1e840b2710 2022-08-01T23:02:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e84148850 con 0x7f1e840b2710 2022-08-01T23:02:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.376+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e84067640 con 0x7f1e840b2710 2022-08-01T23:02:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.380+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e84157630 con 0x7f1e840b2710 2022-08-01T23:02:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.380+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1e7c00f070 con 0x7f1e840b2710 2022-08-01T23:02:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.380+0000 7f1e76ffd700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e64053b30 0x7f1e64055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.380+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1e7c06cd60 con 0x7f1e840b2710 2022-08-01T23:02:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.380+0000 7f1e89bc9700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e64053b30 0x7f1e64055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.380+0000 7f1e89bc9700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e64053b30 0x7f1e64055ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f1e8000d000 tx=0x7f1e80007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.384+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e7c043b20 con 0x7f1e840b2710 2022-08-01T23:02:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.512+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e840574e0 con 0x7f1e840b2710 2022-08-01T23:02:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.512+0000 7f1e76ffd700 1 -- 172.21.15.32:0/712732031 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f1e7c024070 con 0x7f1e840b2710 2022-08-01T23:02:44.519 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:44.519 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:43.969160+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:02:42.386170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T23:02:43.649622+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e64053b30 msgr2=0x7f1e64055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e64053b30 0x7f1e64055ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f1e8000d000 tx=0x7f1e80007bc0).stop 2022-08-01T23:02:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 msgr2=0x7f1e84066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 0x7f1e84066fb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f1e78001110 tx=0x7f1e7c016000).stop 2022-08-01T23:02:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 shutdown_connections 2022-08-01T23:02:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e64053b30 0x7f1e64055ff0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e840b2710 0x7f1e84066fb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e840b01a0 0x7f1e84066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 --2- 172.21.15.32:0/712732031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e840a8c50 0x7f1e840640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 >> 172.21.15.32:0/712732031 conn(0x7f1e84060310 msgr2=0x7f1e8405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 shutdown_connections 2022-08-01T23:02:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.516+0000 7f1e8abcb700 1 -- 172.21.15.32:0/712732031 wait complete. 2022-08-01T23:02:44.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:02:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 Processor -- start 2022-08-01T23:02:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- start start 2022-08-01T23:02:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f561805c960 0x7f56180b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f561813f520 0x7f56180b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561805cd40 con 0x7f561805d240 2022-08-01T23:02:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56180ad6d0 con 0x7f561805c960 2022-08-01T23:02:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5618067f00 con 0x7f561813f520 2022-08-01T23:02:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f5616ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f561805d240 0x7f56180acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33158/0 (socket says 172.21.15.32:33158) 2022-08-01T23:02:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f561805c960 0x7f56180b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f5616ffd700 1 -- 172.21.15.32:0/3285375774 learned_addr learned my addr 172.21.15.32:0/3285375774 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f5617fff700 1 --2- 172.21.15.32:0/3285375774 >> v2:172.21.15.55:3300/0 conn(0x7f561813f520 0x7f56180b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56177fe700 1 -- 172.21.15.32:0/3285375774 >> v2:172.21.15.55:3300/0 conn(0x7f561813f520 msgr2=0x7f56180b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56177fe700 1 --2- 172.21.15.32:0/3285375774 >> v2:172.21.15.55:3300/0 conn(0x7f561813f520 0x7f56180b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56177fe700 1 -- 172.21.15.32:0/3285375774 >> v1:172.21.15.32:6789/0 conn(0x7f561805d240 legacy=0x7f56180acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56177fe700 1 -- 172.21.15.32:0/3285375774 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56180b1920 con 0x7f561805c960 2022-08-01T23:02:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56177fe700 1 --2- 172.21.15.32:0/3285375774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805c960 0x7f56180b1280 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f5608002f70 tx=0x7f5608006df0).ready entity=mon.0 client_cookie=ed569c746d1c9cc3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56167fc700 1 -- 172.21.15.32:0/3285375774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f560800d5c0 con 0x7f561805c960 2022-08-01T23:02:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56167fc700 1 -- 172.21.15.32:0/3285375774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608000e90 con 0x7f561805c960 2022-08-01T23:02:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f56167fc700 1 -- 172.21.15.32:0/3285375774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f560800d910 con 0x7f561805c960 2022-08-01T23:02:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- 172.21.15.32:0/3285375774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805c960 msgr2=0x7f56180b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 --2- 172.21.15.32:0/3285375774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805c960 0x7f56180b1280 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f5608002f70 tx=0x7f5608006df0).stop 2022-08-01T23:02:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- 172.21.15.32:0/3285375774 shutdown_connections 2022-08-01T23:02:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 --2- 172.21.15.32:0/3285375774 >> v2:172.21.15.55:3300/0 conn(0x7f561813f520 0x7f56180b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 --2- 172.21.15.32:0/3285375774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805c960 0x7f56180b1280 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- 172.21.15.32:0/3285375774 >> 172.21.15.32:0/3285375774 conn(0x7f5618060310 msgr2=0x7f56180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- 172.21.15.32:0/3285375774 shutdown_connections 2022-08-01T23:02:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.640+0000 7f561cc09700 1 -- 172.21.15.32:0/3285375774 wait complete. 2022-08-01T23:02:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 Processor -- start 2022-08-01T23:02:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 -- start start 2022-08-01T23:02:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561805c960 0x7f5618142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 0x7f5618136210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f561813f520 0x7f5618135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180b48a0 con 0x7f561805d240 2022-08-01T23:02:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5618150bd0 con 0x7f561805c960 2022-08-01T23:02:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f561cc09700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5618150e70 con 0x7f561813f520 2022-08-01T23:02:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 0x7f5618136210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 0x7f5618136210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45624/0 (socket says 172.21.15.32:45624) 2022-08-01T23:02:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 -- 172.21.15.32:0/2004392784 learned_addr learned my addr 172.21.15.32:0/2004392784 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f56177fe700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561805c960 0x7f5618142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5617fff700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f561813f520 0x7f5618135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 -- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f561813f520 msgr2=0x7f5618135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f561813f520 0x7f5618135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 -- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561805c960 msgr2=0x7f5618142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561805c960 0x7f5618142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 -- 172.21.15.32:0/2004392784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560800b180 con 0x7f561805d240 2022-08-01T23:02:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5617fff700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f561813f520 0x7f5618135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5616ffd700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 0x7f5618136210 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f5618060810 tx=0x7f560c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.644+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f560c0070d0 con 0x7f561805d240 2022-08-01T23:02:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c007230 con 0x7f561805d240 2022-08-01T23:02:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56180b4b40 con 0x7f561805d240 2022-08-01T23:02:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f560c005610 con 0x7f561805d240 2022-08-01T23:02:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56180b6140 con 0x7f561805d240 2022-08-01T23:02:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618151030 con 0x7f561805d240 2022-08-01T23:02:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f560c010070 con 0x7f561805d240 2022-08-01T23:02:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f5614ff9700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053a70 0x7f55f8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f560c06b580 con 0x7f561805d240 2022-08-01T23:02:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.648+0000 7f56177fe700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053a70 0x7f55f8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.652+0000 7f56177fe700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053a70 0x7f55f8055f30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f560801a000 tx=0x7f5608014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.656+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f560c042340 con 0x7f561805d240 2022-08-01T23:02:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.816+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56180b1bc0 con 0x7f561805d240 2022-08-01T23:02:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.984+0000 7f5614ff9700 1 -- 172.21.15.32:0/2004392784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f560c045960 con 0x7f561805d240 2022-08-01T23:02:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053a70 msgr2=0x7f55f8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053a70 0x7f55f8055f30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f560801a000 tx=0x7f5608014000).stop 2022-08-01T23:02:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 msgr2=0x7f5618136210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 0x7f5618136210 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f5618060810 tx=0x7f560c007c00).stop 2022-08-01T23:02:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 shutdown_connections 2022-08-01T23:02:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f55f8053a70 0x7f55f8055f30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f561813f520 0x7f5618135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f561805d240 0x7f5618136210 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 --2- 172.21.15.32:0/2004392784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561805c960 0x7f5618142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 >> 172.21.15.32:0/2004392784 conn(0x7f5618060310 msgr2=0x7f56181455c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 shutdown_connections 2022-08-01T23:02:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:44.988+0000 7f561cc09700 1 -- 172.21.15.32:0/2004392784 wait complete. 2022-08-01T23:02:45.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.108+0000 7f87d5539700 1 Processor -- start 2022-08-01T23:02:45.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.108+0000 7f87d5539700 1 -- start start 2022-08-01T23:02:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87d00614b0 0x7f87d0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00ad750 0x7f87d00b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87d01405d0 0x7f87d0140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d00aee80 con 0x7f87d00614b0 2022-08-01T23:02:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d00b70b0 con 0x7f87d01405d0 2022-08-01T23:02:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00af0b0 con 0x7f87d00ad750 2022-08-01T23:02:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87d00614b0 0x7f87d0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87d00614b0 0x7f87d0061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45632/0 (socket says 172.21.15.32:45632) 2022-08-01T23:02:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 -- 172.21.15.32:0/2775142948 learned_addr learned my addr 172.21.15.32:0/2775142948 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/2775142948 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00ad750 0x7f87d00b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d4d38700 1 --2- 172.21.15.32:0/2775142948 >> v2:172.21.15.55:3300/0 conn(0x7f87d01405d0 0x7f87d0140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 -- 172.21.15.32:0/2775142948 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00ad750 msgr2=0x7f87d00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 --2- 172.21.15.32:0/2775142948 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00ad750 0x7f87d00b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 -- 172.21.15.32:0/2775142948 >> v2:172.21.15.55:3300/0 conn(0x7f87d01405d0 msgr2=0x7f87d0140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 --2- 172.21.15.32:0/2775142948 >> v2:172.21.15.55:3300/0 conn(0x7f87d01405d0 0x7f87d0140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 -- 172.21.15.32:0/2775142948 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00af350 con 0x7f87d00614b0 2022-08-01T23:02:45.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87cffff700 1 --2- 172.21.15.32:0/2775142948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00614b0 0x7f87d0061890 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f87c4002f70 tx=0x7f87c4006eb0).ready entity=mon.0 client_cookie=d8f583fcd9860d2b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87ceffd700 1 -- 172.21.15.32:0/2775142948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c400d570 con 0x7f87d00614b0 2022-08-01T23:02:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87ceffd700 1 -- 172.21.15.32:0/2775142948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4000e90 con 0x7f87d00614b0 2022-08-01T23:02:45.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87ceffd700 1 -- 172.21.15.32:0/2775142948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c400d900 con 0x7f87d00614b0 2022-08-01T23:02:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- 172.21.15.32:0/2775142948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00614b0 msgr2=0x7f87d0061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- 172.21.15.32:0/2775142948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00614b0 0x7f87d0061890 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f87c4002f70 tx=0x7f87c4006eb0).stop 2022-08-01T23:02:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- 172.21.15.32:0/2775142948 shutdown_connections 2022-08-01T23:02:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- 172.21.15.32:0/2775142948 >> v2:172.21.15.55:3300/0 conn(0x7f87d01405d0 0x7f87d0140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- 172.21.15.32:0/2775142948 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00ad750 0x7f87d00b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 --2- 172.21.15.32:0/2775142948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00614b0 0x7f87d0061890 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- 172.21.15.32:0/2775142948 >> 172.21.15.32:0/2775142948 conn(0x7f87d0066f70 msgr2=0x7f87d0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- 172.21.15.32:0/2775142948 shutdown_connections 2022-08-01T23:02:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.112+0000 7f87d5539700 1 -- 172.21.15.32:0/2775142948 wait complete. 2022-08-01T23:02:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 Processor -- start 2022-08-01T23:02:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 -- start start 2022-08-01T23:02:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00614b0 0x7f87d0145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 0x7f87d00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d01405d0 0x7f87d00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00b1a50 con 0x7f87d00ad750 2022-08-01T23:02:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0153890 con 0x7f87d01405d0 2022-08-01T23:02:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d5539700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d01535f0 con 0x7f87d00614b0 2022-08-01T23:02:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 0x7f87d00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 0x7f87d00afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45636/0 (socket says 172.21.15.32:45636) 2022-08-01T23:02:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 -- 172.21.15.32:0/1359829079 learned_addr learned my addr 172.21.15.32:0/1359829079 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d4d38700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d01405d0 0x7f87d00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cffff700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00614b0 0x7f87d0145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 -- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00614b0 msgr2=0x7f87d0145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00614b0 0x7f87d0145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 -- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d01405d0 msgr2=0x7f87d00affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d01405d0 0x7f87d00affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 -- 172.21.15.32:0/1359829079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c400b1e0 con 0x7f87d00ad750 2022-08-01T23:02:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87d4d38700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d01405d0 0x7f87d00affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 0x7f87d00afac0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f87c000c000 tx=0x7f87c0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c000c830 con 0x7f87d00ad750 2022-08-01T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.116+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c000c990 con 0x7f87d00ad750 2022-08-01T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d009efb0 con 0x7f87d00ad750 2022-08-01T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c000ccf0 con 0x7f87d00ad750 2022-08-01T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d0153310 con 0x7f87d00ad750 2022-08-01T23:02:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d00af5f0 con 0x7f87d00ad750 2022-08-01T23:02:45.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f87c001c460 con 0x7f87d00ad750 2022-08-01T23:02:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87cd7fa700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053950 0x7f87b0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f87c006c480 con 0x7f87d00ad750 2022-08-01T23:02:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87cffff700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053950 0x7f87b0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.120+0000 7f87cffff700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053950 0x7f87b0055e10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f87c401a000 tx=0x7f87c4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.124+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87c00431e0 con 0x7f87d00ad750 2022-08-01T23:02:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.284+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87d00574e0 con 0x7f87d00ad750 2022-08-01T23:02:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.284+0000 7f87cd7fa700 1 -- 172.21.15.32:0/1359829079 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f87c0046800 con 0x7f87d00ad750 2022-08-01T23:02:45.291 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:45.291 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:44.973242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6666":{"gid":6666,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4046550885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4046550885},{"type":"v1","addr":"172.21.15.32:6835","nonce":4046550885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053950 msgr2=0x7f87b0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053950 0x7f87b0055e10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f87c401a000 tx=0x7f87c4015000).stop 2022-08-01T23:02:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 msgr2=0x7f87d00afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 0x7f87d00afac0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f87c000c000 tx=0x7f87c0006df0).stop 2022-08-01T23:02:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 shutdown_connections 2022-08-01T23:02:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053950 0x7f87b0055e10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d01405d0 0x7f87d00affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d00ad750 0x7f87d00afac0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 --2- 172.21.15.32:0/1359829079 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d00614b0 0x7f87d0145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 >> 172.21.15.32:0/1359829079 conn(0x7f87d0066f70 msgr2=0x7f87d01469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 shutdown_connections 2022-08-01T23:02:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.288+0000 7f87d5539700 1 -- 172.21.15.32:0/1359829079 wait complete. 2022-08-01T23:02:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2022-08-01T23:02:45.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:45.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa287fff700 1 Processor -- start 2022-08-01T23:02:45.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa287fff700 1 -- start start 2022-08-01T23:02:45.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa287fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2880b01a0 0x7fa2880a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa287fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2880a7250 con 0x7fa2880b2710 2022-08-01T23:02:45.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa287fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2880a7480 con 0x7fa2880afd60 2022-08-01T23:02:45.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa287fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2880a7720 con 0x7fa2880b01a0 2022-08-01T23:02:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2880b01a0 0x7fa2880a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2880b01a0 0x7fa2880a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45644/0 (socket says 172.21.15.32:45644) 2022-08-01T23:02:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 -- 172.21.15.32:0/2668269391 learned_addr learned my addr 172.21.15.32:0/2668269391 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa286ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2880b2710 0x7fa28814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33186/0 (socket says 172.21.15.32:33186) 2022-08-01T23:02:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 -- 172.21.15.32:0/2668269391 >> v1:172.21.15.55:6789/0 conn(0x7fa2880afd60 legacy=0x7fa288151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 -- 172.21.15.32:0/2668269391 >> v1:172.21.15.32:6789/0 conn(0x7fa2880b2710 legacy=0x7fa28814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa285ffb700 1 -- 172.21.15.32:0/2668269391 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542130028 0 0) 0x7fa2880a7250 con 0x7fa2880b2710 2022-08-01T23:02:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 -- 172.21.15.32:0/2668269391 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2880a79c0 con 0x7fa2880b01a0 2022-08-01T23:02:45.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa2867fc700 1 --2- 172.21.15.32:0/2668269391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880b01a0 0x7fa2880a8540 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fa2780032f0 tx=0x7fa278009290).ready entity=mon.0 client_cookie=b3adc812dda053fd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa285ffb700 1 -- 172.21.15.32:0/2668269391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27800b8a0 con 0x7fa2880b01a0 2022-08-01T23:02:45.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa285ffb700 1 -- 172.21.15.32:0/2668269391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27800ba00 con 0x7fa2880b01a0 2022-08-01T23:02:45.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.404+0000 7fa285ffb700 1 -- 172.21.15.32:0/2668269391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27800bb70 con 0x7fa2880b01a0 2022-08-01T23:02:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- 172.21.15.32:0/2668269391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880b01a0 msgr2=0x7fa2880a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 --2- 172.21.15.32:0/2668269391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880b01a0 0x7fa2880a8540 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fa2780032f0 tx=0x7fa278009290).stop 2022-08-01T23:02:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- 172.21.15.32:0/2668269391 shutdown_connections 2022-08-01T23:02:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 --2- 172.21.15.32:0/2668269391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880b01a0 0x7fa2880a8540 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- 172.21.15.32:0/2668269391 >> 172.21.15.32:0/2668269391 conn(0x7fa288060310 msgr2=0x7fa2880b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- 172.21.15.32:0/2668269391 shutdown_connections 2022-08-01T23:02:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- 172.21.15.32:0/2668269391 wait complete. 2022-08-01T23:02:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 Processor -- start 2022-08-01T23:02:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- start start 2022-08-01T23:02:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 0x7fa28815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2880b01a0 0x7fa2880b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2880b2710 0x7fa2880a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880aa200 con 0x7fa2880afd60 2022-08-01T23:02:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa288142b60 con 0x7fa2880b01a0 2022-08-01T23:02:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa287fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880aa420 con 0x7fa2880b2710 2022-08-01T23:02:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 0x7fa28815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 0x7fa28815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45650/0 (socket says 172.21.15.32:45650) 2022-08-01T23:02:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 -- 172.21.15.32:0/3436967780 learned_addr learned my addr 172.21.15.32:0/3436967780 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa2867fc700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2880b01a0 0x7fa2880b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa2877fe700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2880b2710 0x7fa2880a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 -- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2880b2710 msgr2=0x7fa2880a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2880b2710 0x7fa2880a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 -- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2880b01a0 msgr2=0x7fa2880b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2880b01a0 0x7fa2880b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 -- 172.21.15.32:0/3436967780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa278009530 con 0x7fa2880afd60 2022-08-01T23:02:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa286ffd700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 0x7fa28815a5b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fa27c000ed0 tx=0x7fa27c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.408+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27c0070d0 con 0x7fa2880afd60 2022-08-01T23:02:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c007230 con 0x7fa2880afd60 2022-08-01T23:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2880aa6c0 con 0x7fa2880afd60 2022-08-01T23:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27c005550 con 0x7fa2880afd60 2022-08-01T23:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2880671d0 con 0x7fa2880afd60 2022-08-01T23:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2880574e0 con 0x7fa2880afd60 2022-08-01T23:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa27c010070 con 0x7fa2880afd60 2022-08-01T23:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa277fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa264053950 0x7fa264055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa27c06b770 con 0x7fa2880afd60 2022-08-01T23:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.412+0000 7fa2867fc700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa264053950 0x7fa264055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.416+0000 7fa2867fc700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa264053950 0x7fa264055e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fa278019000 tx=0x7fa278016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.420+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa27c042530 con 0x7fa2880afd60 2022-08-01T23:02:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.540+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa288069a70 con 0x7fa2880afd60 2022-08-01T23:02:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa277fff700 1 -- 172.21.15.32:0/3436967780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fa27c045b50 con 0x7fa2880afd60 2022-08-01T23:02:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:45.548 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:44.972186+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:02:42.386170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T23:02:43.649622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa264053950 msgr2=0x7fa264055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa264053950 0x7fa264055e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fa278019000 tx=0x7fa278016000).stop 2022-08-01T23:02:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 msgr2=0x7fa28815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 0x7fa28815a5b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fa27c000ed0 tx=0x7fa27c007c80).stop 2022-08-01T23:02:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 shutdown_connections 2022-08-01T23:02:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa264053950 0x7fa264055e10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2880b2710 0x7fa2880a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2880b01a0 0x7fa2880b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 --2- 172.21.15.32:0/3436967780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2880afd60 0x7fa28815a5b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 >> 172.21.15.32:0/3436967780 conn(0x7fa288060310 msgr2=0x7fa2881460c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 shutdown_connections 2022-08-01T23:02:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.544+0000 7fa287fff700 1 -- 172.21.15.32:0/3436967780 wait complete. 2022-08-01T23:02:45.561 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:02:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 Processor -- start 2022-08-01T23:02:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 -- start start 2022-08-01T23:02:45.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a40b6af0 0x7f97a412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a4144240 con 0x7f97a40b4550 2022-08-01T23:02:45.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a4143d00 con 0x7f97a40b6af0 2022-08-01T23:02:45.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a8beb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4143fa0 con 0x7f97a40b6710 2022-08-01T23:02:45.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a40b6af0 0x7f97a412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a40b6af0 0x7f97a412bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45662/0 (socket says 172.21.15.32:45662) 2022-08-01T23:02:45.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 -- 172.21.15.32:0/2098940378 learned_addr learned my addr 172.21.15.32:0/2098940378 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a259c700 1 --2- 172.21.15.32:0/2098940378 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 -- 172.21.15.32:0/2098940378 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 msgr2=0x7f97a40b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 --2- 172.21.15.32:0/2098940378 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a40b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 -- 172.21.15.32:0/2098940378 >> v1:172.21.15.55:6789/0 conn(0x7f97a40b4550 legacy=0x7f97a4061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 -- 172.21.15.32:0/2098940378 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4061780 con 0x7f97a40b6af0 2022-08-01T23:02:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a2d9d700 1 --2- 172.21.15.32:0/2098940378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b6af0 0x7f97a412bb90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f979c0032f0 tx=0x7f979c009290).ready entity=mon.0 client_cookie=c72a312ba95ed5db server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a159a700 1 -- 172.21.15.32:0/2098940378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c00b8a0 con 0x7f97a40b6af0 2022-08-01T23:02:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a159a700 1 -- 172.21.15.32:0/2098940378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00ba00 con 0x7f97a40b6af0 2022-08-01T23:02:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.668+0000 7f97a159a700 1 -- 172.21.15.32:0/2098940378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c00bb70 con 0x7f97a40b6af0 2022-08-01T23:02:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- 172.21.15.32:0/2098940378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b6af0 msgr2=0x7f97a412bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 --2- 172.21.15.32:0/2098940378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b6af0 0x7f97a412bb90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f979c0032f0 tx=0x7f979c009290).stop 2022-08-01T23:02:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- 172.21.15.32:0/2098940378 shutdown_connections 2022-08-01T23:02:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 --2- 172.21.15.32:0/2098940378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b6af0 0x7f97a412bb90 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 --2- 172.21.15.32:0/2098940378 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a40b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- 172.21.15.32:0/2098940378 >> 172.21.15.32:0/2098940378 conn(0x7f97a405c620 msgr2=0x7f97a405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- 172.21.15.32:0/2098940378 shutdown_connections 2022-08-01T23:02:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- 172.21.15.32:0/2098940378 wait complete. 2022-08-01T23:02:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 Processor -- start 2022-08-01T23:02:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- start start 2022-08-01T23:02:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 0x7f97a40b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a4153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a40b6af0 0x7f97a40b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41413e0 con 0x7f97a40b4550 2022-08-01T23:02:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40b07d0 con 0x7f97a40b6af0 2022-08-01T23:02:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a8beb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4152740 con 0x7f97a40b6710 2022-08-01T23:02:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 0x7f97a40b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 0x7f97a40b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45668/0 (socket says 172.21.15.32:45668) 2022-08-01T23:02:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a259c700 1 -- 172.21.15.32:0/639128879 learned_addr learned my addr 172.21.15.32:0/639128879 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a1d9b700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a4153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.672+0000 7f97a2d9d700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a40b6af0 0x7f97a40b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a259c700 1 -- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 msgr2=0x7f97a4153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a259c700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a4153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a259c700 1 -- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a40b6af0 msgr2=0x7f97a40b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a259c700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a40b6af0 0x7f97a40b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a259c700 1 -- 172.21.15.32:0/639128879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c009530 con 0x7f97a40b4550 2022-08-01T23:02:45.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a259c700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 0x7f97a40b01a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f978c00c000 tx=0x7f978c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978c010780 con 0x7f97a40b4550 2022-08-01T23:02:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c0108e0 con 0x7f97a40b4550 2022-08-01T23:02:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a41529e0 con 0x7f97a40b4550 2022-08-01T23:02:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f978c010c40 con 0x7f97a40b4550 2022-08-01T23:02:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a40afde0 con 0x7f97a40b4550 2022-08-01T23:02:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4061a20 con 0x7f97a40b4550 2022-08-01T23:02:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f978c01c460 con 0x7f97a40b4550 2022-08-01T23:02:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f979b7fe700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9788053b30 0x7f9788055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.676+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f978c06c500 con 0x7f97a40b4550 2022-08-01T23:02:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.680+0000 7f97a1d9b700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9788053b30 0x7f9788055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.680+0000 7f97a1d9b700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9788053b30 0x7f9788055ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f979400d000 tx=0x7f9794007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.684+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f978c043150 con 0x7f97a40b4550 2022-08-01T23:02:45.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.848+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f97a40574e0 con 0x7f97a40b4550 2022-08-01T23:02:45.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f979b7fe700 1 -- 172.21.15.32:0/639128879 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f978c046770 con 0x7f97a40b4550 2022-08-01T23:02:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9788053b30 msgr2=0x7f9788055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9788053b30 0x7f9788055ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f979400d000 tx=0x7f9794007b90).stop 2022-08-01T23:02:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 msgr2=0x7f97a40b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 0x7f97a40b01a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f978c00c000 tx=0x7f978c006df0).stop 2022-08-01T23:02:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 shutdown_connections 2022-08-01T23:02:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9788053b30 0x7f9788055ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a40b6af0 0x7f97a40b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a40b6710 0x7f97a4153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 --2- 172.21.15.32:0/639128879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a40b4550 0x7f97a40b01a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 >> 172.21.15.32:0/639128879 conn(0x7f97a405c620 msgr2=0x7f97a4142340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:45.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 shutdown_connections 2022-08-01T23:02:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.860+0000 7f97a8beb700 1 -- 172.21.15.32:0/639128879 wait complete. 2022-08-01T23:02:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:02:45.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:02:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 Processor -- start 2022-08-01T23:02:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 -- start start 2022-08-01T23:02:45.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51180b1bc0 0x7f5118066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f511813e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:45.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511805f8e0 con 0x7f5118066780 2022-08-01T23:02:45.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51180b1630 con 0x7f51180b1bc0 2022-08-01T23:02:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.988+0000 7f511c859700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511805fb10 con 0x7f5118140e40 2022-08-01T23:02:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51180b1bc0 0x7f5118066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51180b1bc0 0x7f5118066b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45674/0 (socket says 172.21.15.32:45674) 2022-08-01T23:02:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 -- 172.21.15.32:0/1091323916 learned_addr learned my addr 172.21.15.32:0/1091323916 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511759e700 1 --2- 172.21.15.32:0/1091323916 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f511813e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 -- 172.21.15.32:0/1091323916 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 msgr2=0x7f511813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 --2- 172.21.15.32:0/1091323916 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f511813e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 -- 172.21.15.32:0/1091323916 >> v1:172.21.15.55:6789/0 conn(0x7f5118066780 legacy=0x7f511805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 -- 172.21.15.32:0/1091323916 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180b18d0 con 0x7f51180b1bc0 2022-08-01T23:02:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511659c700 1 -- 172.21.15.32:0/1091323916 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749121263 0 0) 0x7f511805f8e0 con 0x7f5118066780 2022-08-01T23:02:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f510ffff700 1 --2- 172.21.15.32:0/1091323916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118066b60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f5100002f70 tx=0x7f5100006eb0).ready entity=mon.0 client_cookie=f533600d2b043868 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:46.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511659c700 1 -- 172.21.15.32:0/1091323916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510000d570 con 0x7f51180b1bc0 2022-08-01T23:02:46.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511659c700 1 -- 172.21.15.32:0/1091323916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5100000e90 con 0x7f51180b1bc0 2022-08-01T23:02:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511659c700 1 -- 172.21.15.32:0/1091323916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510000d8c0 con 0x7f51180b1bc0 2022-08-01T23:02:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 -- 172.21.15.32:0/1091323916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 msgr2=0x7f5118066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 --2- 172.21.15.32:0/1091323916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118066b60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f5100002f70 tx=0x7f5100006eb0).stop 2022-08-01T23:02:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 -- 172.21.15.32:0/1091323916 shutdown_connections 2022-08-01T23:02:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 --2- 172.21.15.32:0/1091323916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118066b60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 --2- 172.21.15.32:0/1091323916 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f511813e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 -- 172.21.15.32:0/1091323916 >> 172.21.15.32:0/1091323916 conn(0x7f5118063870 msgr2=0x7f5118063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 -- 172.21.15.32:0/1091323916 shutdown_connections 2022-08-01T23:02:46.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.992+0000 7f511c859700 1 -- 172.21.15.32:0/1091323916 wait complete. 2022-08-01T23:02:46.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 Processor -- start 2022-08-01T23:02:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 -- start start 2022-08-01T23:02:46.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5118066780 0x7f511813bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f51181396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181425d0 con 0x7f51180b1bc0 2022-08-01T23:02:46.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181544b0 con 0x7f5118066780 2022-08-01T23:02:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511c859700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51181427c0 con 0x7f5118140e40 2022-08-01T23:02:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45680/0 (socket says 172.21.15.32:45680) 2022-08-01T23:02:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 -- 172.21.15.32:0/2985643876 learned_addr learned my addr 172.21.15.32:0/2985643876 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511759e700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f51181396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f5116d9d700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5118066780 0x7f511813bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 -- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 msgr2=0x7f51181396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f51181396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 -- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5118066780 msgr2=0x7f511813bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5118066780 0x7f511813bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f510ffff700 1 -- 172.21.15.32:0/2985643876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510000b1e0 con 0x7f51180b1bc0 2022-08-01T23:02:46.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:45.996+0000 7f511759e700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f51181396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f510ffff700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118153fc0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f5100002f70 tx=0x7f5100015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51000068b0 con 0x7f51180b1bc0 2022-08-01T23:02:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000d8f0 con 0x7f51180b1bc0 2022-08-01T23:02:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5118142a60 con 0x7f51180b1bc0 2022-08-01T23:02:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5100011e20 con 0x7f51180b1bc0 2022-08-01T23:02:46.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5118142fa0 con 0x7f51180b1bc0 2022-08-01T23:02:46.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51180b12b0 con 0x7f51180b1bc0 2022-08-01T23:02:46.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f510001a040 con 0x7f51180b1bc0 2022-08-01T23:02:46.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5114d99700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc0539e0 0x7f50fc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f510006cee0 con 0x7f51180b1bc0 2022-08-01T23:02:46.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5116d9d700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc0539e0 0x7f50fc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.000+0000 7f5116d9d700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc0539e0 0x7f50fc055ea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f510800d000 tx=0x7f5108007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:46.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.004+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5100043c40 con 0x7f51180b1bc0 2022-08-01T23:02:46.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.172+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f51180574e0 con 0x7f51180b1bc0 2022-08-01T23:02:46.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f5114d99700 1 -- 172.21.15.32:0/2985643876 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5100023020 con 0x7f51180b1bc0 2022-08-01T23:02:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc0539e0 msgr2=0x7f50fc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc0539e0 0x7f50fc055ea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f510800d000 tx=0x7f5108007a20).stop 2022-08-01T23:02:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 msgr2=0x7f5118153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118153fc0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f5100002f70 tx=0x7f5100015000).stop 2022-08-01T23:02:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 shutdown_connections 2022-08-01T23:02:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc0539e0 0x7f50fc055ea0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5118140e40 0x7f51181396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51180b1bc0 0x7f5118153fc0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 --2- 172.21.15.32:0/2985643876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5118066780 0x7f511813bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 >> 172.21.15.32:0/2985643876 conn(0x7f5118063870 msgr2=0x7f511813e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 shutdown_connections 2022-08-01T23:02:46.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.180+0000 7f511c859700 1 -- 172.21.15.32:0/2985643876 wait complete. 2022-08-01T23:02:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:02:46.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 Processor -- start 2022-08-01T23:02:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 -- start start 2022-08-01T23:02:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6380b31d0 0x7fb638066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb6380a9da0 0x7fb638060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb638143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb638064a70 con 0x7fb6380b31d0 2022-08-01T23:02:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb638061f60 con 0x7fb6380a9da0 2022-08-01T23:02:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb637fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb638067030 con 0x7fb6381434b0 2022-08-01T23:02:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6380b31d0 0x7fb638066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6380b31d0 0x7fb638066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45688/0 (socket says 172.21.15.32:45688) 2022-08-01T23:02:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 -- 172.21.15.32:0/2636069449 learned_addr learned my addr 172.21.15.32:0/2636069449 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb6367fc700 1 --2- 172.21.15.32:0/2636069449 >> v2:172.21.15.55:3300/0 conn(0x7fb6380a9da0 0x7fb638060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb6377fe700 1 --2- 172.21.15.32:0/2636069449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb638143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 -- 172.21.15.32:0/2636069449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 msgr2=0x7fb638143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 --2- 172.21.15.32:0/2636069449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb638143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 -- 172.21.15.32:0/2636069449 >> v2:172.21.15.55:3300/0 conn(0x7fb6380a9da0 msgr2=0x7fb638060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 --2- 172.21.15.32:0/2636069449 >> v2:172.21.15.55:3300/0 conn(0x7fb6380a9da0 0x7fb638060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 -- 172.21.15.32:0/2636069449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380b35b0 con 0x7fb6380b31d0 2022-08-01T23:02:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb636ffd700 1 --2- 172.21.15.32:0/2636069449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380b31d0 0x7fb638066020 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fb620002f70 tx=0x7fb620006eb0).ready entity=mon.0 client_cookie=3841847cc9bd0023 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb635ffb700 1 -- 172.21.15.32:0/2636069449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62000d570 con 0x7fb6380b31d0 2022-08-01T23:02:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb635ffb700 1 -- 172.21.15.32:0/2636069449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620000e90 con 0x7fb6380b31d0 2022-08-01T23:02:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.292+0000 7fb635ffb700 1 -- 172.21.15.32:0/2636069449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62000d900 con 0x7fb6380b31d0 2022-08-01T23:02:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- 172.21.15.32:0/2636069449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380b31d0 msgr2=0x7fb638066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- 172.21.15.32:0/2636069449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380b31d0 0x7fb638066020 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fb620002f70 tx=0x7fb620006eb0).stop 2022-08-01T23:02:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- 172.21.15.32:0/2636069449 shutdown_connections 2022-08-01T23:02:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- 172.21.15.32:0/2636069449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb638143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- 172.21.15.32:0/2636069449 >> v2:172.21.15.55:3300/0 conn(0x7fb6380a9da0 0x7fb638060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- 172.21.15.32:0/2636069449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380b31d0 0x7fb638066020 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- 172.21.15.32:0/2636069449 >> 172.21.15.32:0/2636069449 conn(0x7fb638063870 msgr2=0x7fb638063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- 172.21.15.32:0/2636069449 shutdown_connections 2022-08-01T23:02:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- 172.21.15.32:0/2636069449 wait complete. 2022-08-01T23:02:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 Processor -- start 2022-08-01T23:02:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- start start 2022-08-01T23:02:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 0x7fb63813d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6380b31d0 0x7fb6380af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb6380afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63813e310 con 0x7fb6380a9da0 2022-08-01T23:02:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380b43f0 con 0x7fb6380b31d0 2022-08-01T23:02:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb637fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb638152060 con 0x7fb6381434b0 2022-08-01T23:02:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 0x7fb63813d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 0x7fb63813d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45692/0 (socket says 172.21.15.32:45692) 2022-08-01T23:02:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 -- 172.21.15.32:0/2304921575 learned_addr learned my addr 172.21.15.32:0/2304921575 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb6367fc700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6380b31d0 0x7fb6380af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb6377fe700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb6380afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 -- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 msgr2=0x7fb6380afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb6380afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 -- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6380b31d0 msgr2=0x7fb6380af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6380b31d0 0x7fb6380af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 -- 172.21.15.32:0/2304921575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62000b1e0 con 0x7fb6380a9da0 2022-08-01T23:02:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.296+0000 7fb636ffd700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 0x7fb63813d870 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fb620002f70 tx=0x7fb620015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62000dea0 con 0x7fb6380a9da0 2022-08-01T23:02:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620006b40 con 0x7fb6380a9da0 2022-08-01T23:02:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb620011d70 con 0x7fb6380a9da0 2022-08-01T23:02:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6380b0240 con 0x7fb6380a9da0 2022-08-01T23:02:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63813dd60 con 0x7fb6380a9da0 2022-08-01T23:02:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6380b45f0 con 0x7fb6380a9da0 2022-08-01T23:02:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb62001a040 con 0x7fb6380a9da0 2022-08-01T23:02:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb62ffff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb61c053980 0x7fb61c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb62006cd60 con 0x7fb6380a9da0 2022-08-01T23:02:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb6367fc700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb61c053980 0x7fb61c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.300+0000 7fb6367fc700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb61c053980 0x7fb61c055e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fb62800b000 tx=0x7fb6280091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.308+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb620043b60 con 0x7fb6380a9da0 2022-08-01T23:02:46.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.472+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb638066af0 con 0x7fb6380a9da0 2022-08-01T23:02:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.472+0000 7fb62ffff700 1 -- 172.21.15.32:0/2304921575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fb620023020 con 0x7fb6380a9da0 2022-08-01T23:02:46.477 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:46.477 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:44.973242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6666":{"gid":6666,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4046550885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4046550885},{"type":"v1","addr":"172.21.15.32:6835","nonce":4046550885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb61c053980 msgr2=0x7fb61c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb61c053980 0x7fb61c055e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fb62800b000 tx=0x7fb6280091d0).stop 2022-08-01T23:02:46.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 msgr2=0x7fb63813d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:46.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 0x7fb63813d870 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fb620002f70 tx=0x7fb620015000).stop 2022-08-01T23:02:46.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 shutdown_connections 2022-08-01T23:02:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb61c053980 0x7fb61c055e40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb6381434b0 0x7fb6380afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6380b31d0 0x7fb6380af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 --2- 172.21.15.32:0/2304921575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6380a9da0 0x7fb63813d870 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 >> 172.21.15.32:0/2304921575 conn(0x7fb638063870 msgr2=0x7fb638063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 shutdown_connections 2022-08-01T23:02:46.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:46.476+0000 7fb637fff700 1 -- 172.21.15.32:0/2304921575 wait complete. 2022-08-01T23:02:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2022-08-01T23:02:46.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:02:43.969500+0000', 'modified': '2022-08-01T23:02:44.973242+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6666}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6666': {'gid': 6666, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4046550885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4046550885}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4046550885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:02:46.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:02:46.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:02:46.918 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6666,"want_state":"up:active","state":"up:active","rank_uptime":2.9347652559999999,"mdsmap_epoch":39,"osdmap_epoch":87,"osdmap_epoch_barrier":87,"uptime":7.2703826239999998} 2022-08-01T23:02:46.919 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6666, "mdsmap_epoch": 39, "osdmap_epoch": 87, "osdmap_epoch_barrier": 87, "rank_uptime": 2.934765256, "state": "up:active", "uptime": 7.270382624, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:02:46.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.012+0000 7fefb126b700 1 Processor -- start 2022-08-01T23:02:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.012+0000 7fefb126b700 1 -- start start 2022-08-01T23:02:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefac148530 0x7fefac0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0b04e0 con 0x7fefac0a7250 2022-08-01T23:02:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefac0a9250 con 0x7fefac148530 2022-08-01T23:02:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0b0780 con 0x7fefac0b1f40 2022-08-01T23:02:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefac148530 0x7fefac0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefac148530 0x7fefac0b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:55486/0 (socket says 172.21.15.32:55486) 2022-08-01T23:02:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 -- 172.21.15.32:0/586694125 learned_addr learned my addr 172.21.15.32:0/586694125 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 -- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 msgr2=0x7fefac0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 --2- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 -- 172.21.15.32:0/586694125 >> v1:172.21.15.55:6789/0 conn(0x7fefac0a7250 legacy=0x7fefac1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 -- 172.21.15.32:0/586694125 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0b0a20 con 0x7fefac148530 2022-08-01T23:02:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefaa7fc700 1 --2- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac0b2320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fef9c0032f0 tx=0x7fef9c009290).ready entity=mon.1 client_cookie=f5b877dbecac07c7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefa9ffb700 1 -- 172.21.15.32:0/586694125 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef9c00b8a0 con 0x7fefac148530 2022-08-01T23:02:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefa9ffb700 1 -- 172.21.15.32:0/586694125 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00ba00 con 0x7fefac148530 2022-08-01T23:02:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefa9ffb700 1 -- 172.21.15.32:0/586694125 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef9c00bb70 con 0x7fefac148530 2022-08-01T23:02:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 msgr2=0x7fefac0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 --2- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac0b2320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fef9c0032f0 tx=0x7fef9c009290).stop 2022-08-01T23:02:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- 172.21.15.32:0/586694125 shutdown_connections 2022-08-01T23:02:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 --2- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac0b2320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 --2- 172.21.15.32:0/586694125 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- 172.21.15.32:0/586694125 >> 172.21.15.32:0/586694125 conn(0x7fefac05f840 msgr2=0x7fefac05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- 172.21.15.32:0/586694125 shutdown_connections 2022-08-01T23:02:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.016+0000 7fefb126b700 1 -- 172.21.15.32:0/586694125 wait complete. 2022-08-01T23:02:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 Processor -- start 2022-08-01T23:02:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 -- start start 2022-08-01T23:02:47.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 0x7fefac148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac149de0 con 0x7fefac0a7250 2022-08-01T23:02:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac14a5f0 con 0x7fefac148530 2022-08-01T23:02:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefb126b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0ae7e0 con 0x7fefac0b1f40 2022-08-01T23:02:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.020+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 0x7fefac148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 0x7fefac148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45706/0 (socket says 172.21.15.32:45706) 2022-08-01T23:02:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 -- 172.21.15.32:0/550240723 learned_addr learned my addr 172.21.15.32:0/550240723 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefab7fe700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaa7fc700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 -- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 msgr2=0x7fefac141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 -- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 msgr2=0x7fefac141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac141b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 -- 172.21.15.32:0/550240723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c009530 con 0x7fefac0a7250 2022-08-01T23:02:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaa7fc700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefaaffd700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 0x7fefac148fd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fefac14a540 tx=0x7fef94006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef94010780 con 0x7fefac0a7250 2022-08-01T23:02:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef940108e0 con 0x7fefac0a7250 2022-08-01T23:02:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac0af5d0 con 0x7fefac0a7250 2022-08-01T23:02:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef94010c40 con 0x7fefac0a7250 2022-08-01T23:02:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac0ae220 con 0x7fefac0a7250 2022-08-01T23:02:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.024+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac0b0cc0 con 0x7fefac0a7250 2022-08-01T23:02:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.028+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef9401c460 con 0x7fefac0a7250 2022-08-01T23:02:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.028+0000 7fefa3fff700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef90053950 0x7fef90055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.028+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fef9406c4d0 con 0x7fefac0a7250 2022-08-01T23:02:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.028+0000 7fefaa7fc700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef90053950 0x7fef90055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.028+0000 7fefaa7fc700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef90053950 0x7fef90055e10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fef9c019000 tx=0x7fef9c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.036+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef94043230 con 0x7fefac0a7250 2022-08-01T23:02:47.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.160+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefac0574e0 con 0x7fefac0a7250 2022-08-01T23:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.160+0000 7fefa3fff700 1 -- 172.21.15.32:0/550240723 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fef94046850 con 0x7fefac0a7250 2022-08-01T23:02:47.167 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:47.168 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:44.972186+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:02:42.386170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T23:02:43.649622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef90053950 msgr2=0x7fef90055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef90053950 0x7fef90055e10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fef9c019000 tx=0x7fef9c016000).stop 2022-08-01T23:02:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 msgr2=0x7fefac148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 0x7fefac148fd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fefac14a540 tx=0x7fef94006df0).stop 2022-08-01T23:02:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 shutdown_connections 2022-08-01T23:02:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fef90053950 0x7fef90055e10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefac148530 0x7fefac141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fefac0b1f40 0x7fefac141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 --2- 172.21.15.32:0/550240723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefac0a7250 0x7fefac148fd0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 >> 172.21.15.32:0/550240723 conn(0x7fefac05f840 msgr2=0x7fefac0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 shutdown_connections 2022-08-01T23:02:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.164+0000 7fefb126b700 1 -- 172.21.15.32:0/550240723 wait complete. 2022-08-01T23:02:47.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:02:42.386170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:02:43.649622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:02:47.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:02:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 Processor -- start 2022-08-01T23:02:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 -- start start 2022-08-01T23:02:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe9ac0ad750 0x7fe9ac0b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac0aee80 con 0x7fe9ac1405d0 2022-08-01T23:02:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b70b0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9abfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0af0b0 con 0x7fe9ac0614b0 2022-08-01T23:02:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9aa7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe9ac0ad750 0x7fe9ac0b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe9ac0ad750 0x7fe9ac0b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:55500/0 (socket says 172.21.15.32:55500) 2022-08-01T23:02:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.288+0000 7fe9ab7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe9ac1405d0 0x7fe9ac1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:54166/0 (socket says 172.21.15.32:54166) 2022-08-01T23:02:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4073649807 learned_addr learned my addr 172.21.15.32:0/4073649807 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 msgr2=0x7fe9ac061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 --2- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4073649807 >> v1:172.21.15.55:6789/0 conn(0x7fe9ac1405d0 legacy=0x7fe9ac1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4073649807 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0af350 con 0x7fe9ac0ad750 2022-08-01T23:02:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9a9ffb700 1 -- 172.21.15.32:0/4073649807 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115243806 0 0) 0x7fe9ac0aee80 con 0x7fe9ac1405d0 2022-08-01T23:02:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9aa7fc700 1 --2- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b9ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fe99c002f70 tx=0x7fe99c006eb0).ready entity=mon.1 client_cookie=a04da58212260f33 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9a9ffb700 1 -- 172.21.15.32:0/4073649807 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe99c00d570 con 0x7fe9ac0ad750 2022-08-01T23:02:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9a9ffb700 1 -- 172.21.15.32:0/4073649807 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c000e90 con 0x7fe9ac0ad750 2022-08-01T23:02:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9a9ffb700 1 -- 172.21.15.32:0/4073649807 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe99c00d8c0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 -- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac0ad750 msgr2=0x7fe9ac0b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b9ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fe99c002f70 tx=0x7fe99c006eb0).stop 2022-08-01T23:02:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 -- 172.21.15.32:0/4073649807 shutdown_connections 2022-08-01T23:02:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b9ff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4073649807 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 -- 172.21.15.32:0/4073649807 >> 172.21.15.32:0/4073649807 conn(0x7fe9ac066f70 msgr2=0x7fe9ac067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 -- 172.21.15.32:0/4073649807 shutdown_connections 2022-08-01T23:02:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 -- 172.21.15.32:0/4073649807 wait complete. 2022-08-01T23:02:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 Processor -- start 2022-08-01T23:02:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.292+0000 7fe9abfff700 1 -- start start 2022-08-01T23:02:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac1422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac1405d0 0x7fe9ac1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac144eb0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac145150 con 0x7fe9ac1405d0 2022-08-01T23:02:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b0af0 con 0x7fe9ac0614b0 2022-08-01T23:02:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45720/0 (socket says 172.21.15.32:45720) 2022-08-01T23:02:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4043985194 learned_addr learned my addr 172.21.15.32:0/4043985194 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9ab7fe700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac1405d0 0x7fe9ac1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aaffd700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac1422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 msgr2=0x7fe9ac1422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac1422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac1405d0 msgr2=0x7fe9ac1526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac1405d0 0x7fe9ac1526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 -- 172.21.15.32:0/4043985194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c00b1e0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9aa7fc700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b0f90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe99c002f70 tx=0x7fe99c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe99c00d8f0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c011c30 con 0x7fe9ac0ad750 2022-08-01T23:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe99c01e460 con 0x7fe9ac0ad750 2022-08-01T23:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac144bb0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac1457e0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.296+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac0af5f0 con 0x7fe9ac0ad750 2022-08-01T23:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.300+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe99c01a040 con 0x7fe9ac0ad750 2022-08-01T23:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.300+0000 7fe9a3fff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe9900539e0 0x7fe990055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.300+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe99c06d020 con 0x7fe9ac0ad750 2022-08-01T23:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.300+0000 7fe9aaffd700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe9900539e0 0x7fe990055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.300+0000 7fe9aaffd700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe9900539e0 0x7fe990055ea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fe99400b000 tx=0x7fe994006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.304+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe99c043d40 con 0x7fe9ac0ad750 2022-08-01T23:02:47.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe9ac0b6c70 con 0x7fe9ac0ad750 2022-08-01T23:02:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9a3fff700 1 -- 172.21.15.32:0/4043985194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v87) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe99c023020 con 0x7fe9ac0ad750 2022-08-01T23:02:47.437 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:02:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe9900539e0 msgr2=0x7fe990055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe9900539e0 0x7fe990055ea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fe99400b000 tx=0x7fe994006df0).stop 2022-08-01T23:02:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 msgr2=0x7fe9ac0b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b0f90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe99c002f70 tx=0x7fe99c015000).stop 2022-08-01T23:02:47.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 shutdown_connections 2022-08-01T23:02:47.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe9900539e0 0x7fe990055ea0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9ac1405d0 0x7fe9ac1526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9ac0ad750 0x7fe9ac0b0f90 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 --2- 172.21.15.32:0/4043985194 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe9ac0614b0 0x7fe9ac1422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.432+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 >> 172.21.15.32:0/4043985194 conn(0x7fe9ac066f70 msgr2=0x7fe9ac146880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.436+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 shutdown_connections 2022-08-01T23:02:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.436+0000 7fe9abfff700 1 -- 172.21.15.32:0/4043985194 wait complete. 2022-08-01T23:02:47.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:02:47.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.556+0000 7fb052d33700 1 Processor -- start 2022-08-01T23:02:47.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.556+0000 7fb052d33700 1 -- start start 2022-08-01T23:02:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb04c14b040 0x7fb04c14b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c0b2c20 con 0x7fb04c0b3070 2022-08-01T23:02:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c061830 con 0x7fb04c14b040 2022-08-01T23:02:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c065c10 con 0x7fb04c0b7840 2022-08-01T23:02:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb04c14b040 0x7fb04c14b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb051530700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb04c0b3070 0x7fb04c061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33270/0 (socket says 172.21.15.32:33270) 2022-08-01T23:02:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb051530700 1 -- 172.21.15.32:0/2864702699 learned_addr learned my addr 172.21.15.32:0/2864702699 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb04c14b040 0x7fb04c14b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45732/0 (socket says 172.21.15.32:45732) 2022-08-01T23:02:47.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb051d31700 1 --2- 172.21.15.32:0/2864702699 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 -- 172.21.15.32:0/2864702699 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 msgr2=0x7fb04c065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 --2- 172.21.15.32:0/2864702699 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 -- 172.21.15.32:0/2864702699 >> v1:172.21.15.32:6789/0 conn(0x7fb04c0b3070 legacy=0x7fb04c061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 -- 172.21.15.32:0/2864702699 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0b7c20 con 0x7fb04c14b040 2022-08-01T23:02:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb050d2f700 1 -- 172.21.15.32:0/2864702699 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611236781 0 0) 0x7fb04c0b2c20 con 0x7fb04c0b3070 2022-08-01T23:02:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052532700 1 --2- 172.21.15.32:0/2864702699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c14b4f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fb044002f70 tx=0x7fb044006df0).ready entity=mon.0 client_cookie=d0d78a1a29fca039 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb050d2f700 1 -- 172.21.15.32:0/2864702699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04400d570 con 0x7fb04c14b040 2022-08-01T23:02:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb050d2f700 1 -- 172.21.15.32:0/2864702699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044000e90 con 0x7fb04c14b040 2022-08-01T23:02:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb050d2f700 1 -- 172.21.15.32:0/2864702699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04400d8c0 con 0x7fb04c14b040 2022-08-01T23:02:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- 172.21.15.32:0/2864702699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 msgr2=0x7fb04c14b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 --2- 172.21.15.32:0/2864702699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c14b4f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fb044002f70 tx=0x7fb044006df0).stop 2022-08-01T23:02:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- 172.21.15.32:0/2864702699 shutdown_connections 2022-08-01T23:02:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 --2- 172.21.15.32:0/2864702699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c14b4f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 --2- 172.21.15.32:0/2864702699 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c065720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- 172.21.15.32:0/2864702699 >> 172.21.15.32:0/2864702699 conn(0x7fb04c05f840 msgr2=0x7fb04c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- 172.21.15.32:0/2864702699 shutdown_connections 2022-08-01T23:02:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.560+0000 7fb052d33700 1 -- 172.21.15.32:0/2864702699 wait complete. 2022-08-01T23:02:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 Processor -- start 2022-08-01T23:02:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 -- start start 2022-08-01T23:02:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb04c0b3070 0x7fb04c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c0b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c09f0f0 con 0x7fb04c14b040 2022-08-01T23:02:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0afc70 con 0x7fb04c0b3070 2022-08-01T23:02:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c14d810 con 0x7fb04c0b7840 2022-08-01T23:02:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45738/0 (socket says 172.21.15.32:45738) 2022-08-01T23:02:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 -- 172.21.15.32:0/724226743 learned_addr learned my addr 172.21.15.32:0/724226743 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb051530700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c0b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb051d31700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb04c0b3070 0x7fb04c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 -- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 msgr2=0x7fb04c0b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c0b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 -- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb04c0b3070 msgr2=0x7fb04c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb04c0b3070 0x7fb04c14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 -- 172.21.15.32:0/724226743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04400b180 con 0x7fb04c14b040 2022-08-01T23:02:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052532700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c0b0600 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fb044002f70 tx=0x7fb044014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04401e400 con 0x7fb04c14b040 2022-08-01T23:02:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04401e560 con 0x7fb04c14b040 2022-08-01T23:02:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0a2360 con 0x7fb04c14b040 2022-08-01T23:02:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04401e8c0 con 0x7fb04c14b040 2022-08-01T23:02:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.564+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c0b1890 con 0x7fb04c14b040 2022-08-01T23:02:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.568+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c09f260 con 0x7fb04c14b040 2022-08-01T23:02:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.568+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb044025030 con 0x7fb04c14b040 2022-08-01T23:02:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.568+0000 7fb03effd700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb02c053b30 0x7fb02c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.568+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb04406cf10 con 0x7fb04c14b040 2022-08-01T23:02:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.568+0000 7fb051d31700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb02c053b30 0x7fb02c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.568+0000 7fb051d31700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb02c053b30 0x7fb02c055ff0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fb048006000 tx=0x7fb048008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.576+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb04c09f260 con 0x7fb04c14b040 2022-08-01T23:02:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.700+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb04c09f230 con 0x7fb04c14b040 2022-08-01T23:02:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.700+0000 7fb03effd700 1 -- 172.21.15.32:0/724226743 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb044023020 con 0x7fb04c14b040 2022-08-01T23:02:47.706 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb02c053b30 msgr2=0x7fb02c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb02c053b30 0x7fb02c055ff0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fb048006000 tx=0x7fb048008040).stop 2022-08-01T23:02:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 msgr2=0x7fb04c0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c0b0600 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fb044002f70 tx=0x7fb044014000).stop 2022-08-01T23:02:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 shutdown_connections 2022-08-01T23:02:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb02c053b30 0x7fb02c055ff0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04c14b040 0x7fb04c0b0600 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb04c0b7840 0x7fb04c0b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 --2- 172.21.15.32:0/724226743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb04c0b3070 0x7fb04c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 >> 172.21.15.32:0/724226743 conn(0x7fb04c05f840 msgr2=0x7fb04c14c800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 shutdown_connections 2022-08-01T23:02:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.704+0000 7fb052d33700 1 -- 172.21.15.32:0/724226743 wait complete. 2022-08-01T23:02:47.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:02:47.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe7432ab700 1 Processor -- start 2022-08-01T23:02:47.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe7432ab700 1 -- start start 2022-08-01T23:02:47.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe7432ab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe73c0b0140 0x7fe73c0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe7432ab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0ae650 con 0x7fe73c0ad1c0 2022-08-01T23:02:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe7432ab700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0b1b90 con 0x7fe73c0b22b0 2022-08-01T23:02:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe7432ab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0ae880 con 0x7fe73c0b0140 2022-08-01T23:02:47.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe741848700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe73c0b0140 0x7fe73c0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe740846700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe73c0ad1c0 0x7fe73c0b0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33284/0 (socket says 172.21.15.32:33284) 2022-08-01T23:02:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe740846700 1 -- 172.21.15.32:0/1625925524 learned_addr learned my addr 172.21.15.32:0/1625925524 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe733fff700 1 -- 172.21.15.32:0/1625925524 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868935178 0 0) 0x7fe73c0ae650 con 0x7fe73c0ad1c0 2022-08-01T23:02:47.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe733fff700 1 -- 172.21.15.32:0/1625925524 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe720003260 con 0x7fe73c0ad1c0 2022-08-01T23:02:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe733fff700 1 -- 172.21.15.32:0/1625925524 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021004950 0 0) 0x7fe720003260 con 0x7fe73c0ad1c0 2022-08-01T23:02:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe741848700 1 -- 172.21.15.32:0/1625925524 >> v1:172.21.15.55:6789/0 conn(0x7fe73c0b22b0 legacy=0x7fe73c143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe741848700 1 -- 172.21.15.32:0/1625925524 >> v1:172.21.15.32:6789/0 conn(0x7fe73c0ad1c0 legacy=0x7fe73c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe741848700 1 -- 172.21.15.32:0/1625925524 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0aeb20 con 0x7fe73c0b0140 2022-08-01T23:02:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.824+0000 7fe741848700 1 --2- 172.21.15.32:0/1625925524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b0140 0x7fe73c0b16a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fe738001b80 tx=0x7fe734006df0).ready entity=mon.0 client_cookie=c3e5bf47314df8f4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe733fff700 1 -- 172.21.15.32:0/1625925524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7380019e0 con 0x7fe73c0b0140 2022-08-01T23:02:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe733fff700 1 -- 172.21.15.32:0/1625925524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe734000e90 con 0x7fe73c0b0140 2022-08-01T23:02:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe733fff700 1 -- 172.21.15.32:0/1625925524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73400d4f0 con 0x7fe73c0b0140 2022-08-01T23:02:47.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- 172.21.15.32:0/1625925524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b0140 msgr2=0x7fe73c0b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 --2- 172.21.15.32:0/1625925524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b0140 0x7fe73c0b16a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fe738001b80 tx=0x7fe734006df0).stop 2022-08-01T23:02:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- 172.21.15.32:0/1625925524 shutdown_connections 2022-08-01T23:02:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 --2- 172.21.15.32:0/1625925524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b0140 0x7fe73c0b16a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- 172.21.15.32:0/1625925524 >> 172.21.15.32:0/1625925524 conn(0x7fe73c05f840 msgr2=0x7fe73c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- 172.21.15.32:0/1625925524 shutdown_connections 2022-08-01T23:02:47.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- 172.21.15.32:0/1625925524 wait complete. 2022-08-01T23:02:47.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 Processor -- start 2022-08-01T23:02:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- start start 2022-08-01T23:02:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73c0ad1c0 0x7fe73c09ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe73c0b0140 0x7fe73c13c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 0x7fe73c13cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0a18a0 con 0x7fe73c0b22b0 2022-08-01T23:02:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c155980 con 0x7fe73c0ad1c0 2022-08-01T23:02:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.828+0000 7fe7432ab700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c155c20 con 0x7fe73c0b0140 2022-08-01T23:02:47.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 0x7fe73c13cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 0x7fe73c13cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45752/0 (socket says 172.21.15.32:45752) 2022-08-01T23:02:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 -- 172.21.15.32:0/3422829461 learned_addr learned my addr 172.21.15.32:0/3422829461 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741047700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73c0ad1c0 0x7fe73c09ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe740846700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe73c0b0140 0x7fe73c13c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 -- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe73c0b0140 msgr2=0x7fe73c13c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe73c0b0140 0x7fe73c13c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 -- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73c0ad1c0 msgr2=0x7fe73c09ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73c0ad1c0 0x7fe73c09ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 -- 172.21.15.32:0/3422829461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73400b180 con 0x7fe73c0b22b0 2022-08-01T23:02:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741047700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73c0ad1c0 0x7fe73c09ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe741848700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 0x7fe73c13cd00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fe738001b80 tx=0x7fe734014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe734016030 con 0x7fe73c0b22b0 2022-08-01T23:02:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe734013cd0 con 0x7fe73c0b22b0 2022-08-01T23:02:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c155ec0 con 0x7fe73c0b22b0 2022-08-01T23:02:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73400d4f0 con 0x7fe73c0b22b0 2022-08-01T23:02:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c0a3110 con 0x7fe73c0b22b0 2022-08-01T23:02:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c0574e0 con 0x7fe73c0b22b0 2022-08-01T23:02:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe734024020 con 0x7fe73c0b22b0 2022-08-01T23:02:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7327fc700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe724053b00 0x7fe724055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.832+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe734017070 con 0x7fe73c0b22b0 2022-08-01T23:02:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.836+0000 7fe741047700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe724053b00 0x7fe724055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.836+0000 7fe741047700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe724053b00 0x7fe724055fc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fe72c00b000 tx=0x7fe72c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.840+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe734043c90 con 0x7fe73c0b22b0 2022-08-01T23:02:47.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.967+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe73c0a3110 con 0x7fe73c0b22b0 2022-08-01T23:02:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.967+0000 7fe7327fc700 1 -- 172.21.15.32:0/3422829461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe73c0a3110 con 0x7fe73c0b22b0 2022-08-01T23:02:47.974 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:47.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe724053b00 msgr2=0x7fe724055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe724053b00 0x7fe724055fc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fe72c00b000 tx=0x7fe72c009040).stop 2022-08-01T23:02:47.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 msgr2=0x7fe73c13cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:47.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 0x7fe73c13cd00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fe738001b80 tx=0x7fe734014000).stop 2022-08-01T23:02:47.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 shutdown_connections 2022-08-01T23:02:47.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe724053b00 0x7fe724055fc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73c0b22b0 0x7fe73c13cd00 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe73c0b0140 0x7fe73c13c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 --2- 172.21.15.32:0/3422829461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73c0ad1c0 0x7fe73c09ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 >> 172.21.15.32:0/3422829461 conn(0x7fe73c05f840 msgr2=0x7fe73c140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 shutdown_connections 2022-08-01T23:02:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:47.971+0000 7fe7432ab700 1 -- 172.21.15.32:0/3422829461 wait complete. 2022-08-01T23:02:47.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:02:48.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 Processor -- start 2022-08-01T23:02:48.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 -- start start 2022-08-01T23:02:48.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94100b7840 0x7f9410060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f94100610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100671e0 con 0x7f9410146720 2022-08-01T23:02:48.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9410065540 con 0x7f94100b7840 2022-08-01T23:02:48.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100b7c20 con 0x7f94100b2930 2022-08-01T23:02:48.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9410146720 0x7f9410146b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33300/0 (socket says 172.21.15.32:33300) 2022-08-01T23:02:48.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940ed9d700 1 -- 172.21.15.32:0/1932928841 learned_addr learned my addr 172.21.15.32:0/1932928841 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:48.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940dd9b700 1 --2- 172.21.15.32:0/1932928841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f94100610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940e59c700 1 --2- 172.21.15.32:0/1932928841 >> v2:172.21.15.55:3300/0 conn(0x7f94100b7840 0x7f9410060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940d59a700 1 -- 172.21.15.32:0/1932928841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878974420 0 0) 0x7f94100671e0 con 0x7f9410146720 2022-08-01T23:02:48.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940d59a700 1 -- 172.21.15.32:0/1932928841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f4003260 con 0x7f9410146720 2022-08-01T23:02:48.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940d59a700 1 -- 172.21.15.32:0/1932928841 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1506880256 0 0) 0x7f93f4003260 con 0x7f9410146720 2022-08-01T23:02:48.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940dd9b700 1 -- 172.21.15.32:0/1932928841 >> v2:172.21.15.55:3300/0 conn(0x7f94100b7840 msgr2=0x7f9410060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:48.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940dd9b700 1 --2- 172.21.15.32:0/1932928841 >> v2:172.21.15.55:3300/0 conn(0x7f94100b7840 0x7f9410060be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940dd9b700 1 -- 172.21.15.32:0/1932928841 >> v1:172.21.15.32:6789/0 conn(0x7f9410146720 legacy=0x7f9410146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.095+0000 7f940dd9b700 1 -- 172.21.15.32:0/1932928841 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410066ce0 con 0x7f94100b2930 2022-08-01T23:02:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940dd9b700 1 --2- 172.21.15.32:0/1932928841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f94100610d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f9408001b80 tx=0x7f9400006df0).ready entity=mon.2 client_cookie=19f7d65154679d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940d59a700 1 -- 172.21.15.32:0/1932928841 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94080019e0 con 0x7f94100b2930 2022-08-01T23:02:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940d59a700 1 -- 172.21.15.32:0/1932928841 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400000e90 con 0x7f94100b2930 2022-08-01T23:02:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940d59a700 1 -- 172.21.15.32:0/1932928841 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9400012460 con 0x7f94100b2930 2022-08-01T23:02:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- 172.21.15.32:0/1932928841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 msgr2=0x7f94100610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 --2- 172.21.15.32:0/1932928841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f94100610d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f9408001b80 tx=0x7f9400006df0).stop 2022-08-01T23:02:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- 172.21.15.32:0/1932928841 shutdown_connections 2022-08-01T23:02:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 --2- 172.21.15.32:0/1932928841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f94100610d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 --2- 172.21.15.32:0/1932928841 >> v2:172.21.15.55:3300/0 conn(0x7f94100b7840 0x7f9410060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- 172.21.15.32:0/1932928841 >> 172.21.15.32:0/1932928841 conn(0x7f941005f840 msgr2=0x7f941005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- 172.21.15.32:0/1932928841 shutdown_connections 2022-08-01T23:02:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- 172.21.15.32:0/1932928841 wait complete. 2022-08-01T23:02:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 Processor -- start 2022-08-01T23:02:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- start start 2022-08-01T23:02:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f9410148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 0x7f9410152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410146720 0x7f941014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941013faf0 con 0x7f94100b7840 2022-08-01T23:02:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410140ba0 con 0x7f9410146720 2022-08-01T23:02:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.099+0000 7f940f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9410140dd0 con 0x7f94100b2930 2022-08-01T23:02:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 0x7f9410152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 0x7f9410152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45764/0 (socket says 172.21.15.32:45764) 2022-08-01T23:02:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 -- 172.21.15.32:0/2373834105 learned_addr learned my addr 172.21.15.32:0/2373834105 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940ed9d700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410146720 0x7f941014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940e59c700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f9410148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 -- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 msgr2=0x7f9410148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f9410148de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 -- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410146720 msgr2=0x7f941014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410146720 0x7f941014a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 -- 172.21.15.32:0/2373834105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940000b180 con 0x7f94100b7840 2022-08-01T23:02:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940dd9b700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 0x7f9410152340 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f9408001b80 tx=0x7f9400014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9400016070 con 0x7f94100b7840 2022-08-01T23:02:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940000bec0 con 0x7f94100b7840 2022-08-01T23:02:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f940000dd40 con 0x7f94100b7840 2022-08-01T23:02:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9410141070 con 0x7f94100b7840 2022-08-01T23:02:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941014a6e0 con 0x7f94100b7840 2022-08-01T23:02:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100574e0 con 0x7f94100b7840 2022-08-01T23:02:48.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.103+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9400025030 con 0x7f94100b7840 2022-08-01T23:02:48.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.107+0000 7f93ff7fe700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93ec053aa0 0x7f93ec055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.107+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f940006cfb0 con 0x7f94100b7840 2022-08-01T23:02:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.107+0000 7f940e59c700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93ec053aa0 0x7f93ec055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.107+0000 7f940e59c700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93ec053aa0 0x7f93ec055f60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f9404006000 tx=0x7f9404006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.111+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9400043c60 con 0x7f94100b7840 2022-08-01T23:02:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.235+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9410060330 con 0x7f94100b7840 2022-08-01T23:02:48.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.235+0000 7f93ff7fe700 1 -- 172.21.15.32:0/2373834105 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9400023020 con 0x7f94100b7840 2022-08-01T23:02:48.244 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93ec053aa0 msgr2=0x7f93ec055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93ec053aa0 0x7f93ec055f60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f9404006000 tx=0x7f9404006d90).stop 2022-08-01T23:02:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 msgr2=0x7f9410152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 0x7f9410152340 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f9408001b80 tx=0x7f9400014000).stop 2022-08-01T23:02:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 shutdown_connections 2022-08-01T23:02:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93ec053aa0 0x7f93ec055f60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410146720 0x7f941014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94100b7840 0x7f9410152340 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 --2- 172.21.15.32:0/2373834105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f94100b2930 0x7f9410148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 >> 172.21.15.32:0/2373834105 conn(0x7f941005f840 msgr2=0x7f94100b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 shutdown_connections 2022-08-01T23:02:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.239+0000 7f940f59e700 1 -- 172.21.15.32:0/2373834105 wait complete. 2022-08-01T23:02:48.257 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:02:48.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.351+0000 7fa3e359b700 1 Processor -- start 2022-08-01T23:02:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.351+0000 7fa3e359b700 1 -- start start 2022-08-01T23:02:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa3dc0ad750 0x7fa3dc061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc1405d0 0x7fa3dc140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0aee80 con 0x7fa3dc0614b0 2022-08-01T23:02:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b70b0 con 0x7fa3dc0ad750 2022-08-01T23:02:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0af0b0 con 0x7fa3dc1405d0 2022-08-01T23:02:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e1d98700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa3dc0ad750 0x7fa3dc061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2d9a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc1405d0 0x7fa3dc140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2599700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa3dc0614b0 0x7fa3dc0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:54218/0 (socket says 172.21.15.32:54218) 2022-08-01T23:02:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2599700 1 -- 172.21.15.32:0/3998211045 learned_addr learned my addr 172.21.15.32:0/3998211045 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e1597700 1 -- 172.21.15.32:0/3998211045 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181580128 0 0) 0x7fa3dc0aee80 con 0x7fa3dc0614b0 2022-08-01T23:02:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e1597700 1 -- 172.21.15.32:0/3998211045 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c4003260 con 0x7fa3dc0614b0 2022-08-01T23:02:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2d9a700 1 -- 172.21.15.32:0/3998211045 >> v2:172.21.15.55:3300/0 conn(0x7fa3dc0ad750 msgr2=0x7fa3dc061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2d9a700 1 --2- 172.21.15.32:0/3998211045 >> v2:172.21.15.55:3300/0 conn(0x7fa3dc0ad750 0x7fa3dc061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2d9a700 1 -- 172.21.15.32:0/3998211045 >> v1:172.21.15.55:6789/0 conn(0x7fa3dc0614b0 legacy=0x7fa3dc0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2d9a700 1 -- 172.21.15.32:0/3998211045 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0af350 con 0x7fa3dc1405d0 2022-08-01T23:02:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e2d9a700 1 --2- 172.21.15.32:0/3998211045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc1405d0 0x7fa3dc140a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fa3d40032f0 tx=0x7fa3d40091d0).ready entity=mon.2 client_cookie=5e574a51f046e65 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e1597700 1 -- 172.21.15.32:0/3998211045 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d400b600 con 0x7fa3dc1405d0 2022-08-01T23:02:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e1597700 1 -- 172.21.15.32:0/3998211045 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400b760 con 0x7fa3dc1405d0 2022-08-01T23:02:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e1597700 1 -- 172.21.15.32:0/3998211045 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d400b8d0 con 0x7fa3dc1405d0 2022-08-01T23:02:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- 172.21.15.32:0/3998211045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc1405d0 msgr2=0x7fa3dc140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3998211045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc1405d0 0x7fa3dc140a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fa3d40032f0 tx=0x7fa3d40091d0).stop 2022-08-01T23:02:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- 172.21.15.32:0/3998211045 shutdown_connections 2022-08-01T23:02:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3998211045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc1405d0 0x7fa3dc140a80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3998211045 >> v2:172.21.15.55:3300/0 conn(0x7fa3dc0ad750 0x7fa3dc061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- 172.21.15.32:0/3998211045 >> 172.21.15.32:0/3998211045 conn(0x7fa3dc066f70 msgr2=0x7fa3dc067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- 172.21.15.32:0/3998211045 shutdown_connections 2022-08-01T23:02:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.355+0000 7fa3e359b700 1 -- 172.21.15.32:0/3998211045 wait complete. 2022-08-01T23:02:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 Processor -- start 2022-08-01T23:02:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 -- start start 2022-08-01T23:02:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 0x7fa3dc0b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc0ad750 0x7fa3dc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3dc1405d0 0x7fa3dc1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b27a0 con 0x7fa3dc0614b0 2022-08-01T23:02:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc155f10 con 0x7fa3dc1405d0 2022-08-01T23:02:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1561b0 con 0x7fa3dc0ad750 2022-08-01T23:02:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 0x7fa3dc0b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 0x7fa3dc0b7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45776/0 (socket says 172.21.15.32:45776) 2022-08-01T23:02:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 -- 172.21.15.32:0/3278961989 learned_addr learned my addr 172.21.15.32:0/3278961989 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e1d98700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc0ad750 0x7fa3dc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2d9a700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3dc1405d0 0x7fa3dc1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 -- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc0ad750 msgr2=0x7fa3dc155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc0ad750 0x7fa3dc155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 -- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3dc1405d0 msgr2=0x7fa3dc1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3dc1405d0 0x7fa3dc1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 -- 172.21.15.32:0/3278961989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009470 con 0x7fa3dc0614b0 2022-08-01T23:02:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e2599700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 0x7fa3dc0b7b20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fa3d8001110 tx=0x7fa3d8007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d800ce20 con 0x7fa3dc0614b0 2022-08-01T23:02:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d80072e0 con 0x7fa3dc0614b0 2022-08-01T23:02:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc156450 con 0x7fa3dc0614b0 2022-08-01T23:02:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d8005800 con 0x7fa3dc0614b0 2022-08-01T23:02:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.359+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc0b3cc0 con 0x7fa3dc0614b0 2022-08-01T23:02:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.363+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc152350 con 0x7fa3dc0614b0 2022-08-01T23:02:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.363+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa3d8010070 con 0x7fa3dc0614b0 2022-08-01T23:02:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.363+0000 7fa3cb7fe700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3bc0539e0 0x7fa3bc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.363+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa3d806b780 con 0x7fa3dc0614b0 2022-08-01T23:02:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.363+0000 7fa3e1d98700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3bc0539e0 0x7fa3bc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.363+0000 7fa3e1d98700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3bc0539e0 0x7fa3bc055ea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fa3cc00b000 tx=0x7fa3cc006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.371+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3d80424a0 con 0x7fa3dc0614b0 2022-08-01T23:02:48.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.531+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3dc0574e0 con 0x7fa3dc0614b0 2022-08-01T23:02:48.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.531+0000 7fa3cb7fe700 1 -- 172.21.15.32:0/3278961989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fa3d8018020 con 0x7fa3dc0614b0 2022-08-01T23:02:48.538 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:48.538 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:44.973242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6666":{"gid":6666,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4046550885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4046550885},{"type":"v1","addr":"172.21.15.32:6835","nonce":4046550885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:48.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.531+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3bc0539e0 msgr2=0x7fa3bc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.531+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3bc0539e0 0x7fa3bc055ea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fa3cc00b000 tx=0x7fa3cc006df0).stop 2022-08-01T23:02:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 msgr2=0x7fa3dc0b7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 0x7fa3dc0b7b20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fa3d8001110 tx=0x7fa3d8007ad0).stop 2022-08-01T23:02:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 shutdown_connections 2022-08-01T23:02:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa3bc0539e0 0x7fa3bc055ea0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3dc1405d0 0x7fa3dc1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3dc0ad750 0x7fa3dc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 --2- 172.21.15.32:0/3278961989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3dc0614b0 0x7fa3dc0b7b20 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 >> 172.21.15.32:0/3278961989 conn(0x7fa3dc066f70 msgr2=0x7fa3dc1493d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 shutdown_connections 2022-08-01T23:02:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:48.535+0000 7fa3e359b700 1 -- 172.21.15.32:0/3278961989 wait complete. 2022-08-01T23:02:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2022-08-01T23:02:48.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:02:43.969500+0000', 'modified': '2022-08-01T23:02:44.973242+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6666}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6666': {'gid': 6666, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4046550885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4046550885}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4046550885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:02:48.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:02:48.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:02:48.973 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6666,"want_state":"up:active","state":"up:active","rank_uptime":4.9896125380000003,"mdsmap_epoch":39,"osdmap_epoch":87,"osdmap_epoch_barrier":87,"uptime":9.3252294429999996} 2022-08-01T23:02:48.974 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6666, "mdsmap_epoch": 39, "osdmap_epoch": 87, "osdmap_epoch_barrier": 87, "rank_uptime": 4.989612538, "state": "up:active", "uptime": 9.325229443, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:02:48.974 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:02:48.975 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:02:48.975 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:02:48.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:48.976 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:48.985 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:02:49.044 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:49.045 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:49.060 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:49.060 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:49.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:49.062 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:02:49.068 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:02:49.069 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:49.072 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:02:49.087 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:49.087 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:49.097 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:49.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:49.099 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:02:49.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:02:49.137+0000 7f34664ca080 -1 init, newargv = 0x558e3ce9b980 newargc=9ceph-fuse[23141]: starting ceph client 2022-08-01T23:02:49.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr: 2022-08-01T23:02:49.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23141]: starting fuse 2022-08-01T23:02:50.207 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:02:50.225 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:50.226 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:50.282 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:50.283 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:50.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:50.285 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:02:50.290 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:02:50.290 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:02:50.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:50.291 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:02:50.291 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:02:50.292 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:02:50.292 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:02:50.292 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:02:50.293 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:50.293 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:02:50.294 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:02:50.294 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:02:50.295 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:50.295 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:02:50.295 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:02:50.296 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:02:50.296 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:02:50.297 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:50.297 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:02:50.297 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:02:50.298 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:02:50.298 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:02:50.299 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:02:50.299 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:02:50.300 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:02:50.300 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:02:50.374 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23141.asok 2022-08-01T23:02:50.379 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23141.asok 2022-08-01T23:02:50.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:50.380 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23141.asok status 2022-08-01T23:02:50.470 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:02:50.470 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:02:50.471 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:02:50.471 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:02:50.471 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:02:50.471 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:02:50.472 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:02:50.472 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23141", 2022-08-01T23:02:50.472 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:02:50.472 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:50.472 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:02:50.473 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:02:50.473 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 6104, 2022-08-01T23:02:50.473 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:02:50.473 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:02:50.474 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:02:50.474 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 6104 2022-08-01T23:02:50.474 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:50.474 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:02:50.475 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:02:50.475 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:02:50.475 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3273122521 2022-08-01T23:02:50.475 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:02:50.475 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:50.476 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:02:50.476 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:02:50.476 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:02:50.476 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3273122521 2022-08-01T23:02:50.477 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:02:50.477 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.6104 172.21.15.130:0/3273122521", 2022-08-01T23:02:50.477 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/3273122521", 2022-08-01T23:02:50.477 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:02:50.477 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 39, 2022-08-01T23:02:50.478 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 87, 2022-08-01T23:02:50.478 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:02:50.478 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:02:50.478 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:02:50.478 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:02:50.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:50.480 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:50.481 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:50.488 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:02:50.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:02:50.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:50.490 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:50.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:02:50.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe89c157700 1 Processor -- start 2022-08-01T23:02:50.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe89c157700 1 -- start start 2022-08-01T23:02:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe89c157700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 0x7fe8940acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe89c157700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8940b6e70 con 0x7fe8940b6a90 2022-08-01T23:02:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe89c157700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe894061370 con 0x7fe8940aa760 2022-08-01T23:02:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe89c157700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe894061610 con 0x7fe894060aa0 2022-08-01T23:02:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe899ef3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8940b6a90 0x7fe8940aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33326/0 (socket says 172.21.15.32:33326) 2022-08-01T23:02:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe899ef3700 1 -- 172.21.15.32:0/1499136080 learned_addr learned my addr 172.21.15.32:0/1499136080 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe8996f2700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 0x7fe8940acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362235079 0 0) 0x7fe8940b6e70 con 0x7fe8940b6a90 2022-08-01T23:02:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe87c003260 con 0x7fe8940b6a90 2022-08-01T23:02:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441502420 0 0) 0x7fe87c003260 con 0x7fe8940b6a90 2022-08-01T23:02:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 msgr2=0x7fe8940acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 0x7fe8940acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 >> v1:172.21.15.55:6789/0 conn(0x7fe8940aa760 legacy=0x7fe89413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940618b0 con 0x7fe8940b6a90 2022-08-01T23:02:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fe890001680 con 0x7fe8940b6a90 2022-08-01T23:02:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.655+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 >> v1:172.21.15.32:6789/0 conn(0x7fe8940b6a90 legacy=0x7fe8940aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c003780 0x7fe87c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c004500 0x7fe87c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 0x7fe87c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894061610 con 0x7fe87c004500 2022-08-01T23:02:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c005720 con 0x7fe87c003780 2022-08-01T23:02:50.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0058e0 con 0x7fe894060aa0 2022-08-01T23:02:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c004500 0x7fe87c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe899ef3700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c003780 0x7fe87c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89a6f4700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 0x7fe87c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 -- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 msgr2=0x7fe87c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe894060aa0 0x7fe87c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 -- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c003780 msgr2=0x7fe87c003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c003780 0x7fe87c003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 -- 172.21.15.32:0/1499136080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c003260 con 0x7fe87c004500 2022-08-01T23:02:50.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe899ef3700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c003780 0x7fe87c003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:50.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe8996f2700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c004500 0x7fe87c004980 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fe88400c000 tx=0x7fe884006df0).ready entity=mon.0 client_cookie=5b10f894021cfaca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:50.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe88400e830 con 0x7fe87c004500 2022-08-01T23:02:50.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88400e990 con 0x7fe87c004500 2022-08-01T23:02:50.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe898ef1700 1 -- 172.21.15.32:0/1499136080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe88400ecf0 con 0x7fe87c004500 2022-08-01T23:02:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89c157700 1 -- 172.21.15.32:0/1499136080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c004500 msgr2=0x7fe87c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89c157700 1 --2- 172.21.15.32:0/1499136080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c004500 0x7fe87c004980 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fe88400c000 tx=0x7fe884006df0).stop 2022-08-01T23:02:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89a6f4700 1 -- 172.21.15.32:0/1499136080 reap_dead start 2022-08-01T23:02:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89c157700 1 -- 172.21.15.32:0/1499136080 shutdown_connections 2022-08-01T23:02:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89c157700 1 -- 172.21.15.32:0/1499136080 >> 172.21.15.32:0/1499136080 conn(0x7fe8940b3610 msgr2=0x7fe8940b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89c157700 1 -- 172.21.15.32:0/1499136080 shutdown_connections 2022-08-01T23:02:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.659+0000 7fe89c157700 1 -- 172.21.15.32:0/1499136080 wait complete. 2022-08-01T23:02:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 Processor -- start 2022-08-01T23:02:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 -- start start 2022-08-01T23:02:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c004500 0x7fe8940b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 0x7fe89413ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8940ace90 0x7fe894151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89413c770 con 0x7fe89413c160 2022-08-01T23:02:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89413bed0 con 0x7fe87c004500 2022-08-01T23:02:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940add20 con 0x7fe8940ace90 2022-08-01T23:02:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 0x7fe89413ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe899ef3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c004500 0x7fe8940b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 0x7fe89413ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45798/0 (socket says 172.21.15.32:45798) 2022-08-01T23:02:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 -- 172.21.15.32:0/2548087897 learned_addr learned my addr 172.21.15.32:0/2548087897 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89a6f4700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8940ace90 0x7fe894151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 -- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8940ace90 msgr2=0x7fe894151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8940ace90 0x7fe894151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 -- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c004500 msgr2=0x7fe8940b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c004500 0x7fe8940b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 -- 172.21.15.32:0/2548087897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940adfc0 con 0x7fe89413c160 2022-08-01T23:02:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe8996f2700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 0x7fe89413ce40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fe88400c000 tx=0x7fe884019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe88401b820 con 0x7fe89413c160 2022-08-01T23:02:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe884000e90 con 0x7fe89413c160 2022-08-01T23:02:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89413c9a0 con 0x7fe89413c160 2022-08-01T23:02:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe88401bb70 con 0x7fe89413c160 2022-08-01T23:02:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.663+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8940ad340 con 0x7fe89413c160 2022-08-01T23:02:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.667+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89413d330 con 0x7fe89413c160 2022-08-01T23:02:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.667+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe88401bcd0 con 0x7fe89413c160 2022-08-01T23:02:50.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.667+0000 7fe882ffd700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe874053b00 0x7fe874055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.667+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe88406d580 con 0x7fe89413c160 2022-08-01T23:02:50.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.667+0000 7fe899ef3700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe874053b00 0x7fe874055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.667+0000 7fe899ef3700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe874053b00 0x7fe874055fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fe890005000 tx=0x7fe8940ac120).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.671+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe884044340 con 0x7fe89413c160 2022-08-01T23:02:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.795+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe89413cb30 con 0x7fe89413c160 2022-08-01T23:02:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.795+0000 7fe882ffd700 1 -- 172.21.15.32:0/2548087897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fe884047960 con 0x7fe89413c160 2022-08-01T23:02:50.803 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:50.803 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:02:44.972186+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:02:42.386170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T23:02:43.649622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:02:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe874053b00 msgr2=0x7fe874055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe874053b00 0x7fe874055fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fe890005000 tx=0x7fe8940ac120).stop 2022-08-01T23:02:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 msgr2=0x7fe89413ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 0x7fe89413ce40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fe88400c000 tx=0x7fe884019000).stop 2022-08-01T23:02:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 shutdown_connections 2022-08-01T23:02:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe874053b00 0x7fe874055fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8940ace90 0x7fe894151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89413c160 0x7fe89413ce40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 --2- 172.21.15.32:0/2548087897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87c004500 0x7fe8940b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 >> 172.21.15.32:0/2548087897 conn(0x7fe8940b3610 msgr2=0x7fe89413f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 shutdown_connections 2022-08-01T23:02:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.799+0000 7fe89c157700 1 -- 172.21.15.32:0/2548087897 wait complete. 2022-08-01T23:02:50.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:02:42.386170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:02:43.649622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:02:50.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:02:50.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.923+0000 7fa9f1227700 1 Processor -- start 2022-08-01T23:02:50.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.923+0000 7fa9f1227700 1 -- start start 2022-08-01T23:02:50.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa9ec0b7500 0x7fa9ec0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9ec0b31f0 0x7fa9ec13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0b2510 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ec1443e0 con 0x7fa9ec0b31f0 2022-08-01T23:02:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b3700 con 0x7fa9ec0b7500 2022-08-01T23:02:50.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9ec0b31f0 0x7fa9ec13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f0a26700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9ec13e8a0 0x7fa9ec064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33350/0 (socket says 172.21.15.32:33350) 2022-08-01T23:02:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f0a26700 1 -- 172.21.15.32:0/3818623727 learned_addr learned my addr 172.21.15.32:0/3818623727 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:50.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9ebfff700 1 --2- 172.21.15.32:0/3818623727 >> v2:172.21.15.55:3300/0 conn(0x7fa9ec0b7500 0x7fa9ec0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9e3fff700 1 -- 172.21.15.32:0/3818623727 >> v2:172.21.15.55:3300/0 conn(0x7fa9ec0b7500 msgr2=0x7fa9ec0b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:50.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9e3fff700 1 --2- 172.21.15.32:0/3818623727 >> v2:172.21.15.55:3300/0 conn(0x7fa9ec0b7500 0x7fa9ec0b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9e3fff700 1 -- 172.21.15.32:0/3818623727 >> v1:172.21.15.32:6789/0 conn(0x7fa9ec13e8a0 legacy=0x7fa9ec064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9e3fff700 1 -- 172.21.15.32:0/3818623727 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec0b2740 con 0x7fa9ec0b31f0 2022-08-01T23:02:50.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9eb7fe700 1 -- 172.21.15.32:0/3818623727 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713825231 0 0) 0x7fa9ec0b2510 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9e3fff700 1 --2- 172.21.15.32:0/3818623727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec13e3b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7fa9d4002f70 tx=0x7fa9d4006df0).ready entity=mon.0 client_cookie=8e9b37c42669c87a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:50.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9eb7fe700 1 -- 172.21.15.32:0/3818623727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9d400d570 con 0x7fa9ec0b31f0 2022-08-01T23:02:50.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9eb7fe700 1 -- 172.21.15.32:0/3818623727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4000e90 con 0x7fa9ec0b31f0 2022-08-01T23:02:50.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- 172.21.15.32:0/3818623727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b31f0 msgr2=0x7fa9ec13e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3818623727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec13e3b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7fa9d4002f70 tx=0x7fa9d4006df0).stop 2022-08-01T23:02:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- 172.21.15.32:0/3818623727 shutdown_connections 2022-08-01T23:02:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3818623727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec13e3b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3818623727 >> v2:172.21.15.55:3300/0 conn(0x7fa9ec0b7500 0x7fa9ec0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- 172.21.15.32:0/3818623727 >> 172.21.15.32:0/3818623727 conn(0x7fa9ec05f840 msgr2=0x7fa9ec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- 172.21.15.32:0/3818623727 shutdown_connections 2022-08-01T23:02:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.927+0000 7fa9f1227700 1 -- 172.21.15.32:0/3818623727 wait complete. 2022-08-01T23:02:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 Processor -- start 2022-08-01T23:02:50.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 -- start start 2022-08-01T23:02:50.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec0ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b7500 0x7fa9ec0afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 0x7fa9ec0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec150bf0 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b08c0 con 0x7fa9ec0b31f0 2022-08-01T23:02:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b0e90 con 0x7fa9ec0b7500 2022-08-01T23:02:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 0x7fa9ec0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 0x7fa9ec0b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45814/0 (socket says 172.21.15.32:45814) 2022-08-01T23:02:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 -- 172.21.15.32:0/3203505400 learned_addr learned my addr 172.21.15.32:0/3203505400 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9ebfff700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec0ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9e3fff700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b7500 0x7fa9ec0afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 -- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b7500 msgr2=0x7fa9ec0afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b7500 0x7fa9ec0afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 -- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0b31f0 msgr2=0x7fa9ec0ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec0ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 -- 172.21.15.32:0/3203505400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d400b180 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f0a26700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 0x7fa9ec0b03d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fa9e4001250 tx=0x7fa9e4001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e4005ba0 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4005d00 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec0af470 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e4004a90 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.931+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec0ac6d0 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.935+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec150d60 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.935+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9e4004bf0 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.935+0000 7fa9e9ffb700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053b00 0x7fa9d0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.935+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa9e406bb80 con 0x7fa9ec13e8a0 2022-08-01T23:02:50.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.935+0000 7fa9ebfff700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053b00 0x7fa9d0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.935+0000 7fa9ebfff700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053b00 0x7fa9d0055fc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fa9dc006000 tx=0x7fa9dc006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:50.939+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9e4042940 con 0x7fa9ec13e8a0 2022-08-01T23:02:51.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa9ec0af600 con 0x7fa9ec13e8a0 2022-08-01T23:02:51.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9e9ffb700 1 -- 172.21.15.32:0/3203505400 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v87) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa9e4045f60 con 0x7fa9ec13e8a0 2022-08-01T23:02:51.073 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:02:51.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053b00 msgr2=0x7fa9d0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053b00 0x7fa9d0055fc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fa9dc006000 tx=0x7fa9dc006d90).stop 2022-08-01T23:02:51.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 msgr2=0x7fa9ec0b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 0x7fa9ec0b03d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fa9e4001250 tx=0x7fa9e4001360).stop 2022-08-01T23:02:51.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 shutdown_connections 2022-08-01T23:02:51.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053b00 0x7fa9d0055fc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec13e8a0 0x7fa9ec0b03d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9ec0b7500 0x7fa9ec0afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 --2- 172.21.15.32:0/3203505400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0b31f0 0x7fa9ec0ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 >> 172.21.15.32:0/3203505400 conn(0x7fa9ec05f840 msgr2=0x7fa9ec140240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 shutdown_connections 2022-08-01T23:02:51.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.067+0000 7fa9f1227700 1 -- 172.21.15.32:0/3203505400 wait complete. 2022-08-01T23:02:51.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:02:51.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.187+0000 7fdb8b120700 1 Processor -- start 2022-08-01T23:02:51.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.187+0000 7fdb8b120700 1 -- start start 2022-08-01T23:02:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb840614b0 0x7fdb84061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb840ad750 0x7fdb840b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb840aee80 con 0x7fdb841405d0 2022-08-01T23:02:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840b70b0 con 0x7fdb840614b0 2022-08-01T23:02:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840af0b0 con 0x7fdb840ad750 2022-08-01T23:02:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a11e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb840614b0 0x7fdb84061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a91f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdb841405d0 0x7fdb841466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33362/0 (socket says 172.21.15.32:33362) 2022-08-01T23:02:51.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a91f700 1 -- 172.21.15.32:0/2930142817 learned_addr learned my addr 172.21.15.32:0/2930142817 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:51.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8991d700 1 --2- 172.21.15.32:0/2930142817 >> v2:172.21.15.55:3300/0 conn(0x7fdb840ad750 0x7fdb840b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a11e700 1 -- 172.21.15.32:0/2930142817 >> v2:172.21.15.55:3300/0 conn(0x7fdb840ad750 msgr2=0x7fdb840b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a11e700 1 --2- 172.21.15.32:0/2930142817 >> v2:172.21.15.55:3300/0 conn(0x7fdb840ad750 0x7fdb840b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a11e700 1 -- 172.21.15.32:0/2930142817 >> v1:172.21.15.32:6789/0 conn(0x7fdb841405d0 legacy=0x7fdb841466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8911c700 1 -- 172.21.15.32:0/2930142817 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386981736 0 0) 0x7fdb840aee80 con 0x7fdb841405d0 2022-08-01T23:02:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a11e700 1 -- 172.21.15.32:0/2930142817 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840af350 con 0x7fdb840614b0 2022-08-01T23:02:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8a11e700 1 --2- 172.21.15.32:0/2930142817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb84061890 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fdb840aed30 tx=0x7fdb78006df0).ready entity=mon.0 client_cookie=3675e6c8932e0a9e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8911c700 1 -- 172.21.15.32:0/2930142817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb7800d5c0 con 0x7fdb840614b0 2022-08-01T23:02:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8911c700 1 -- 172.21.15.32:0/2930142817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb78000e90 con 0x7fdb840614b0 2022-08-01T23:02:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8911c700 1 -- 172.21.15.32:0/2930142817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb7800d910 con 0x7fdb840614b0 2022-08-01T23:02:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- 172.21.15.32:0/2930142817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 msgr2=0x7fdb84061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 --2- 172.21.15.32:0/2930142817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb84061890 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fdb840aed30 tx=0x7fdb78006df0).stop 2022-08-01T23:02:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- 172.21.15.32:0/2930142817 shutdown_connections 2022-08-01T23:02:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 --2- 172.21.15.32:0/2930142817 >> v2:172.21.15.55:3300/0 conn(0x7fdb840ad750 0x7fdb840b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 --2- 172.21.15.32:0/2930142817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb84061890 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- 172.21.15.32:0/2930142817 >> 172.21.15.32:0/2930142817 conn(0x7fdb84066f70 msgr2=0x7fdb84067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- 172.21.15.32:0/2930142817 shutdown_connections 2022-08-01T23:02:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.191+0000 7fdb8b120700 1 -- 172.21.15.32:0/2930142817 wait complete. 2022-08-01T23:02:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 Processor -- start 2022-08-01T23:02:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 -- start start 2022-08-01T23:02:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb840afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb840ad750 0x7fdb840b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb841405d0 0x7fdb840b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840b1530 con 0x7fdb840614b0 2022-08-01T23:02:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb84156580 con 0x7fdb840ad750 2022-08-01T23:02:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8b120700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb84141f90 con 0x7fdb841405d0 2022-08-01T23:02:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8a11e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb840afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8a11e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb840afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45824/0 (socket says 172.21.15.32:45824) 2022-08-01T23:02:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8a11e700 1 -- 172.21.15.32:0/1633578069 learned_addr learned my addr 172.21.15.32:0/1633578069 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8a91f700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb841405d0 0x7fdb840b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.195+0000 7fdb8991d700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb840ad750 0x7fdb840b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8a11e700 1 -- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb841405d0 msgr2=0x7fdb840b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8a11e700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb841405d0 0x7fdb840b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8a11e700 1 -- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb840ad750 msgr2=0x7fdb840b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8a11e700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb840ad750 0x7fdb840b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8a11e700 1 -- 172.21.15.32:0/1633578069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7800b180 con 0x7fdb840614b0 2022-08-01T23:02:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8a11e700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb840afe00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fdb840aed30 tx=0x7fdb78014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb78016070 con 0x7fdb840614b0 2022-08-01T23:02:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb84156820 con 0x7fdb840614b0 2022-08-01T23:02:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7800de40 con 0x7fdb840614b0 2022-08-01T23:02:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb840b0410 con 0x7fdb840614b0 2022-08-01T23:02:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb78013e10 con 0x7fdb840614b0 2022-08-01T23:02:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.199+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb841563c0 con 0x7fdb840614b0 2022-08-01T23:02:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.203+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdb7800f030 con 0x7fdb840614b0 2022-08-01T23:02:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.203+0000 7fdb777fe700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdb640539e0 0x7fdb64055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.203+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdb7806d1c0 con 0x7fdb840614b0 2022-08-01T23:02:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.203+0000 7fdb8991d700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdb640539e0 0x7fdb64055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.203+0000 7fdb8991d700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdb640539e0 0x7fdb64055ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fdb8000b000 tx=0x7fdb80006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.207+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdb78043f80 con 0x7fdb840614b0 2022-08-01T23:02:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.335+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdb840574e0 con 0x7fdb840614b0 2022-08-01T23:02:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.335+0000 7fdb777fe700 1 -- 172.21.15.32:0/1633578069 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdb78023070 con 0x7fdb840614b0 2022-08-01T23:02:51.343 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdb640539e0 msgr2=0x7fdb64055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdb640539e0 0x7fdb64055ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fdb8000b000 tx=0x7fdb80006df0).stop 2022-08-01T23:02:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 msgr2=0x7fdb840afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb840afe00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fdb840aed30 tx=0x7fdb78014000).stop 2022-08-01T23:02:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 shutdown_connections 2022-08-01T23:02:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdb640539e0 0x7fdb64055ea0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdb841405d0 0x7fdb840b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb840ad750 0x7fdb840b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 --2- 172.21.15.32:0/1633578069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb840614b0 0x7fdb840afe00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 >> 172.21.15.32:0/1633578069 conn(0x7fdb84066f70 msgr2=0x7fdb840b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 shutdown_connections 2022-08-01T23:02:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.339+0000 7fdb8b120700 1 -- 172.21.15.32:0/1633578069 wait complete. 2022-08-01T23:02:51.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:02:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.455+0000 7fc110a1c700 1 Processor -- start 2022-08-01T23:02:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.455+0000 7fc110a1c700 1 -- start start 2022-08-01T23:02:51.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc110a1c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc10c0a9da0 0x7fc10c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc110a1c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c067030 con 0x7fc10c1402d0 2022-08-01T23:02:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc110a1c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c0b35b0 con 0x7fc10c0b31d0 2022-08-01T23:02:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc110a1c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c064a70 con 0x7fc10c0a9da0 2022-08-01T23:02:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10b59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc10c1402d0 0x7fc10c140720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33376/0 (socket says 172.21.15.32:33376) 2022-08-01T23:02:51.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10b59e700 1 -- 172.21.15.32:0/2865079908 learned_addr learned my addr 172.21.15.32:0/2865079908 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:51.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10259c700 1 --2- 172.21.15.32:0/2865079908 >> v2:172.21.15.55:3300/0 conn(0x7fc10c0a9da0 0x7fc10c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10a59c700 1 -- 172.21.15.32:0/2865079908 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336292132 0 0) 0x7fc10c067030 con 0x7fc10c1402d0 2022-08-01T23:02:51.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10a59c700 1 -- 172.21.15.32:0/2865079908 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0ec003260 con 0x7fc10c1402d0 2022-08-01T23:02:51.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10259c700 1 -- 172.21.15.32:0/2865079908 >> v1:172.21.15.55:6789/0 conn(0x7fc10c0b31d0 legacy=0x7fc10c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10259c700 1 -- 172.21.15.32:0/2865079908 >> v1:172.21.15.32:6789/0 conn(0x7fc10c1402d0 legacy=0x7fc10c140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10259c700 1 -- 172.21.15.32:0/2865079908 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c066af0 con 0x7fc10c0a9da0 2022-08-01T23:02:51.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10259c700 1 --2- 172.21.15.32:0/2865079908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c0a9da0 0x7fc10c060950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc0f4002f70 tx=0x7fc0f4006df0).ready entity=mon.1 client_cookie=ab126b26e02aaaf1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10a59c700 1 -- 172.21.15.32:0/2865079908 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0f400d610 con 0x7fc10c0a9da0 2022-08-01T23:02:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10a59c700 1 -- 172.21.15.32:0/2865079908 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4000e90 con 0x7fc10c0a9da0 2022-08-01T23:02:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.459+0000 7fc10a59c700 1 -- 172.21.15.32:0/2865079908 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0f400d960 con 0x7fc10c0a9da0 2022-08-01T23:02:51.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- 172.21.15.32:0/2865079908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c0a9da0 msgr2=0x7fc10c060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 --2- 172.21.15.32:0/2865079908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c0a9da0 0x7fc10c060950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc0f4002f70 tx=0x7fc0f4006df0).stop 2022-08-01T23:02:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- 172.21.15.32:0/2865079908 shutdown_connections 2022-08-01T23:02:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 --2- 172.21.15.32:0/2865079908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c0a9da0 0x7fc10c060950 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- 172.21.15.32:0/2865079908 >> 172.21.15.32:0/2865079908 conn(0x7fc10c063870 msgr2=0x7fc10c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- 172.21.15.32:0/2865079908 shutdown_connections 2022-08-01T23:02:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- 172.21.15.32:0/2865079908 wait complete. 2022-08-01T23:02:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 Processor -- start 2022-08-01T23:02:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- start start 2022-08-01T23:02:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 0x7fc10c142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc10c0b31d0 0x7fc10c13d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c1402d0 0x7fc10c13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c143b00 con 0x7fc10c0a9da0 2022-08-01T23:02:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c140ad0 con 0x7fc10c1402d0 2022-08-01T23:02:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc110a1c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0ab870 con 0x7fc10c0b31d0 2022-08-01T23:02:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc10ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 0x7fc10c142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc10ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 0x7fc10c142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45838/0 (socket says 172.21.15.32:45838) 2022-08-01T23:02:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc10ad9d700 1 -- 172.21.15.32:0/3575079941 learned_addr learned my addr 172.21.15.32:0/3575079941 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc10b59e700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c1402d0 0x7fc10c13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.463+0000 7fc10259c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc10c0b31d0 0x7fc10c13d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10ad9d700 1 -- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc10c0b31d0 msgr2=0x7fc10c13d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10ad9d700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc10c0b31d0 0x7fc10c13d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10ad9d700 1 -- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c1402d0 msgr2=0x7fc10c13dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10ad9d700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c1402d0 0x7fc10c13dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10ad9d700 1 -- 172.21.15.32:0/3575079941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f400b180 con 0x7fc10c0a9da0 2022-08-01T23:02:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10ad9d700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 0x7fc10c142580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fc10c141580 tx=0x7fc0fc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0fc007030 con 0x7fc10c0a9da0 2022-08-01T23:02:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc007190 con 0x7fc10c0a9da0 2022-08-01T23:02:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c140d00 con 0x7fc10c0a9da0 2022-08-01T23:02:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0fc006510 con 0x7fc10c0a9da0 2022-08-01T23:02:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c0aad50 con 0x7fc10c0a9da0 2022-08-01T23:02:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c140e90 con 0x7fc10c0a9da0 2022-08-01T23:02:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc0fc010030 con 0x7fc10c0a9da0 2022-08-01T23:02:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc108d99700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc0fc06bba0 con 0x7fc10c0a9da0 2022-08-01T23:02:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.467+0000 7fc10259c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.471+0000 7fc10259c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fc0f401a000 tx=0x7fc0f4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.475+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc0fc042960 con 0x7fc10c0a9da0 2022-08-01T23:02:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.599+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc10c066d90 con 0x7fc10c0a9da0 2022-08-01T23:02:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc108d99700 1 -- 172.21.15.32:0/3575079941 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc0fc045f80 con 0x7fc10c0a9da0 2022-08-01T23:02:51.608 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 msgr2=0x7fc0f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fc0f401a000 tx=0x7fc0f4014000).stop 2022-08-01T23:02:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 msgr2=0x7fc10c142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 0x7fc10c142580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fc10c141580 tx=0x7fc0fc007c10).stop 2022-08-01T23:02:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 shutdown_connections 2022-08-01T23:02:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc0f0053b00 0x7fc0f0055fc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc10c1402d0 0x7fc10c13dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc10c0b31d0 0x7fc10c13d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 --2- 172.21.15.32:0/3575079941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10c0a9da0 0x7fc10c142580 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 >> 172.21.15.32:0/3575079941 conn(0x7fc10c063870 msgr2=0x7fc10c0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 shutdown_connections 2022-08-01T23:02:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.603+0000 7fc110a1c700 1 -- 172.21.15.32:0/3575079941 wait complete. 2022-08-01T23:02:51.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:02:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.727+0000 7fc355ebb700 1 Processor -- start 2022-08-01T23:02:51.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.727+0000 7fc355ebb700 1 -- start start 2022-08-01T23:02:51.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 0x7fc3501401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3500b0270 0x7fc3501406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500ac880 0x7fc350143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3500ad740 con 0x7fc3500b2710 2022-08-01T23:02:51.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc350067a10 con 0x7fc3500b0270 2022-08-01T23:02:51.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc350067cb0 con 0x7fc3500ac880 2022-08-01T23:02:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc354eb9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 0x7fc3501401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3500b0270 0x7fc3501406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc354eb9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 0x7fc3501401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45848/0 (socket says 172.21.15.32:45848) 2022-08-01T23:02:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc354eb9700 1 -- 172.21.15.32:0/3023176185 learned_addr learned my addr 172.21.15.32:0/3023176185 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc3556ba700 1 --2- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500ac880 0x7fc350143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 -- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500ac880 msgr2=0x7fc350143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 --2- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500ac880 0x7fc350143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 -- 172.21.15.32:0/3023176185 >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 msgr2=0x7fc3501401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 --2- 172.21.15.32:0/3023176185 >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 0x7fc3501401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 -- 172.21.15.32:0/3023176185 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350147160 con 0x7fc3500b0270 2022-08-01T23:02:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc354eb9700 1 --2- 172.21.15.32:0/3023176185 >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 0x7fc3501401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34ffff700 1 --2- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500b0270 0x7fc3501406d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fc3400032f0 tx=0x7fc3400091d0).ready entity=mon.1 client_cookie=dd2c8221b5787730 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34f7fe700 1 -- 172.21.15.32:0/3023176185 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34000b600 con 0x7fc3500b0270 2022-08-01T23:02:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34f7fe700 1 -- 172.21.15.32:0/3023176185 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34000b760 con 0x7fc3500b0270 2022-08-01T23:02:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc34f7fe700 1 -- 172.21.15.32:0/3023176185 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34000b8d0 con 0x7fc3500b0270 2022-08-01T23:02:51.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500b0270 msgr2=0x7fc3501406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500b0270 0x7fc3501406d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fc3400032f0 tx=0x7fc3400091d0).stop 2022-08-01T23:02:51.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- 172.21.15.32:0/3023176185 shutdown_connections 2022-08-01T23:02:51.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500ac880 0x7fc350143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- 172.21.15.32:0/3023176185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500b0270 0x7fc3501406d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 --2- 172.21.15.32:0/3023176185 >> v2:172.21.15.32:3300/0 conn(0x7fc3500b2710 0x7fc3501401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- 172.21.15.32:0/3023176185 >> 172.21.15.32:0/3023176185 conn(0x7fc350060310 msgr2=0x7fc3500b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- 172.21.15.32:0/3023176185 shutdown_connections 2022-08-01T23:02:51.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.731+0000 7fc355ebb700 1 -- 172.21.15.32:0/3023176185 wait complete. 2022-08-01T23:02:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 Processor -- start 2022-08-01T23:02:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 -- start start 2022-08-01T23:02:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500ac880 0x7fc350146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 0x7fc35013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500b2710 0x7fc35013fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350144350 con 0x7fc3500b0270 2022-08-01T23:02:51.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350143c20 con 0x7fc3500ac880 2022-08-01T23:02:51.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35009f630 con 0x7fc3500b2710 2022-08-01T23:02:51.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 0x7fc35013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc3556ba700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500b2710 0x7fc35013fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 0x7fc35013f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45856/0 (socket says 172.21.15.32:45856) 2022-08-01T23:02:51.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc354eb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500ac880 0x7fc350146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 -- 172.21.15.32:0/16359174 learned_addr learned my addr 172.21.15.32:0/16359174 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 -- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500b2710 msgr2=0x7fc35013fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500b2710 0x7fc35013fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 -- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500ac880 msgr2=0x7fc350146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500ac880 0x7fc350146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 -- 172.21.15.32:0/16359174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34400b1e0 con 0x7fc3500b0270 2022-08-01T23:02:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34ffff700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 0x7fc35013f760 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fc3400032f0 tx=0x7fc340016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34000b600 con 0x7fc3500b0270 2022-08-01T23:02:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34001f5e0 con 0x7fc3500b0270 2022-08-01T23:02:51.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc340009470 con 0x7fc3500b0270 2022-08-01T23:02:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.735+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34001f940 con 0x7fc3500b0270 2022-08-01T23:02:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.739+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35009eef0 con 0x7fc3500b0270 2022-08-01T23:02:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.739+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc350143a50 con 0x7fc3500b0270 2022-08-01T23:02:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.739+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc340026030 con 0x7fc3500b0270 2022-08-01T23:02:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.739+0000 7fc34dffb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc330053950 0x7fc330055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.739+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc34006cb20 con 0x7fc3500b0270 2022-08-01T23:02:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.739+0000 7fc354eb9700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc330053950 0x7fc330055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.743+0000 7fc354eb9700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc330053950 0x7fc330055e10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fc34400b000 tx=0x7fc344006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.747+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc340043880 con 0x7fc3500b0270 2022-08-01T23:02:51.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.871+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc35009f330 con 0x7fc3500b0270 2022-08-01T23:02:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.871+0000 7fc34dffb700 1 -- 172.21.15.32:0/16359174 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc340024020 con 0x7fc3500b0270 2022-08-01T23:02:51.879 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:02:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc330053950 msgr2=0x7fc330055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc330053950 0x7fc330055e10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fc34400b000 tx=0x7fc344006eb0).stop 2022-08-01T23:02:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 msgr2=0x7fc35013f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 0x7fc35013f760 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fc3400032f0 tx=0x7fc340016000).stop 2022-08-01T23:02:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 shutdown_connections 2022-08-01T23:02:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc330053950 0x7fc330055e10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3500b2710 0x7fc35013fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3500b0270 0x7fc35013f760 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 --2- 172.21.15.32:0/16359174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3500ac880 0x7fc350146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 >> 172.21.15.32:0/16359174 conn(0x7fc350060310 msgr2=0x7fc3500adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 shutdown_connections 2022-08-01T23:02:51.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.875+0000 7fc355ebb700 1 -- 172.21.15.32:0/16359174 wait complete. 2022-08-01T23:02:51.892 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:02:51.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.991+0000 7f355759e700 1 Processor -- start 2022-08-01T23:02:52.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.991+0000 7f355759e700 1 -- start start 2022-08-01T23:02:52.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3558061890 0x7f35581466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580aee80 con 0x7f35580ad750 2022-08-01T23:02:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580b70b0 con 0x7f35580614b0 2022-08-01T23:02:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35580af0b0 con 0x7f3558061890 2022-08-01T23:02:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f354ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f35580ad750 0x7f35581405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33402/0 (socket says 172.21.15.32:33402) 2022-08-01T23:02:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f354ffff700 1 -- 172.21.15.32:0/1361998672 learned_addr learned my addr 172.21.15.32:0/1361998672 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3556d9d700 1 --2- 172.21.15.32:0/1361998672 >> v2:172.21.15.55:3300/0 conn(0x7f3558061890 0x7f35581466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3555d9b700 1 -- 172.21.15.32:0/1361998672 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839990410 0 0) 0x7f35580aee80 con 0x7f35580ad750 2022-08-01T23:02:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3555d9b700 1 -- 172.21.15.32:0/1361998672 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f353c003260 con 0x7f35580ad750 2022-08-01T23:02:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3556d9d700 1 -- 172.21.15.32:0/1361998672 >> v1:172.21.15.55:6789/0 conn(0x7f35580614b0 legacy=0x7f35580b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3556d9d700 1 -- 172.21.15.32:0/1361998672 >> v1:172.21.15.32:6789/0 conn(0x7f35580ad750 legacy=0x7f35581405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3556d9d700 1 -- 172.21.15.32:0/1361998672 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580af350 con 0x7f3558061890 2022-08-01T23:02:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3556d9d700 1 --2- 172.21.15.32:0/1361998672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3558061890 0x7f35581466b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f35500032f0 tx=0x7f35500091d0).ready entity=mon.1 client_cookie=5db1a108fac18e4e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3555d9b700 1 -- 172.21.15.32:0/1361998672 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f355000b6c0 con 0x7f3558061890 2022-08-01T23:02:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3555d9b700 1 -- 172.21.15.32:0/1361998672 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355000b820 con 0x7f3558061890 2022-08-01T23:02:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f3555d9b700 1 -- 172.21.15.32:0/1361998672 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f355000b990 con 0x7f3558061890 2022-08-01T23:02:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- 172.21.15.32:0/1361998672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3558061890 msgr2=0x7f35581466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 --2- 172.21.15.32:0/1361998672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3558061890 0x7f35581466b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f35500032f0 tx=0x7f35500091d0).stop 2022-08-01T23:02:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- 172.21.15.32:0/1361998672 shutdown_connections 2022-08-01T23:02:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 --2- 172.21.15.32:0/1361998672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3558061890 0x7f35581466b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- 172.21.15.32:0/1361998672 >> 172.21.15.32:0/1361998672 conn(0x7f3558066f70 msgr2=0x7f3558067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- 172.21.15.32:0/1361998672 shutdown_connections 2022-08-01T23:02:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.995+0000 7f355759e700 1 -- 172.21.15.32:0/1361998672 wait complete. 2022-08-01T23:02:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 Processor -- start 2022-08-01T23:02:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 -- start start 2022-08-01T23:02:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 0x7f35581529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558061890 0x7f3558146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 0x7f35580ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558153220 con 0x7f3558061890 2022-08-01T23:02:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35581534c0 con 0x7f35580ad750 2022-08-01T23:02:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355809eae0 con 0x7f35580614b0 2022-08-01T23:02:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f354ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558061890 0x7f3558146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 0x7f35581529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f3556d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 0x7f35580ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 0x7f35581529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:57604/0 (socket says 172.21.15.32:57604) 2022-08-01T23:02:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f3556d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 0x7f35580ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:55656/0 (socket says 172.21.15.32:55656) 2022-08-01T23:02:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 -- 172.21.15.32:0/255446230 learned_addr learned my addr 172.21.15.32:0/255446230 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f354ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558061890 0x7f3558146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45868/0 (socket says 172.21.15.32:45868) 2022-08-01T23:02:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 -- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 msgr2=0x7f35580ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 0x7f35580ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 -- 172.21.15.32:0/255446230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558061890 msgr2=0x7f3558146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558061890 0x7f3558146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 -- 172.21.15.32:0/255446230 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3550009470 con 0x7f35580614b0 2022-08-01T23:02:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f3556d9d700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 0x7f35580ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355659c700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 0x7f35581529e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f35580af970 tx=0x7f3548007be0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3548007030 con 0x7f35580614b0 2022-08-01T23:02:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548007190 con 0x7f35580614b0 2022-08-01T23:02:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3548006510 con 0x7f35580614b0 2022-08-01T23:02:52.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355809ef80 con 0x7f35580614b0 2022-08-01T23:02:52.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:51.999+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3558145f40 con 0x7f35580614b0 2022-08-01T23:02:52.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.003+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355809eca0 con 0x7f35580614b0 2022-08-01T23:02:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.003+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3548006670 con 0x7f35580614b0 2022-08-01T23:02:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.003+0000 7f354f7fe700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3530053a70 0x7f3530055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:52.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.003+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f354806b9f0 con 0x7f35580614b0 2022-08-01T23:02:52.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.003+0000 7f354ffff700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3530053a70 0x7f3530055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:52.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.003+0000 7f354ffff700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3530053a70 0x7f3530055f30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f354000d000 tx=0x7f3540009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:52.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.015+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3548042640 con 0x7f35580614b0 2022-08-01T23:02:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.179+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35580574e0 con 0x7f35580614b0 2022-08-01T23:02:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.179+0000 7f354f7fe700 1 -- 172.21.15.32:0/255446230 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f3548045c60 con 0x7f35580614b0 2022-08-01T23:02:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:52.195 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:44.973242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6666":{"gid":6666,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4046550885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4046550885},{"type":"v1","addr":"172.21.15.32:6835","nonce":4046550885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:52.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3530053a70 msgr2=0x7f3530055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3530053a70 0x7f3530055f30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f354000d000 tx=0x7f3540009040).stop 2022-08-01T23:02:52.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 msgr2=0x7f35581529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 0x7f35581529e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f35580af970 tx=0x7f3548007be0).stop 2022-08-01T23:02:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 shutdown_connections 2022-08-01T23:02:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3530053a70 0x7f3530055f30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35580ad750 0x7f35580ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3558061890 0x7f3558146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 --2- 172.21.15.32:0/255446230 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35580614b0 0x7f35581529e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 >> 172.21.15.32:0/255446230 conn(0x7f3558066f70 msgr2=0x7f3558141a30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 shutdown_connections 2022-08-01T23:02:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:52.183+0000 7f355759e700 1 -- 172.21.15.32:0/255446230 wait complete. 2022-08-01T23:02:52.199 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2022-08-01T23:02:52.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:02:43.969500+0000', 'modified': '2022-08-01T23:02:44.973242+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6666}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6666': {'gid': 6666, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/4046550885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 4046550885}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 4046550885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:02:52.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:02:52.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:02:52.618 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6666,"want_state":"up:active","state":"up:active","rank_uptime":8.6348644449999998,"mdsmap_epoch":39,"osdmap_epoch":87,"osdmap_epoch_barrier":87,"uptime":12.970473318} 2022-08-01T23:02:52.618 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6666, "mdsmap_epoch": 39, "osdmap_epoch": 87, "osdmap_epoch_barrier": 87, "rank_uptime": 8.634864445, "state": "up:active", "uptime": 12.970473318, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:02:52.619 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:02:52.619 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:02:52.620 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:02:52.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:52.621 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:52.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:02:52.643 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:52.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:52.655 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:52.657 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:52.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:52.658 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:02:52.706 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:02:52.706 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:52.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:02:52.726 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:52.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:52.737 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:52.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:52.739 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:02:52.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:02:52.776+0000 7faf72d5f080 -1 init, newargv = 0x55db1feaf9e0 newargc=9ceph-fuse[22676]: starting ceph client 2022-08-01T23:02:52.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr: 2022-08-01T23:02:52.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22676]: starting fuse 2022-08-01T23:02:53.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:02:53.907 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:02:53.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:02:53.967 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:02:53.967 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:02:53.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:53.969 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:02:53.974 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:02:53.975 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:02:53.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:53.976 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:02:53.976 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:02:53.977 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:02:53.977 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:02:53.978 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:02:53.978 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:02:53.979 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:02:53.979 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:02:53.979 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:02:53.980 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:02:53.980 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:02:53.981 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:02:53.981 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:02:53.982 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:02:53.982 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:02:53.983 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:02:53.983 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:02:53.983 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:02:53.984 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:02:53.984 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:02:53.985 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:02:53.985 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:02:53.986 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:02:54.030 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.22676.asok 2022-08-01T23:02:54.034 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22676.asok 2022-08-01T23:02:54.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:54.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22676.asok status 2022-08-01T23:02:54.169 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:02:54.169 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:02:54.169 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:02:54.170 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:02:54.170 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:02:54.170 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:02:54.170 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:02:54.171 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "22676", 2022-08-01T23:02:54.171 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:02:54.171 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:02:54.172 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:02:54.172 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:02:54.172 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6134, 2022-08-01T23:02:54.172 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:02:54.173 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:02:54.173 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:02:54.173 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6134 2022-08-01T23:02:54.173 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:02:54.174 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:02:54.174 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:02:54.174 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:02:54.174 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2254014482 2022-08-01T23:02:54.175 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:02:54.175 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:02:54.175 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:02:54.175 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:02:54.176 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:02:54.176 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2254014482 2022-08-01T23:02:54.176 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:02:54.176 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6134 172.21.15.157:0/2254014482", 2022-08-01T23:02:54.177 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/2254014482", 2022-08-01T23:02:54.177 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:02:54.177 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 39, 2022-08-01T23:02:54.177 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 87, 2022-08-01T23:02:54.178 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:02:54.178 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:02:54.178 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:02:54.178 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:02:54.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:54.180 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:54.181 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:54.188 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:02:54.189 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:02:54.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:54.190 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:54.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 Processor -- start 2022-08-01T23:02:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 -- start start 2022-08-01T23:02:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14fc0aa2b0 0x7f14fc0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc0b2c80 con 0x7f14fc0b0a10 2022-08-01T23:02:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14fc0b2900 con 0x7f14fc0aa2b0 2022-08-01T23:02:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f150145a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0b2eb0 con 0x7f14fc0b0630 2022-08-01T23:02:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f14fa7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14fc0aa2b0 0x7f14fc0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f14fa7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14fc0aa2b0 0x7f14fc0ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45876/0 (socket says 172.21.15.32:45876) 2022-08-01T23:02:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f14fa7fc700 1 -- 172.21.15.32:0/558878236 learned_addr learned my addr 172.21.15.32:0/558878236 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f14fb7fe700 1 --1- 172.21.15.32:0/558878236 >> v1:172.21.15.55:6789/0 conn(0x7f14fc0b0a10 0x7f14fc1438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:54326/0 (socket says 172.21.15.32:54326) 2022-08-01T23:02:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.359+0000 7f14faffd700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14f9ffb700 1 -- 172.21.15.32:0/558878236 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812116319 0 0) 0x7f14fc0b2c80 con 0x7f14fc0b0a10 2022-08-01T23:02:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14f9ffb700 1 -- 172.21.15.32:0/558878236 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e0003260 con 0x7f14fc0b0a10 2022-08-01T23:02:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14fa7fc700 1 -- 172.21.15.32:0/558878236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 msgr2=0x7f14fc0adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14fa7fc700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc0adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14fa7fc700 1 -- 172.21.15.32:0/558878236 >> v1:172.21.15.55:6789/0 conn(0x7f14fc0b0a10 legacy=0x7f14fc1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14fa7fc700 1 -- 172.21.15.32:0/558878236 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc0a9d00 con 0x7f14fc0aa2b0 2022-08-01T23:02:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14faffd700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc0adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:02:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14fa7fc700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0ae3b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f14f0002f70 tx=0x7f14f0006f40).ready entity=mon.0 client_cookie=184556757f497903 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14f9ffb700 1 -- 172.21.15.32:0/558878236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14f000d9a0 con 0x7f14fc0aa2b0 2022-08-01T23:02:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14f9ffb700 1 -- 172.21.15.32:0/558878236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0000e90 con 0x7f14fc0aa2b0 2022-08-01T23:02:54.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f14f9ffb700 1 -- 172.21.15.32:0/558878236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14f000dd30 con 0x7f14fc0aa2b0 2022-08-01T23:02:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 -- 172.21.15.32:0/558878236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0aa2b0 msgr2=0x7f14fc0ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0ae3b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f14f0002f70 tx=0x7f14f0006f40).stop 2022-08-01T23:02:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 -- 172.21.15.32:0/558878236 shutdown_connections 2022-08-01T23:02:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0ae3b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 --2- 172.21.15.32:0/558878236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc0adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 -- 172.21.15.32:0/558878236 >> 172.21.15.32:0/558878236 conn(0x7f14fc060950 msgr2=0x7f14fc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 -- 172.21.15.32:0/558878236 shutdown_connections 2022-08-01T23:02:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.363+0000 7f150145a700 1 -- 172.21.15.32:0/558878236 wait complete. 2022-08-01T23:02:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 Processor -- start 2022-08-01T23:02:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 -- start start 2022-08-01T23:02:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc13e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 0x7f14fc0aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0ac6c0 con 0x7f14fc0b0a10 2022-08-01T23:02:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0ac130 con 0x7f14fc0aa2b0 2022-08-01T23:02:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc144ed0 con 0x7f14fc0b0630 2022-08-01T23:02:54.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 0x7f14fc0aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:54.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 0x7f14fc0aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45884/0 (socket says 172.21.15.32:45884) 2022-08-01T23:02:54.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 -- 172.21.15.32:0/1675303371 learned_addr learned my addr 172.21.15.32:0/1675303371 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:54.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14faffd700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fa7fc700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc13e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 -- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 msgr2=0x7f14fc13e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc13e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 -- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14fc0aa2b0 msgr2=0x7f14fc0a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 -- 172.21.15.32:0/1675303371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec00b180 con 0x7f14fc0b0a10 2022-08-01T23:02:54.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fa7fc700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:02:54.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14fb7fe700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 0x7f14fc0aa6d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f14f4001490 tx=0x7f14f4007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14f4005bb0 con 0x7f14fc0b0a10 2022-08-01T23:02:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f4005d10 con 0x7f14fc0b0a10 2022-08-01T23:02:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f000b460 con 0x7f14fc0b0a10 2022-08-01T23:02:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14f40072c0 con 0x7f14fc0b0a10 2022-08-01T23:02:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.367+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc145130 con 0x7f14fc0b0a10 2022-08-01T23:02:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.371+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14fc0574e0 con 0x7f14fc0b0a10 2022-08-01T23:02:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.371+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f14f4010070 con 0x7f14fc0b0a10 2022-08-01T23:02:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.371+0000 7f14ebfff700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14d80539e0 0x7f14d8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.371+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f14f406ba60 con 0x7f14fc0b0a10 2022-08-01T23:02:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.371+0000 7f14faffd700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14d80539e0 0x7f14d8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:54.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.371+0000 7f14faffd700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14d80539e0 0x7f14d8055ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f14ec00b000 tx=0x7f14ec006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:54.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.375+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14f4042780 con 0x7f14fc0b0a10 2022-08-01T23:02:54.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:54.433 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:54.435 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:54.438 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:02:54.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.543+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14fc0447e0 con 0x7f14fc0b0a10 2022-08-01T23:02:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f14ebfff700 1 -- 172.21.15.32:0/1675303371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f14f4045da0 con 0x7f14fc0b0a10 2022-08-01T23:02:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:54.553 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:44.973242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6666":{"gid":6666,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4046550885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4046550885},{"type":"v1","addr":"172.21.15.32:6835","nonce":4046550885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14d80539e0 msgr2=0x7f14d8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14d80539e0 0x7f14d8055ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f14ec00b000 tx=0x7f14ec006df0).stop 2022-08-01T23:02:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 msgr2=0x7f14fc0aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 0x7f14fc0aa6d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f14f4001490 tx=0x7f14f4007de0).stop 2022-08-01T23:02:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 shutdown_connections 2022-08-01T23:02:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14d80539e0 0x7f14d8055ea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc0b0a10 0x7f14fc0aa6d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14fc0b0630 0x7f14fc13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 --2- 172.21.15.32:0/1675303371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14fc0aa2b0 0x7f14fc0a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:54.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 >> 172.21.15.32:0/1675303371 conn(0x7f14fc060950 msgr2=0x7f14fc0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:54.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 shutdown_connections 2022-08-01T23:02:54.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:54.547+0000 7f150145a700 1 -- 172.21.15.32:0/1675303371 wait complete. 2022-08-01T23:02:54.558 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2022-08-01T23:02:54.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2022-08-01T23:02:54.670 INFO:teuthology.orchestra.run.smithi032.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2022-08-01T23:02:54.244506+0000","btime":"2022-08-01T23:02:43.973761+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-08-01T23:02:43.973761+0000","atime":"0.000000","time_warp_seq":0,"change_attr":2,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-08-01T23:02:54.244506+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-08-01T23:02:43.973761+0000"},"version":4,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6104,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":6134,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2022-08-01T23:02:54.671 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2022-08-01T23:02:43.973761+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 3, "btime": "2022-08-01T23:02:43.973761+0000", "change_attr": 2, "client_caps": [ { "client_id": 6104, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 6134, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-08-01T23:02:54.244506+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2022-08-01T23:02:43.973761+0000", "nestlock": {}, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2022-08-01T23:02:54.244506+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2022-08-01T23:02:54.671 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:02:54.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:54.691 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:54.712 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:54.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:02:54.754 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:02:54.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:54.755 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:54.756 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:54.783 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:02:54.784 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:02:54.784 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:02:54.785 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:02:54.785 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:54.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23141]: fuse finished with error 0 and tester_r 0 2022-08-01T23:02:54.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:54.803 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:54.803 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:54.810 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:02:54.812 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:02:54.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:02:54.930 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:02:54.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:54.937 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:54.937 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:54.945 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:02:54.947 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:02:54.947 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:02:54.948 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:02:54.948 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:55.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22676]: fuse finished with error 0 and tester_r 0 2022-08-01T23:02:55.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:02:55.005 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:02:55.005 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:55.013 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:02:55.014 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:02:55.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:02:55.133 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:02:55.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:02:55.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 Processor -- start 2022-08-01T23:02:55.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 -- start start 2022-08-01T23:02:55.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b2710 0x7fb75414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7540a8c50 0x7fb75414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7540a7250 con 0x7fb7540b01a0 2022-08-01T23:02:55.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7540a7480 con 0x7fb7540a8c50 2022-08-01T23:02:55.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb759e7e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540a7720 con 0x7fb7540b2710 2022-08-01T23:02:55.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7540a8c50 0x7fb75414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7540a8c50 0x7fb75414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45892/0 (socket says 172.21.15.32:45892) 2022-08-01T23:02:55.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 -- 172.21.15.32:0/3624181665 learned_addr learned my addr 172.21.15.32:0/3624181665 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:55.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb758e7c700 1 --2- 172.21.15.32:0/3624181665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b2710 0x7fb75414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb7537fe700 1 -- 172.21.15.32:0/3624181665 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751322933 0 0) 0x7fb7540a7250 con 0x7fb7540b01a0 2022-08-01T23:02:55.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb7537fe700 1 -- 172.21.15.32:0/3624181665 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb740003260 con 0x7fb7540b01a0 2022-08-01T23:02:55.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 -- 172.21.15.32:0/3624181665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b2710 msgr2=0x7fb75414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:55.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 --2- 172.21.15.32:0/3624181665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b2710 0x7fb75414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 -- 172.21.15.32:0/3624181665 >> v1:172.21.15.55:6789/0 conn(0x7fb7540b01a0 legacy=0x7fb7540a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 -- 172.21.15.32:0/3624181665 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7540a79c0 con 0x7fb7540a8c50 2022-08-01T23:02:55.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.247+0000 7fb75967d700 1 --2- 172.21.15.32:0/3624181665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb75414b4c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fb74c0032f0 tx=0x7fb74c009290).ready entity=mon.0 client_cookie=c554d91bf5a370ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:55.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb7537fe700 1 -- 172.21.15.32:0/3624181665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb74c00b6c0 con 0x7fb7540a8c50 2022-08-01T23:02:55.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb7537fe700 1 -- 172.21.15.32:0/3624181665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c00b820 con 0x7fb7540a8c50 2022-08-01T23:02:55.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb7537fe700 1 -- 172.21.15.32:0/3624181665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb74c00b990 con 0x7fb7540a8c50 2022-08-01T23:02:55.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 -- 172.21.15.32:0/3624181665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 msgr2=0x7fb75414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 --2- 172.21.15.32:0/3624181665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb75414b4c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7fb74c0032f0 tx=0x7fb74c009290).stop 2022-08-01T23:02:55.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 -- 172.21.15.32:0/3624181665 shutdown_connections 2022-08-01T23:02:55.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 --2- 172.21.15.32:0/3624181665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb75414b4c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 --2- 172.21.15.32:0/3624181665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b2710 0x7fb75414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 -- 172.21.15.32:0/3624181665 >> 172.21.15.32:0/3624181665 conn(0x7fb754060310 msgr2=0x7fb7540b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:55.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 -- 172.21.15.32:0/3624181665 shutdown_connections 2022-08-01T23:02:55.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.251+0000 7fb759e7e700 1 -- 172.21.15.32:0/3624181665 wait complete. 2022-08-01T23:02:55.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 Processor -- start 2022-08-01T23:02:55.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 -- start start 2022-08-01T23:02:55.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb754143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b01a0 0x7fb7540b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7540b2710 0x7fb754159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75414d120 con 0x7fb7540a8c50 2022-08-01T23:02:55.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75414d350 con 0x7fb7540b2710 2022-08-01T23:02:55.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75414d5f0 con 0x7fb7540b01a0 2022-08-01T23:02:55.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb754143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb754143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45896/0 (socket says 172.21.15.32:45896) 2022-08-01T23:02:55.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 -- 172.21.15.32:0/829830045 learned_addr learned my addr 172.21.15.32:0/829830045 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:55.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb75967d700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7540b2710 0x7fb754159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb753fff700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b01a0 0x7fb7540b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 -- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b01a0 msgr2=0x7fb7540b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b01a0 0x7fb7540b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 -- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7540b2710 msgr2=0x7fb754159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7540b2710 0x7fb754159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 -- 172.21.15.32:0/829830045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c009530 con 0x7fb7540a8c50 2022-08-01T23:02:55.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb758e7c700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb754143e30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fb74800c000 tx=0x7fb748006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:55.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb748011750 con 0x7fb7540a8c50 2022-08-01T23:02:55.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7480118b0 con 0x7fb7540a8c50 2022-08-01T23:02:55.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb748011c10 con 0x7fb7540a8c50 2022-08-01T23:02:55.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7540a9ca0 con 0x7fb7540a8c50 2022-08-01T23:02:55.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.255+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7540b3890 con 0x7fb7540a8c50 2022-08-01T23:02:55.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.259+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75414d7b0 con 0x7fb7540a8c50 2022-08-01T23:02:55.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.259+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb748011d70 con 0x7fb7540a8c50 2022-08-01T23:02:55.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.259+0000 7fb751ffb700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb734053b30 0x7fb734055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.259+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb74806c520 con 0x7fb7540a8c50 2022-08-01T23:02:55.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.259+0000 7fb753fff700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb734053b30 0x7fb734055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.259+0000 7fb753fff700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb734053b30 0x7fb734055ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fb74400d000 tx=0x7fb744007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:55.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.267+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb748043170 con 0x7fb7540a8c50 2022-08-01T23:02:55.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.395+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb7540574e0 con 0x7fb7540a8c50 2022-08-01T23:02:55.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb751ffb700 1 -- 172.21.15.32:0/829830045 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v87) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fb748046790 con 0x7fb7540a8c50 2022-08-01T23:02:55.406 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:55.406 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:02:55.406 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 87, 2022-08-01T23:02:55.407 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:02:55.407 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:02:55.407 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:02:44.972186+0000", 2022-08-01T23:02:55.408 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:02:55.408 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:02:55.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:02:55.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:02:55.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:02:55.409 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:02:55.409 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:02:55.410 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:02:55.410 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:02:55.410 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:55.410 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:02:55.411 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:02:55.411 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:02:55.411 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:02:55.411 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:02:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 16, 2022-08-01T23:02:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:02:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:02:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:02:55.413 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:02:55.413 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:02:55.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.414 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:02:55.414 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:02:55.414 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:02:55.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:55.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:55.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:55.415 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:55.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:55.416 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:55.416 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:55.417 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:55.417 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:02:55.418 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:02:55.418 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:02:55.419 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:02:55.419 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:02:55.420 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:55.420 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:55.421 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:55.421 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:55.422 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:55.422 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:55.423 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:55.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:02:55.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:55.425 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:55.425 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:55.426 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:55.426 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:55.427 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:55.427 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:55.428 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:55.428 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:55.429 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:55.429 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:55.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:55.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:55.431 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:55.432 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:55.432 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:55.433 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:55.433 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:55.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:55.434 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:55.435 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:55.435 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:55.436 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:55.436 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:55.437 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:55.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:55.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.438 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:55.439 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:55.440 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:55.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:55.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:55.441 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:55.442 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:55.442 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:55.443 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:55.443 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:55.444 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:55.444 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:02:55.445 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:02:55.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.446 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:55.446 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:02:55.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.448 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:02:55.449 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:02:55.449 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:02:55.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:55.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:55.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:55.451 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:55.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:55.452 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:55.453 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:55.453 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:55.454 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:55.454 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:55.455 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:55.456 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:55.456 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:55.456 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:55.457 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:55.457 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:55.458 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:55.458 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:55.459 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:55.460 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:55.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.460 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:02:55.461 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:55.461 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:55.462 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:55.462 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:55.463 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:55.463 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:55.464 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:55.464 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:55.465 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:55.465 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:55.466 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:55.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:55.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:55.468 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:55.468 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:55.469 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:55.469 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:55.470 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:55.470 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:55.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:55.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:55.472 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:55.472 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:55.473 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:55.473 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:55.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:55.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.475 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:55.475 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:55.476 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:55.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:55.477 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:55.477 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:55.477 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:55.478 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:55.478 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:55.478 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:55.479 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:55.479 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:02:55.479 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:55.479 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:02:55.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.480 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 15, 2022-08-01T23:02:55.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:02:55.481 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:02:42.386170+0000", 2022-08-01T23:02:55.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:55.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:55.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:55.482 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:55.482 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:55.482 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:55.483 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:55.483 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:55.483 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:55.483 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:55.484 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:55.484 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:55.484 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:55.484 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:55.485 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:55.485 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:55.485 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:55.485 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:55.485 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:55.486 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:55.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.486 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "86", 2022-08-01T23:02:55.486 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:55.487 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:55.487 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:55.487 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:55.487 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:55.488 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:55.488 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:55.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:55.488 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:55.489 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:55.489 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:55.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:55.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:55.490 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:55.490 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:55.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:55.490 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:55.491 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:55.491 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:55.491 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:55.491 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:55.492 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:55.492 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:55.492 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:55.492 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:55.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:55.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.493 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:55.493 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:55.494 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:55.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:55.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:55.494 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:55.495 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:55.495 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:55.495 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:55.495 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:55.496 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:55.496 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:02:55.496 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:02:55.496 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:02:55.497 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:02:55.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.497 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:55.497 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:02:55.497 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:02:55.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.499 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 16, 2022-08-01T23:02:55.499 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:02:55.499 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:02:43.649622+0000", 2022-08-01T23:02:55.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:02:55.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:02:55.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:02:55.500 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:02:55.500 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:02:55.501 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:02:55.501 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:02:55.501 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:02:55.501 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:02:55.502 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:02:55.502 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:02:55.502 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:02:55.502 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:02:55.503 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:02:55.503 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:02:55.503 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:02:55.503 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:02:55.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:02:55.504 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:02:55.504 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:02:55.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.505 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "87", 2022-08-01T23:02:55.505 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:02:55.505 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:02:55.505 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:02:55.506 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:02:55.506 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:02:55.506 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:02:55.506 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:02:55.507 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:02:55.507 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:02:55.507 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:02:55.507 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:02:55.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:02:55.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:02:55.508 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:02:55.508 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:02:55.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:02:55.509 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:02:55.509 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:02:55.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:02:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:02:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:02:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:02:55.510 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:02:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:02:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:02:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:02:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:02:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:02:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:02:55.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:02:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:02:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:02:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:02:55.513 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:02:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:02:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:02:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:02:55.514 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:02:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:02:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:02:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:02:55.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:55.516 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:02:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:02:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:02:55.517 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.518 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:02:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.519 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:02:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:02:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.523 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:02:55.524 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:02:55.525 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.526 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:02:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:02:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:02:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:02:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:02:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:02:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:02:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:02:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:02:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:02:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:02:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 84, 2022-08-01T23:02:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:02:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:02:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:02:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:02:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:02:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:02:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:02:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:02:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:02:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:02:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:02:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:02:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:02:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:02:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:02:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:02:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:02:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:02:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:02:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:02:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:02:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:02:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:02:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:02:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:02:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:02:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:02:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:02:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:02:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:02:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:02:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:02:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:02:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:02:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:02:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:02:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:02:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:02:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:02:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:02:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:02:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:02:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:02:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:02:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:02:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:02:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:02:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:02:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:02:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:02:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:02:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:02:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:02:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:02:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:02:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:02:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:02:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:02:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:02:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:02:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:02:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:02:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:02:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:02:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:02:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:02:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:02:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:02:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:02:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:02:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:02:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:02:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:02:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:02:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:02:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:02:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:02:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:02:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:02:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:02:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:02:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:02:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:02:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:02:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:02:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:02:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:02:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:02:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:02:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:02:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:02:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:02:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:02:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:02:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:02:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:02:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:02:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:02:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:02:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:02:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:02:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:02:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 73, 2022-08-01T23:02:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:02:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:02:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:02:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:02:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:02:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:02:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:02:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:02:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:02:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:02:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:02:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:02:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:02:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:02:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:02:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:02:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:02:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:02:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:02:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:02:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:02:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:02:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:02:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:02:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:02:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:02:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:02:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:02:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:02:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:02:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:02:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:02:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:02:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:02:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:02:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:02:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:02:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:02:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:02:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:02:55.685 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.685 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.685 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.685 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.686 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.686 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:02:55.686 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:02:55.687 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:02:55.687 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:02:55.687 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:02:55.687 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:02:55.688 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:02:55.688 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:02:55.688 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:02:55.688 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:02:55.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.689 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:02:55.689 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:02:55.689 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:02:55.690 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:02:55.690 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:02:55.690 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:02:55.690 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:02:55.691 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:02:55.691 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:02:55.691 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:02:55.691 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:02:55.691 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:02:55.692 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:02:55.692 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:02:55.692 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:02:55.692 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:02:55.693 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:02:55.693 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:02:55.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:02:55.694 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:02:55.694 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:02:55.694 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:02:55.694 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:02:55.695 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:02:55.695 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:02:55.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:02:55.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:02:55.696 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:02:55.696 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:02:55.696 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:02:55.696 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:02:55.697 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:02:55.697 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:02:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb734053b30 msgr2=0x7fb734055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb734053b30 0x7fb734055ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fb74400d000 tx=0x7fb744007a00).stop 2022-08-01T23:02:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 msgr2=0x7fb754143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb754143e30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fb74800c000 tx=0x7fb748006df0).stop 2022-08-01T23:02:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 shutdown_connections 2022-08-01T23:02:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb734053b30 0x7fb734055ff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7540b2710 0x7fb754159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb7540b01a0 0x7fb7540b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 --2- 172.21.15.32:0/829830045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7540a8c50 0x7fb754143e30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 >> 172.21.15.32:0/829830045 conn(0x7fb754060310 msgr2=0x7fb7541460c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 shutdown_connections 2022-08-01T23:02:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.399+0000 7fb759e7e700 1 -- 172.21.15.32:0/829830045 wait complete. 2022-08-01T23:02:55.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 Processor -- start 2022-08-01T23:02:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- start start 2022-08-01T23:02:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8940b0910 0x7fc8940a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc8940b1ed0 0x7fc8940a9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8940b3440 con 0x7fc8940b26c0 2022-08-01T23:02:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89414b660 con 0x7fc8940b0910 2022-08-01T23:02:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89414b900 con 0x7fc8940b1ed0 2022-08-01T23:02:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8940b0910 0x7fc8940a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8940b0910 0x7fc8940a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45902/0 (socket says 172.21.15.32:45902) 2022-08-01T23:02:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 -- 172.21.15.32:0/2935189244 learned_addr learned my addr 172.21.15.32:0/2935189244 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8937fe700 1 --2- 172.21.15.32:0/2935189244 >> v2:172.21.15.55:3300/0 conn(0x7fc8940b1ed0 0x7fc8940a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 -- 172.21.15.32:0/2935189244 >> v2:172.21.15.55:3300/0 conn(0x7fc8940b1ed0 msgr2=0x7fc8940a9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 --2- 172.21.15.32:0/2935189244 >> v2:172.21.15.55:3300/0 conn(0x7fc8940b1ed0 0x7fc8940a9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 -- 172.21.15.32:0/2935189244 >> v1:172.21.15.55:6789/0 conn(0x7fc8940b26c0 legacy=0x7fc894147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 -- 172.21.15.32:0/2935189244 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894147500 con 0x7fc8940b0910 2022-08-01T23:02:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc8927fc700 1 --2- 172.21.15.32:0/2935189244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940a8d10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fc89414bbd0 tx=0x7fc884009290).ready entity=mon.0 client_cookie=3f6cde888b5932f6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc891ffb700 1 -- 172.21.15.32:0/2935189244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc88400b860 con 0x7fc8940b0910 2022-08-01T23:02:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc891ffb700 1 -- 172.21.15.32:0/2935189244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88400b9c0 con 0x7fc8940b0910 2022-08-01T23:02:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc891ffb700 1 -- 172.21.15.32:0/2935189244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc88400bb30 con 0x7fc8940b0910 2022-08-01T23:02:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- 172.21.15.32:0/2935189244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 msgr2=0x7fc8940a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 --2- 172.21.15.32:0/2935189244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940a8d10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fc89414bbd0 tx=0x7fc884009290).stop 2022-08-01T23:02:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- 172.21.15.32:0/2935189244 shutdown_connections 2022-08-01T23:02:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 --2- 172.21.15.32:0/2935189244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940a8d10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 --2- 172.21.15.32:0/2935189244 >> v2:172.21.15.55:3300/0 conn(0x7fc8940b1ed0 0x7fc8940a9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- 172.21.15.32:0/2935189244 >> 172.21.15.32:0/2935189244 conn(0x7fc894061360 msgr2=0x7fc894062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- 172.21.15.32:0/2935189244 shutdown_connections 2022-08-01T23:02:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.819+0000 7fc893fff700 1 -- 172.21.15.32:0/2935189244 wait complete. 2022-08-01T23:02:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 Processor -- start 2022-08-01T23:02:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 -- start start 2022-08-01T23:02:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc8940b1ed0 0x7fc8941570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8940b26c0 0x7fc8941575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894156aa0 con 0x7fc8940b0910 2022-08-01T23:02:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940ab690 con 0x7fc8940b26c0 2022-08-01T23:02:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc893fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc894156c90 con 0x7fc8940b1ed0 2022-08-01T23:02:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45908/0 (socket says 172.21.15.32:45908) 2022-08-01T23:02:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 -- 172.21.15.32:0/890318036 learned_addr learned my addr 172.21.15.32:0/890318036 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc8927fc700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc8940b1ed0 0x7fc8941570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc8937fe700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8940b26c0 0x7fc8941575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 -- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc8940b1ed0 msgr2=0x7fc8941570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc8940b1ed0 0x7fc8941570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 -- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8940b26c0 msgr2=0x7fc8941575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8940b26c0 0x7fc8941575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 -- 172.21.15.32:0/890318036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009530 con 0x7fc8940b0910 2022-08-01T23:02:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.823+0000 7fc892ffd700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940ae3e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fc894146d40 tx=0x7fc888007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc888007e70 con 0x7fc8940b0910 2022-08-01T23:02:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888007030 con 0x7fc8940b0910 2022-08-01T23:02:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940ab3f0 con 0x7fc8940b0910 2022-08-01T23:02:55.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc888007390 con 0x7fc8940b0910 2022-08-01T23:02:55.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8940650e0 con 0x7fc8940b0910 2022-08-01T23:02:55.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8941477a0 con 0x7fc8940b0910 2022-08-01T23:02:55.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc888010030 con 0x7fc8940b0910 2022-08-01T23:02:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc883fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc870053a70 0x7fc870055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc88806bab0 con 0x7fc8940b0910 2022-08-01T23:02:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.827+0000 7fc8927fc700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc870053a70 0x7fc870055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:55.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.831+0000 7fc8927fc700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc870053a70 0x7fc870055f30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fc88401b000 tx=0x7fc884016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:55.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:55.835+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc888042870 con 0x7fc8940b0910 2022-08-01T23:02:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.003+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8940574e0 con 0x7fc8940b0910 2022-08-01T23:02:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.003+0000 7fc883fff700 1 -- 172.21.15.32:0/890318036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fc888045e90 con 0x7fc8940b0910 2022-08-01T23:02:56.011 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:56.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:44.973242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6666":{"gid":6666,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/4046550885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4046550885},{"type":"v1","addr":"172.21.15.32:6835","nonce":4046550885}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc870053a70 msgr2=0x7fc870055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc870053a70 0x7fc870055f30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fc88401b000 tx=0x7fc884016000).stop 2022-08-01T23:02:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 msgr2=0x7fc8940ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940ae3e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fc894146d40 tx=0x7fc888007c00).stop 2022-08-01T23:02:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 shutdown_connections 2022-08-01T23:02:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc870053a70 0x7fc870055f30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8940b26c0 0x7fc8941575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc8940b1ed0 0x7fc8941570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 --2- 172.21.15.32:0/890318036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8940b0910 0x7fc8940ae3e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 >> 172.21.15.32:0/890318036 conn(0x7fc894061360 msgr2=0x7fc8940af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:56.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 shutdown_connections 2022-08-01T23:02:56.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.007+0000 7fc893fff700 1 -- 172.21.15.32:0/890318036 wait complete. 2022-08-01T23:02:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2022-08-01T23:02:56.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:02:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 Processor -- start 2022-08-01T23:02:56.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 -- start start 2022-08-01T23:02:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b5630 0x7f6b800af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6b800b07a0 0x7f6b80147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b80067730 0x7f6b800acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b80145470 con 0x7f6b80067730 2022-08-01T23:02:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800ae180 con 0x7f6b800b07a0 2022-08-01T23:02:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800b5a10 con 0x7f6b800b5630 2022-08-01T23:02:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b80067730 0x7f6b800acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b80067730 0x7f6b800acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45916/0 (socket says 172.21.15.32:45916) 2022-08-01T23:02:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 -- 172.21.15.32:0/2174204131 learned_addr learned my addr 172.21.15.32:0/2174204131 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7e7fc700 1 --2- 172.21.15.32:0/2174204131 >> v2:172.21.15.55:3300/0 conn(0x7f6b800b07a0 0x7f6b80147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7effd700 1 --2- 172.21.15.32:0/2174204131 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b5630 0x7f6b800af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 -- 172.21.15.32:0/2174204131 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b5630 msgr2=0x7f6b800af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 --2- 172.21.15.32:0/2174204131 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b5630 0x7f6b800af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 -- 172.21.15.32:0/2174204131 >> v2:172.21.15.55:3300/0 conn(0x7f6b800b07a0 msgr2=0x7f6b80147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 --2- 172.21.15.32:0/2174204131 >> v2:172.21.15.55:3300/0 conn(0x7f6b800b07a0 0x7f6b80147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 -- 172.21.15.32:0/2174204131 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800ac990 con 0x7f6b80067730 2022-08-01T23:02:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7f7fe700 1 --2- 172.21.15.32:0/2174204131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b800acdf0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f6b78002f70 tx=0x7f6b78006eb0).ready entity=mon.0 client_cookie=2811f48c455500de server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7dffb700 1 -- 172.21.15.32:0/2174204131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7800d570 con 0x7f6b80067730 2022-08-01T23:02:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7dffb700 1 -- 172.21.15.32:0/2174204131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b78000e90 con 0x7f6b80067730 2022-08-01T23:02:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.123+0000 7f6b7dffb700 1 -- 172.21.15.32:0/2174204131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7800d900 con 0x7f6b80067730 2022-08-01T23:02:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2174204131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 msgr2=0x7f6b800acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2174204131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b800acdf0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f6b78002f70 tx=0x7f6b78006eb0).stop 2022-08-01T23:02:56.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2174204131 shutdown_connections 2022-08-01T23:02:56.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2174204131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b800acdf0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2174204131 >> v2:172.21.15.55:3300/0 conn(0x7f6b800b07a0 0x7f6b80147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2174204131 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b5630 0x7f6b800af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2174204131 >> 172.21.15.32:0/2174204131 conn(0x7f6b800626f0 msgr2=0x7f6b80062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2174204131 shutdown_connections 2022-08-01T23:02:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2174204131 wait complete. 2022-08-01T23:02:56.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 Processor -- start 2022-08-01T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- start start 2022-08-01T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b801475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b07a0 0x7f6b80146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b800b5630 0x7f6b80156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80065650 con 0x7f6b80067730 2022-08-01T23:02:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80066b70 con 0x7f6b800b5630 2022-08-01T23:02:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80065e70 con 0x7f6b800b07a0 2022-08-01T23:02:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b801475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b801475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45926/0 (socket says 172.21.15.32:45926) 2022-08-01T23:02:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7effd700 1 -- 172.21.15.32:0/2958076252 learned_addr learned my addr 172.21.15.32:0/2958076252 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7f7fe700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b800b5630 0x7f6b80156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.127+0000 7f6b7e7fc700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b07a0 0x7f6b80146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7effd700 1 -- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b07a0 msgr2=0x7f6b80146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7effd700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b07a0 0x7f6b80146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7effd700 1 -- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b800b5630 msgr2=0x7f6b80156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:56.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7effd700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b800b5630 0x7f6b80156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7effd700 1 -- 172.21.15.32:0/2958076252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7800b1e0 con 0x7f6b80067730 2022-08-01T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7effd700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b801475a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f6b7400c000 tx=0x7f6b74006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b74010780 con 0x7f6b80067730 2022-08-01T23:02:56.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b740108e0 con 0x7f6b80067730 2022-08-01T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b74010c40 con 0x7f6b80067730 2022-08-01T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b80067e30 con 0x7f6b80067730 2022-08-01T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b80066340 con 0x7f6b80067730 2022-08-01T23:02:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b80146fe0 con 0x7f6b80067730 2022-08-01T23:02:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.131+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b7401c460 con 0x7f6b80067730 2022-08-01T23:02:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.135+0000 7f6b6ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b5c053a70 0x7f6b5c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.135+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6b7406c400 con 0x7f6b80067730 2022-08-01T23:02:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.135+0000 7f6b7e7fc700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b5c053a70 0x7f6b5c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.135+0000 7f6b7e7fc700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b5c053a70 0x7f6b5c055f30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f6b7000b000 tx=0x7f6b700091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.139+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b740430c0 con 0x7f6b80067730 2022-08-01T23:02:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:56.299+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b800b1680 con 0x7f6b80067730 2022-08-01T23:02:57.205 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:02:57.199+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:02:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.207+0000 7f6b6ffff700 1 -- 172.21.15.32:0/2958076252 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b740466e0 con 0x7f6b80067730 2022-08-01T23:02:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.207+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b5c053a70 msgr2=0x7f6b5c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.207+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b5c053a70 0x7f6b5c055f30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f6b7000b000 tx=0x7f6b700091d0).stop 2022-08-01T23:02:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.207+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 msgr2=0x7f6b801475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.207+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b801475a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f6b7400c000 tx=0x7f6b74006df0).stop 2022-08-01T23:02:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 shutdown_connections 2022-08-01T23:02:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b5c053a70 0x7f6b5c055f30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b800b5630 0x7f6b80156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b800b07a0 0x7f6b80146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 --2- 172.21.15.32:0/2958076252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b80067730 0x7f6b801475a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 >> 172.21.15.32:0/2958076252 conn(0x7f6b800626f0 msgr2=0x7f6b80148990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 shutdown_connections 2022-08-01T23:02:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.211+0000 7f6b7ffff700 1 -- 172.21.15.32:0/2958076252 wait complete. 2022-08-01T23:02:57.220 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:02:57.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:02:57.239 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:02:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.331+0000 7f4292cff700 1 Processor -- start 2022-08-01T23:02:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.331+0000 7f4292cff700 1 -- start start 2022-08-01T23:02:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f428c0b68f0 0x7f428c13ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f428c05ff40 0x7f428c146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f428c13d860 con 0x7f428c05ff40 2022-08-01T23:02:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f428c0ac6e0 con 0x7f428c0b68f0 2022-08-01T23:02:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c0ac980 con 0x7f428c0aa210 2022-08-01T23:02:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f428c05ff40 0x7f428c146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f428c05ff40 0x7f428c146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45946/0 (socket says 172.21.15.32:45946) 2022-08-01T23:02:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 -- 172.21.15.32:0/3994048748 learned_addr learned my addr 172.21.15.32:0/3994048748 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f428bfff700 1 --2- 172.21.15.32:0/3994048748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4290a9b700 1 --2- 172.21.15.32:0/3994048748 >> v2:172.21.15.55:3300/0 conn(0x7f428c0b68f0 0x7f428c13ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 -- 172.21.15.32:0/3994048748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 msgr2=0x7f428c13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 --2- 172.21.15.32:0/3994048748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 -- 172.21.15.32:0/3994048748 >> v2:172.21.15.55:3300/0 conn(0x7f428c0b68f0 msgr2=0x7f428c13ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 --2- 172.21.15.32:0/3994048748 >> v2:172.21.15.55:3300/0 conn(0x7f428c0b68f0 0x7f428c13ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 -- 172.21.15.32:0/3994048748 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c0acc20 con 0x7f428c05ff40 2022-08-01T23:02:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f429129c700 1 --2- 172.21.15.32:0/3994048748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c146970 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f42840032f0 tx=0x7f4284009290).ready entity=mon.0 client_cookie=b7ea5fa418b35b3f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:57.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f428b7fe700 1 -- 172.21.15.32:0/3994048748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f428400b680 con 0x7f428c05ff40 2022-08-01T23:02:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f428b7fe700 1 -- 172.21.15.32:0/3994048748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428400b7e0 con 0x7f428c05ff40 2022-08-01T23:02:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f428b7fe700 1 -- 172.21.15.32:0/3994048748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f428400b950 con 0x7f428c05ff40 2022-08-01T23:02:57.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- 172.21.15.32:0/3994048748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 msgr2=0x7f428c146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- 172.21.15.32:0/3994048748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c146970 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f42840032f0 tx=0x7f4284009290).stop 2022-08-01T23:02:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- 172.21.15.32:0/3994048748 shutdown_connections 2022-08-01T23:02:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- 172.21.15.32:0/3994048748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c146970 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- 172.21.15.32:0/3994048748 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 --2- 172.21.15.32:0/3994048748 >> v2:172.21.15.55:3300/0 conn(0x7f428c0b68f0 0x7f428c13ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- 172.21.15.32:0/3994048748 >> 172.21.15.32:0/3994048748 conn(0x7f428c0b3610 msgr2=0x7f428c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- 172.21.15.32:0/3994048748 shutdown_connections 2022-08-01T23:02:57.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.335+0000 7f4292cff700 1 -- 172.21.15.32:0/3994048748 wait complete. 2022-08-01T23:02:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 Processor -- start 2022-08-01T23:02:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 -- start start 2022-08-01T23:02:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c0b68f0 0x7f428c1400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c0b26f0 con 0x7f428c05ff40 2022-08-01T23:02:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c1458a0 con 0x7f428c0b68f0 2022-08-01T23:02:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c145b40 con 0x7f428c0aa210 2022-08-01T23:02:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c0b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45950/0 (socket says 172.21.15.32:45950) 2022-08-01T23:02:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 -- 172.21.15.32:0/3878294953 learned_addr learned my addr 172.21.15.32:0/3878294953 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f428bfff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f429129c700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c0b68f0 0x7f428c1400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 -- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 msgr2=0x7f428c13fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 -- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c0b68f0 msgr2=0x7f428c1400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c0b68f0 0x7f428c1400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 -- 172.21.15.32:0/3878294953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009530 con 0x7f428c05ff40 2022-08-01T23:02:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4290a9b700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c0b50e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f428000c000 tx=0x7f4280006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4280010780 con 0x7f428c05ff40 2022-08-01T23:02:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42800108e0 con 0x7f428c05ff40 2022-08-01T23:02:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c0652c0 con 0x7f428c05ff40 2022-08-01T23:02:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.339+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4280010c40 con 0x7f428c05ff40 2022-08-01T23:02:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.343+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c144940 con 0x7f428c05ff40 2022-08-01T23:02:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.343+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c145d00 con 0x7f428c05ff40 2022-08-01T23:02:57.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.343+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f428001c460 con 0x7f428c05ff40 2022-08-01T23:02:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.343+0000 7f4289ffb700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f426c053a10 0x7f426c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.343+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f428006c570 con 0x7f428c05ff40 2022-08-01T23:02:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.343+0000 7f428bfff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f426c053a10 0x7f426c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.347+0000 7f428bfff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f426c053a10 0x7f426c055ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f427800b000 tx=0x7f4278006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.351+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4280043170 con 0x7f428c05ff40 2022-08-01T23:02:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.515+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f428c0b2830 con 0x7f428c05ff40 2022-08-01T23:02:57.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.515+0000 7f4289ffb700 1 -- 172.21.15.32:0/3878294953 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f4280046790 con 0x7f428c05ff40 2022-08-01T23:02:57.523 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:02:57.523 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:02:43.969500+0000","modified":"2022-08-01T23:02:57.203391+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T23:02:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f426c053a10 msgr2=0x7f426c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f426c053a10 0x7f426c055ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f427800b000 tx=0x7f4278006df0).stop 2022-08-01T23:02:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 msgr2=0x7f428c0b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c0b50e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f428000c000 tx=0x7f4280006df0).stop 2022-08-01T23:02:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 shutdown_connections 2022-08-01T23:02:57.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f426c053a10 0x7f426c055ed0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c0b68f0 0x7f428c1400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f428c0aa210 0x7f428c13fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 --2- 172.21.15.32:0/3878294953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428c05ff40 0x7f428c0b50e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 >> 172.21.15.32:0/3878294953 conn(0x7f428c0b3610 msgr2=0x7f428c0b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 shutdown_connections 2022-08-01T23:02:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.519+0000 7f4292cff700 1 -- 172.21.15.32:0/3878294953 wait complete. 2022-08-01T23:02:57.529 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 40 2022-08-01T23:02:57.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 Processor -- start 2022-08-01T23:02:57.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 -- start start 2022-08-01T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee400b2710 0x7fee4014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 0x7fee4014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee400a7250 con 0x7fee400b01a0 2022-08-01T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400a7480 con 0x7fee400b2710 2022-08-01T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee45030700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee400a7720 con 0x7fee400a8c50 2022-08-01T23:02:57.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee400b01a0 0x7fee400a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33498/0 (socket says 172.21.15.32:33498) 2022-08-01T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3f7fe700 1 -- 172.21.15.32:0/1856177580 learned_addr learned my addr 172.21.15.32:0/1856177580 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee4482f700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 0x7fee4014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1856177580 >> v2:172.21.15.55:3300/0 conn(0x7fee400b2710 0x7fee4014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283864734 0 0) 0x7fee400a7250 con 0x7fee400b01a0 2022-08-01T23:02:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee2c003260 con 0x7fee400b01a0 2022-08-01T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 265138025 0 0) 0x7fee2c003260 con 0x7fee400b01a0 2022-08-01T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 msgr2=0x7fee4014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 0x7fee4014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 >> v2:172.21.15.55:3300/0 conn(0x7fee400b2710 msgr2=0x7fee4014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 --2- 172.21.15.32:0/1856177580 >> v2:172.21.15.55:3300/0 conn(0x7fee400b2710 0x7fee4014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400a79c0 con 0x7fee400b01a0 2022-08-01T23:02:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fee300019e0 con 0x7fee400b01a0 2022-08-01T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 >> v1:172.21.15.32:6789/0 conn(0x7fee400b01a0 legacy=0x7fee400a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c003750 0x7fee2c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2c004500 0x7fee2c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 0x7fee2c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400a7480 con 0x7fee2c003750 2022-08-01T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400a7720 con 0x7fee2c004500 2022-08-01T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.635+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c00b750 con 0x7fee400a8c50 2022-08-01T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c003750 0x7fee2c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3f7fe700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2c004500 0x7fee2c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee4482f700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 0x7fee2c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 -- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 msgr2=0x7fee2c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee400a8c50 0x7fee2c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 -- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2c004500 msgr2=0x7fee2c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2c004500 0x7fee2c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 -- 172.21.15.32:0/1856177580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c003260 con 0x7fee2c003750 2022-08-01T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c003750 0x7fee2c003b70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fee3400c000 tx=0x7fee34006df0).ready entity=mon.0 client_cookie=44eb0c623349079e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:57.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3400e830 con 0x7fee2c003750 2022-08-01T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3400e990 con 0x7fee2c003750 2022-08-01T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee3effd700 1 -- 172.21.15.32:0/1856177580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3400ecf0 con 0x7fee2c003750 2022-08-01T23:02:57.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 -- 172.21.15.32:0/1856177580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c003750 msgr2=0x7fee2c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 --2- 172.21.15.32:0/1856177580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c003750 0x7fee2c003b70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fee3400c000 tx=0x7fee34006df0).stop 2022-08-01T23:02:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee4482f700 1 -- 172.21.15.32:0/1856177580 reap_dead start 2022-08-01T23:02:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 -- 172.21.15.32:0/1856177580 shutdown_connections 2022-08-01T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 -- 172.21.15.32:0/1856177580 >> 172.21.15.32:0/1856177580 conn(0x7fee40060310 msgr2=0x7fee400b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 -- 172.21.15.32:0/1856177580 shutdown_connections 2022-08-01T23:02:57.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 -- 172.21.15.32:0/1856177580 wait complete. 2022-08-01T23:02:57.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 Processor -- start 2022-08-01T23:02:57.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.639+0000 7fee45030700 1 -- start start 2022-08-01T23:02:57.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee40149ed0 0x7fee40156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4015a6a0 0x7fee40064690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 0x7fee400aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400ac630 con 0x7fee40064b80 2022-08-01T23:02:57.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee40156d70 con 0x7fee4015a6a0 2022-08-01T23:02:57.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4014a2b0 con 0x7fee40149ed0 2022-08-01T23:02:57.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 0x7fee400aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 0x7fee400aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45970/0 (socket says 172.21.15.32:45970) 2022-08-01T23:02:57.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 -- 172.21.15.32:0/1784653069 learned_addr learned my addr 172.21.15.32:0/1784653069 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:57.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee3f7fe700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4015a6a0 0x7fee40064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee40149ed0 0x7fee40156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 -- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee40149ed0 msgr2=0x7fee40156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee40149ed0 0x7fee40156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 -- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4015a6a0 msgr2=0x7fee40064690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:57.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4015a6a0 0x7fee40064690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:57.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 -- 172.21.15.32:0/1784653069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400ac870 con 0x7fee40064b80 2022-08-01T23:02:57.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee4482f700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 0x7fee400aa260 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fee3800b000 tx=0x7fee38006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:57.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee38019070 con 0x7fee40064b80 2022-08-01T23:02:57.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee38000e90 con 0x7fee40064b80 2022-08-01T23:02:57.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee400ac390 con 0x7fee40064b80 2022-08-01T23:02:57.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee38014840 con 0x7fee40064b80 2022-08-01T23:02:57.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4015a200 con 0x7fee40064b80 2022-08-01T23:02:57.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4015a3c0 con 0x7fee40064b80 2022-08-01T23:02:57.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.643+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee380149a0 con 0x7fee40064b80 2022-08-01T23:02:57.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.647+0000 7fee3d7fa700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee20053b00 0x7fee20055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:57.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.647+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fee3806bcc0 con 0x7fee40064b80 2022-08-01T23:02:57.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.647+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee20053b00 0x7fee20055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:57.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.647+0000 7fee3ffff700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee20053b00 0x7fee20055fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fee3401b000 tx=0x7fee34019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:57.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.651+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee38042a30 con 0x7fee40064b80 2022-08-01T23:02:57.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:57.819+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fee400574e0 con 0x7fee40064b80 2022-08-01T23:02:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.443+0000 7fee3d7fa700 1 -- 172.21.15.32:0/1784653069 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee38046050 con 0x7fee40064b80 2022-08-01T23:02:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee20053b00 msgr2=0x7fee20055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee20053b00 0x7fee20055fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fee3401b000 tx=0x7fee34019000).stop 2022-08-01T23:02:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 msgr2=0x7fee400aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 0x7fee400aa260 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fee3800b000 tx=0x7fee38006df0).stop 2022-08-01T23:02:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 shutdown_connections 2022-08-01T23:02:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee20053b00 0x7fee20055fc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee40064b80 0x7fee400aa260 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee4015a6a0 0x7fee40064690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 --2- 172.21.15.32:0/1784653069 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee40149ed0 0x7fee40156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 >> 172.21.15.32:0/1784653069 conn(0x7fee40060310 msgr2=0x7fee4005ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 shutdown_connections 2022-08-01T23:02:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.447+0000 7fee45030700 1 -- 172.21.15.32:0/1784653069 wait complete. 2022-08-01T23:02:58.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.579+0000 7fd2b338d700 1 Processor -- start 2022-08-01T23:02:58.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 -- start start 2022-08-01T23:02:58.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac0b6710 0x7fd2ac0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac144240 con 0x7fd2ac0b4550 2022-08-01T23:02:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac143d00 con 0x7fd2ac0b6af0 2022-08-01T23:02:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac143fa0 con 0x7fd2ac0b6710 2022-08-01T23:02:58.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b1129700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2ac0b6710 0x7fd2ac0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b0928700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd2ac0b4550 0x7fd2ac061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33520/0 (socket says 172.21.15.32:33520) 2022-08-01T23:02:58.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b0928700 1 -- 172.21.15.32:0/1347249117 learned_addr learned my addr 172.21.15.32:0/1347249117 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:02:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b1129700 1 -- 172.21.15.32:0/1347249117 >> v1:172.21.15.55:6789/0 conn(0x7fd2ac0b6af0 legacy=0x7fd2ac12bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b1129700 1 -- 172.21.15.32:0/1347249117 >> v1:172.21.15.32:6789/0 conn(0x7fd2ac0b4550 legacy=0x7fd2ac061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd29ffff700 1 -- 172.21.15.32:0/1347249117 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121434757 0 0) 0x7fd2ac144240 con 0x7fd2ac0b4550 2022-08-01T23:02:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b1129700 1 -- 172.21.15.32:0/1347249117 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac061780 con 0x7fd2ac0b6710 2022-08-01T23:02:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b1129700 1 --2- 172.21.15.32:0/1347249117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b5d80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fd2ac1432b0 tx=0x7fd2a8006df0).ready entity=mon.0 client_cookie=e601263712e6eef6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd29ffff700 1 -- 172.21.15.32:0/1347249117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a800d610 con 0x7fd2ac0b6710 2022-08-01T23:02:58.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd29ffff700 1 -- 172.21.15.32:0/1347249117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8000e90 con 0x7fd2ac0b6710 2022-08-01T23:02:58.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd29ffff700 1 -- 172.21.15.32:0/1347249117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a800d960 con 0x7fd2ac0b6710 2022-08-01T23:02:58.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 -- 172.21.15.32:0/1347249117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 msgr2=0x7fd2ac0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1347249117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b5d80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fd2ac1432b0 tx=0x7fd2a8006df0).stop 2022-08-01T23:02:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 -- 172.21.15.32:0/1347249117 shutdown_connections 2022-08-01T23:02:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.583+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1347249117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b5d80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- 172.21.15.32:0/1347249117 >> 172.21.15.32:0/1347249117 conn(0x7fd2ac05c620 msgr2=0x7fd2ac05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:58.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- 172.21.15.32:0/1347249117 shutdown_connections 2022-08-01T23:02:58.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- 172.21.15.32:0/1347249117 wait complete. 2022-08-01T23:02:58.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 Processor -- start 2022-08-01T23:02:58.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- start start 2022-08-01T23:02:58.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2ac0b4550 0x7fd2ac127320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:58.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:58.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd2ac0b6af0 0x7fd2ac0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:58.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac121720 con 0x7fd2ac0b6710 2022-08-01T23:02:58.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac12bfc0 con 0x7fd2ac0b4550 2022-08-01T23:02:58.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b338d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac12c260 con 0x7fd2ac0b6af0 2022-08-01T23:02:58.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b0928700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:58.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b0928700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45984/0 (socket says 172.21.15.32:45984) 2022-08-01T23:02:58.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b0928700 1 -- 172.21.15.32:0/1179579497 learned_addr learned my addr 172.21.15.32:0/1179579497 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:58.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b1129700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2ac0b4550 0x7fd2ac127320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:58.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.587+0000 7fd2b192a700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd2ac0b6af0 0x7fd2ac0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:58.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b0928700 1 -- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd2ac0b6af0 msgr2=0x7fd2ac0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b0928700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd2ac0b6af0 0x7fd2ac0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b0928700 1 -- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2ac0b4550 msgr2=0x7fd2ac127320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:58.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b0928700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2ac0b4550 0x7fd2ac127320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:58.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b0928700 1 -- 172.21.15.32:0/1179579497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a800b180 con 0x7fd2ac0b6710 2022-08-01T23:02:58.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b0928700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b2a30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7fd2a0000bc0 tx=0x7fd2a0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:58.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a00070d0 con 0x7fd2ac0b6710 2022-08-01T23:02:58.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0007230 con 0x7fd2ac0b6710 2022-08-01T23:02:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac12c500 con 0x7fd2ac0b6710 2022-08-01T23:02:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a0005610 con 0x7fd2ac0b6710 2022-08-01T23:02:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac127e00 con 0x7fd2ac0b6710 2022-08-01T23:02:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac061a20 con 0x7fd2ac0b6710 2022-08-01T23:02:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2a0010070 con 0x7fd2ac0b6710 2022-08-01T23:02:58.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd29e7fc700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd28c053a70 0x7fd28c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:58.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd2a006b790 con 0x7fd2ac0b6710 2022-08-01T23:02:58.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.591+0000 7fd2b1129700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd28c053a70 0x7fd28c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:58.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.595+0000 7fd2b1129700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd28c053a70 0x7fd28c055f30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd2a801c000 tx=0x7fd2a8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:58.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.595+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2a0042460 con 0x7fd2ac0b6710 2022-08-01T23:02:58.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:58.727+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2ac0574e0 con 0x7fd2ac0b6710 2022-08-01T23:02:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.451+0000 7fd29e7fc700 1 -- 172.21.15.32:0/1179579497 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd2a0045a80 con 0x7fd2ac0b6710 2022-08-01T23:02:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd28c053a70 msgr2=0x7fd28c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd28c053a70 0x7fd28c055f30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd2a801c000 tx=0x7fd2a8015000).stop 2022-08-01T23:02:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 msgr2=0x7fd2ac0b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b2a30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7fd2a0000bc0 tx=0x7fd2a0007a40).stop 2022-08-01T23:02:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 shutdown_connections 2022-08-01T23:02:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd28c053a70 0x7fd28c055f30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd2ac0b6af0 0x7fd2ac0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2ac0b6710 0x7fd2ac0b2a30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 --2- 172.21.15.32:0/1179579497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2ac0b4550 0x7fd2ac127320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 >> 172.21.15.32:0/1179579497 conn(0x7fd2ac05c620 msgr2=0x7fd2ac1432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 shutdown_connections 2022-08-01T23:02:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.455+0000 7fd2b338d700 1 -- 172.21.15.32:0/1179579497 wait complete. 2022-08-01T23:02:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:02:59.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:02:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 Processor -- start 2022-08-01T23:02:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 -- start start 2022-08-01T23:02:59.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd880af550 0x7fdd880b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd880a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880aa630 con 0x7fdd88060330 2022-08-01T23:02:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880b15e0 con 0x7fdd880af550 2022-08-01T23:02:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8d683700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880af930 con 0x7fdd8805c6f0 2022-08-01T23:02:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd87fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd880af550 0x7fdd880b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd87fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd880af550 0x7fdd880b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45990/0 (socket says 172.21.15.32:45990) 2022-08-01T23:02:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd87fff700 1 -- 172.21.15.32:0/4047024459 learned_addr learned my addr 172.21.15.32:0/4047024459 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.583+0000 7fdd8ce82700 1 --2- 172.21.15.32:0/4047024459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd880a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd87fff700 1 -- 172.21.15.32:0/4047024459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 msgr2=0x7fdd880a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:02:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd87fff700 1 --2- 172.21.15.32:0/4047024459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd880a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd87fff700 1 -- 172.21.15.32:0/4047024459 >> v1:172.21.15.55:6789/0 conn(0x7fdd88060330 legacy=0x7fdd881445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd87fff700 1 -- 172.21.15.32:0/4047024459 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8805fd10 con 0x7fdd880af550 2022-08-01T23:02:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd86ffd700 1 -- 172.21.15.32:0/4047024459 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524798608 0 0) 0x7fdd880aa630 con 0x7fdd88060330 2022-08-01T23:02:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd87fff700 1 --2- 172.21.15.32:0/4047024459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd880af550 0x7fdd880b0e40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fdd78002f70 tx=0x7fdd78006eb0).ready entity=mon.0 client_cookie=564ca81a2c2f8648 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd86ffd700 1 -- 172.21.15.32:0/4047024459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd7800d570 con 0x7fdd880af550 2022-08-01T23:02:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd86ffd700 1 -- 172.21.15.32:0/4047024459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78000e90 con 0x7fdd880af550 2022-08-01T23:02:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd86ffd700 1 -- 172.21.15.32:0/4047024459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd7800d8c0 con 0x7fdd880af550 2022-08-01T23:02:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 -- 172.21.15.32:0/4047024459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd880af550 msgr2=0x7fdd880b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 --2- 172.21.15.32:0/4047024459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd880af550 0x7fdd880b0e40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fdd78002f70 tx=0x7fdd78006eb0).stop 2022-08-01T23:02:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 -- 172.21.15.32:0/4047024459 shutdown_connections 2022-08-01T23:02:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 --2- 172.21.15.32:0/4047024459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd880a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 --2- 172.21.15.32:0/4047024459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd880af550 0x7fdd880b0e40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 -- 172.21.15.32:0/4047024459 >> 172.21.15.32:0/4047024459 conn(0x7fdd880612d0 msgr2=0x7fdd88062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:02:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 -- 172.21.15.32:0/4047024459 shutdown_connections 2022-08-01T23:02:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 -- 172.21.15.32:0/4047024459 wait complete. 2022-08-01T23:02:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 Processor -- start 2022-08-01T23:02:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.587+0000 7fdd8d683700 1 -- start start 2022-08-01T23:02:59.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd88063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:59.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 0x7fdd88142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:59.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd880af550 0x7fdd88153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:59.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88066840 con 0x7fdd88060330 2022-08-01T23:02:59.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88066ae0 con 0x7fdd880af550 2022-08-01T23:02:59.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88066d80 con 0x7fdd8805c6f0 2022-08-01T23:02:59.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 0x7fdd88142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:59.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 0x7fdd88142c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45998/0 (socket says 172.21.15.32:45998) 2022-08-01T23:02:59.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 -- 172.21.15.32:0/395544774 learned_addr learned my addr 172.21.15.32:0/395544774 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:02:59.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd87fff700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd88063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:59.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8ce82700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd880af550 0x7fdd88153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:59.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 -- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 msgr2=0x7fdd88063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:59.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd88063ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 -- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd880af550 msgr2=0x7fdd88153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:02:59.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd880af550 0x7fdd88153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:02:59.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 -- 172.21.15.32:0/395544774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7800b1e0 con 0x7fdd88060330 2022-08-01T23:02:59.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd877fe700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 0x7fdd88142c20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fdd7c000ed0 tx=0x7fdd7c007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:59.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd7c00ce00 con 0x7fdd88060330 2022-08-01T23:02:59.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c00f470 con 0x7fdd88060330 2022-08-01T23:02:59.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8805f840 con 0x7fdd88060330 2022-08-01T23:02:59.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd7c00f7d0 con 0x7fdd88060330 2022-08-01T23:02:59.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd88145710 con 0x7fdd88060330 2022-08-01T23:02:59.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.591+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8805fa70 con 0x7fdd88060330 2022-08-01T23:02:59.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.595+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd7c019030 con 0x7fdd88060330 2022-08-01T23:02:59.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.595+0000 7fdd857fa700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd68053b30 0x7fdd68055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:02:59.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.595+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fdd7c06b2b0 con 0x7fdd88060330 2022-08-01T23:02:59.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.595+0000 7fdd87fff700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd68053b30 0x7fdd68055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:02:59.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.595+0000 7fdd87fff700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd68053b30 0x7fdd68055ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fdd7801a000 tx=0x7fdd78015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:02:59.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.603+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd7c042280 con 0x7fdd88060330 2022-08-01T23:02:59.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:02:59.727+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdd880574e0 con 0x7fdd88060330 2022-08-01T23:03:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.475+0000 7fdd857fa700 1 -- 172.21.15.32:0/395544774 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdd7c0458a0 con 0x7fdd88060330 2022-08-01T23:03:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd68053b30 msgr2=0x7fdd68055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd68053b30 0x7fdd68055ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fdd7801a000 tx=0x7fdd78015000).stop 2022-08-01T23:03:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 msgr2=0x7fdd88142c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 0x7fdd88142c20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fdd7c000ed0 tx=0x7fdd7c007ab0).stop 2022-08-01T23:03:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 shutdown_connections 2022-08-01T23:03:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd68053b30 0x7fdd68055ff0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd880af550 0x7fdd88153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd88060330 0x7fdd88142c20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 --2- 172.21.15.32:0/395544774 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd8805c6f0 0x7fdd88063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 >> 172.21.15.32:0/395544774 conn(0x7fdd880612d0 msgr2=0x7fdd88140f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 shutdown_connections 2022-08-01T23:03:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.479+0000 7fdd8d683700 1 -- 172.21.15.32:0/395544774 wait complete. 2022-08-01T23:03:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:03:00.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:03:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 Processor -- start 2022-08-01T23:03:00.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 -- start start 2022-08-01T23:03:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5d00600f0 0x7fb5d01405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d0140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d0140fb0 con 0x7fb5d0061360 2022-08-01T23:03:00.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d0064aa0 con 0x7fb5d00600f0 2022-08-01T23:03:00.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5d659e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0064d40 con 0x7fb5d00b28a0 2022-08-01T23:03:00.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5d00600f0 0x7fb5d01405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5d00600f0 0x7fb5d01405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46004/0 (socket says 172.21.15.32:46004) 2022-08-01T23:03:00.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 -- 172.21.15.32:0/749099248 learned_addr learned my addr 172.21.15.32:0/749099248 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:00.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cf7fe700 1 --2- 172.21.15.32:0/749099248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d0140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 -- 172.21.15.32:0/749099248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 msgr2=0x7fb5d0140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:00.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 --2- 172.21.15.32:0/749099248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d0140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 -- 172.21.15.32:0/749099248 >> v1:172.21.15.55:6789/0 conn(0x7fb5d0061360 legacy=0x7fb5d01432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 -- 172.21.15.32:0/749099248 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0064fe0 con 0x7fb5d00600f0 2022-08-01T23:03:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.603+0000 7fb5cffff700 1 --2- 172.21.15.32:0/749099248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d00600f0 0x7fb5d01405d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fb5c0002f70 tx=0x7fb5c0006eb0).ready entity=mon.0 client_cookie=5d31a505d7cd2bed server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5ceffd700 1 -- 172.21.15.32:0/749099248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c000d570 con 0x7fb5d00600f0 2022-08-01T23:03:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5ceffd700 1 -- 172.21.15.32:0/749099248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0000e90 con 0x7fb5d00600f0 2022-08-01T23:03:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5ceffd700 1 -- 172.21.15.32:0/749099248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c000d900 con 0x7fb5d00600f0 2022-08-01T23:03:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 -- 172.21.15.32:0/749099248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d00600f0 msgr2=0x7fb5d01405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 --2- 172.21.15.32:0/749099248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d00600f0 0x7fb5d01405d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fb5c0002f70 tx=0x7fb5c0006eb0).stop 2022-08-01T23:03:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 -- 172.21.15.32:0/749099248 shutdown_connections 2022-08-01T23:03:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 --2- 172.21.15.32:0/749099248 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d0140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 --2- 172.21.15.32:0/749099248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d00600f0 0x7fb5d01405d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 -- 172.21.15.32:0/749099248 >> 172.21.15.32:0/749099248 conn(0x7fb5d005d0f0 msgr2=0x7fb5d005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 -- 172.21.15.32:0/749099248 shutdown_connections 2022-08-01T23:03:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 -- 172.21.15.32:0/749099248 wait complete. 2022-08-01T23:03:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 Processor -- start 2022-08-01T23:03:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.607+0000 7fb5d659e700 1 -- start start 2022-08-01T23:03:00.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 0x7fb5d01525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 0x7fb5d00b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d01561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00acfc0 con 0x7fb5d0061360 2022-08-01T23:03:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0155b10 con 0x7fb5d00600f0 2022-08-01T23:03:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0146160 con 0x7fb5d00b28a0 2022-08-01T23:03:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 0x7fb5d00b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 0x7fb5d01525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 0x7fb5d00b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46012/0 (socket says 172.21.15.32:46012) 2022-08-01T23:03:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 0x7fb5d01525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:55796/0 (socket says 172.21.15.32:55796) 2022-08-01T23:03:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 -- 172.21.15.32:0/798275923 learned_addr learned my addr 172.21.15.32:0/798275923 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d4b3b700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d01561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 -- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 msgr2=0x7fb5d01561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d01561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 -- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 msgr2=0x7fb5d01525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 0x7fb5d01525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 -- 172.21.15.32:0/798275923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c000b1e0 con 0x7fb5d0061360 2022-08-01T23:03:00.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cffff700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 0x7fb5d01525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:00.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cf7fe700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 0x7fb5d00b5b10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fb5d00ac3f0 tx=0x7fb5c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:00.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c4010780 con 0x7fb5d0061360 2022-08-01T23:03:00.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c40108e0 con 0x7fb5d0061360 2022-08-01T23:03:00.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d0146400 con 0x7fb5d0061360 2022-08-01T23:03:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c4010c40 con 0x7fb5d0061360 2022-08-01T23:03:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.611+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d00b6000 con 0x7fb5d0061360 2022-08-01T23:03:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.615+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d0155f40 con 0x7fb5d0061360 2022-08-01T23:03:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.615+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5c401c460 con 0x7fb5d0061360 2022-08-01T23:03:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.615+0000 7fb5cd7fa700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5b0053b00 0x7fb5b0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.615+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb5c406c1a0 con 0x7fb5d0061360 2022-08-01T23:03:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.615+0000 7fb5cffff700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5b0053b00 0x7fb5b0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.615+0000 7fb5cffff700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5b0053b00 0x7fb5b0055fc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fb5c001c000 tx=0x7fb5c0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.619+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5d0155f40 con 0x7fb5d0061360 2022-08-01T23:03:00.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.747+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5d0155f40 con 0x7fb5d0061360 2022-08-01T23:03:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.755+0000 7fb5cd7fa700 1 -- 172.21.15.32:0/798275923 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5d0155f40 con 0x7fb5d0061360 2022-08-01T23:03:00.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5b0053b00 msgr2=0x7fb5b0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5b0053b00 0x7fb5b0055fc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fb5c001c000 tx=0x7fb5c0015000).stop 2022-08-01T23:03:00.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 msgr2=0x7fb5d00b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 0x7fb5d00b5b10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fb5d00ac3f0 tx=0x7fb5c4006df0).stop 2022-08-01T23:03:00.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 shutdown_connections 2022-08-01T23:03:00.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb5b0053b00 0x7fb5b0055fc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5d00b28a0 0x7fb5d01561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5d0061360 0x7fb5d00b5b10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 --2- 172.21.15.32:0/798275923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00600f0 0x7fb5d01525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 >> 172.21.15.32:0/798275923 conn(0x7fb5d005d0f0 msgr2=0x7fb5d0065f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:00.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 shutdown_connections 2022-08-01T23:03:00.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.759+0000 7fb5d659e700 1 -- 172.21.15.32:0/798275923 wait complete. 2022-08-01T23:03:00.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-08-01T23:03:00.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 Processor -- start 2022-08-01T23:03:00.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 -- start start 2022-08-01T23:03:00.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86c060410 0x7fc86c066a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c066f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc86c0a9930 0x7fc86c0a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c0ab4a0 con 0x7fc86c060410 2022-08-01T23:03:00.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c0ac030 con 0x7fc86c0a9930 2022-08-01T23:03:00.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc873343700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c140490 con 0x7fc86c0b1b30 2022-08-01T23:03:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc8710df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86c060410 0x7fc86c066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc8710df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86c060410 0x7fc86c066a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46020/0 (socket says 172.21.15.32:46020) 2022-08-01T23:03:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc8710df700 1 -- 172.21.15.32:0/975760019 learned_addr learned my addr 172.21.15.32:0/975760019 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:00.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc8708de700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c066f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.875+0000 7fc8718e0700 1 --2- 172.21.15.32:0/975760019 >> v2:172.21.15.55:3300/0 conn(0x7fc86c0a9930 0x7fc86c0a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8710df700 1 -- 172.21.15.32:0/975760019 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 msgr2=0x7fc86c066f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8710df700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c066f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8710df700 1 -- 172.21.15.32:0/975760019 >> v2:172.21.15.55:3300/0 conn(0x7fc86c0a9930 msgr2=0x7fc86c0a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8710df700 1 --2- 172.21.15.32:0/975760019 >> v2:172.21.15.55:3300/0 conn(0x7fc86c0a9930 0x7fc86c0a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8710df700 1 -- 172.21.15.32:0/975760019 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c140730 con 0x7fc86c060410 2022-08-01T23:03:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8708de700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c066f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:00.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc8710df700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c066a40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fc868002f70 tx=0x7fc868006eb0).ready entity=mon.0 client_cookie=967f932f49344026 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc85bfff700 1 -- 172.21.15.32:0/975760019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86800d570 con 0x7fc86c060410 2022-08-01T23:03:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc85bfff700 1 -- 172.21.15.32:0/975760019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc868000e90 con 0x7fc86c060410 2022-08-01T23:03:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc85bfff700 1 -- 172.21.15.32:0/975760019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86800d900 con 0x7fc86c060410 2022-08-01T23:03:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 -- 172.21.15.32:0/975760019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 msgr2=0x7fc86c066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c066a40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fc868002f70 tx=0x7fc868006eb0).stop 2022-08-01T23:03:00.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 -- 172.21.15.32:0/975760019 shutdown_connections 2022-08-01T23:03:00.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 --2- 172.21.15.32:0/975760019 >> v2:172.21.15.55:3300/0 conn(0x7fc86c0a9930 0x7fc86c0a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c066f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 --2- 172.21.15.32:0/975760019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c066a40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 -- 172.21.15.32:0/975760019 >> 172.21.15.32:0/975760019 conn(0x7fc86c0b5570 msgr2=0x7fc86c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 -- 172.21.15.32:0/975760019 shutdown_connections 2022-08-01T23:03:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 -- 172.21.15.32:0/975760019 wait complete. 2022-08-01T23:03:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 Processor -- start 2022-08-01T23:03:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.879+0000 7fc873343700 1 -- start start 2022-08-01T23:03:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0a9930 0x7fc86c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c062760 con 0x7fc86c060410 2022-08-01T23:03:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0ada70 con 0x7fc86c0a9930 2022-08-01T23:03:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0af230 con 0x7fc86c0b1b30 2022-08-01T23:03:00.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c143870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46024/0 (socket says 172.21.15.32:46024) 2022-08-01T23:03:00.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 -- 172.21.15.32:0/1943409598 learned_addr learned my addr 172.21.15.32:0/1943409598 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:00.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8708de700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0a9930 0x7fc86c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8718e0700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 -- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 msgr2=0x7fc86c1509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c1509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 -- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0a9930 msgr2=0x7fc86c150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0a9930 0x7fc86c150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 -- 172.21.15.32:0/1943409598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86800b1e0 con 0x7fc86c060410 2022-08-01T23:03:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8718e0700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc8710df700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c143870 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fc86c0acf00 tx=0x7fc868015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:00.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc868006be0 con 0x7fc86c060410 2022-08-01T23:03:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8680068b0 con 0x7fc86c060410 2022-08-01T23:03:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c062990 con 0x7fc86c060410 2022-08-01T23:03:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc868011de0 con 0x7fc86c060410 2022-08-01T23:03:00.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c150ee0 con 0x7fc86c060410 2022-08-01T23:03:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c0af370 con 0x7fc86c060410 2022-08-01T23:03:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.883+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc868025030 con 0x7fc86c060410 2022-08-01T23:03:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.887+0000 7fc85a7fc700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc84c053b00 0x7fc84c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.887+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc86806ced0 con 0x7fc86c060410 2022-08-01T23:03:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.887+0000 7fc8708de700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc84c053b00 0x7fc84c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.887+0000 7fc8708de700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc84c053b00 0x7fc84c055fc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc85c00c000 tx=0x7fc85c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:00.891+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc868047e60 con 0x7fc86c060410 2022-08-01T23:03:01.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.055+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7fc86c0a9490 con 0x7fc86c060410 2022-08-01T23:03:01.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.475+0000 7fc85a7fc700 1 -- 172.21.15.32:0/1943409598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v312) v1 ==== 126+0+0 (secure 0 0 0) 0x7fc868023020 con 0x7fc86c060410 2022-08-01T23:03:01.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.475+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc84c053b00 msgr2=0x7fc84c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:01.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.475+0000 7fc873343700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc84c053b00 0x7fc84c055fc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc85c00c000 tx=0x7fc85c008040).stop 2022-08-01T23:03:01.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.475+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 msgr2=0x7fc86c143870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:01.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.475+0000 7fc873343700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c143870 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fc86c0acf00 tx=0x7fc868015000).stop 2022-08-01T23:03:01.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 shutdown_connections 2022-08-01T23:03:01.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc84c053b00 0x7fc84c055fc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc86c0b1b30 0x7fc86c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0a9930 0x7fc86c150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 --2- 172.21.15.32:0/1943409598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c060410 0x7fc86c143870 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 >> 172.21.15.32:0/1943409598 conn(0x7fc86c0b5570 msgr2=0x7fc86c0afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:01.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 shutdown_connections 2022-08-01T23:03:01.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.479+0000 7fc873343700 1 -- 172.21.15.32:0/1943409598 wait complete. 2022-08-01T23:03:01.501 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:03:01.502 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:03:01.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-08-01T23:03:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.603+0000 7f488bfff700 1 Processor -- start 2022-08-01T23:03:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.603+0000 7f488bfff700 1 -- start start 2022-08-01T23:03:01.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f488c0accd0 0x7f488c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:01.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f488c0b1100 0x7f488c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:01.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c0b06c0 con 0x7f488c0b2580 2022-08-01T23:03:01.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c0b2960 con 0x7f488c0b1100 2022-08-01T23:03:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c0ae360 con 0x7f488c0accd0 2022-08-01T23:03:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f488c0b1100 0x7f488c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488a7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f488c0accd0 0x7f488c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f488c0b1100 0x7f488c1466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46036/0 (socket says 172.21.15.32:46036) 2022-08-01T23:03:01.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 -- 172.21.15.32:0/1988673467 learned_addr learned my addr 172.21.15.32:0/1988673467 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:01.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 -- 172.21.15.32:0/1988673467 >> v2:172.21.15.55:3300/0 conn(0x7f488c0accd0 msgr2=0x7f488c0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:01.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 --2- 172.21.15.32:0/1988673467 >> v2:172.21.15.55:3300/0 conn(0x7f488c0accd0 0x7f488c0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 -- 172.21.15.32:0/1988673467 >> v1:172.21.15.55:6789/0 conn(0x7f488c0b2580 legacy=0x7f488c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:01.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 -- 172.21.15.32:0/1988673467 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0b0330 con 0x7f488c0b1100 2022-08-01T23:03:01.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488a7fc700 1 --2- 172.21.15.32:0/1988673467 >> v2:172.21.15.55:3300/0 conn(0x7f488c0accd0 0x7f488c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488b7fe700 1 --2- 172.21.15.32:0/1988673467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b1100 0x7f488c1466f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f48840032f0 tx=0x7f4884009290).ready entity=mon.0 client_cookie=23122d147052eb0d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f4889ffb700 1 -- 172.21.15.32:0/1988673467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f488400b870 con 0x7f488c0b1100 2022-08-01T23:03:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f4889ffb700 1 -- 172.21.15.32:0/1988673467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b9d0 con 0x7f488c0b1100 2022-08-01T23:03:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f4889ffb700 1 -- 172.21.15.32:0/1988673467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f488400bb40 con 0x7f488c0b1100 2022-08-01T23:03:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- 172.21.15.32:0/1988673467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b1100 msgr2=0x7f488c1466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 --2- 172.21.15.32:0/1988673467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b1100 0x7f488c1466f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f48840032f0 tx=0x7f4884009290).stop 2022-08-01T23:03:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- 172.21.15.32:0/1988673467 shutdown_connections 2022-08-01T23:03:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 --2- 172.21.15.32:0/1988673467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b1100 0x7f488c1466f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 --2- 172.21.15.32:0/1988673467 >> v2:172.21.15.55:3300/0 conn(0x7f488c0accd0 0x7f488c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- 172.21.15.32:0/1988673467 >> 172.21.15.32:0/1988673467 conn(0x7f488c05c620 msgr2=0x7f488c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- 172.21.15.32:0/1988673467 shutdown_connections 2022-08-01T23:03:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.607+0000 7f488bfff700 1 -- 172.21.15.32:0/1988673467 wait complete. 2022-08-01T23:03:01.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 Processor -- start 2022-08-01T23:03:01.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 -- start start 2022-08-01T23:03:01.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f488c0accd0 0x7f488c09ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:01.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f488c0b1100 0x7f488c09f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:01.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 0x7f488c1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c09f930 con 0x7f488c0b2580 2022-08-01T23:03:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0aaaf0 con 0x7f488c0accd0 2022-08-01T23:03:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0aad90 con 0x7f488c0b1100 2022-08-01T23:03:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 0x7f488c1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:01.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 0x7f488c1423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46040/0 (socket says 172.21.15.32:46040) 2022-08-01T23:03:01.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 -- 172.21.15.32:0/418274556 learned_addr learned my addr 172.21.15.32:0/418274556 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:01.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488affd700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f488c0accd0 0x7f488c09ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:01.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488a7fc700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f488c0b1100 0x7f488c09f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 -- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f488c0b1100 msgr2=0x7f488c09f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f488c0b1100 0x7f488c09f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 -- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f488c0accd0 msgr2=0x7f488c09ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f488c0accd0 0x7f488c09ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 -- 172.21.15.32:0/418274556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c00b180 con 0x7f488c0b2580 2022-08-01T23:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488affd700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f488c0accd0 0x7f488c09ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f488b7fe700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 0x7f488c1423d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f48840032f0 tx=0x7f4884016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.611+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f488400b870 con 0x7f488c0b2580 2022-08-01T23:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4884021630 con 0x7f488c0b2580 2022-08-01T23:03:01.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4884021990 con 0x7f488c0b2580 2022-08-01T23:03:01.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4884009530 con 0x7f488c0b2580 2022-08-01T23:03:01.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c0ab2d0 con 0x7f488c0b2580 2022-08-01T23:03:01.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c0ab490 con 0x7f488c0b2580 2022-08-01T23:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4884027020 con 0x7f488c0b2580 2022-08-01T23:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f487bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4868053b00 0x7f4868055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f488406d020 con 0x7f488c0b2580 2022-08-01T23:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.615+0000 7f488affd700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4868053b00 0x7f4868055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.619+0000 7f488affd700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4868053b00 0x7f4868055fc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f488000d000 tx=0x7f4880009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.623+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48840440c0 con 0x7f488c0b2580 2022-08-01T23:03:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:01.791+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f488c0b08f0 con 0x7f488c0b2580 2022-08-01T23:03:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.475+0000 7f487bfff700 1 -- 172.21.15.32:0/418274556 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v313) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4884015070 con 0x7f488c0b2580 2022-08-01T23:03:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4868053b00 msgr2=0x7f4868055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4868053b00 0x7f4868055fc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f488000d000 tx=0x7f4880009040).stop 2022-08-01T23:03:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 msgr2=0x7f488c1423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 0x7f488c1423d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f48840032f0 tx=0x7f4884016000).stop 2022-08-01T23:03:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 shutdown_connections 2022-08-01T23:03:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4868053b00 0x7f4868055fc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f488c0b2580 0x7f488c1423d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f488c0b1100 0x7f488c09f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 --2- 172.21.15.32:0/418274556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f488c0accd0 0x7f488c09ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 >> 172.21.15.32:0/418274556 conn(0x7f488c05c620 msgr2=0x7f488c146650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 shutdown_connections 2022-08-01T23:03:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.479+0000 7f488bfff700 1 -- 172.21.15.32:0/418274556 wait complete. 2022-08-01T23:03:02.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:03:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.615+0000 7f65b359e700 1 Processor -- start 2022-08-01T23:03:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.615+0000 7f65b359e700 1 -- start start 2022-08-01T23:03:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f65b40b31d0 0x7f65b4066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b40a9da0 0x7f65b4060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b4067030 con 0x7f65b41434b0 2022-08-01T23:03:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b4064a70 con 0x7f65b40a9da0 2022-08-01T23:03:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b4061f60 con 0x7f65b40b31d0 2022-08-01T23:03:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b40a9da0 0x7f65b4060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65b41434b0 0x7f65b4143900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33592/0 (socket says 172.21.15.32:33592) 2022-08-01T23:03:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b40a9da0 0x7f65b4060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46050/0 (socket says 172.21.15.32:46050) 2022-08-01T23:03:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b2d9d700 1 -- 172.21.15.32:0/1134492515 learned_addr learned my addr 172.21.15.32:0/1134492515 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b259c700 1 --2- 172.21.15.32:0/1134492515 >> v2:172.21.15.55:3300/0 conn(0x7f65b40b31d0 0x7f65b4066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 -- 172.21.15.32:0/1134492515 >> v2:172.21.15.55:3300/0 conn(0x7f65b40b31d0 msgr2=0x7f65b4066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 --2- 172.21.15.32:0/1134492515 >> v2:172.21.15.55:3300/0 conn(0x7f65b40b31d0 0x7f65b4066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 -- 172.21.15.32:0/1134492515 >> v1:172.21.15.32:6789/0 conn(0x7f65b41434b0 legacy=0x7f65b4143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 -- 172.21.15.32:0/1134492515 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b40b35b0 con 0x7f65b40a9da0 2022-08-01T23:03:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65a9d9b700 1 --2- 172.21.15.32:0/1134492515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b40a9da0 0x7f65b4060950 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f659c002f70 tx=0x7f659c006df0).ready entity=mon.0 client_cookie=a4436da7fc9d603f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b1d9b700 1 -- 172.21.15.32:0/1134492515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659c00d570 con 0x7f65b40a9da0 2022-08-01T23:03:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b1d9b700 1 -- 172.21.15.32:0/1134492515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c000e90 con 0x7f65b40a9da0 2022-08-01T23:03:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b1d9b700 1 -- 172.21.15.32:0/1134492515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659c00d8c0 con 0x7f65b40a9da0 2022-08-01T23:03:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- 172.21.15.32:0/1134492515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b40a9da0 msgr2=0x7f65b4060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 --2- 172.21.15.32:0/1134492515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b40a9da0 0x7f65b4060950 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f659c002f70 tx=0x7f659c006df0).stop 2022-08-01T23:03:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- 172.21.15.32:0/1134492515 shutdown_connections 2022-08-01T23:03:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 --2- 172.21.15.32:0/1134492515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b40a9da0 0x7f65b4060950 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 --2- 172.21.15.32:0/1134492515 >> v2:172.21.15.55:3300/0 conn(0x7f65b40b31d0 0x7f65b4066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- 172.21.15.32:0/1134492515 >> 172.21.15.32:0/1134492515 conn(0x7f65b4063870 msgr2=0x7f65b4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- 172.21.15.32:0/1134492515 shutdown_connections 2022-08-01T23:03:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.619+0000 7f65b359e700 1 -- 172.21.15.32:0/1134492515 wait complete. 2022-08-01T23:03:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 Processor -- start 2022-08-01T23:03:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 -- start start 2022-08-01T23:03:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65b40a9da0 0x7f65b4140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65b40b31d0 0x7f65b40abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 0x7f65b40ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40ac660 con 0x7f65b41434b0 2022-08-01T23:03:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40ac940 con 0x7f65b40b31d0 2022-08-01T23:03:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40aa690 con 0x7f65b40a9da0 2022-08-01T23:03:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 0x7f65b40ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 0x7f65b40ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46058/0 (socket says 172.21.15.32:46058) 2022-08-01T23:03:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 -- 172.21.15.32:0/1873234767 learned_addr learned my addr 172.21.15.32:0/1873234767 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65a9d9b700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65b40b31d0 0x7f65b40abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b259c700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65b40a9da0 0x7f65b4140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 -- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65b40a9da0 msgr2=0x7f65b4140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65b40a9da0 0x7f65b4140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 -- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65b40b31d0 msgr2=0x7f65b40abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65b40b31d0 0x7f65b40abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 -- 172.21.15.32:0/1873234767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c00b180 con 0x7f65b41434b0 2022-08-01T23:03:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b2d9d700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 0x7f65b40ac170 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f65b40aa820 tx=0x7f65ac0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65ac005ba0 con 0x7f65b41434b0 2022-08-01T23:03:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac005d00 con 0x7f65b41434b0 2022-08-01T23:03:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b40aa3f0 con 0x7f65b41434b0 2022-08-01T23:03:02.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65ac0072c0 con 0x7f65b41434b0 2022-08-01T23:03:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.623+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b41369b0 con 0x7f65b41434b0 2022-08-01T23:03:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.627+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b4066af0 con 0x7f65b41434b0 2022-08-01T23:03:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.627+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f65ac01a060 con 0x7f65b41434b0 2022-08-01T23:03:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.627+0000 7f65abfff700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65980539e0 0x7f6598055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.627+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f65ac06b700 con 0x7f65b41434b0 2022-08-01T23:03:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.627+0000 7f65b259c700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65980539e0 0x7f6598055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.627+0000 7f65b259c700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65980539e0 0x7f6598055ea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f65a400b000 tx=0x7f65a4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.635+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65ac0427a0 con 0x7f65b41434b0 2022-08-01T23:03:02.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.763+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f65b40574e0 con 0x7f65b41434b0 2022-08-01T23:03:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65abfff700 1 -- 172.21.15.32:0/1873234767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f65ac0180c0 con 0x7f65b41434b0 2022-08-01T23:03:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65980539e0 msgr2=0x7f6598055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65980539e0 0x7f6598055ea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f65a400b000 tx=0x7f65a4006df0).stop 2022-08-01T23:03:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 msgr2=0x7f65b40ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 0x7f65b40ac170 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f65b40aa820 tx=0x7f65ac0013a0).stop 2022-08-01T23:03:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 shutdown_connections 2022-08-01T23:03:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65980539e0 0x7f6598055ea0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b41434b0 0x7f65b40ac170 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65b40b31d0 0x7f65b40abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 --2- 172.21.15.32:0/1873234767 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65b40a9da0 0x7f65b4140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 >> 172.21.15.32:0/1873234767 conn(0x7f65b4063870 msgr2=0x7f65b413af90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 shutdown_connections 2022-08-01T23:03:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:02.775+0000 7f65b359e700 1 -- 172.21.15.32:0/1873234767 wait complete. 2022-08-01T23:03:02.795 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:03:02.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:02.813 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:02.835 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:02.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:02.878 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:03:02.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:02.879 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:02.879 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:02.906 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:03:02.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:03:02.907 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:02.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:02.908 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:02.909 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:02.926 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:03:02.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:03:02.928 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:02.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:03:03.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 Processor -- start 2022-08-01T23:03:03.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- start start 2022-08-01T23:03:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f010c148530 0x7f010c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c0b04e0 con 0x7f010c0a7250 2022-08-01T23:03:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c0b0780 con 0x7f010c0b1f40 2022-08-01T23:03:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c0a9250 con 0x7f010c148530 2022-08-01T23:03:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f010c148530 0x7f010c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f010c148530 0x7f010c0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46064/0 (socket says 172.21.15.32:46064) 2022-08-01T23:03:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 -- 172.21.15.32:0/3118378062 learned_addr learned my addr 172.21.15.32:0/3118378062 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 -- 172.21.15.32:0/3118378062 >> v1:172.21.15.55:6789/0 conn(0x7f010c0b1f40 legacy=0x7f010c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 -- 172.21.15.32:0/3118378062 >> v1:172.21.15.32:6789/0 conn(0x7f010c0a7250 legacy=0x7f010c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 -- 172.21.15.32:0/3118378062 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c0b0a20 con 0x7f010c148530 2022-08-01T23:03:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f0110dc5700 1 --2- 172.21.15.32:0/3118378062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c0b1160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f010c0b3300 tx=0x7f0104009290).ready entity=mon.0 client_cookie=cb2948c95a3187a1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f010bfff700 1 -- 172.21.15.32:0/3118378062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f010400b820 con 0x7f010c148530 2022-08-01T23:03:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f010bfff700 1 -- 172.21.15.32:0/3118378062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010400b980 con 0x7f010c148530 2022-08-01T23:03:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f010bfff700 1 -- 172.21.15.32:0/3118378062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f010400baf0 con 0x7f010c148530 2022-08-01T23:03:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- 172.21.15.32:0/3118378062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 msgr2=0x7f010c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 --2- 172.21.15.32:0/3118378062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c0b1160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f010c0b3300 tx=0x7f0104009290).stop 2022-08-01T23:03:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- 172.21.15.32:0/3118378062 shutdown_connections 2022-08-01T23:03:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 --2- 172.21.15.32:0/3118378062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c0b1160 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- 172.21.15.32:0/3118378062 >> 172.21.15.32:0/3118378062 conn(0x7f010c05f840 msgr2=0x7f010c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.031+0000 7f011382a700 1 -- 172.21.15.32:0/3118378062 shutdown_connections 2022-08-01T23:03:03.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 -- 172.21.15.32:0/3118378062 wait complete. 2022-08-01T23:03:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 Processor -- start 2022-08-01T23:03:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 -- start start 2022-08-01T23:03:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f010c0a7250 0x7f010c0aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b1f40 0x7f010c0ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c0aaae0 con 0x7f010c148530 2022-08-01T23:03:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c0ab240 con 0x7f010c0b1f40 2022-08-01T23:03:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c0ab4e0 con 0x7f010c0a7250 2022-08-01T23:03:03.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0110dc5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b1f40 0x7f010c0ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46072/0 (socket says 172.21.15.32:46072) 2022-08-01T23:03:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 -- 172.21.15.32:0/218722771 learned_addr learned my addr 172.21.15.32:0/218722771 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f01115c6700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f010c0a7250 0x7f010c0aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 -- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f010c0a7250 msgr2=0x7f010c0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f010c0a7250 0x7f010c0aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 -- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b1f40 msgr2=0x7f010c0ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b1f40 0x7f010c0ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 -- 172.21.15.32:0/218722771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104009530 con 0x7f010c148530 2022-08-01T23:03:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f0111dc7700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c156b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f010c0acff0 tx=0x7f00fc007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00fc012400 con 0x7f010c148530 2022-08-01T23:03:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc012560 con 0x7f010c148530 2022-08-01T23:03:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c0abe70 con 0x7f010c148530 2022-08-01T23:03:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.035+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00fc0128c0 con 0x7f010c148530 2022-08-01T23:03:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c14b9e0 con 0x7f010c148530 2022-08-01T23:03:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f00fc019030 con 0x7f010c148530 2022-08-01T23:03:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c0ab6a0 con 0x7f010c148530 2022-08-01T23:03:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f010a7fc700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00ec0539e0 0x7f00ec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f00fc06b140 con 0x7f010c148530 2022-08-01T23:03:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f01115c6700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00ec0539e0 0x7f00ec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.039+0000 7f01115c6700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00ec0539e0 0x7f00ec055ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f010000d000 tx=0x7f0100007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.043+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00fc0420d0 con 0x7f010c148530 2022-08-01T23:03:03.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.171+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f010c0b0cc0 con 0x7f010c148530 2022-08-01T23:03:03.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.171+0000 7f010a7fc700 1 -- 172.21.15.32:0/218722771 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v90) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f00fc0456f0 con 0x7f010c148530 2022-08-01T23:03:03.181 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:03.181 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:03.181 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 90, 2022-08-01T23:03:03.181 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:03:03.182 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:03:03.182 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:03:00.462045+0000", 2022-08-01T23:03:03.182 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:03.182 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:03.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:03:03.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:03:03.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:03:03.184 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:03:03.184 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:03:03.184 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:03:03.184 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:03:03.185 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:03.185 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:03:03.185 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:03:03.185 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:03:03.186 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:03:03.186 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:03:03.186 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 16, 2022-08-01T23:03:03.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:03:03.186 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:03:03.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:03:03.187 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:03:03.187 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:03:03.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:03:03.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:03:03.188 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:03:03.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:03.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:03.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:03.189 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:03.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:03.190 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:03.190 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:03.190 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:03.190 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:03:03.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:03:03.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:03:03.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:03:03.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:03:03.191 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:03.192 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:03.192 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:03.192 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:03.192 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:03.193 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:03.193 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:03.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:03:03.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:03.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:03.194 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:03.194 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:03.195 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:03.195 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:03.195 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:03.195 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:03.196 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:03.196 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:03.196 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:03.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:03.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:03.197 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:03.197 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:03.197 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:03.197 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:03.198 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:03.198 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:03.198 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:03.199 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:03.199 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:03.199 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:03.199 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:03.200 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:03.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:03.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.200 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:03.201 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:03.201 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:03.201 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:03.201 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:03.202 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:03.202 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:03.202 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:03.202 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:03.203 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:03.203 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:03.203 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:03.203 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:03:03.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.204 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:03.204 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:03:03.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.205 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:03:03.205 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:03:03.205 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:03:03.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:03.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:03.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:03.206 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:03.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:03.207 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:03.207 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:03.207 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:03.207 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:03.208 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:03.208 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:03.208 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:03.208 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:03.210 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:03.210 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:03.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.210 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:03:03.210 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:03.211 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:03.211 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:03.211 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:03.211 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:03.212 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:03.212 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:03.212 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:03.212 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:03.213 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:03.213 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:03.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:03.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:03.213 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:03.214 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:03.214 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:03.214 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:03.214 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:03.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:03.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:03.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:03.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:03.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:03.216 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:03.216 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:03.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:03.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.217 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:03.217 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:03.217 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:03.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:03.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:03.218 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:03.218 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:03.218 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:03.219 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:03.219 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:03.219 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:03.219 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:03.219 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:03.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:03:03.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.220 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:03.221 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:03:03.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.221 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:03.221 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:03:03.221 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.222 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.222 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.222 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.223 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.223 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.223 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.223 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:03.224 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.224 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.224 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.224 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.225 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:03:03.225 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.226 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:03:03.226 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.227 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.228 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:03:03.229 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.229 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:03:03.230 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.230 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.230 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.231 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.231 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:03:03.232 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.232 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:03:03.233 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.233 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.234 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:03:03.235 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.235 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:03:03.236 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:03.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.236 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:03:03.237 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:03:03.237 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:03:03.237 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:03:03.237 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.238 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.238 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.238 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.239 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:03.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:03:03.239 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.239 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.240 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.240 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.240 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.240 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.240 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.241 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 84, 2022-08-01T23:03:03.241 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.241 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.242 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.242 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.242 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:03:03.243 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.244 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:03:03.244 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.245 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.245 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.246 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:03:03.246 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:03:03.247 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.248 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.248 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:03:03.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.250 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:03:03.250 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.251 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.251 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.252 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:03:03.252 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.253 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:03:03.253 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:03.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.253 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.254 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:03:03.254 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:03:03.254 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:03:03.254 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:03:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:03:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:03:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:03:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:03:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:03:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.266 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:03:03.266 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:03:03.268 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.268 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.269 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.269 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:03:03.270 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.270 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:03:03.271 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:03.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.271 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:03:03.272 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:03:03.272 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:03:03.272 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:03:03.272 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.273 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.273 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:03.274 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:03:03.274 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.274 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.275 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.275 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.275 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.275 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.276 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.276 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:03.276 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.276 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.276 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:03:03.277 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:03:03.279 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.280 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:03:03.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:03:03.282 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:03:03.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:03:03.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.286 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:03:03.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:03:03.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:03.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.289 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:03:03.289 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:03:03.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:03:03.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:03:03.290 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.290 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.291 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:03.291 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:03:03.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.292 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.292 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.292 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:03.293 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.293 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.294 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:03:03.295 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:03:03.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.296 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.297 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:03:03.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:03:03.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.300 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:03:03.301 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:03:03.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.303 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:03:03.304 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:03:03.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:03.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.306 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:03:03.306 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:03:03.307 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:03:03.307 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:03:03.307 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.307 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.308 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.308 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:03.309 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:03:03.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.309 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.309 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.309 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.310 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.310 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:03.311 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.311 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.311 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.311 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:03:03.312 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:03:03.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.314 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.314 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.315 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:03:03.315 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:03:03.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.317 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:03:03.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:03:03.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.320 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.320 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.321 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:03:03.321 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:03:03.322 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:03.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.323 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:03:03.323 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:03:03.323 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:03:03.324 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:03:03.324 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.324 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.324 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.325 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:03.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:03:03.326 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.326 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.326 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.326 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.328 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.328 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:03:03.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:03:03.330 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.331 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:03:03.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:03:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:03:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:03:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:03:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:03:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:03:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:03:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:03:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:03:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:03:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 73, 2022-08-01T23:03:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:03:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:03:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:03:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:03:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:03:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:03:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:03:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:03:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:03:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:03:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:03:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:03:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:03:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:03:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:03:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:03:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:03:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:03:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:03:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:03:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:03:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:03:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:03:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:03:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:03:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:03:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/4046550885": "2022-08-02T23:02:56.305968+0000", 2022-08-01T23:03:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/4046550885": "2022-08-02T23:02:56.305968+0000" 2022-08-01T23:03:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:03:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:03:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:03:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:03:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:03:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:03:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:03:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:03:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:03:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:03:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:03:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:03:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:03:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:03:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:03:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:03:03.394 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00ec0539e0 msgr2=0x7f00ec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00ec0539e0 0x7f00ec055ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f010000d000 tx=0x7f0100007b70).stop 2022-08-01T23:03:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 msgr2=0x7f010c156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c156b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f010c0acff0 tx=0x7f00fc007bc0).stop 2022-08-01T23:03:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 shutdown_connections 2022-08-01T23:03:03.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f00ec0539e0 0x7f00ec055ea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f010c148530 0x7f010c156b60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b1f40 0x7f010c0ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 --2- 172.21.15.32:0/218722771 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f010c0a7250 0x7f010c0aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 >> 172.21.15.32:0/218722771 conn(0x7f010c05f840 msgr2=0x7f010c143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:03.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 shutdown_connections 2022-08-01T23:03:03.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.175+0000 7f011382a700 1 -- 172.21.15.32:0/218722771 wait complete. 2022-08-01T23:03:03.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.502+0000 7ff322d27700 1 Processor -- start 2022-08-01T23:03:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.502+0000 7ff322d27700 1 -- start start 2022-08-01T23:03:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff31c0b6420 0x7ff31c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff31c142ff0 0x7ff31c143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c0b60e0 con 0x7ff31c0b41f0 2022-08-01T23:03:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c0b2100 con 0x7ff31c0b6420 2022-08-01T23:03:03.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c0b23a0 con 0x7ff31c142ff0 2022-08-01T23:03:03.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff31c0b6420 0x7ff31c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322526700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff31c142ff0 0x7ff31c143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff31c0b6420 0x7ff31c061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46078/0 (socket says 172.21.15.32:46078) 2022-08-01T23:03:03.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 -- 172.21.15.32:0/83308994 learned_addr learned my addr 172.21.15.32:0/83308994 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:03.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 -- 172.21.15.32:0/83308994 >> v2:172.21.15.55:3300/0 conn(0x7ff31c142ff0 msgr2=0x7ff31c143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:03.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 --2- 172.21.15.32:0/83308994 >> v2:172.21.15.55:3300/0 conn(0x7ff31c142ff0 0x7ff31c143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 -- 172.21.15.32:0/83308994 >> v1:172.21.15.55:6789/0 conn(0x7ff31c0b41f0 legacy=0x7ff31c061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff320d23700 1 -- 172.21.15.32:0/83308994 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469620441 0 0) 0x7ff310001110 con 0x7ff31c0b41f0 2022-08-01T23:03:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 -- 172.21.15.32:0/83308994 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0b2640 con 0x7ff31c0b6420 2022-08-01T23:03:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff321d25700 1 --2- 172.21.15.32:0/83308994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c0b6420 0x7ff31c061600 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7ff30c002f70 tx=0x7ff30c006eb0).ready entity=mon.0 client_cookie=f8b97cbdd0771a30 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff320d23700 1 -- 172.21.15.32:0/83308994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30c00d5c0 con 0x7ff31c0b6420 2022-08-01T23:03:03.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff320d23700 1 -- 172.21.15.32:0/83308994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c000e90 con 0x7ff31c0b6420 2022-08-01T23:03:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff320d23700 1 -- 172.21.15.32:0/83308994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30c00d910 con 0x7ff31c0b6420 2022-08-01T23:03:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- 172.21.15.32:0/83308994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c0b6420 msgr2=0x7ff31c061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 --2- 172.21.15.32:0/83308994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c0b6420 0x7ff31c061600 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7ff30c002f70 tx=0x7ff30c006eb0).stop 2022-08-01T23:03:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- 172.21.15.32:0/83308994 shutdown_connections 2022-08-01T23:03:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 --2- 172.21.15.32:0/83308994 >> v2:172.21.15.55:3300/0 conn(0x7ff31c142ff0 0x7ff31c143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 --2- 172.21.15.32:0/83308994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c0b6420 0x7ff31c061600 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- 172.21.15.32:0/83308994 >> 172.21.15.32:0/83308994 conn(0x7ff31c05c620 msgr2=0x7ff31c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- 172.21.15.32:0/83308994 shutdown_connections 2022-08-01T23:03:03.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.506+0000 7ff322d27700 1 -- 172.21.15.32:0/83308994 wait complete. 2022-08-01T23:03:03.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 Processor -- start 2022-08-01T23:03:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 -- start start 2022-08-01T23:03:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff31c0b41f0 0x7ff31c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff31c0b6420 0x7ff31c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 0x7ff31c152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0b0570 con 0x7ff31c142ff0 2022-08-01T23:03:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0b0810 con 0x7ff31c0b41f0 2022-08-01T23:03:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c152940 con 0x7ff31c0b6420 2022-08-01T23:03:03.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 0x7ff31c152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 0x7ff31c152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46084/0 (socket says 172.21.15.32:46084) 2022-08-01T23:03:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 -- 172.21.15.32:0/1973611041 learned_addr learned my addr 172.21.15.32:0/1973611041 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff321d25700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff31c0b41f0 0x7ff31c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff321524700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff31c0b6420 0x7ff31c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 -- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff31c0b6420 msgr2=0x7ff31c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff31c0b6420 0x7ff31c0afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 -- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff31c0b41f0 msgr2=0x7ff31c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff31c0b41f0 0x7ff31c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 -- 172.21.15.32:0/1973611041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c00b1e0 con 0x7ff31c142ff0 2022-08-01T23:03:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322526700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 0x7ff31c152de0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7ff314006e00 tx=0x7ff3140091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff314012600 con 0x7ff31c142ff0 2022-08-01T23:03:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff314012760 con 0x7ff31c142ff0 2022-08-01T23:03:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c0b0290 con 0x7ff31c142ff0 2022-08-01T23:03:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff314012ac0 con 0x7ff31c142ff0 2022-08-01T23:03:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.510+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c1532d0 con 0x7ff31c142ff0 2022-08-01T23:03:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.514+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c153580 con 0x7ff31c142ff0 2022-08-01T23:03:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.514+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff314022070 con 0x7ff31c142ff0 2022-08-01T23:03:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.514+0000 7ff31affd700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff2fc053b00 0x7ff2fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.514+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff31406c320 con 0x7ff31c142ff0 2022-08-01T23:03:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.514+0000 7ff321d25700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff2fc053b00 0x7ff2fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.514+0000 7ff321d25700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff2fc053b00 0x7ff2fc055fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7ff30c01c000 tx=0x7ff30c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.522+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff314043320 con 0x7ff31c142ff0 2022-08-01T23:03:03.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff31c140970 con 0x7ff31c142ff0 2022-08-01T23:03:03.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff31affd700 1 -- 172.21.15.32:0/1973611041 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1562 (secure 0 0 0) 0x7ff31c140970 con 0x7ff31c142ff0 2022-08-01T23:03:03.698 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37},{"gid":6888,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/3985732189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3985732189},{"type":"v1","addr":"172.21.15.32:6835","nonce":3985732189}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[]} 2022-08-01T23:03:03.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff2fc053b00 msgr2=0x7ff2fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff2fc053b00 0x7ff2fc055fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7ff30c01c000 tx=0x7ff30c015000).stop 2022-08-01T23:03:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 msgr2=0x7ff31c152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 0x7ff31c152de0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7ff314006e00 tx=0x7ff3140091d0).stop 2022-08-01T23:03:03.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 shutdown_connections 2022-08-01T23:03:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff2fc053b00 0x7ff2fc055fc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31c142ff0 0x7ff31c152de0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff31c0b6420 0x7ff31c0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 --2- 172.21.15.32:0/1973611041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff31c0b41f0 0x7ff31c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 >> 172.21.15.32:0/1973611041 conn(0x7ff31c05c620 msgr2=0x7ff31c0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 shutdown_connections 2022-08-01T23:03:03.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.690+0000 7ff322d27700 1 -- 172.21.15.32:0/1973611041 wait complete. 2022-08-01T23:03:03.704 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2022-08-01T23:03:03.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 Processor -- start 2022-08-01T23:03:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 -- start start 2022-08-01T23:03:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ea00b01a0 0x7f5ea00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 0x7f5ea00a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea00a7250 con 0x7f5ea00b2710 2022-08-01T23:03:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00a7480 con 0x7f5ea00b01a0 2022-08-01T23:03:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea5840700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00a7720 con 0x7f5ea00afd60 2022-08-01T23:03:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5e9ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ea00b01a0 0x7f5ea00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea503f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 0x7f5ea00a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea503f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 0x7f5ea00a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:57832/0 (socket says 172.21.15.32:57832) 2022-08-01T23:03:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5e9ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ea00b01a0 0x7f5ea00a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:55876/0 (socket says 172.21.15.32:55876) 2022-08-01T23:03:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea503f700 1 -- 172.21.15.32:0/2911590425 learned_addr learned my addr 172.21.15.32:0/2911590425 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.810+0000 7f5ea483e700 1 --1- 172.21.15.32:0/2911590425 >> v1:172.21.15.55:6789/0 conn(0x7f5ea00b2710 0x7f5ea014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:54540/0 (socket says 172.21.15.32:54540) 2022-08-01T23:03:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea503f700 1 -- 172.21.15.32:0/2911590425 >> v2:172.21.15.55:3300/0 conn(0x7f5ea00b01a0 msgr2=0x7f5ea00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea503f700 1 --2- 172.21.15.32:0/2911590425 >> v2:172.21.15.55:3300/0 conn(0x7f5ea00b01a0 0x7f5ea00a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea503f700 1 -- 172.21.15.32:0/2911590425 >> v1:172.21.15.55:6789/0 conn(0x7f5ea00b2710 legacy=0x7f5ea014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea503f700 1 -- 172.21.15.32:0/2911590425 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00a79c0 con 0x7f5ea00afd60 2022-08-01T23:03:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea503f700 1 --2- 172.21.15.32:0/2911590425 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 0x7f5ea00a8a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f5e980032f0 tx=0x7f5e98009290).ready entity=mon.2 client_cookie=d8e13ae77242ec5f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2911590425 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9800b8a0 con 0x7f5ea00afd60 2022-08-01T23:03:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2911590425 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9800ba00 con 0x7f5ea00afd60 2022-08-01T23:03:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2911590425 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9800bb70 con 0x7f5ea00afd60 2022-08-01T23:03:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 -- 172.21.15.32:0/2911590425 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 msgr2=0x7f5ea00a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 --2- 172.21.15.32:0/2911590425 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 0x7f5ea00a8a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f5e980032f0 tx=0x7f5e98009290).stop 2022-08-01T23:03:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 -- 172.21.15.32:0/2911590425 shutdown_connections 2022-08-01T23:03:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 --2- 172.21.15.32:0/2911590425 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00afd60 0x7f5ea00a8a30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 --2- 172.21.15.32:0/2911590425 >> v2:172.21.15.55:3300/0 conn(0x7f5ea00b01a0 0x7f5ea00a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 -- 172.21.15.32:0/2911590425 >> 172.21.15.32:0/2911590425 conn(0x7f5ea0060310 msgr2=0x7f5ea00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:03.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 -- 172.21.15.32:0/2911590425 shutdown_connections 2022-08-01T23:03:03.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 -- 172.21.15.32:0/2911590425 wait complete. 2022-08-01T23:03:03.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 Processor -- start 2022-08-01T23:03:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.814+0000 7f5ea5840700 1 -- start start 2022-08-01T23:03:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 0x7f5ea0142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea00b01a0 0x7f5ea00a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00b2710 0x7f5ea00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0157640 con 0x7f5ea00afd60 2022-08-01T23:03:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b58d0 con 0x7f5ea00b01a0 2022-08-01T23:03:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea014ae90 con 0x7f5ea00b2710 2022-08-01T23:03:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 0x7f5ea0142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 0x7f5ea0142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46096/0 (socket says 172.21.15.32:46096) 2022-08-01T23:03:03.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 -- 172.21.15.32:0/4016739708 learned_addr learned my addr 172.21.15.32:0/4016739708 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:03.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9ffff700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea00b01a0 0x7f5ea00a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea503f700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00b2710 0x7f5ea00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 -- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00b2710 msgr2=0x7f5ea00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00b2710 0x7f5ea00a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 -- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea00b01a0 msgr2=0x7f5ea00a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea00b01a0 0x7f5ea00a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 -- 172.21.15.32:0/4016739708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009530 con 0x7f5ea00afd60 2022-08-01T23:03:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea483e700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 0x7f5ea0142200 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f5ea006b950 tx=0x7f5e94007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e940070d0 con 0x7f5ea00afd60 2022-08-01T23:03:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94007230 con 0x7f5ea00afd60 2022-08-01T23:03:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea01573a0 con 0x7f5ea00afd60 2022-08-01T23:03:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e94005550 con 0x7f5ea00afd60 2022-08-01T23:03:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea0157050 con 0x7f5ea00afd60 2022-08-01T23:03:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea00574e0 con 0x7f5ea00afd60 2022-08-01T23:03:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e94010070 con 0x7f5ea00afd60 2022-08-01T23:03:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9dffb700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e80053a70 0x7f5e80055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5e9406b200 con 0x7f5ea00afd60 2022-08-01T23:03:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.818+0000 7f5e9ffff700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e80053a70 0x7f5e80055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.822+0000 7f5e9ffff700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e80053a70 0x7f5e80055f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f5e8c00b000 tx=0x7f5e8c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.826+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e940422a0 con 0x7f5ea00afd60 2022-08-01T23:03:03.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.986+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea0142890 con 0x7f5ea00afd60 2022-08-01T23:03:03.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5e9dffb700 1 -- 172.21.15.32:0/4016739708 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f5e940458c0 con 0x7f5ea00afd60 2022-08-01T23:03:03.997 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:03.997 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1896218987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1896218987},{"type":"v1","addr":"172.21.15.55:6835","nonce":1896218987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6663,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1158016799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1158016799},{"type":"v1","addr":"172.21.15.32:6833","nonce":1158016799}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37},{"gid":6888,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/3985732189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3985732189},{"type":"v1","addr":"172.21.15.32:6835","nonce":3985732189}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[]} 2022-08-01T23:03:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e80053a70 msgr2=0x7f5e80055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e80053a70 0x7f5e80055f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f5e8c00b000 tx=0x7f5e8c006df0).stop 2022-08-01T23:03:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 msgr2=0x7f5ea0142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 0x7f5ea0142200 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f5ea006b950 tx=0x7f5e94007c60).stop 2022-08-01T23:03:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 shutdown_connections 2022-08-01T23:03:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e80053a70 0x7f5e80055f30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ea00b2710 0x7f5ea00a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ea00b01a0 0x7f5ea00a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 --2- 172.21.15.32:0/4016739708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea00afd60 0x7f5ea0142200 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 >> 172.21.15.32:0/4016739708 conn(0x7f5ea0060310 msgr2=0x7f5ea0148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:04.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 shutdown_connections 2022-08-01T23:03:04.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:03.990+0000 7f5ea5840700 1 -- 172.21.15.32:0/4016739708 wait complete. 2022-08-01T23:03:04.003 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2022-08-01T23:03:04.011 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:03:04.011 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:03:04.012 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:03:04.012 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:04.012 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:03:04.013 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:03:04.013 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:03:04.013 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:04.014 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:03:04.014 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:03:04.015 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:03:04.015 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:04.026 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:03:04.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:03:04.028 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:03:04.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:03:04.029 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:03:04.030 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:03:04.032 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:03:04.033 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:04.070 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:03:04.070 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:04.071 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:03:04.071 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:04.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:03:04.105 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:03:04.137 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:03:04.139 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:03:04.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d2321d700 1 Processor -- start 2022-08-01T23:03:04.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d2321d700 1 -- start start 2022-08-01T23:03:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d2321d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d1c0b1790 0x7f7d1c141c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d2321d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c142180 con 0x7f7d1c13eae0 2022-08-01T23:03:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d2321d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c144ee0 con 0x7f7d1c0ab9e0 2022-08-01T23:03:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d2321d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c1456c0 con 0x7f7d1c0b1790 2022-08-01T23:03:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d1c13eae0 0x7f7d1c141580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33672/0 (socket says 172.21.15.32:33672) 2022-08-01T23:03:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1bfff700 1 -- 172.21.15.32:0/764172887 learned_addr learned my addr 172.21.15.32:0/764172887 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d217ba700 1 --2- 172.21.15.32:0/764172887 >> v2:172.21.15.55:3300/0 conn(0x7f7d1c0b1790 0x7f7d1c141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079601199 0 0) 0x7f7d1c142180 con 0x7f7d1c13eae0 2022-08-01T23:03:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cfc003260 con 0x7f7d1c13eae0 2022-08-01T23:03:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090396167 0 0) 0x7f7d1c144ee0 con 0x7f7d1c0ab9e0 2022-08-01T23:03:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1c142180 con 0x7f7d1c0ab9e0 2022-08-01T23:03:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909490874 0 0) 0x7f7cfc003260 con 0x7f7d1c13eae0 2022-08-01T23:03:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 >> v2:172.21.15.55:3300/0 conn(0x7f7d1c0b1790 msgr2=0x7f7d1c141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:04.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 --2- 172.21.15.32:0/764172887 >> v2:172.21.15.55:3300/0 conn(0x7f7d1c0b1790 0x7f7d1c141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 >> v1:172.21.15.55:6789/0 conn(0x7f7d1c0ab9e0 legacy=0x7f7d1c13e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:04.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c145180 con 0x7f7d1c13eae0 2022-08-01T23:03:04.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f7d0c001660 con 0x7f7d1c13eae0 2022-08-01T23:03:04.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 >> v1:172.21.15.32:6789/0 conn(0x7f7d1c13eae0 legacy=0x7f7d1c141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:04.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc0038e0 0x7f7cfc003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cfc004500 0x7f7cfc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc004e70 0x7f7cfc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1456c0 con 0x7f7cfc0038e0 2022-08-01T23:03:04.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc00bca0 con 0x7f7cfc004e70 2022-08-01T23:03:04.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc00be60 con 0x7f7cfc004500 2022-08-01T23:03:04.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc0038e0 0x7f7cfc003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d217ba700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc004e70 0x7f7cfc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.170+0000 7f7d1bfff700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cfc004500 0x7f7cfc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d20fb9700 1 -- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cfc004500 msgr2=0x7f7cfc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:04.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cfc004500 0x7f7cfc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d20fb9700 1 -- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc004e70 msgr2=0x7f7cfc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:04.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc004e70 0x7f7cfc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d20fb9700 1 -- 172.21.15.32:0/764172887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c145180 con 0x7f7cfc0038e0 2022-08-01T23:03:04.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d217ba700 1 -- 172.21.15.32:0/764172887 reap_dead start 2022-08-01T23:03:04.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d1bfff700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cfc004500 0x7f7cfc004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc0038e0 0x7f7cfc003ce0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f7d04001490 tx=0x7f7d04007d80).ready entity=mon.0 client_cookie=66ff1a43502f9242 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d04001780 con 0x7f7cfc0038e0 2022-08-01T23:03:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04006890 con 0x7f7cfc0038e0 2022-08-01T23:03:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d1b7fe700 1 -- 172.21.15.32:0/764172887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d04006bf0 con 0x7f7cfc0038e0 2022-08-01T23:03:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 -- 172.21.15.32:0/764172887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc0038e0 msgr2=0x7f7cfc003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc0038e0 0x7f7cfc003ce0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f7d04001490 tx=0x7f7d04007d80).stop 2022-08-01T23:03:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 -- 172.21.15.32:0/764172887 shutdown_connections 2022-08-01T23:03:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 --2- 172.21.15.32:0/764172887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cfc0038e0 0x7f7cfc003ce0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 -- 172.21.15.32:0/764172887 >> 172.21.15.32:0/764172887 conn(0x7f7d1c0b5570 msgr2=0x7f7d1c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 -- 172.21.15.32:0/764172887 shutdown_connections 2022-08-01T23:03:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 -- 172.21.15.32:0/764172887 wait complete. 2022-08-01T23:03:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 Processor -- start 2022-08-01T23:03:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.174+0000 7f7d2321d700 1 -- start start 2022-08-01T23:03:04.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0038e0 0x7f7d1c148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d1c142fc0 0x7f7d1c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 0x7f7d1c0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c151270 con 0x7f7d1c150df0 2022-08-01T23:03:04.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c142ba0 con 0x7f7cfc0038e0 2022-08-01T23:03:04.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c142660 con 0x7f7d1c142fc0 2022-08-01T23:03:04.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 0x7f7d1c0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 0x7f7d1c0ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46144/0 (socket says 172.21.15.32:46144) 2022-08-01T23:03:04.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 -- 172.21.15.32:0/1441980575 learned_addr learned my addr 172.21.15.32:0/1441980575 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:04.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d1bfff700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d1c142fc0 0x7f7d1c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0038e0 0x7f7d1c148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 -- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d1c142fc0 msgr2=0x7f7d1c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:04.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d1c142fc0 0x7f7d1c150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 -- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0038e0 msgr2=0x7f7d1c148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0038e0 0x7f7d1c148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 -- 172.21.15.32:0/1441980575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c142900 con 0x7f7d1c150df0 2022-08-01T23:03:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d217ba700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 0x7f7d1c0ae7e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f7d1400b000 tx=0x7f7d14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:04.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d14018070 con 0x7f7d1c150df0 2022-08-01T23:03:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14010ec0 con 0x7f7d1c150df0 2022-08-01T23:03:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c150460 con 0x7f7d1c150df0 2022-08-01T23:03:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d140137d0 con 0x7f7d1c150df0 2022-08-01T23:03:04.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c143510 con 0x7f7d1c150df0 2022-08-01T23:03:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.178+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c1505f0 con 0x7f7d1c150df0 2022-08-01T23:03:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.182+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d14013930 con 0x7f7d1c150df0 2022-08-01T23:03:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.182+0000 7f7d19ffb700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053aa0 0x7f7d00055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:04.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.182+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7d1406bf90 con 0x7f7d1c150df0 2022-08-01T23:03:04.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.182+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053aa0 0x7f7d00055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:04.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.182+0000 7f7d20fb9700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053aa0 0x7f7d00055f60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f7d0401b000 tx=0x7f7d04015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:04.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.186+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d14042fd0 con 0x7f7d1c150df0 2022-08-01T23:03:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:04.322+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f7d1c0574e0 con 0x7f7d1c150df0 2022-08-01T23:03:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.166+0000 7f7d19ffb700 1 -- 172.21.15.32:0/1441980575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7d140465f0 con 0x7f7d1c150df0 2022-08-01T23:03:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053aa0 msgr2=0x7f7d00055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053aa0 0x7f7d00055f60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f7d0401b000 tx=0x7f7d04015000).stop 2022-08-01T23:03:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 msgr2=0x7f7d1c0ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 0x7f7d1c0ae7e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f7d1400b000 tx=0x7f7d14008040).stop 2022-08-01T23:03:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 shutdown_connections 2022-08-01T23:03:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7d00053aa0 0x7f7d00055f60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d1c150df0 0x7f7d1c0ae7e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7d1c142fc0 0x7f7d1c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 --2- 172.21.15.32:0/1441980575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0038e0 0x7f7d1c148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 >> 172.21.15.32:0/1441980575 conn(0x7f7d1c0b5570 msgr2=0x7f7d1c0608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 shutdown_connections 2022-08-01T23:03:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.170+0000 7f7d2321d700 1 -- 172.21.15.32:0/1441980575 wait complete. 2022-08-01T23:03:05.182 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:03:05.195 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:03:05.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:03:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 Processor -- start 2022-08-01T23:03:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 -- start start 2022-08-01T23:03:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8980b2710 0x7fb8981401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8980ac880 0x7fb8981406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb898067a10 con 0x7fb8980b0270 2022-08-01T23:03:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8980ad740 con 0x7fb8980ac880 2022-08-01T23:03:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d85c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb898067cb0 con 0x7fb8980b2710 2022-08-01T23:03:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb897fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8980b0270 0x7fb898143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33692/0 (socket says 172.21.15.32:33692) 2022-08-01T23:03:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb897fff700 1 -- 172.21.15.32:0/1211570104 learned_addr learned my addr 172.21.15.32:0/1211570104 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.290+0000 7fb89d05b700 1 --2- 172.21.15.32:0/1211570104 >> v2:172.21.15.32:3300/0 conn(0x7fb8980ac880 0x7fb8981406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89c85a700 1 --2- 172.21.15.32:0/1211570104 >> v2:172.21.15.55:3300/0 conn(0x7fb8980b2710 0x7fb8981401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d05b700 1 -- 172.21.15.32:0/1211570104 >> v2:172.21.15.55:3300/0 conn(0x7fb8980b2710 msgr2=0x7fb8981401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d05b700 1 --2- 172.21.15.32:0/1211570104 >> v2:172.21.15.55:3300/0 conn(0x7fb8980b2710 0x7fb8981401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d05b700 1 -- 172.21.15.32:0/1211570104 >> v1:172.21.15.32:6789/0 conn(0x7fb8980b0270 legacy=0x7fb898143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d05b700 1 -- 172.21.15.32:0/1211570104 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898147160 con 0x7fb8980ac880 2022-08-01T23:03:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d05b700 1 --2- 172.21.15.32:0/1211570104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980ac880 0x7fb8981406d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fb8900032f0 tx=0x7fb8900091d0).ready entity=mon.0 client_cookie=10bdb089682bb712 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb8977fe700 1 -- 172.21.15.32:0/1211570104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89000b680 con 0x7fb8980ac880 2022-08-01T23:03:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb8977fe700 1 -- 172.21.15.32:0/1211570104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000b7e0 con 0x7fb8980ac880 2022-08-01T23:03:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb8977fe700 1 -- 172.21.15.32:0/1211570104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89000b950 con 0x7fb8980ac880 2022-08-01T23:03:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 -- 172.21.15.32:0/1211570104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980ac880 msgr2=0x7fb8981406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 --2- 172.21.15.32:0/1211570104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980ac880 0x7fb8981406d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fb8900032f0 tx=0x7fb8900091d0).stop 2022-08-01T23:03:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 -- 172.21.15.32:0/1211570104 shutdown_connections 2022-08-01T23:03:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 --2- 172.21.15.32:0/1211570104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980ac880 0x7fb8981406d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 --2- 172.21.15.32:0/1211570104 >> v2:172.21.15.55:3300/0 conn(0x7fb8980b2710 0x7fb8981401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 -- 172.21.15.32:0/1211570104 >> 172.21.15.32:0/1211570104 conn(0x7fb898060310 msgr2=0x7fb8980b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 -- 172.21.15.32:0/1211570104 shutdown_connections 2022-08-01T23:03:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 -- 172.21.15.32:0/1211570104 wait complete. 2022-08-01T23:03:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 Processor -- start 2022-08-01T23:03:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.294+0000 7fb89d85c700 1 -- start start 2022-08-01T23:03:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8980ac880 0x7fb8981416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8980b0270 0x7fb898155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 0x7fb8980abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980ac370 con 0x7fb8980b2710 2022-08-01T23:03:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb898156970 con 0x7fb8980b0270 2022-08-01T23:03:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb898156430 con 0x7fb8980ac880 2022-08-01T23:03:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 0x7fb8980abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 0x7fb8980abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46160/0 (socket says 172.21.15.32:46160) 2022-08-01T23:03:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 -- 172.21.15.32:0/2087183881 learned_addr learned my addr 172.21.15.32:0/2087183881 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89c85a700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8980ac880 0x7fb8981416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb897fff700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8980b0270 0x7fb898155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 -- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8980ac880 msgr2=0x7fb8981416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8980ac880 0x7fb8981416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 -- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8980b0270 msgr2=0x7fb898155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8980b0270 0x7fb898155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 -- 172.21.15.32:0/2087183881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890009470 con 0x7fb8980b2710 2022-08-01T23:03:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d05b700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 0x7fb8980abe80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fb8900032f0 tx=0x7fb890016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89000b680 con 0x7fb8980b2710 2022-08-01T23:03:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890021630 con 0x7fb8980b2710 2022-08-01T23:03:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8981566d0 con 0x7fb8980b2710 2022-08-01T23:03:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89000bee0 con 0x7fb8980b2710 2022-08-01T23:03:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.298+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb898140ea0 con 0x7fb8980b2710 2022-08-01T23:03:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.302+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8980a9ff0 con 0x7fb8980b2710 2022-08-01T23:03:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.302+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb890027020 con 0x7fb8980b2710 2022-08-01T23:03:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.302+0000 7fb895ffb700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb8780539e0 0x7fb878055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.302+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb89006d120 con 0x7fb8980b2710 2022-08-01T23:03:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.302+0000 7fb89c85a700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb8780539e0 0x7fb878055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.302+0000 7fb89c85a700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb8780539e0 0x7fb878055ea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fb88c00b000 tx=0x7fb88c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.306+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8980a9ff0 con 0x7fb8980b2710 2022-08-01T23:03:05.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.470+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb8980a9ff0 con 0x7fb8980b2710 2022-08-01T23:03:05.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.470+0000 7fb895ffb700 1 -- 172.21.15.32:0/2087183881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fb8980a9ff0 con 0x7fb8980b2710 2022-08-01T23:03:05.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:05.480 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:05.480 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:03:05.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.481 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:03:05.481 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:03:05.481 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:05.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.483 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:03:05.483 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:03:05.484 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:05.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.484 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.485 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:03:05.486 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:03:05.486 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:05.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.487 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.488 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:03:05.488 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:03:05.488 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.488 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.488 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.489 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.490 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:03:05.490 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:03:05.490 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.490 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.490 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.491 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.492 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:03:05.492 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:03:05.492 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.492 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.492 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.493 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.494 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:03:05.494 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:03:05.494 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.494 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.494 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.495 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.495 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:03:05.496 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:03:05.496 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.496 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.496 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.497 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.497 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:03:05.498 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:03:05.498 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.498 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.498 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.498 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.499 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:03:05.500 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:03:05.500 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.500 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.500 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.501 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.502 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:03:05.502 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:03:05.502 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.502 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.503 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.503 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.504 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:03:05.504 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:03:05.504 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.504 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:05.504 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:03:05.505 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:03:05.505 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.506 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:03:05.506 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:03:05.506 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.506 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:05.507 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:03:05.507 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:03:05.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.508 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:03:05.508 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:03:05.508 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.509 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:05.509 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:05.509 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:05.509 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.509 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.510 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:03:05.510 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:03:05.510 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.511 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:05.511 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:03:05.511 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:05.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:05.512 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:03:05.512 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:03:05.512 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:05.513 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:05.513 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:03:05.513 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:05.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:05.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:05.514 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb8780539e0 msgr2=0x7fb878055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb8780539e0 0x7fb878055ea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fb88c00b000 tx=0x7fb88c006df0).stop 2022-08-01T23:03:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 msgr2=0x7fb8980abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 0x7fb8980abe80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fb8900032f0 tx=0x7fb890016000).stop 2022-08-01T23:03:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 shutdown_connections 2022-08-01T23:03:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb8780539e0 0x7fb878055ea0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8980b2710 0x7fb8980abe80 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8980b0270 0x7fb898155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 --2- 172.21.15.32:0/2087183881 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8980ac880 0x7fb8981416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 >> 172.21.15.32:0/2087183881 conn(0x7fb898060310 msgr2=0x7fb89805c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 shutdown_connections 2022-08-01T23:03:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.474+0000 7fb89d85c700 1 -- 172.21.15.32:0/2087183881 wait complete. 2022-08-01T23:03:05.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d1ccf700 1 Processor -- start 2022-08-01T23:03:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d1ccf700 1 -- start start 2022-08-01T23:03:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d1ccf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc1466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d1ccf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc0aee80 con 0x7f33cc0ad750 2022-08-01T23:03:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d1ccf700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc0b70b0 con 0x7f33cc0614b0 2022-08-01T23:03:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d1ccf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0af0b0 con 0x7f33cc061890 2022-08-01T23:03:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33cc0ad750 0x7f33cc1405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33706/0 (socket says 172.21.15.32:33706) 2022-08-01T23:03:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cbfff700 1 -- 172.21.15.32:0/2575216097 learned_addr learned my addr 172.21.15.32:0/2575216097 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d14ce700 1 --2- 172.21.15.32:0/2575216097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc1466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cb7fe700 1 -- 172.21.15.32:0/2575216097 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249871905 0 0) 0x7f33cc0aee80 con 0x7f33cc0ad750 2022-08-01T23:03:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cb7fe700 1 -- 172.21.15.32:0/2575216097 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b8003260 con 0x7f33cc0ad750 2022-08-01T23:03:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cb7fe700 1 -- 172.21.15.32:0/2575216097 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2314661158 0 0) 0x7f33b8003260 con 0x7f33cc0ad750 2022-08-01T23:03:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d14ce700 1 -- 172.21.15.32:0/2575216097 >> v1:172.21.15.55:6789/0 conn(0x7f33cc0614b0 legacy=0x7f33cc0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d14ce700 1 -- 172.21.15.32:0/2575216097 >> v1:172.21.15.32:6789/0 conn(0x7f33cc0ad750 legacy=0x7f33cc1405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d14ce700 1 -- 172.21.15.32:0/2575216097 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc0af350 con 0x7f33cc061890 2022-08-01T23:03:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33d14ce700 1 --2- 172.21.15.32:0/2575216097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc1466b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f33bc001b80 tx=0x7f33c4006df0).ready entity=mon.2 client_cookie=102faf1d14660ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cb7fe700 1 -- 172.21.15.32:0/2575216097 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33bc0019e0 con 0x7f33cc061890 2022-08-01T23:03:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cb7fe700 1 -- 172.21.15.32:0/2575216097 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4000e90 con 0x7f33cc061890 2022-08-01T23:03:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.618+0000 7f33cb7fe700 1 -- 172.21.15.32:0/2575216097 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c400d4a0 con 0x7f33cc061890 2022-08-01T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 -- 172.21.15.32:0/2575216097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 msgr2=0x7f33cc1466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/2575216097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc1466b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f33bc001b80 tx=0x7f33c4006df0).stop 2022-08-01T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 -- 172.21.15.32:0/2575216097 shutdown_connections 2022-08-01T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/2575216097 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc1466b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 -- 172.21.15.32:0/2575216097 >> 172.21.15.32:0/2575216097 conn(0x7f33cc066f70 msgr2=0x7f33cc067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 -- 172.21.15.32:0/2575216097 shutdown_connections 2022-08-01T23:03:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 -- 172.21.15.32:0/2575216097 wait complete. 2022-08-01T23:03:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 Processor -- start 2022-08-01T23:03:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.622+0000 7f33d1ccf700 1 -- start start 2022-08-01T23:03:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33cc0614b0 0x7f33cc146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 0x7f33cc1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc1565b0 con 0x7f33cc0ad750 2022-08-01T23:03:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0a2110 con 0x7f33cc0614b0 2022-08-01T23:03:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc09fa00 con 0x7f33cc061890 2022-08-01T23:03:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 0x7f33cc1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 0x7f33cc1560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46174/0 (socket says 172.21.15.32:46174) 2022-08-01T23:03:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 -- 172.21.15.32:0/3019971227 learned_addr learned my addr 172.21.15.32:0/3019971227 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33cbfff700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d0ccd700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33cc0614b0 0x7f33cc146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 -- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 msgr2=0x7f33cc155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 -- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33cc0614b0 msgr2=0x7f33cc146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33cc0614b0 0x7f33cc146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 -- 172.21.15.32:0/3019971227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c400b180 con 0x7f33cc0ad750 2022-08-01T23:03:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d14ce700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 0x7f33cc1560c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f33bc001b80 tx=0x7f33cc0b6720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c4015070 con 0x7f33cc0ad750 2022-08-01T23:03:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4012770 con 0x7f33cc0ad750 2022-08-01T23:03:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc0a2340 con 0x7f33cc0ad750 2022-08-01T23:03:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c4012ad0 con 0x7f33cc0ad750 2022-08-01T23:03:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.626+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc0b2f80 con 0x7f33cc0ad750 2022-08-01T23:03:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.630+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33c4023030 con 0x7f33cc0ad750 2022-08-01T23:03:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.630+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc09fc00 con 0x7f33cc0ad750 2022-08-01T23:03:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.630+0000 7f33c9ffb700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33ac053aa0 0x7f33ac055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.630+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f33c406c310 con 0x7f33cc0ad750 2022-08-01T23:03:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.630+0000 7f33d0ccd700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33ac053aa0 0x7f33ac055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.630+0000 7f33d0ccd700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33ac053aa0 0x7f33ac055f60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f33c000d000 tx=0x7f33c0007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.634+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33c4043410 con 0x7f33cc0ad750 2022-08-01T23:03:05.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.758+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33cc0af5f0 con 0x7f33cc0ad750 2022-08-01T23:03:05.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33c9ffb700 1 -- 172.21.15.32:0/3019971227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f33c4021090 con 0x7f33cc0ad750 2022-08-01T23:03:05.770 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:05.770 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:05.159758+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33ac053aa0 msgr2=0x7f33ac055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33ac053aa0 0x7f33ac055f60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f33c000d000 tx=0x7f33c0007b70).stop 2022-08-01T23:03:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 msgr2=0x7f33cc1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 0x7f33cc1560c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f33bc001b80 tx=0x7f33cc0b6720).stop 2022-08-01T23:03:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 shutdown_connections 2022-08-01T23:03:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33ac053aa0 0x7f33ac055f60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33cc0ad750 0x7f33cc1560c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33cc061890 0x7f33cc155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 --2- 172.21.15.32:0/3019971227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33cc0614b0 0x7f33cc146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 >> 172.21.15.32:0/3019971227 conn(0x7f33cc066f70 msgr2=0x7f33cc145e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 shutdown_connections 2022-08-01T23:03:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.762+0000 7f33d1ccf700 1 -- 172.21.15.32:0/3019971227 wait complete. 2022-08-01T23:03:05.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:03:05.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:05.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 Processor -- start 2022-08-01T23:03:05.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- start start 2022-08-01T23:03:05.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72a40b01a0 0x7f72a414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40a7250 con 0x7f72a40afd60 2022-08-01T23:03:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a40a7480 con 0x7f72a40b01a0 2022-08-01T23:03:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40a7720 con 0x7f72a40b2710 2022-08-01T23:03:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72a40afd60 0x7f72a40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33724/0 (socket says 172.21.15.32:33724) 2022-08-01T23:03:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a37fe700 1 -- 172.21.15.32:0/4074908704 learned_addr learned my addr 172.21.15.32:0/4074908704 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a27fc700 1 --2- 172.21.15.32:0/4074908704 >> v2:172.21.15.55:3300/0 conn(0x7f72a40b01a0 0x7f72a414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a2ffd700 1 --2- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a1ffb700 1 -- 172.21.15.32:0/4074908704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202604355 0 0) 0x7f72a40a7250 con 0x7f72a40afd60 2022-08-01T23:03:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a1ffb700 1 -- 172.21.15.32:0/4074908704 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7288003260 con 0x7f72a40afd60 2022-08-01T23:03:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a27fc700 1 -- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 msgr2=0x7f72a414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a27fc700 1 --2- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a27fc700 1 -- 172.21.15.32:0/4074908704 >> v1:172.21.15.32:6789/0 conn(0x7f72a40afd60 legacy=0x7f72a40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a27fc700 1 -- 172.21.15.32:0/4074908704 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40a79c0 con 0x7f72a40b01a0 2022-08-01T23:03:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a27fc700 1 --2- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40b01a0 0x7f72a414b4c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f7294002f70 tx=0x7f7294006df0).ready entity=mon.1 client_cookie=b0fe0b48a9381f32 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a1ffb700 1 -- 172.21.15.32:0/4074908704 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729400d4a0 con 0x7f72a40b01a0 2022-08-01T23:03:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a1ffb700 1 -- 172.21.15.32:0/4074908704 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294000e90 con 0x7f72a40b01a0 2022-08-01T23:03:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a1ffb700 1 -- 172.21.15.32:0/4074908704 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729400d7f0 con 0x7f72a40b01a0 2022-08-01T23:03:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40b01a0 msgr2=0x7f72a414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 --2- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40b01a0 0x7f72a414b4c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f7294002f70 tx=0x7f7294006df0).stop 2022-08-01T23:03:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- 172.21.15.32:0/4074908704 shutdown_connections 2022-08-01T23:03:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 --2- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40b01a0 0x7f72a414b4c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 --2- 172.21.15.32:0/4074908704 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- 172.21.15.32:0/4074908704 >> 172.21.15.32:0/4074908704 conn(0x7f72a4060310 msgr2=0x7f72a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- 172.21.15.32:0/4074908704 shutdown_connections 2022-08-01T23:03:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.882+0000 7f72a3fff700 1 -- 172.21.15.32:0/4074908704 wait complete. 2022-08-01T23:03:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 Processor -- start 2022-08-01T23:03:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 -- start start 2022-08-01T23:03:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40afd60 0x7f72a40a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 0x7f72a40a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a4067400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40b5e00 con 0x7f72a40b01a0 2022-08-01T23:03:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40aa450 con 0x7f72a40afd60 2022-08-01T23:03:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a414baa0 con 0x7f72a40b2710 2022-08-01T23:03:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 0x7f72a40a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 0x7f72a40a9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46186/0 (socket says 172.21.15.32:46186) 2022-08-01T23:03:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 -- 172.21.15.32:0/2483398853 learned_addr learned my addr 172.21.15.32:0/2483398853 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a2ffd700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40afd60 0x7f72a40a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a37fe700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a4067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 -- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 msgr2=0x7f72a4067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a4067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 -- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40afd60 msgr2=0x7f72a40a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40afd60 0x7f72a40a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 -- 172.21.15.32:0/2483398853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729400b180 con 0x7f72a40b01a0 2022-08-01T23:03:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a37fe700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a4067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a2ffd700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40afd60 0x7f72a40a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a27fc700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 0x7f72a40a9f60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f72a4157540 tx=0x7f7294014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7294016070 con 0x7f72a40b01a0 2022-08-01T23:03:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294013a00 con 0x7f72a40b01a0 2022-08-01T23:03:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a4156fd0 con 0x7f72a40b01a0 2022-08-01T23:03:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7294013d60 con 0x7f72a40b01a0 2022-08-01T23:03:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.886+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a414d8d0 con 0x7f72a40b01a0 2022-08-01T23:03:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.890+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a4157380 con 0x7f72a40b01a0 2022-08-01T23:03:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.890+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7294025030 con 0x7f72a40b01a0 2022-08-01T23:03:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.890+0000 7f7293fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7280053aa0 0x7f7280055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.890+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f729406cd10 con 0x7f72a40b01a0 2022-08-01T23:03:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.890+0000 7f72a2ffd700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7280053aa0 0x7f7280055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.890+0000 7f72a2ffd700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7280053aa0 0x7f7280055f60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f729800b000 tx=0x7f7298008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:05.894+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7294072020 con 0x7f72a40b01a0 2022-08-01T23:03:06.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.022+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f72a40b32a0 con 0x7f72a40b01a0 2022-08-01T23:03:06.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.022+0000 7f7293fff700 1 -- 172.21.15.32:0/2483398853 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v92) v1 ==== 110+0+10 (secure 0 0 0) 0x7f72a40b32a0 con 0x7f72a40b01a0 2022-08-01T23:03:06.031 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7280053aa0 msgr2=0x7f7280055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7280053aa0 0x7f7280055f60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f729800b000 tx=0x7f7298008040).stop 2022-08-01T23:03:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 msgr2=0x7f72a40a9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 0x7f72a40a9f60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f72a4157540 tx=0x7f7294014000).stop 2022-08-01T23:03:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 shutdown_connections 2022-08-01T23:03:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7280053aa0 0x7f7280055f60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72a40b2710 0x7f72a4067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40b01a0 0x7f72a40a9f60 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 --2- 172.21.15.32:0/2483398853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72a40afd60 0x7f72a40a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 >> 172.21.15.32:0/2483398853 conn(0x7f72a4060310 msgr2=0x7f72a4149740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 shutdown_connections 2022-08-01T23:03:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.026+0000 7f72a3fff700 1 -- 172.21.15.32:0/2483398853 wait complete. 2022-08-01T23:03:06.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:06.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc296574700 1 Processor -- start 2022-08-01T23:03:06.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc296574700 1 -- start start 2022-08-01T23:03:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc296574700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2900614b0 0x7fc290061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc296574700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2900aee80 con 0x7fc2900ad750 2022-08-01T23:03:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc296574700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2900b70b0 con 0x7fc2901405d0 2022-08-01T23:03:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc296574700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2900af0b0 con 0x7fc2900614b0 2022-08-01T23:03:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc294d71700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2900ad750 0x7fc2900b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33734/0 (socket says 172.21.15.32:33734) 2022-08-01T23:03:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc294d71700 1 -- 172.21.15.32:0/910937545 learned_addr learned my addr 172.21.15.32:0/910937545 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295572700 1 --2- 172.21.15.32:0/910937545 >> v2:172.21.15.32:3300/0 conn(0x7fc2900614b0 0x7fc290061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056587907 0 0) 0x7fc2900aee80 con 0x7fc2900ad750 2022-08-01T23:03:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc274003260 con 0x7fc2900ad750 2022-08-01T23:03:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1644131513 0 0) 0x7fc274003260 con 0x7fc2900ad750 2022-08-01T23:03:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 >> v2:172.21.15.32:3300/0 conn(0x7fc2900614b0 msgr2=0x7fc290061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 --2- 172.21.15.32:0/910937545 >> v2:172.21.15.32:3300/0 conn(0x7fc2900614b0 0x7fc290061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 >> v1:172.21.15.55:6789/0 conn(0x7fc2901405d0 legacy=0x7fc2901466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2900af350 con 0x7fc2900ad750 2022-08-01T23:03:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fc2880019e0 con 0x7fc2900ad750 2022-08-01T23:03:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 >> v1:172.21.15.32:6789/0 conn(0x7fc2900ad750 legacy=0x7fc2900b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc274003750 0x7fc274003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc274004500 0x7fc274004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc27400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900af0b0 con 0x7fc274004e70 2022-08-01T23:03:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400bc70 con 0x7fc274004500 2022-08-01T23:03:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400be30 con 0x7fc274003750 2022-08-01T23:03:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295572700 1 --2- 172.21.15.32:0/910937545 >> v2:172.21.15.32:3300/0 conn(0x7fc2900614b0 0x7fc290061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc27400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc294d71700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc274004500 0x7fc274004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295572700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc274003750 0x7fc274003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 -- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc274003750 msgr2=0x7fc274003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:06.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc274003750 0x7fc274003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 -- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc274004500 msgr2=0x7fc274004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc274004500 0x7fc274004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 -- 172.21.15.32:0/910937545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274003260 con 0x7fc274004e70 2022-08-01T23:03:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295572700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc274003750 0x7fc274003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.142+0000 7fc295d73700 1 -- 172.21.15.32:0/910937545 reap_dead start 2022-08-01T23:03:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295d73700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc27400b750 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fc28c001020 tx=0x7fc28c0079e0).ready entity=mon.0 client_cookie=407f22e0b39a3f26 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc28c007cf0 con 0x7fc274004e70 2022-08-01T23:03:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c007030 con 0x7fc274004e70 2022-08-01T23:03:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc283fff700 1 -- 172.21.15.32:0/910937545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc28c007390 con 0x7fc274004e70 2022-08-01T23:03:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- 172.21.15.32:0/910937545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 msgr2=0x7fc27400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc27400b750 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fc28c001020 tx=0x7fc28c0079e0).stop 2022-08-01T23:03:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- 172.21.15.32:0/910937545 shutdown_connections 2022-08-01T23:03:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 --2- 172.21.15.32:0/910937545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc27400b750 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- 172.21.15.32:0/910937545 >> 172.21.15.32:0/910937545 conn(0x7fc290066f70 msgr2=0x7fc290067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- 172.21.15.32:0/910937545 shutdown_connections 2022-08-01T23:03:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- 172.21.15.32:0/910937545 wait complete. 2022-08-01T23:03:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 Processor -- start 2022-08-01T23:03:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- start start 2022-08-01T23:03:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc290140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2900b3d30 0x7fc290152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2900b1b80 0x7fc29013f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc290153320 con 0x7fc274004e70 2022-08-01T23:03:06.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900b1290 con 0x7fc2900b1b80 2022-08-01T23:03:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc296574700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900b14c0 con 0x7fc2900b3d30 2022-08-01T23:03:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295572700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc290140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295572700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc290140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46204/0 (socket says 172.21.15.32:46204) 2022-08-01T23:03:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295572700 1 -- 172.21.15.32:0/1359838958 learned_addr learned my addr 172.21.15.32:0/1359838958 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc294d71700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2900b3d30 0x7fc290152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295d73700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2900b1b80 0x7fc29013f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295572700 1 -- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2900b3d30 msgr2=0x7fc290152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295572700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2900b3d30 0x7fc290152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.146+0000 7fc295572700 1 -- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2900b1b80 msgr2=0x7fc29013f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc295572700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2900b1b80 0x7fc29013f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc295572700 1 -- 172.21.15.32:0/1359838958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290143d60 con 0x7fc274004e70 2022-08-01T23:03:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc294d71700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2900b3d30 0x7fc290152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc295572700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc290140f80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fc2901430f0 tx=0x7fc284008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2840115a0 con 0x7fc274004e70 2022-08-01T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284000e90 con 0x7fc274004e70 2022-08-01T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2900b25f0 con 0x7fc274004e70 2022-08-01T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2840118f0 con 0x7fc274004e70 2022-08-01T23:03:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc290142c90 con 0x7fc274004e70 2022-08-01T23:03:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2900b1f90 con 0x7fc274004e70 2022-08-01T23:03:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc284019020 con 0x7fc274004e70 2022-08-01T23:03:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc2827fc700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2780539e0 0x7fc278055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc28406b5e0 con 0x7fc274004e70 2022-08-01T23:03:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc294d71700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2780539e0 0x7fc278055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.150+0000 7fc294d71700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2780539e0 0x7fc278055ea0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fc288005000 tx=0x7fc288009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.158+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2840426e0 con 0x7fc274004e70 2022-08-01T23:03:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.286+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc290144300 con 0x7fc274004e70 2022-08-01T23:03:06.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.286+0000 7fc2827fc700 1 -- 172.21.15.32:0/1359838958 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc284045d00 con 0x7fc274004e70 2022-08-01T23:03:06.294 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.286+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2780539e0 msgr2=0x7fc278055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.286+0000 7fc296574700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2780539e0 0x7fc278055ea0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fc288005000 tx=0x7fc288009040).stop 2022-08-01T23:03:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.286+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 msgr2=0x7fc290140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.286+0000 7fc296574700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc290140f80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fc2901430f0 tx=0x7fc284008040).stop 2022-08-01T23:03:06.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 shutdown_connections 2022-08-01T23:03:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc2780539e0 0x7fc278055ea0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2900b1b80 0x7fc29013f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc2900b3d30 0x7fc290152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 --2- 172.21.15.32:0/1359838958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc274004e70 0x7fc290140f80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 >> 172.21.15.32:0/1359838958 conn(0x7fc290066f70 msgr2=0x7fc290067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 shutdown_connections 2022-08-01T23:03:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.290+0000 7fc296574700 1 -- 172.21.15.32:0/1359838958 wait complete. 2022-08-01T23:03:06.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:03:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 Processor -- start 2022-08-01T23:03:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 -- start start 2022-08-01T23:03:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f17480b5630 0x7f17480af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f17480acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17480b5a10 con 0x7f17480b07a0 2022-08-01T23:03:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1748145470 con 0x7f17480b5630 2022-08-01T23:03:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174df65700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17480ae180 con 0x7f1748147730 2022-08-01T23:03:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f1747fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17480b07a0 0x7f1748067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33754/0 (socket says 172.21.15.32:33754) 2022-08-01T23:03:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f1747fff700 1 -- 172.21.15.32:0/3475133199 learned_addr learned my addr 172.21.15.32:0/3475133199 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174d764700 1 --2- 172.21.15.32:0/3475133199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f17480acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f174cf63700 1 --2- 172.21.15.32:0/3475133199 >> v2:172.21.15.55:3300/0 conn(0x7f17480b5630 0x7f17480af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f17477fe700 1 -- 172.21.15.32:0/3475133199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827810215 0 0) 0x7f17480b5a10 con 0x7f17480b07a0 2022-08-01T23:03:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.410+0000 7f17477fe700 1 -- 172.21.15.32:0/3475133199 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1734003260 con 0x7f17480b07a0 2022-08-01T23:03:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174d764700 1 -- 172.21.15.32:0/3475133199 >> v2:172.21.15.55:3300/0 conn(0x7f17480b5630 msgr2=0x7f17480af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174d764700 1 --2- 172.21.15.32:0/3475133199 >> v2:172.21.15.55:3300/0 conn(0x7f17480b5630 0x7f17480af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174d764700 1 -- 172.21.15.32:0/3475133199 >> v1:172.21.15.32:6789/0 conn(0x7f17480b07a0 legacy=0x7f1748067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f17477fe700 1 -- 172.21.15.32:0/3475133199 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653440867 0 0) 0x7f1734003260 con 0x7f17480b07a0 2022-08-01T23:03:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174d764700 1 -- 172.21.15.32:0/3475133199 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17480ac990 con 0x7f1748147730 2022-08-01T23:03:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174d764700 1 --2- 172.21.15.32:0/3475133199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f17480acdf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1740002f70 tx=0x7f1740006df0).ready entity=mon.2 client_cookie=f35cb889cb3fc1a8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f17477fe700 1 -- 172.21.15.32:0/3475133199 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174000d4a0 con 0x7f1748147730 2022-08-01T23:03:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f17477fe700 1 -- 172.21.15.32:0/3475133199 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740000e90 con 0x7f1748147730 2022-08-01T23:03:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f17477fe700 1 -- 172.21.15.32:0/3475133199 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174000d7f0 con 0x7f1748147730 2022-08-01T23:03:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 -- 172.21.15.32:0/3475133199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 msgr2=0x7f17480acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 --2- 172.21.15.32:0/3475133199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f17480acdf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1740002f70 tx=0x7f1740006df0).stop 2022-08-01T23:03:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 -- 172.21.15.32:0/3475133199 shutdown_connections 2022-08-01T23:03:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 --2- 172.21.15.32:0/3475133199 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f17480acdf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 --2- 172.21.15.32:0/3475133199 >> v2:172.21.15.55:3300/0 conn(0x7f17480b5630 0x7f17480af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 -- 172.21.15.32:0/3475133199 >> 172.21.15.32:0/3475133199 conn(0x7f17480626f0 msgr2=0x7f1748062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 -- 172.21.15.32:0/3475133199 shutdown_connections 2022-08-01T23:03:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 -- 172.21.15.32:0/3475133199 wait complete. 2022-08-01T23:03:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.414+0000 7f174df65700 1 Processor -- start 2022-08-01T23:03:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 -- start start 2022-08-01T23:03:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b07a0 0x7f1748145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 0x7f17481445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f1748141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748141140 con 0x7f17480b5630 2022-08-01T23:03:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174813f8e0 con 0x7f17480b07a0 2022-08-01T23:03:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1748140ea0 con 0x7f1748147730 2022-08-01T23:03:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 0x7f17481445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 0x7f17481445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46220/0 (socket says 172.21.15.32:46220) 2022-08-01T23:03:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 -- 172.21.15.32:0/406718321 learned_addr learned my addr 172.21.15.32:0/406718321 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174d764700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f1748141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174cf63700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b07a0 0x7f1748145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 -- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 msgr2=0x7f1748141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f1748141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 -- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b07a0 msgr2=0x7f1748145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b07a0 0x7f1748145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 -- 172.21.15.32:0/406718321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174000b180 con 0x7f17480b5630 2022-08-01T23:03:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1747fff700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 0x7f17481445f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f1738001250 tx=0x7f1738007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1738004ab0 con 0x7f17480b5630 2022-08-01T23:03:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738004c10 con 0x7f17480b5630 2022-08-01T23:03:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17481574f0 con 0x7f17480b5630 2022-08-01T23:03:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1738007170 con 0x7f17480b5630 2022-08-01T23:03:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.418+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748141880 con 0x7f17480b5630 2022-08-01T23:03:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.422+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17480574e0 con 0x7f17480b5630 2022-08-01T23:03:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.422+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f17380072d0 con 0x7f17480b5630 2022-08-01T23:03:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.422+0000 7f1745ffb700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1728053950 0x7f1728055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.422+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f173806bae0 con 0x7f17480b5630 2022-08-01T23:03:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.422+0000 7f174cf63700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1728053950 0x7f1728055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.422+0000 7f174cf63700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1728053950 0x7f1728055e10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f173c00b000 tx=0x7f173c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.426+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1738042ad0 con 0x7f17480b5630 2022-08-01T23:03:06.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.558+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f174805f3d0 con 0x7f17480b5630 2022-08-01T23:03:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.558+0000 7f1745ffb700 1 -- 172.21.15.32:0/406718321 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+143 (secure 0 0 0) 0x7f17380460f0 con 0x7f17480b5630 2022-08-01T23:03:06.569 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:06.569 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:03:06.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:06.569 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:03:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:03:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:03:06.570 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:03:06.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:06.571 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:03:06.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1728053950 msgr2=0x7f1728055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1728053950 0x7f1728055e10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f173c00b000 tx=0x7f173c006df0).stop 2022-08-01T23:03:06.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 msgr2=0x7f17481445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 0x7f17481445f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f1738001250 tx=0x7f1738007bc0).stop 2022-08-01T23:03:06.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 shutdown_connections 2022-08-01T23:03:06.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1728053950 0x7f1728055e10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748147730 0x7f1748141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480b5630 0x7f17481445f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 --2- 172.21.15.32:0/406718321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b07a0 0x7f1748145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 >> 172.21.15.32:0/406718321 conn(0x7f17480626f0 msgr2=0x7f1748147cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 shutdown_connections 2022-08-01T23:03:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.562+0000 7f174df65700 1 -- 172.21.15.32:0/406718321 wait complete. 2022-08-01T23:03:06.581 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:03:06.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:03:06.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 Processor -- start 2022-08-01T23:03:06.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- start start 2022-08-01T23:03:06.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5cc0b2710 0x7fe5cc14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b01a0 0x7fe5cc14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5cc0a7250 con 0x7fe5cc0afd60 2022-08-01T23:03:06.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc0a7480 con 0x7fe5cc0b2710 2022-08-01T23:03:06.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0a7720 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5cc0b2710 0x7fe5cc14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5cc0b2710 0x7fe5cc14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46226/0 (socket says 172.21.15.32:46226) 2022-08-01T23:03:06.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 -- 172.21.15.32:0/1286460249 learned_addr learned my addr 172.21.15.32:0/1286460249 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:06.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cb7fe700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b01a0 0x7fe5cc14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 -- 172.21.15.32:0/1286460249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b01a0 msgr2=0x7fe5cc14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:06.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b01a0 0x7fe5cc14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 -- 172.21.15.32:0/1286460249 >> v1:172.21.15.55:6789/0 conn(0x7fe5cc0afd60 legacy=0x7fe5cc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 -- 172.21.15.32:0/1286460249 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc0a79c0 con 0x7fe5cc0b2710 2022-08-01T23:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cb7fe700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b01a0 0x7fe5cc14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5cbfff700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b2710 0x7fe5cc14afd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fe5bc002f70 tx=0x7fe5bc006eb0).ready entity=mon.0 client_cookie=8b3e5f262aba4d5b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5caffd700 1 -- 172.21.15.32:0/1286460249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5bc00d5c0 con 0x7fe5cc0b2710 2022-08-01T23:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5caffd700 1 -- 172.21.15.32:0/1286460249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc000e90 con 0x7fe5cc0b2710 2022-08-01T23:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5caffd700 1 -- 172.21.15.32:0/1286460249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5bc00d950 con 0x7fe5cc0b2710 2022-08-01T23:03:06.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- 172.21.15.32:0/1286460249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b2710 msgr2=0x7fe5cc14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b2710 0x7fe5cc14afd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fe5bc002f70 tx=0x7fe5bc006eb0).stop 2022-08-01T23:03:06.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- 172.21.15.32:0/1286460249 shutdown_connections 2022-08-01T23:03:06.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b01a0 0x7fe5cc14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 --2- 172.21.15.32:0/1286460249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b2710 0x7fe5cc14afd0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- 172.21.15.32:0/1286460249 >> 172.21.15.32:0/1286460249 conn(0x7fe5cc060310 msgr2=0x7fe5cc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:06.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- 172.21.15.32:0/1286460249 shutdown_connections 2022-08-01T23:03:06.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.682+0000 7fe5d133f700 1 -- 172.21.15.32:0/1286460249 wait complete. 2022-08-01T23:03:06.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 Processor -- start 2022-08-01T23:03:06.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 -- start start 2022-08-01T23:03:06.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5cc0afd60 0x7fe5cc0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 0x7fe5cc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b2710 0x7fe5cc14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0b4550 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0b4780 con 0x7fe5cc0afd60 2022-08-01T23:03:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0b4a20 con 0x7fe5cc0b2710 2022-08-01T23:03:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 0x7fe5cc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 0x7fe5cc0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46232/0 (socket says 172.21.15.32:46232) 2022-08-01T23:03:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 -- 172.21.15.32:0/913833813 learned_addr learned my addr 172.21.15.32:0/913833813 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cbfff700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5cc0afd60 0x7fe5cc0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d0b3e700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b2710 0x7fe5cc14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 -- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b2710 msgr2=0x7fe5cc14b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b2710 0x7fe5cc14b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 -- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5cc0afd60 msgr2=0x7fe5cc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5cc0afd60 0x7fe5cc0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 -- 172.21.15.32:0/913833813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc00b1e0 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cbfff700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5cc0afd60 0x7fe5cc0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5cb7fe700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 0x7fe5cc0b4f30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fe5c0006940 tx=0x7fe5c0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5c0011780 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c00118e0 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc0b3b20 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5c0011c40 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.686+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc0b34e0 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.690+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe5c0012420 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.690+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc0b4c20 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.690+0000 7fe5c97fa700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5ac053b00 0x7fe5ac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.690+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe5c006bb40 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.690+0000 7fe5cbfff700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5ac053b00 0x7fe5ac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.690+0000 7fe5cbfff700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5ac053b00 0x7fe5ac055fc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fe5bc01a000 tx=0x7fe5bc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.694+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5c0042be0 con 0x7fe5cc0b01a0 2022-08-01T23:03:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:06.822+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe5cc0574e0 con 0x7fe5cc0b01a0 2022-08-01T23:03:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.502+0000 7fe5c97fa700 1 -- 172.21.15.32:0/913833813 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v93) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe5c0046200 con 0x7fe5cc0b01a0 2022-08-01T23:03:07.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5ac053b00 msgr2=0x7fe5ac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:07.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5ac053b00 0x7fe5ac055fc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fe5bc01a000 tx=0x7fe5bc015000).stop 2022-08-01T23:03:07.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 msgr2=0x7fe5cc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:07.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 0x7fe5cc0b4f30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fe5c0006940 tx=0x7fe5c0008040).stop 2022-08-01T23:03:07.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 shutdown_connections 2022-08-01T23:03:07.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe5ac053b00 0x7fe5ac055fc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe5cc0b2710 0x7fe5cc14b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0b01a0 0x7fe5cc0b4f30 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 --2- 172.21.15.32:0/913833813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5cc0afd60 0x7fe5cc0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 >> 172.21.15.32:0/913833813 conn(0x7fe5cc060310 msgr2=0x7fe5cc146130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:07.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 shutdown_connections 2022-08-01T23:03:07.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.506+0000 7fe5d133f700 1 -- 172.21.15.32:0/913833813 wait complete. 2022-08-01T23:03:07.518 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:03:07.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:03:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 Processor -- start 2022-08-01T23:03:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- start start 2022-08-01T23:03:07.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b1460 0x7f46740b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46740b2430 0x7f46741413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46740a7960 con 0x7f46740b2050 2022-08-01T23:03:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46740b18d0 con 0x7f46740b2430 2022-08-01T23:03:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46740b07e0 con 0x7f46740b1460 2022-08-01T23:03:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46740b2430 0x7f46741413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:07.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46740b2430 0x7f46741413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46242/0 (socket says 172.21.15.32:46242) 2022-08-01T23:03:07.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 -- 172.21.15.32:0/3902335950 learned_addr learned my addr 172.21.15.32:0/3902335950 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:07.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4678dc2700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b1460 0x7f46740b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:07.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 -- 172.21.15.32:0/3902335950 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b1460 msgr2=0x7f46740b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:07.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b1460 0x7f46740b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 -- 172.21.15.32:0/3902335950 >> v1:172.21.15.55:6789/0 conn(0x7f46740b2050 legacy=0x7f46740a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:07.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 -- 172.21.15.32:0/3902335950 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740b0a80 con 0x7f46740b2430 2022-08-01T23:03:07.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4678dc2700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b1460 0x7f46740b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:07.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4679dc4700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2430 0x7f46741413e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f466c0032f0 tx=0x7f466c009290).ready entity=mon.0 client_cookie=f5fd7a3650d305b7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:07.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4667fff700 1 -- 172.21.15.32:0/3902335950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f466c00b8a0 con 0x7f46740b2430 2022-08-01T23:03:07.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4667fff700 1 -- 172.21.15.32:0/3902335950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c00ba00 con 0x7f46740b2430 2022-08-01T23:03:07.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f4667fff700 1 -- 172.21.15.32:0/3902335950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f466c00bb70 con 0x7f46740b2430 2022-08-01T23:03:07.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- 172.21.15.32:0/3902335950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2430 msgr2=0x7f46741413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:07.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2430 0x7f46741413e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f466c0032f0 tx=0x7f466c009290).stop 2022-08-01T23:03:07.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- 172.21.15.32:0/3902335950 shutdown_connections 2022-08-01T23:03:07.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2430 0x7f46741413e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 --2- 172.21.15.32:0/3902335950 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b1460 0x7f46740b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- 172.21.15.32:0/3902335950 >> 172.21.15.32:0/3902335950 conn(0x7f467405f840 msgr2=0x7f467405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:07.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- 172.21.15.32:0/3902335950 shutdown_connections 2022-08-01T23:03:07.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.638+0000 7f467b827700 1 -- 172.21.15.32:0/3902335950 wait complete. 2022-08-01T23:03:07.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 Processor -- start 2022-08-01T23:03:07.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 -- start start 2022-08-01T23:03:07.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46740b1460 0x7f4674140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:07.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 0x7f46741433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:07.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b2430 0x7f4674144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:07.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46741438e0 con 0x7f46740b2050 2022-08-01T23:03:07.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46740abac0 con 0x7f46740b1460 2022-08-01T23:03:07.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f467b827700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46740abd60 con 0x7f46740b2430 2022-08-01T23:03:07.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 0x7f46741433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:07.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 0x7f46741433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46248/0 (socket says 172.21.15.32:46248) 2022-08-01T23:03:07.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 -- 172.21.15.32:0/1697778649 learned_addr learned my addr 172.21.15.32:0/1697778649 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:07.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f46795c3700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46740b1460 0x7f4674140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:07.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4679dc4700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b2430 0x7f4674144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:07.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 -- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b2430 msgr2=0x7f4674144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b2430 0x7f4674144740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 -- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46740b1460 msgr2=0x7f4674140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:07.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46740b1460 0x7f4674140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:07.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 -- 172.21.15.32:0/1697778649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c009530 con 0x7f46740b2050 2022-08-01T23:03:07.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.642+0000 7f4678dc2700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 0x7f46741433f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f4670006940 tx=0x7f4670008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:07.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4670011850 con 0x7f46740b2050 2022-08-01T23:03:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46700119b0 con 0x7f46740b2050 2022-08-01T23:03:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46740ab800 con 0x7f46740b2050 2022-08-01T23:03:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4670011d10 con 0x7f46740b2050 2022-08-01T23:03:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46740aacc0 con 0x7f46740b2050 2022-08-01T23:03:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4674142e80 con 0x7f46740b2050 2022-08-01T23:03:07.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.646+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4670006030 con 0x7f46740b2050 2022-08-01T23:03:07.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.650+0000 7f46667fc700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f465c053950 0x7f465c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:07.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.650+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f467006bfe0 con 0x7f46740b2050 2022-08-01T23:03:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.650+0000 7f46795c3700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f465c053950 0x7f465c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:07.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.650+0000 7f46795c3700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f465c053950 0x7f465c055e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f466800d000 tx=0x7f4668009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:07.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.654+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4670046e60 con 0x7f46740b2050 2022-08-01T23:03:07.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:07.778+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f46740ac0e0 con 0x7f46740b2050 2022-08-01T23:03:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.154+0000 7f46667fc700 1 -- 172.21.15.32:0/1697778649 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v94) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4674142e80 con 0x7f46740b2050 2022-08-01T23:03:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f465c053950 msgr2=0x7f465c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f465c053950 0x7f465c055e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f466800d000 tx=0x7f4668009040).stop 2022-08-01T23:03:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 msgr2=0x7f46741433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 0x7f46741433f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f4670006940 tx=0x7f4670008040).stop 2022-08-01T23:03:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 shutdown_connections 2022-08-01T23:03:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f465c053950 0x7f465c055e10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46740b2430 0x7f4674144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46740b2050 0x7f46741433f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 --2- 172.21.15.32:0/1697778649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46740b1460 0x7f4674140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.158+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 >> 172.21.15.32:0/1697778649 conn(0x7f467405f840 msgr2=0x7f4674146860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.162+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 shutdown_connections 2022-08-01T23:03:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.162+0000 7f467b827700 1 -- 172.21.15.32:0/1697778649 wait complete. 2022-08-01T23:03:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:03:08.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:03:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.282+0000 7f05a672e700 1 Processor -- start 2022-08-01T23:03:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.282+0000 7f05a672e700 1 -- start start 2022-08-01T23:03:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6710 0x7f05a00b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05a00b6af0 0x7f05a012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a0144240 con 0x7f05a00b4550 2022-08-01T23:03:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a0143d00 con 0x7f05a00b6af0 2022-08-01T23:03:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0143fa0 con 0x7f05a00b6710 2022-08-01T23:03:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05a00b4550 0x7f05a0061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33796/0 (socket says 172.21.15.32:33796) 2022-08-01T23:03:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059f7fe700 1 -- 172.21.15.32:0/3616014318 learned_addr learned my addr 172.21.15.32:0/3616014318 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059ffff700 1 --2- 172.21.15.32:0/3616014318 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6710 0x7f05a00b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a4ccb700 1 --2- 172.21.15.32:0/3616014318 >> v2:172.21.15.55:3300/0 conn(0x7f05a00b6af0 0x7f05a012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059effd700 1 -- 172.21.15.32:0/3616014318 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442471354 0 0) 0x7f05a0144240 con 0x7f05a00b4550 2022-08-01T23:03:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059effd700 1 -- 172.21.15.32:0/3616014318 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0580003260 con 0x7f05a00b4550 2022-08-01T23:03:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059effd700 1 -- 172.21.15.32:0/3616014318 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782403053 0 0) 0x7f0580003260 con 0x7f05a00b4550 2022-08-01T23:03:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059ffff700 1 -- 172.21.15.32:0/3616014318 >> v2:172.21.15.55:3300/0 conn(0x7f05a00b6af0 msgr2=0x7f05a012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059ffff700 1 --2- 172.21.15.32:0/3616014318 >> v2:172.21.15.55:3300/0 conn(0x7f05a00b6af0 0x7f05a012bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059ffff700 1 -- 172.21.15.32:0/3616014318 >> v1:172.21.15.32:6789/0 conn(0x7f05a00b4550 legacy=0x7f05a0061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059ffff700 1 -- 172.21.15.32:0/3616014318 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a0061780 con 0x7f05a00b6710 2022-08-01T23:03:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a4ccb700 1 --2- 172.21.15.32:0/3616014318 >> v2:172.21.15.55:3300/0 conn(0x7f05a00b6af0 0x7f05a012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059ffff700 1 --2- 172.21.15.32:0/3616014318 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6710 0x7f05a00b5d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f0588001800 tx=0x7f0590006df0).ready entity=mon.2 client_cookie=feb4938d87b82873 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059effd700 1 -- 172.21.15.32:0/3616014318 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0588001660 con 0x7f05a00b6710 2022-08-01T23:03:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059effd700 1 -- 172.21.15.32:0/3616014318 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0590000e90 con 0x7f05a00b6710 2022-08-01T23:03:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f059effd700 1 -- 172.21.15.32:0/3616014318 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0590012460 con 0x7f05a00b6710 2022-08-01T23:03:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- 172.21.15.32:0/3616014318 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6710 msgr2=0x7f05a00b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 --2- 172.21.15.32:0/3616014318 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6710 0x7f05a00b5d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f0588001800 tx=0x7f0590006df0).stop 2022-08-01T23:03:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- 172.21.15.32:0/3616014318 shutdown_connections 2022-08-01T23:03:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 --2- 172.21.15.32:0/3616014318 >> v2:172.21.15.55:3300/0 conn(0x7f05a00b6af0 0x7f05a012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 --2- 172.21.15.32:0/3616014318 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6710 0x7f05a00b5d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- 172.21.15.32:0/3616014318 >> 172.21.15.32:0/3616014318 conn(0x7f05a005c620 msgr2=0x7f05a005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- 172.21.15.32:0/3616014318 shutdown_connections 2022-08-01T23:03:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.286+0000 7f05a672e700 1 -- 172.21.15.32:0/3616014318 wait complete. 2022-08-01T23:03:08.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 Processor -- start 2022-08-01T23:03:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 -- start start 2022-08-01T23:03:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 0x7f05a013f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00b6710 0x7f05a013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6af0 0x7f05a013de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a013c230 con 0x7f05a00b4550 2022-08-01T23:03:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a013f830 con 0x7f05a00b6710 2022-08-01T23:03:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0140330 con 0x7f05a00b6af0 2022-08-01T23:03:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 0x7f05a013f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 0x7f05a013f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46260/0 (socket says 172.21.15.32:46260) 2022-08-01T23:03:08.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 -- 172.21.15.32:0/1127441984 learned_addr learned my addr 172.21.15.32:0/1127441984 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:08.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a4ccb700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6af0 0x7f05a013de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059f7fe700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00b6710 0x7f05a013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 -- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6af0 msgr2=0x7f05a013de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6af0 0x7f05a013de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 -- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00b6710 msgr2=0x7f05a013d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00b6710 0x7f05a013d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 -- 172.21.15.32:0/1127441984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009470 con 0x7f05a00b4550 2022-08-01T23:03:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059ffff700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 0x7f05a013f340 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f05a013cf40 tx=0x7f0590014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0590016070 con 0x7f05a00b4550 2022-08-01T23:03:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059000bec0 con 0x7f05a00b4550 2022-08-01T23:03:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059000de60 con 0x7f05a00b4550 2022-08-01T23:03:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059000b180 con 0x7f05a00b4550 2022-08-01T23:03:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.290+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a0124f00 con 0x7f05a00b4550 2022-08-01T23:03:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.294+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a01258b0 con 0x7f05a00b4550 2022-08-01T23:03:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.294+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0590025030 con 0x7f05a00b4550 2022-08-01T23:03:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.294+0000 7f059d7fa700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05840539e0 0x7f0584055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.294+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f059006cfe0 con 0x7f05a00b4550 2022-08-01T23:03:08.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.294+0000 7f059f7fe700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05840539e0 0x7f0584055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.294+0000 7f059f7fe700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05840539e0 0x7f0584055ea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f058800b000 tx=0x7f0588009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.298+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0590043db0 con 0x7f05a00b4550 2022-08-01T23:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.462+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f05a01250c0 con 0x7f05a00b4550 2022-08-01T23:03:08.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.474+0000 7f059d7fa700 1 -- 172.21.15.32:0/1127441984 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v44) v1 ==== 189+0+0 (secure 0 0 0) 0x7f0590023020 con 0x7f05a00b4550 2022-08-01T23:03:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.474+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05840539e0 msgr2=0x7f0584055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.474+0000 7f05a672e700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05840539e0 0x7f0584055ea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f058800b000 tx=0x7f0588009040).stop 2022-08-01T23:03:08.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.474+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 msgr2=0x7f05a013f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.474+0000 7f05a672e700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 0x7f05a013f340 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f05a013cf40 tx=0x7f0590014000).stop 2022-08-01T23:03:08.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 shutdown_connections 2022-08-01T23:03:08.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05840539e0 0x7f0584055ea0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05a00b6af0 0x7f05a013de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00b6710 0x7f05a013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 --2- 172.21.15.32:0/1127441984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a00b4550 0x7f05a013f340 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 >> 172.21.15.32:0/1127441984 conn(0x7f05a005c620 msgr2=0x7f05a01432d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 shutdown_connections 2022-08-01T23:03:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.478+0000 7f05a672e700 1 -- 172.21.15.32:0/1127441984 wait complete. 2022-08-01T23:03:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 17 and data pool 18 2022-08-01T23:03:08.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:08.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4c850700 1 Processor -- start 2022-08-01T23:03:08.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4c850700 1 -- start start 2022-08-01T23:03:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4c850700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd48066e70 0x7fdd4814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4c850700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd48060670 con 0x7fdd48060290 2022-08-01T23:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4c850700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd48060e00 con 0x7fdd480617e0 2022-08-01T23:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4c850700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480672c0 con 0x7fdd48066e70 2022-08-01T23:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd46d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd48066e70 0x7fdd4814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd48060290 0x7fdd48065b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33820/0 (socket says 172.21.15.32:33820) 2022-08-01T23:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd46d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd48066e70 0x7fdd4814eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46282/0 (socket says 172.21.15.32:46282) 2022-08-01T23:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd4659c700 1 -- 172.21.15.32:0/76113458 learned_addr learned my addr 172.21.15.32:0/76113458 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd46d9d700 1 -- 172.21.15.32:0/76113458 >> v1:172.21.15.55:6789/0 conn(0x7fdd480617e0 legacy=0x7fdd48066760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd46d9d700 1 -- 172.21.15.32:0/76113458 >> v1:172.21.15.32:6789/0 conn(0x7fdd48060290 legacy=0x7fdd48065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd46d9d700 1 -- 172.21.15.32:0/76113458 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480610a0 con 0x7fdd48066e70 2022-08-01T23:03:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd45d9b700 1 -- 172.21.15.32:0/76113458 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816337991 0 0) 0x7fdd48060670 con 0x7fdd48060290 2022-08-01T23:03:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.586+0000 7fdd46d9d700 1 --2- 172.21.15.32:0/76113458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd4814eb70 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fdd400032f0 tx=0x7fdd400091d0).ready entity=mon.0 client_cookie=6ecbc7784dfc0c3a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd45d9b700 1 -- 172.21.15.32:0/76113458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4000b8a0 con 0x7fdd48066e70 2022-08-01T23:03:08.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd45d9b700 1 -- 172.21.15.32:0/76113458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4000ba00 con 0x7fdd48066e70 2022-08-01T23:03:08.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd45d9b700 1 -- 172.21.15.32:0/76113458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4000bb70 con 0x7fdd48066e70 2022-08-01T23:03:08.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 -- 172.21.15.32:0/76113458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 msgr2=0x7fdd4814eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 --2- 172.21.15.32:0/76113458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd4814eb70 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fdd400032f0 tx=0x7fdd400091d0).stop 2022-08-01T23:03:08.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 -- 172.21.15.32:0/76113458 shutdown_connections 2022-08-01T23:03:08.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 --2- 172.21.15.32:0/76113458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd4814eb70 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 -- 172.21.15.32:0/76113458 >> 172.21.15.32:0/76113458 conn(0x7fdd4805d0c0 msgr2=0x7fdd4805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:08.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 -- 172.21.15.32:0/76113458 shutdown_connections 2022-08-01T23:03:08.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 -- 172.21.15.32:0/76113458 wait complete. 2022-08-01T23:03:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 Processor -- start 2022-08-01T23:03:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.590+0000 7fdd4c850700 1 -- start start 2022-08-01T23:03:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd48060290 0x7fdd480a8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd480617e0 0x7fdd480a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd480a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48156bc0 con 0x7fdd48066e70 2022-08-01T23:03:08.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480abe20 con 0x7fdd48060290 2022-08-01T23:03:08.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480ac0c0 con 0x7fdd480617e0 2022-08-01T23:03:08.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd480a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd480a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46288/0 (socket says 172.21.15.32:46288) 2022-08-01T23:03:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 -- 172.21.15.32:0/429202121 learned_addr learned my addr 172.21.15.32:0/429202121 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4659c700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd48060290 0x7fdd480a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd3dd9b700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd480617e0 0x7fdd480a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 -- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd480617e0 msgr2=0x7fdd480a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd480617e0 0x7fdd480a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 -- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd48060290 msgr2=0x7fdd480a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd48060290 0x7fdd480a8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 -- 172.21.15.32:0/429202121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40009470 con 0x7fdd48066e70 2022-08-01T23:03:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd3dd9b700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd480617e0 0x7fdd480a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd46d9d700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd480a9e40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fdd400032f0 tx=0x7fdd40016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4000bb70 con 0x7fdd48066e70 2022-08-01T23:03:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4001f3f0 con 0x7fdd48066e70 2022-08-01T23:03:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd480a8a20 con 0x7fdd48066e70 2022-08-01T23:03:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4001f750 con 0x7fdd48066e70 2022-08-01T23:03:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd480a7f90 con 0x7fdd48066e70 2022-08-01T23:03:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.594+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd48156dc0 con 0x7fdd48066e70 2022-08-01T23:03:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.598+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd40026030 con 0x7fdd48066e70 2022-08-01T23:03:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.598+0000 7fdd3ffff700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd2c0539e0 0x7fdd2c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.598+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdd4006ce30 con 0x7fdd48066e70 2022-08-01T23:03:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.598+0000 7fdd4659c700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd2c0539e0 0x7fdd2c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.598+0000 7fdd4659c700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd2c0539e0 0x7fdd2c055ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fdd3800d000 tx=0x7fdd380011a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.606+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd40043bf0 con 0x7fdd48066e70 2022-08-01T23:03:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.734+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd480ac280 con 0x7fdd48066e70 2022-08-01T23:03:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.734+0000 7fdd3ffff700 1 -- 172.21.15.32:0/429202121 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fdd40015090 con 0x7fdd48066e70 2022-08-01T23:03:08.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:08.744 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:08.468842+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:06.881558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:07.840459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd2c0539e0 msgr2=0x7fdd2c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd2c0539e0 0x7fdd2c055ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fdd3800d000 tx=0x7fdd380011a0).stop 2022-08-01T23:03:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 msgr2=0x7fdd480a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd480a9e40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fdd400032f0 tx=0x7fdd40016000).stop 2022-08-01T23:03:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 shutdown_connections 2022-08-01T23:03:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdd2c0539e0 0x7fdd2c055ea0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd48066e70 0x7fdd480a9e40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdd480617e0 0x7fdd480a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 --2- 172.21.15.32:0/429202121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd48060290 0x7fdd480a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 >> 172.21.15.32:0/429202121 conn(0x7fdd4805d0c0 msgr2=0x7fdd48147b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 shutdown_connections 2022-08-01T23:03:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.738+0000 7fdd4c850700 1 -- 172.21.15.32:0/429202121 wait complete. 2022-08-01T23:03:08.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 Processor -- start 2022-08-01T23:03:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- start start 2022-08-01T23:03:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f59000b28b0 0x7f590013e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b1180 0x7f590013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000aac20 con 0x7f5900141210 2022-08-01T23:03:08.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900141db0 con 0x7f59000b28b0 2022-08-01T23:03:08.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5900140e20 con 0x7f59000b1180 2022-08-01T23:03:08.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f59000b28b0 0x7f590013e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f59000b28b0 0x7f590013e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:56078/0 (socket says 172.21.15.32:56078) 2022-08-01T23:03:08.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 -- 172.21.15.32:0/2219388285 learned_addr learned my addr 172.21.15.32:0/2219388285 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58ff7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5900141210 0x7f59001416a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:54742/0 (socket says 172.21.15.32:54742) 2022-08-01T23:03:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fe7fc700 1 --2- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b1180 0x7f590013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 -- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b1180 msgr2=0x7f590013e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 --2- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b1180 0x7f590013e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 -- 172.21.15.32:0/2219388285 >> v1:172.21.15.55:6789/0 conn(0x7f5900141210 legacy=0x7f59001416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 -- 172.21.15.32:0/2219388285 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000b2c90 con 0x7f59000b28b0 2022-08-01T23:03:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58feffd700 1 --2- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b28b0 0x7f590013e1c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f58f0002f70 tx=0x7f58f0006eb0).ready entity=mon.1 client_cookie=4dacb85756436611 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fdffb700 1 -- 172.21.15.32:0/2219388285 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000d570 con 0x7f59000b28b0 2022-08-01T23:03:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fdffb700 1 -- 172.21.15.32:0/2219388285 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0000e90 con 0x7f59000b28b0 2022-08-01T23:03:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fdffb700 1 -- 172.21.15.32:0/2219388285 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000d900 con 0x7f59000b28b0 2022-08-01T23:03:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b28b0 msgr2=0x7f590013e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 --2- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b28b0 0x7f590013e1c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f58f0002f70 tx=0x7f58f0006eb0).stop 2022-08-01T23:03:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- 172.21.15.32:0/2219388285 shutdown_connections 2022-08-01T23:03:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 --2- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b1180 0x7f590013e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 --2- 172.21.15.32:0/2219388285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b28b0 0x7f590013e1c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- 172.21.15.32:0/2219388285 >> 172.21.15.32:0/2219388285 conn(0x7f5900060310 msgr2=0x7f59000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- 172.21.15.32:0/2219388285 shutdown_connections 2022-08-01T23:03:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.858+0000 7f58fffff700 1 -- 172.21.15.32:0/2219388285 wait complete. 2022-08-01T23:03:08.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 Processor -- start 2022-08-01T23:03:08.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 -- start start 2022-08-01T23:03:08.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b1180 0x7f59000b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b28b0 0x7f5900155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 0x7f5900155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900156900 con 0x7f5900141210 2022-08-01T23:03:08.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900145780 con 0x7f59000b1180 2022-08-01T23:03:08.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5900156400 con 0x7f59000b28b0 2022-08-01T23:03:08.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 0x7f5900155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58feffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b1180 0x7f59000b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 0x7f5900155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46302/0 (socket says 172.21.15.32:46302) 2022-08-01T23:03:08.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 -- 172.21.15.32:0/1925200330 learned_addr learned my addr 172.21.15.32:0/1925200330 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:08.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fe7fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b28b0 0x7f5900155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 -- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b28b0 msgr2=0x7f5900155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b28b0 0x7f5900155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 -- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b1180 msgr2=0x7f59000b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:08.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b1180 0x7f59000b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:08.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 -- 172.21.15.32:0/1925200330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f000b1e0 con 0x7f5900141210 2022-08-01T23:03:08.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58ff7fe700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 0x7f5900155f10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f590006ae60 tx=0x7f58f8007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f8005ae0 con 0x7f5900141210 2022-08-01T23:03:08.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8005c40 con 0x7f5900141210 2022-08-01T23:03:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5900156660 con 0x7f5900141210 2022-08-01T23:03:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f80070d0 con 0x7f5900141210 2022-08-01T23:03:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.862+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5900142770 con 0x7f5900141210 2022-08-01T23:03:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.866+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59000574e0 con 0x7f5900141210 2022-08-01T23:03:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.866+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58f8010070 con 0x7f5900141210 2022-08-01T23:03:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.866+0000 7f58effff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58dc0539e0 0x7f58dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.866+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f58f806b6c0 con 0x7f5900141210 2022-08-01T23:03:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.866+0000 7f58feffd700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58dc0539e0 0x7f58dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:08.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.866+0000 7f58feffd700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58dc0539e0 0x7f58dc055ea0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f58f001c000 tx=0x7f58f0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:08.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:08.870+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58f8042480 con 0x7f5900141210 2022-08-01T23:03:09.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.002+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5900069a70 con 0x7f5900141210 2022-08-01T23:03:09.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.002+0000 7f58effff700 1 -- 172.21.15.32:0/1925200330 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f58f8045aa0 con 0x7f5900141210 2022-08-01T23:03:09.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:09.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:08.468842+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:06.881558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:07.840459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58dc0539e0 msgr2=0x7f58dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58dc0539e0 0x7f58dc055ea0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f58f001c000 tx=0x7f58f0015000).stop 2022-08-01T23:03:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 msgr2=0x7f5900155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 0x7f5900155f10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f590006ae60 tx=0x7f58f8007f60).stop 2022-08-01T23:03:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 shutdown_connections 2022-08-01T23:03:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58dc0539e0 0x7f58dc055ea0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5900141210 0x7f5900155f10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f59000b28b0 0x7f5900155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 --2- 172.21.15.32:0/1925200330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59000b1180 0x7f59000b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 >> 172.21.15.32:0/1925200330 conn(0x7f5900060310 msgr2=0x7f59000afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 shutdown_connections 2022-08-01T23:03:09.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.006+0000 7f58fffff700 1 -- 172.21.15.32:0/1925200330 wait complete. 2022-08-01T23:03:09.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:03:09.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 Processor -- start 2022-08-01T23:03:09.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 -- start start 2022-08-01T23:03:09.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7c00b31d0 0x7ff7c0066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7c01402d0 0x7ff7c0140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c0067030 con 0x7ff7c00a9da0 2022-08-01T23:03:09.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c0064a70 con 0x7ff7c01402d0 2022-08-01T23:03:09.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5d98700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c0061f60 con 0x7ff7c00b31d0 2022-08-01T23:03:09.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5597700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7c01402d0 0x7ff7c0140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7b7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7c00a9da0 0x7ff7c0060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33850/0 (socket says 172.21.15.32:33850) 2022-08-01T23:03:09.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7b7fff700 1 -- 172.21.15.32:0/3830208562 learned_addr learned my addr 172.21.15.32:0/3830208562 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:09.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c4d96700 1 --2- 172.21.15.32:0/3830208562 >> v2:172.21.15.55:3300/0 conn(0x7ff7c00b31d0 0x7ff7c0066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5597700 1 -- 172.21.15.32:0/3830208562 >> v2:172.21.15.55:3300/0 conn(0x7ff7c00b31d0 msgr2=0x7ff7c0066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:09.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5597700 1 --2- 172.21.15.32:0/3830208562 >> v2:172.21.15.55:3300/0 conn(0x7ff7c00b31d0 0x7ff7c0066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7b77fe700 1 -- 172.21.15.32:0/3830208562 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65068799 0 0) 0x7ff7c0067030 con 0x7ff7c00a9da0 2022-08-01T23:03:09.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5597700 1 -- 172.21.15.32:0/3830208562 >> v1:172.21.15.32:6789/0 conn(0x7ff7c00a9da0 legacy=0x7ff7c0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5597700 1 -- 172.21.15.32:0/3830208562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c00b35b0 con 0x7ff7c01402d0 2022-08-01T23:03:09.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7c5597700 1 --2- 172.21.15.32:0/3830208562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c01402d0 0x7ff7c0140760 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7ff7c00625d0 tx=0x7ff7b80091d0).ready entity=mon.0 client_cookie=e81a52140f9fb5ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7b77fe700 1 -- 172.21.15.32:0/3830208562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7b800b830 con 0x7ff7c01402d0 2022-08-01T23:03:09.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7b77fe700 1 -- 172.21.15.32:0/3830208562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b800b990 con 0x7ff7c01402d0 2022-08-01T23:03:09.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.130+0000 7ff7b77fe700 1 -- 172.21.15.32:0/3830208562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7b800bb00 con 0x7ff7c01402d0 2022-08-01T23:03:09.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- 172.21.15.32:0/3830208562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c01402d0 msgr2=0x7ff7c0140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/3830208562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c01402d0 0x7ff7c0140760 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7ff7c00625d0 tx=0x7ff7b80091d0).stop 2022-08-01T23:03:09.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- 172.21.15.32:0/3830208562 shutdown_connections 2022-08-01T23:03:09.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/3830208562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c01402d0 0x7ff7c0140760 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/3830208562 >> v2:172.21.15.55:3300/0 conn(0x7ff7c00b31d0 0x7ff7c0066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- 172.21.15.32:0/3830208562 >> 172.21.15.32:0/3830208562 conn(0x7ff7c0063870 msgr2=0x7ff7c0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:09.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- 172.21.15.32:0/3830208562 shutdown_connections 2022-08-01T23:03:09.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- 172.21.15.32:0/3830208562 wait complete. 2022-08-01T23:03:09.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 Processor -- start 2022-08-01T23:03:09.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- start start 2022-08-01T23:03:09.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff7c00a9da0 0x7ff7c0145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 0x7ff7c0151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7c01402d0 0x7ff7c01516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01412e0 con 0x7ff7c00b31d0 2022-08-01T23:03:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c0151be0 con 0x7ff7c01402d0 2022-08-01T23:03:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5d98700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c014e790 con 0x7ff7c00a9da0 2022-08-01T23:03:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 0x7ff7c0151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 0x7ff7c0151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46316/0 (socket says 172.21.15.32:46316) 2022-08-01T23:03:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 -- 172.21.15.32:0/113761358 learned_addr learned my addr 172.21.15.32:0/113761358 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5597700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7c01402d0 0x7ff7c01516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c4d96700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff7c00a9da0 0x7ff7c0145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 -- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff7c00a9da0 msgr2=0x7ff7c0145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff7c00a9da0 0x7ff7c0145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 -- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7c01402d0 msgr2=0x7ff7c01516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7c01402d0 0x7ff7c01516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 -- 172.21.15.32:0/113761358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8009470 con 0x7ff7c00b31d0 2022-08-01T23:03:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7c5597700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7c01402d0 0x7ff7c01516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.134+0000 7ff7b7fff700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 0x7ff7c0151200 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7ff7b0000ed0 tx=0x7ff7b0007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7b00070d0 con 0x7ff7c00b31d0 2022-08-01T23:03:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b0007230 con 0x7ff7c00b31d0 2022-08-01T23:03:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7b0005610 con 0x7ff7c00b31d0 2022-08-01T23:03:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c014ea30 con 0x7ff7c00b31d0 2022-08-01T23:03:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01465c0 con 0x7ff7c00b31d0 2022-08-01T23:03:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c0066af0 con 0x7ff7c00b31d0 2022-08-01T23:03:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff7b0010070 con 0x7ff7c00b31d0 2022-08-01T23:03:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7b5ffb700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff7a0053aa0 0x7ff7a0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff7b006b590 con 0x7ff7c00b31d0 2022-08-01T23:03:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.138+0000 7ff7c4d96700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff7a0053aa0 0x7ff7a0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.142+0000 7ff7c4d96700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff7a0053aa0 0x7ff7a0055f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7ff7bc00c000 tx=0x7ff7bc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.146+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff7b0042350 con 0x7ff7c00b31d0 2022-08-01T23:03:09.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.310+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff7c00574e0 con 0x7ff7c00b31d0 2022-08-01T23:03:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.478+0000 7ff7b5ffb700 1 -- 172.21.15.32:0/113761358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff7b0045970 con 0x7ff7c00b31d0 2022-08-01T23:03:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff7a0053aa0 msgr2=0x7ff7a0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff7a0053aa0 0x7ff7a0055f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7ff7bc00c000 tx=0x7ff7bc008040).stop 2022-08-01T23:03:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 msgr2=0x7ff7c0151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 0x7ff7c0151200 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7ff7b0000ed0 tx=0x7ff7b0007a80).stop 2022-08-01T23:03:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 shutdown_connections 2022-08-01T23:03:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff7a0053aa0 0x7ff7a0055f60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7c01402d0 0x7ff7c01516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c00b31d0 0x7ff7c0151200 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 --2- 172.21.15.32:0/113761358 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff7c00a9da0 0x7ff7c0145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 >> 172.21.15.32:0/113761358 conn(0x7ff7c0063870 msgr2=0x7ff7c00605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 shutdown_connections 2022-08-01T23:03:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.482+0000 7ff7c5d98700 1 -- 172.21.15.32:0/113761358 wait complete. 2022-08-01T23:03:09.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 Processor -- start 2022-08-01T23:03:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- start start 2022-08-01T23:03:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efc700af450 0x7efc700af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc701432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc701437d0 0x7efc700a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc700aff30 con 0x7efc701437d0 2022-08-01T23:03:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7005c7a0 con 0x7efc700af450 2022-08-01T23:03:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc70140990 con 0x7efc700ad4b0 2022-08-01T23:03:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc701437d0 0x7efc700a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc701437d0 0x7efc700a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46324/0 (socket says 172.21.15.32:46324) 2022-08-01T23:03:09.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 -- 172.21.15.32:0/1617610467 learned_addr learned my addr 172.21.15.32:0/1617610467 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:09.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6dd9b700 1 --2- 172.21.15.32:0/1617610467 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc701432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6e59c700 1 --2- 172.21.15.32:0/1617610467 >> v2:172.21.15.55:3300/0 conn(0x7efc700af450 0x7efc700af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 -- 172.21.15.32:0/1617610467 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 msgr2=0x7efc701432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:09.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 --2- 172.21.15.32:0/1617610467 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc701432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 -- 172.21.15.32:0/1617610467 >> v2:172.21.15.55:3300/0 conn(0x7efc700af450 msgr2=0x7efc700af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:09.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 --2- 172.21.15.32:0/1617610467 >> v2:172.21.15.55:3300/0 conn(0x7efc700af450 0x7efc700af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 -- 172.21.15.32:0/1617610467 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc700ad9e0 con 0x7efc701437d0 2022-08-01T23:03:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6ed9d700 1 --2- 172.21.15.32:0/1617610467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc701437d0 0x7efc700a8e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7efc7013fdc0 tx=0x7efc68009290).ready entity=mon.0 client_cookie=4a8c2fab48ed257 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6d59a700 1 -- 172.21.15.32:0/1617610467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc6800ba50 con 0x7efc701437d0 2022-08-01T23:03:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6d59a700 1 -- 172.21.15.32:0/1617610467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6800bbb0 con 0x7efc701437d0 2022-08-01T23:03:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6d59a700 1 -- 172.21.15.32:0/1617610467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc68010470 con 0x7efc701437d0 2022-08-01T23:03:09.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- 172.21.15.32:0/1617610467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc701437d0 msgr2=0x7efc700a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- 172.21.15.32:0/1617610467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc701437d0 0x7efc700a8e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7efc7013fdc0 tx=0x7efc68009290).stop 2022-08-01T23:03:09.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- 172.21.15.32:0/1617610467 shutdown_connections 2022-08-01T23:03:09.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- 172.21.15.32:0/1617610467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc701437d0 0x7efc700a8e40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- 172.21.15.32:0/1617610467 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc701432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- 172.21.15.32:0/1617610467 >> v2:172.21.15.55:3300/0 conn(0x7efc700af450 0x7efc700af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- 172.21.15.32:0/1617610467 >> 172.21.15.32:0/1617610467 conn(0x7efc700612d0 msgr2=0x7efc70062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:09.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- 172.21.15.32:0/1617610467 shutdown_connections 2022-08-01T23:03:09.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- 172.21.15.32:0/1617610467 wait complete. 2022-08-01T23:03:09.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 Processor -- start 2022-08-01T23:03:09.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- start start 2022-08-01T23:03:09.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc70139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 0x7efc70142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc701437d0 0x7efc700ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc70143280 con 0x7efc700af450 2022-08-01T23:03:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc70142550 con 0x7efc701437d0 2022-08-01T23:03:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc700a9950 con 0x7efc700ad4b0 2022-08-01T23:03:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 0x7efc70142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 0x7efc70142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46330/0 (socket says 172.21.15.32:46330) 2022-08-01T23:03:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6dd9b700 1 -- 172.21.15.32:0/3275167379 learned_addr learned my addr 172.21.15.32:0/3275167379 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.606+0000 7efc6e59c700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc70139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6ed9d700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc701437d0 0x7efc700ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6dd9b700 1 -- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 msgr2=0x7efc70139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6dd9b700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc70139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6dd9b700 1 -- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc701437d0 msgr2=0x7efc700ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6dd9b700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc701437d0 0x7efc700ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6dd9b700 1 -- 172.21.15.32:0/3275167379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009530 con 0x7efc700af450 2022-08-01T23:03:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6dd9b700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 0x7efc70142d90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7efc700aabe0 tx=0x7efc64008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc64011780 con 0x7efc700af450 2022-08-01T23:03:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc640118e0 con 0x7efc700af450 2022-08-01T23:03:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc700a9650 con 0x7efc700af450 2022-08-01T23:03:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc64011c40 con 0x7efc700af450 2022-08-01T23:03:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc700ac8b0 con 0x7efc700af450 2022-08-01T23:03:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc70142c00 con 0x7efc700af450 2022-08-01T23:03:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.610+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efc6401c460 con 0x7efc700af450 2022-08-01T23:03:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.614+0000 7efc5f7fe700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc4c0539e0 0x7efc4c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.614+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efc6406c410 con 0x7efc700af450 2022-08-01T23:03:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.614+0000 7efc6e59c700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc4c0539e0 0x7efc4c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.614+0000 7efc6e59c700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc4c0539e0 0x7efc4c055ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7efc6000b000 tx=0x7efc60006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.626+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efc64043170 con 0x7efc700af450 2022-08-01T23:03:09.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.782+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc70142710 con 0x7efc700af450 2022-08-01T23:03:09.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc5f7fe700 1 -- 172.21.15.32:0/3275167379 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7efc64046790 con 0x7efc700af450 2022-08-01T23:03:09.792 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:09.793 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:09.472203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6981":{"gid":6981,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2562641804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2562641804},{"type":"v1","addr":"172.21.15.32:6835","nonce":2562641804}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc4c0539e0 msgr2=0x7efc4c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc4c0539e0 0x7efc4c055ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7efc6000b000 tx=0x7efc60006df0).stop 2022-08-01T23:03:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 msgr2=0x7efc70142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 0x7efc70142d90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7efc700aabe0 tx=0x7efc64008040).stop 2022-08-01T23:03:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 shutdown_connections 2022-08-01T23:03:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc4c0539e0 0x7efc4c055ea0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc701437d0 0x7efc700ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc700af450 0x7efc70142d90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 --2- 172.21.15.32:0/3275167379 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc700ad4b0 0x7efc70139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 >> 172.21.15.32:0/3275167379 conn(0x7efc700612d0 msgr2=0x7efc70144650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 shutdown_connections 2022-08-01T23:03:09.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.786+0000 7efc6f59e700 1 -- 172.21.15.32:0/3275167379 wait complete. 2022-08-01T23:03:09.799 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2022-08-01T23:03:09.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6517fff700 1 Processor -- start 2022-08-01T23:03:09.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6517fff700 1 -- start start 2022-08-01T23:03:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6517fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 0x7f6518148540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6517fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65180b3570 con 0x7f65180b1a80 2022-08-01T23:03:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6517fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65180b0e00 con 0x7f65180b3190 2022-08-01T23:03:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6517fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65180b10a0 con 0x7f65180a9da0 2022-08-01T23:03:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65167fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65180b1a80 0x7f6518147e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33876/0 (socket says 172.21.15.32:33876) 2022-08-01T23:03:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65167fc700 1 -- 172.21.15.32:0/1038619934 learned_addr learned my addr 172.21.15.32:0/1038619934 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 0x7f6518148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814293337 0 0) 0x7f65180b3570 con 0x7f65180b1a80 2022-08-01T23:03:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64fc003260 con 0x7f65180b1a80 2022-08-01T23:03:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862494430 0 0) 0x7f65180b0e00 con 0x7f65180b3190 2022-08-01T23:03:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65180b3570 con 0x7f65180b3190 2022-08-01T23:03:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284937333 0 0) 0x7f64fc003260 con 0x7f65180b1a80 2022-08-01T23:03:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 msgr2=0x7f6518148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 0x7f6518148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 >> v1:172.21.15.55:6789/0 conn(0x7f65180b3190 legacy=0x7f65180656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180b1340 con 0x7f65180b1a80 2022-08-01T23:03:09.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f6508001660 con 0x7f65180b1a80 2022-08-01T23:03:09.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 >> v1:172.21.15.32:6789/0 conn(0x7f65180b1a80 legacy=0x7f6518147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 0x7f64fc003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64fc003e20 0x7f64fc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f64fc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180b10a0 con 0x7f64fc0049f0 2022-08-01T23:03:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc00b800 con 0x7f64fc003e20 2022-08-01T23:03:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc00b9c0 con 0x7f65180a9da0 2022-08-01T23:03:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f64fc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65167fc700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64fc003e20 0x7f64fc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6516ffd700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 0x7f64fc003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 -- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 msgr2=0x7f64fc003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180a9da0 0x7f64fc003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 -- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64fc003e20 msgr2=0x7f64fc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64fc003e20 0x7f64fc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 -- 172.21.15.32:0/1038619934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc003260 con 0x7f64fc0049f0 2022-08-01T23:03:09.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f65177fe700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f64fc004e70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f651000c000 tx=0x7f65100091d0).ready entity=mon.0 client_cookie=a3358e771564c800 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f651000e7e0 con 0x7f64fc0049f0 2022-08-01T23:03:09.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651000e940 con 0x7f64fc0049f0 2022-08-01T23:03:09.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.910+0000 7f6515ffb700 1 -- 172.21.15.32:0/1038619934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f651000eca0 con 0x7f64fc0049f0 2022-08-01T23:03:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 -- 172.21.15.32:0/1038619934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 msgr2=0x7f64fc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 --2- 172.21.15.32:0/1038619934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f64fc004e70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f651000c000 tx=0x7f65100091d0).stop 2022-08-01T23:03:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f65177fe700 1 -- 172.21.15.32:0/1038619934 reap_dead start 2022-08-01T23:03:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 -- 172.21.15.32:0/1038619934 shutdown_connections 2022-08-01T23:03:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 -- 172.21.15.32:0/1038619934 >> 172.21.15.32:0/1038619934 conn(0x7f6518063870 msgr2=0x7f6518063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 -- 172.21.15.32:0/1038619934 shutdown_connections 2022-08-01T23:03:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 -- 172.21.15.32:0/1038619934 wait complete. 2022-08-01T23:03:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 Processor -- start 2022-08-01T23:03:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.914+0000 7f6517fff700 1 -- start start 2022-08-01T23:03:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f65180aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180b5ff0 0x7f6518141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65180b5630 0x7f65180ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180aced0 con 0x7f64fc0049f0 2022-08-01T23:03:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518157040 con 0x7f65180b5630 2022-08-01T23:03:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65180aaee0 con 0x7f65180b5ff0 2022-08-01T23:03:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f65180aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f65180aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46346/0 (socket says 172.21.15.32:46346) 2022-08-01T23:03:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 -- 172.21.15.32:0/2914228741 learned_addr learned my addr 172.21.15.32:0/2914228741 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f65177fe700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65180b5630 0x7f65180ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f65167fc700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180b5ff0 0x7f6518141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 -- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180b5ff0 msgr2=0x7f6518141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180b5ff0 0x7f6518141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 -- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65180b5630 msgr2=0x7f65180ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65180b5630 0x7f65180ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 -- 172.21.15.32:0/2914228741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180ad100 con 0x7f64fc0049f0 2022-08-01T23:03:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f65167fc700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180b5ff0 0x7f6518141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6516ffd700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f65180aa9f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f65181571d0 tx=0x7f650c007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f650c011590 con 0x7f64fc0049f0 2022-08-01T23:03:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c0116f0 con 0x7f64fc0049f0 2022-08-01T23:03:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65181572b0 con 0x7f64fc0049f0 2022-08-01T23:03:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f650c011a50 con 0x7f64fc0049f0 2022-08-01T23:03:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65180add20 con 0x7f64fc0049f0 2022-08-01T23:03:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.918+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65180574e0 con 0x7f64fc0049f0 2022-08-01T23:03:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.926+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f650c01a030 con 0x7f64fc0049f0 2022-08-01T23:03:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.926+0000 7f6507fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64f4053a10 0x7f64f4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.926+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f650c06be30 con 0x7f64fc0049f0 2022-08-01T23:03:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.926+0000 7f65167fc700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64f4053a10 0x7f64f4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.926+0000 7f65167fc700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64f4053a10 0x7f64f4055ed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f6508005000 tx=0x7f6508009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:09.926+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f650c042ae0 con 0x7f64fc0049f0 2022-08-01T23:03:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.058+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6518157440 con 0x7f64fc0049f0 2022-08-01T23:03:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.058+0000 7f6507fff700 1 -- 172.21.15.32:0/2914228741 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f650c046100 con 0x7f64fc0049f0 2022-08-01T23:03:10.064 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:10.065 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:09.472841+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:06.881558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:07.840459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64f4053a10 msgr2=0x7f64f4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64f4053a10 0x7f64f4055ed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f6508005000 tx=0x7f6508009040).stop 2022-08-01T23:03:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 msgr2=0x7f65180aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f65180aa9f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f65181571d0 tx=0x7f650c007a00).stop 2022-08-01T23:03:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 shutdown_connections 2022-08-01T23:03:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64f4053a10 0x7f64f4055ed0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65180b5630 0x7f65180ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f65180b5ff0 0x7f6518141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 --2- 172.21.15.32:0/2914228741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0049f0 0x7f65180aa9f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 >> 172.21.15.32:0/2914228741 conn(0x7f6518063870 msgr2=0x7f6518145710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 shutdown_connections 2022-08-01T23:03:10.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.062+0000 7f6517fff700 1 -- 172.21.15.32:0/2914228741 wait complete. 2022-08-01T23:03:10.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:03:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4e013e9700 1 Processor -- start 2022-08-01T23:03:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4e013e9700 1 -- start start 2022-08-01T23:03:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4e013e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4dfc0b6710 0x7f4dfc0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4e013e9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dfc144240 con 0x7f4dfc0b4550 2022-08-01T23:03:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4e013e9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dfc143d00 con 0x7f4dfc0b6af0 2022-08-01T23:03:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4e013e9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dfc143fa0 con 0x7f4dfc0b6710 2022-08-01T23:03:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4dfaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4dfc0b6710 0x7f4dfc0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4dfa7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4dfc0b4550 0x7f4dfc061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33896/0 (socket says 172.21.15.32:33896) 2022-08-01T23:03:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4dfa7fc700 1 -- 172.21.15.32:0/2918835018 learned_addr learned my addr 172.21.15.32:0/2918835018 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4df9ffb700 1 -- 172.21.15.32:0/2918835018 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868702798 0 0) 0x7f4dfc144240 con 0x7f4dfc0b4550 2022-08-01T23:03:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.190+0000 7f4df9ffb700 1 -- 172.21.15.32:0/2918835018 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ddc003260 con 0x7f4dfc0b4550 2022-08-01T23:03:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 -- 172.21.15.32:0/2918835018 >> v1:172.21.15.55:6789/0 conn(0x7f4dfc0b6af0 legacy=0x7f4dfc12bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4df9ffb700 1 -- 172.21.15.32:0/2918835018 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094022251 0 0) 0x7f4dfc143d00 con 0x7f4dfc0b6af0 2022-08-01T23:03:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 -- 172.21.15.32:0/2918835018 >> v1:172.21.15.32:6789/0 conn(0x7f4dfc0b4550 legacy=0x7f4dfc061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 -- 172.21.15.32:0/2918835018 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc061780 con 0x7f4dfc0b6710 2022-08-01T23:03:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 --2- 172.21.15.32:0/2918835018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc0b5d80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f4dfc1432b0 tx=0x7f4dec006df0).ready entity=mon.0 client_cookie=37782eb488a499df server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4df9ffb700 1 -- 172.21.15.32:0/2918835018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dec00d610 con 0x7f4dfc0b6710 2022-08-01T23:03:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4df9ffb700 1 -- 172.21.15.32:0/2918835018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec000e90 con 0x7f4dfc0b6710 2022-08-01T23:03:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4df9ffb700 1 -- 172.21.15.32:0/2918835018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dec00d960 con 0x7f4dfc0b6710 2022-08-01T23:03:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- 172.21.15.32:0/2918835018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b6710 msgr2=0x7f4dfc0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 --2- 172.21.15.32:0/2918835018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc0b5d80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f4dfc1432b0 tx=0x7f4dec006df0).stop 2022-08-01T23:03:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- 172.21.15.32:0/2918835018 shutdown_connections 2022-08-01T23:03:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 --2- 172.21.15.32:0/2918835018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc0b5d80 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- 172.21.15.32:0/2918835018 >> 172.21.15.32:0/2918835018 conn(0x7f4dfc05c620 msgr2=0x7f4dfc05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- 172.21.15.32:0/2918835018 shutdown_connections 2022-08-01T23:03:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- 172.21.15.32:0/2918835018 wait complete. 2022-08-01T23:03:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 Processor -- start 2022-08-01T23:03:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- start start 2022-08-01T23:03:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 0x7f4dfc12ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc12a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4dfc0b6af0 0x7f4dfc12be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc12b340 con 0x7f4dfc0b4550 2022-08-01T23:03:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc152f60 con 0x7f4dfc0b6710 2022-08-01T23:03:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4e013e9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc12b570 con 0x7f4dfc0b6af0 2022-08-01T23:03:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 0x7f4dfc12ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 0x7f4dfc12ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46360/0 (socket says 172.21.15.32:46360) 2022-08-01T23:03:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfaffd700 1 -- 172.21.15.32:0/1452565839 learned_addr learned my addr 172.21.15.32:0/1452565839 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfa7fc700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc12a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.194+0000 7f4dfb7fe700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4dfc0b6af0 0x7f4dfc12be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfaffd700 1 -- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4dfc0b6af0 msgr2=0x7f4dfc12be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfaffd700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4dfc0b6af0 0x7f4dfc12be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfaffd700 1 -- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dfc0b6710 msgr2=0x7f4dfc12a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfaffd700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc12a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfaffd700 1 -- 172.21.15.32:0/1452565839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec00b180 con 0x7f4dfc0b4550 2022-08-01T23:03:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfa7fc700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc12a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfb7fe700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4dfc0b6af0 0x7f4dfc12be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4dfaffd700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 0x7f4dfc12ac20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f4dfc1432b0 tx=0x7f4dec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dec016070 con 0x7f4dfc0b4550 2022-08-01T23:03:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec002f30 con 0x7f4dfc0b4550 2022-08-01T23:03:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dfc12b810 con 0x7f4dfc0b4550 2022-08-01T23:03:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dec013d60 con 0x7f4dfc0b4550 2022-08-01T23:03:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc152b60 con 0x7f4dfc0b4550 2022-08-01T23:03:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dfc12a450 con 0x7f4dfc0b4550 2022-08-01T23:03:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4dec025030 con 0x7f4dfc0b4550 2022-08-01T23:03:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4df3fff700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4de0053b00 0x7f4de0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.198+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4dec06d080 con 0x7f4dfc0b4550 2022-08-01T23:03:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.202+0000 7f4dfa7fc700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4de0053b00 0x7f4de0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.202+0000 7f4dfa7fc700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4de0053b00 0x7f4de0055fc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4de400d000 tx=0x7f4de4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.206+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4dfc12a450 con 0x7f4dfc0b4550 2022-08-01T23:03:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.378+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4dfc12a450 con 0x7f4dfc0b4550 2022-08-01T23:03:10.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.386+0000 7f4df3fff700 1 -- 172.21.15.32:0/1452565839 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4dfc12a450 con 0x7f4dfc0b4550 2022-08-01T23:03:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4de0053b00 msgr2=0x7f4de0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4de0053b00 0x7f4de0055fc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4de400d000 tx=0x7f4de4009040).stop 2022-08-01T23:03:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 msgr2=0x7f4dfc12ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 0x7f4dfc12ac20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f4dfc1432b0 tx=0x7f4dec014000).stop 2022-08-01T23:03:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 shutdown_connections 2022-08-01T23:03:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4de0053b00 0x7f4de0055fc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4dfc0b6af0 0x7f4dfc12be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4dfc0b6710 0x7f4dfc12a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 --2- 172.21.15.32:0/1452565839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dfc0b4550 0x7f4dfc12ac20 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 >> 172.21.15.32:0/1452565839 conn(0x7f4dfc05c620 msgr2=0x7f4dfc141120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 shutdown_connections 2022-08-01T23:03:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.390+0000 7f4e013e9700 1 -- 172.21.15.32:0/1452565839 wait complete. 2022-08-01T23:03:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:03:10.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:03:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29c49fe700 1 Processor -- start 2022-08-01T23:03:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29c49fe700 1 -- start start 2022-08-01T23:03:10.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29c49fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29c00b4550 0x7f29c00b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29c49fe700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c0144240 con 0x7f29c00b6710 2022-08-01T23:03:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29c49fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c0143d00 con 0x7f29c00b6af0 2022-08-01T23:03:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29c49fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c0143fa0 con 0x7f29c00b4550 2022-08-01T23:03:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29be59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29c00b6710 0x7f29c0061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33910/0 (socket says 172.21.15.32:33910) 2022-08-01T23:03:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29be59c700 1 -- 172.21.15.32:0/3689780474 learned_addr learned my addr 172.21.15.32:0/3689780474 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:10.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29bdd9b700 1 --2- 172.21.15.32:0/3689780474 >> v2:172.21.15.55:3300/0 conn(0x7f29c00b4550 0x7f29c00b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29bdd9b700 1 -- 172.21.15.32:0/3689780474 >> v1:172.21.15.55:6789/0 conn(0x7f29c00b6af0 legacy=0x7f29c012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29bdd9b700 1 -- 172.21.15.32:0/3689780474 >> v1:172.21.15.32:6789/0 conn(0x7f29c00b6710 legacy=0x7f29c0061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.518+0000 7f29bdd9b700 1 -- 172.21.15.32:0/3689780474 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0061780 con 0x7f29c00b4550 2022-08-01T23:03:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.522+0000 7f29bdd9b700 1 --2- 172.21.15.32:0/3689780474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c00b5d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f29c01432b0 tx=0x7f29a8006df0).ready entity=mon.1 client_cookie=c8d09446c82e4e5a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29bd59a700 1 -- 172.21.15.32:0/3689780474 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a800d610 con 0x7f29c00b4550 2022-08-01T23:03:10.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29bd59a700 1 -- 172.21.15.32:0/3689780474 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8000e90 con 0x7f29c00b4550 2022-08-01T23:03:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29bd59a700 1 -- 172.21.15.32:0/3689780474 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a800d960 con 0x7f29c00b4550 2022-08-01T23:03:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 -- 172.21.15.32:0/3689780474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 msgr2=0x7f29c00b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 --2- 172.21.15.32:0/3689780474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c00b5d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f29c01432b0 tx=0x7f29a8006df0).stop 2022-08-01T23:03:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 -- 172.21.15.32:0/3689780474 shutdown_connections 2022-08-01T23:03:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 --2- 172.21.15.32:0/3689780474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c00b5d80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 -- 172.21.15.32:0/3689780474 >> 172.21.15.32:0/3689780474 conn(0x7f29c005c620 msgr2=0x7f29c005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 -- 172.21.15.32:0/3689780474 shutdown_connections 2022-08-01T23:03:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.526+0000 7f29c49fe700 1 -- 172.21.15.32:0/3689780474 wait complete. 2022-08-01T23:03:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 Processor -- start 2022-08-01T23:03:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 -- start start 2022-08-01T23:03:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c0147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 0x7f29c0140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c00b6af0 0x7f29c0140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0129a90 con 0x7f29c00b6710 2022-08-01T23:03:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0126b00 con 0x7f29c00b4550 2022-08-01T23:03:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29c49fe700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0126fa0 con 0x7f29c00b6af0 2022-08-01T23:03:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29bdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 0x7f29c0140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29bdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 0x7f29c0140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46376/0 (socket says 172.21.15.32:46376) 2022-08-01T23:03:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.530+0000 7f29bdd9b700 1 -- 172.21.15.32:0/1280920667 learned_addr learned my addr 172.21.15.32:0/1280920667 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c0147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29bed9d700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c00b6af0 0x7f29c0140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 -- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c00b6af0 msgr2=0x7f29c0140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c00b6af0 0x7f29c0140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 -- 172.21.15.32:0/1280920667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 msgr2=0x7f29c0140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 0x7f29c0140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 -- 172.21.15.32:0/1280920667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a800b180 con 0x7f29c00b4550 2022-08-01T23:03:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29bdd9b700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 0x7f29c0140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29be59c700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c0147f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f29c00b1e10 tx=0x7f29b0007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b00070d0 con 0x7f29c00b4550 2022-08-01T23:03:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0007230 con 0x7f29c00b4550 2022-08-01T23:03:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c012a420 con 0x7f29c00b4550 2022-08-01T23:03:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b0005610 con 0x7f29c00b4550 2022-08-01T23:03:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00b3b60 con 0x7f29c00b4550 2022-08-01T23:03:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c0061a20 con 0x7f29c00b4550 2022-08-01T23:03:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29b0010070 con 0x7f29c00b4550 2022-08-01T23:03:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.534+0000 7f29b77fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29a4053950 0x7f29a4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.538+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f29b006b4f0 con 0x7f29c00b4550 2022-08-01T23:03:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.538+0000 7f29bdd9b700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29a4053950 0x7f29a4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.538+0000 7f29bdd9b700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29a4053950 0x7f29a4055e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f29a801c000 tx=0x7f29a8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.542+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29b0042210 con 0x7f29c00b4550 2022-08-01T23:03:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.706+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f29c00574e0 con 0x7f29c00b4550 2022-08-01T23:03:10.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29b77fe700 1 -- 172.21.15.32:0/1280920667 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f29b0018070 con 0x7f29c00b4550 2022-08-01T23:03:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29a4053950 msgr2=0x7f29a4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29a4053950 0x7f29a4055e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f29a801c000 tx=0x7f29a8015000).stop 2022-08-01T23:03:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 msgr2=0x7f29c0147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c0147f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f29c00b1e10 tx=0x7f29b0007b80).stop 2022-08-01T23:03:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 shutdown_connections 2022-08-01T23:03:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29a4053950 0x7f29a4055e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c00b6af0 0x7f29c0140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29c00b6710 0x7f29c0140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 --2- 172.21.15.32:0/1280920667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c00b4550 0x7f29c0147f20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 >> 172.21.15.32:0/1280920667 conn(0x7f29c005c620 msgr2=0x7f29c01272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 shutdown_connections 2022-08-01T23:03:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.722+0000 7f29c49fe700 1 -- 172.21.15.32:0/1280920667 wait complete. 2022-08-01T23:03:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:03:10.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:10.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 Processor -- start 2022-08-01T23:03:10.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 -- start start 2022-08-01T23:03:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f417c0b6a90 0x7f417c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f417c0aa760 0x7f417c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c0b6e70 con 0x7f417c060aa0 2022-08-01T23:03:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c061370 con 0x7f417c0aa760 2022-08-01T23:03:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.842+0000 7f41845eb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c061610 con 0x7f417c0b6a90 2022-08-01T23:03:10.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f417c0aa760 0x7f417c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f417c0aa760 0x7f417c13c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46386/0 (socket says 172.21.15.32:46386) 2022-08-01T23:03:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 -- 172.21.15.32:0/2042766203 learned_addr learned my addr 172.21.15.32:0/2042766203 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182387700 1 --2- 172.21.15.32:0/2042766203 >> v2:172.21.15.55:3300/0 conn(0x7f417c0b6a90 0x7f417c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 -- 172.21.15.32:0/2042766203 >> v2:172.21.15.55:3300/0 conn(0x7f417c0b6a90 msgr2=0x7f417c0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:10.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 --2- 172.21.15.32:0/2042766203 >> v2:172.21.15.55:3300/0 conn(0x7f417c0b6a90 0x7f417c0acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 -- 172.21.15.32:0/2042766203 >> v1:172.21.15.55:6789/0 conn(0x7f417c060aa0 legacy=0x7f417c0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 -- 172.21.15.32:0/2042766203 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0618b0 con 0x7f417c0aa760 2022-08-01T23:03:10.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4182b88700 1 --2- 172.21.15.32:0/2042766203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c0aa760 0x7f417c13c600 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f41740032f0 tx=0x7f4174009290).ready entity=mon.0 client_cookie=9414d991f95efda4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4181385700 1 -- 172.21.15.32:0/2042766203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f417400ba60 con 0x7f417c0aa760 2022-08-01T23:03:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4181385700 1 -- 172.21.15.32:0/2042766203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400bbc0 con 0x7f417c0aa760 2022-08-01T23:03:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f4181385700 1 -- 172.21.15.32:0/2042766203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4174010410 con 0x7f417c0aa760 2022-08-01T23:03:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 -- 172.21.15.32:0/2042766203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c0aa760 msgr2=0x7f417c13c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 --2- 172.21.15.32:0/2042766203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c0aa760 0x7f417c13c600 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f41740032f0 tx=0x7f4174009290).stop 2022-08-01T23:03:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 -- 172.21.15.32:0/2042766203 shutdown_connections 2022-08-01T23:03:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 --2- 172.21.15.32:0/2042766203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c0aa760 0x7f417c13c600 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 --2- 172.21.15.32:0/2042766203 >> v2:172.21.15.55:3300/0 conn(0x7f417c0b6a90 0x7f417c0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 -- 172.21.15.32:0/2042766203 >> 172.21.15.32:0/2042766203 conn(0x7f417c0b3610 msgr2=0x7f417c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 -- 172.21.15.32:0/2042766203 shutdown_connections 2022-08-01T23:03:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 -- 172.21.15.32:0/2042766203 wait complete. 2022-08-01T23:03:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 Processor -- start 2022-08-01T23:03:10.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.846+0000 7f41845eb700 1 -- start start 2022-08-01T23:03:10.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 0x7f417c152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f417c0aa760 0x7f417c13e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f417c0b6a90 0x7f417c156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c137210 con 0x7f417c060aa0 2022-08-01T23:03:10.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c155b50 con 0x7f417c0aa760 2022-08-01T23:03:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0b1ff0 con 0x7f417c0b6a90 2022-08-01T23:03:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 0x7f417c152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 0x7f417c152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46388/0 (socket says 172.21.15.32:46388) 2022-08-01T23:03:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 -- 172.21.15.32:0/925321566 learned_addr learned my addr 172.21.15.32:0/925321566 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:10.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182b88700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f417c0b6a90 0x7f417c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4181b86700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f417c0aa760 0x7f417c13e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 -- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f417c0b6a90 msgr2=0x7f417c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f417c0b6a90 0x7f417c156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 -- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f417c0aa760 msgr2=0x7f417c13e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f417c0aa760 0x7f417c13e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 -- 172.21.15.32:0/925321566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174009530 con 0x7f417c060aa0 2022-08-01T23:03:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f4182387700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 0x7f417c152580 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f417800c000 tx=0x7f4178006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4178010780 con 0x7f417c060aa0 2022-08-01T23:03:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41780108e0 con 0x7f417c060aa0 2022-08-01T23:03:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c0b2290 con 0x7f417c060aa0 2022-08-01T23:03:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4178010c40 con 0x7f417c060aa0 2022-08-01T23:03:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c0ae5f0 con 0x7f417c060aa0 2022-08-01T23:03:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.850+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c0acd90 con 0x7f417c060aa0 2022-08-01T23:03:10.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.854+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f417801c460 con 0x7f417c060aa0 2022-08-01T23:03:10.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.854+0000 7f416b7fe700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f415c053ac0 0x7f415c055f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:10.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.854+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f417806c550 con 0x7f417c060aa0 2022-08-01T23:03:10.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.854+0000 7f4181b86700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f415c053ac0 0x7f415c055f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.854+0000 7f4181b86700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f415c053ac0 0x7f415c055f80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f416c00d000 tx=0x7f416c007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:10.858+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f417c0acd90 con 0x7f417c060aa0 2022-08-01T23:03:11.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.022+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f417c0acd90 con 0x7f417c060aa0 2022-08-01T23:03:11.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.022+0000 7f416b7fe700 1 -- 172.21.15.32:0/925321566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f417c0acd90 con 0x7f417c060aa0 2022-08-01T23:03:11.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:11.028 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:09.472203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6981":{"gid":6981,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2562641804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2562641804},{"type":"v1","addr":"172.21.15.32:6835","nonce":2562641804}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:11.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f415c053ac0 msgr2=0x7f415c055f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f415c053ac0 0x7f415c055f80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f416c00d000 tx=0x7f416c007b90).stop 2022-08-01T23:03:11.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 msgr2=0x7f417c152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 0x7f417c152580 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f417800c000 tx=0x7f4178006df0).stop 2022-08-01T23:03:11.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 shutdown_connections 2022-08-01T23:03:11.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f415c053ac0 0x7f415c055f80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f417c0b6a90 0x7f417c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f417c0aa760 0x7f417c13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 --2- 172.21.15.32:0/925321566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417c060aa0 0x7f417c152580 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 >> 172.21.15.32:0/925321566 conn(0x7f417c0b3610 msgr2=0x7f417c142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:11.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 shutdown_connections 2022-08-01T23:03:11.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.026+0000 7f41845eb700 1 -- 172.21.15.32:0/925321566 wait complete. 2022-08-01T23:03:11.034 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2022-08-01T23:03:11.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:08.469158+0000', 'modified': '2022-08-01T23:03:09.472203+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6981}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6981': {'gid': 6981, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2562641804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2562641804}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2562641804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:11.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:11.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:03:11.471 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6981,"want_state":"up:active","state":"up:active","rank_uptime":2.9874871999999999,"mdsmap_epoch":45,"osdmap_epoch":97,"osdmap_epoch_barrier":96,"uptime":7.3209636280000003} 2022-08-01T23:03:11.472 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6981, "mdsmap_epoch": 45, "osdmap_epoch": 97, "osdmap_epoch_barrier": 96, "rank_uptime": 2.9874872, "state": "up:active", "uptime": 7.320963628, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:11.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:11.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 Processor -- start 2022-08-01T23:03:11.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- start start 2022-08-01T23:03:11.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f93700b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93700a7250 0x7f93701448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93700b04e0 con 0x7f93700b1f40 2022-08-01T23:03:11.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93700a9250 con 0x7f93700a7250 2022-08-01T23:03:11.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700b0780 con 0x7f9370148530 2022-08-01T23:03:11.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93700a7250 0x7f93701448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93700a7250 0x7f93701448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46400/0 (socket says 172.21.15.32:46400) 2022-08-01T23:03:11.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 -- 172.21.15.32:0/1397554773 learned_addr learned my addr 172.21.15.32:0/1397554773 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:11.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f9365d9b700 1 --2- 172.21.15.32:0/1397554773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f93700b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 -- 172.21.15.32:0/1397554773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 msgr2=0x7f93700b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:11.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 --2- 172.21.15.32:0/1397554773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f93700b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 -- 172.21.15.32:0/1397554773 >> v1:172.21.15.55:6789/0 conn(0x7f93700b1f40 legacy=0x7f93700b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 -- 172.21.15.32:0/1397554773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700b0a20 con 0x7f93700a7250 2022-08-01T23:03:11.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936dd9b700 1 -- 172.21.15.32:0/1397554773 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011609019 0 0) 0x7f93700b04e0 con 0x7f93700b1f40 2022-08-01T23:03:11.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936ed9d700 1 --2- 172.21.15.32:0/1397554773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700a7250 0x7f93701448b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f93680032f0 tx=0x7f9368009290).ready entity=mon.0 client_cookie=646354a3d908d7a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:11.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936dd9b700 1 -- 172.21.15.32:0/1397554773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f936800b8a0 con 0x7f93700a7250 2022-08-01T23:03:11.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936dd9b700 1 -- 172.21.15.32:0/1397554773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800ba00 con 0x7f93700a7250 2022-08-01T23:03:11.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f936dd9b700 1 -- 172.21.15.32:0/1397554773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f936800bb70 con 0x7f93700a7250 2022-08-01T23:03:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- 172.21.15.32:0/1397554773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700a7250 msgr2=0x7f93701448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 --2- 172.21.15.32:0/1397554773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700a7250 0x7f93701448b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f93680032f0 tx=0x7f9368009290).stop 2022-08-01T23:03:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- 172.21.15.32:0/1397554773 shutdown_connections 2022-08-01T23:03:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 --2- 172.21.15.32:0/1397554773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700a7250 0x7f93701448b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 --2- 172.21.15.32:0/1397554773 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f93700b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- 172.21.15.32:0/1397554773 >> 172.21.15.32:0/1397554773 conn(0x7f937005f840 msgr2=0x7f937005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- 172.21.15.32:0/1397554773 shutdown_connections 2022-08-01T23:03:11.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.578+0000 7f93748e0700 1 -- 172.21.15.32:0/1397554773 wait complete. 2022-08-01T23:03:11.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 Processor -- start 2022-08-01T23:03:11.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 -- start start 2022-08-01T23:03:11.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93700a7250 0x7f9370144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 0x7f9370143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f9370143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9370063a00 con 0x7f93700b1f40 2022-08-01T23:03:11.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700ab930 con 0x7f93700a7250 2022-08-01T23:03:11.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700aaa80 con 0x7f9370148530 2022-08-01T23:03:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 0x7f9370143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 0x7f9370143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46404/0 (socket says 172.21.15.32:46404) 2022-08-01T23:03:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 -- 172.21.15.32:0/485788683 learned_addr learned my addr 172.21.15.32:0/485788683 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f936e59c700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93700a7250 0x7f9370144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f936ed9d700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f9370143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 -- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 msgr2=0x7f9370143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f9370143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 -- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93700a7250 msgr2=0x7f9370144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93700a7250 0x7f9370144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 -- 172.21.15.32:0/485788683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009530 con 0x7f93700b1f40 2022-08-01T23:03:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9365d9b700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 0x7f9370143a60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f935800c000 tx=0x7f9358006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9358011750 con 0x7f93700b1f40 2022-08-01T23:03:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580118b0 con 0x7f93700b1f40 2022-08-01T23:03:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.582+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93700aad20 con 0x7f93700b1f40 2022-08-01T23:03:11.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9358011c10 con 0x7f93700b1f40 2022-08-01T23:03:11.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9370144440 con 0x7f93700b1f40 2022-08-01T23:03:11.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9358011d70 con 0x7f93700b1f40 2022-08-01T23:03:11.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93701568b0 con 0x7f93700b1f40 2022-08-01T23:03:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f9367fff700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93540539e0 0x7f9354055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f935806c340 con 0x7f93700b1f40 2022-08-01T23:03:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f936e59c700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93540539e0 0x7f9354055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.586+0000 7f936e59c700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93540539e0 0x7f9354055ea0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f936000d000 tx=0x7f9360009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.594+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9358043100 con 0x7f93700b1f40 2022-08-01T23:03:11.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.722+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93700574e0 con 0x7f93700b1f40 2022-08-01T23:03:11.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f9367fff700 1 -- 172.21.15.32:0/485788683 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f9358046720 con 0x7f93700b1f40 2022-08-01T23:03:11.730 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:11.731 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:09.472841+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:06.881558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:07.840459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:11.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93540539e0 msgr2=0x7f9354055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93540539e0 0x7f9354055ea0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f936000d000 tx=0x7f9360009040).stop 2022-08-01T23:03:11.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 msgr2=0x7f9370143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 0x7f9370143a60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f935800c000 tx=0x7f9358006df0).stop 2022-08-01T23:03:11.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 shutdown_connections 2022-08-01T23:03:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f93540539e0 0x7f9354055ea0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9370148530 0x7f9370143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93700b1f40 0x7f9370143a60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 --2- 172.21.15.32:0/485788683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93700a7250 0x7f9370144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 >> 172.21.15.32:0/485788683 conn(0x7f937005f840 msgr2=0x7f93700b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:11.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 shutdown_connections 2022-08-01T23:03:11.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.726+0000 7f93748e0700 1 -- 172.21.15.32:0/485788683 wait complete. 2022-08-01T23:03:11.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:03:06.881558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:03:07.840459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:03:11.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:11.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 Processor -- start 2022-08-01T23:03:11.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 -- start start 2022-08-01T23:03:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f8c0b2710 0x7f9f8c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c0a7250 con 0x7f9f8c0afd60 2022-08-01T23:03:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c0a7480 con 0x7f9f8c0b2710 2022-08-01T23:03:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.850+0000 7f9f91cf0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0a7720 con 0x7f9f8c0b01a0 2022-08-01T23:03:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f914ef700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9f8c0afd60 0x7f9f8c0a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33956/0 (socket says 172.21.15.32:33956) 2022-08-01T23:03:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f914ef700 1 -- 172.21.15.32:0/442408255 learned_addr learned my addr 172.21.15.32:0/442408255 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f90cee700 1 --2- 172.21.15.32:0/442408255 >> v2:172.21.15.32:3300/0 conn(0x7f9f8c0b2710 0x7f9f8c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8bfff700 1 --2- 172.21.15.32:0/442408255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8b7fe700 1 -- 172.21.15.32:0/442408255 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165164950 0 0) 0x7f9f8c0a7250 con 0x7f9f8c0afd60 2022-08-01T23:03:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8b7fe700 1 -- 172.21.15.32:0/442408255 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f78003260 con 0x7f9f8c0afd60 2022-08-01T23:03:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f90cee700 1 -- 172.21.15.32:0/442408255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 msgr2=0x7f9f8c14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f90cee700 1 --2- 172.21.15.32:0/442408255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f90cee700 1 -- 172.21.15.32:0/442408255 >> v1:172.21.15.32:6789/0 conn(0x7f9f8c0afd60 legacy=0x7f9f8c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8b7fe700 1 -- 172.21.15.32:0/442408255 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533668106 0 0) 0x7f9f78003260 con 0x7f9f8c0afd60 2022-08-01T23:03:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f90cee700 1 -- 172.21.15.32:0/442408255 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0a79c0 con 0x7f9f8c0b2710 2022-08-01T23:03:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f90cee700 1 --2- 172.21.15.32:0/442408255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c14afd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f9f80002f70 tx=0x7f9f80006df0).ready entity=mon.0 client_cookie=e03f990b5db69656 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:11.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8b7fe700 1 -- 172.21.15.32:0/442408255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f8000d4a0 con 0x7f9f8c0b2710 2022-08-01T23:03:11.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8b7fe700 1 -- 172.21.15.32:0/442408255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f80000e90 con 0x7f9f8c0b2710 2022-08-01T23:03:11.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f8b7fe700 1 -- 172.21.15.32:0/442408255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f8000d7f0 con 0x7f9f8c0b2710 2022-08-01T23:03:11.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 -- 172.21.15.32:0/442408255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 msgr2=0x7f9f8c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/442408255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c14afd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f9f80002f70 tx=0x7f9f80006df0).stop 2022-08-01T23:03:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 -- 172.21.15.32:0/442408255 shutdown_connections 2022-08-01T23:03:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/442408255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/442408255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c14afd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 -- 172.21.15.32:0/442408255 >> 172.21.15.32:0/442408255 conn(0x7f9f8c060310 msgr2=0x7f9f8c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 -- 172.21.15.32:0/442408255 shutdown_connections 2022-08-01T23:03:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.854+0000 7f9f91cf0700 1 -- 172.21.15.32:0/442408255 wait complete. 2022-08-01T23:03:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 Processor -- start 2022-08-01T23:03:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- start start 2022-08-01T23:03:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f8c0afd60 0x7f9f8c143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c156d40 con 0x7f9f8c0b2710 2022-08-01T23:03:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b5ff0 con 0x7f9f8c0afd60 2022-08-01T23:03:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b3610 con 0x7f9f8c0b01a0 2022-08-01T23:03:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46420/0 (socket says 172.21.15.32:46420) 2022-08-01T23:03:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 -- 172.21.15.32:0/345427722 learned_addr learned my addr 172.21.15.32:0/345427722 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f8bfff700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f90cee700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f8c0afd60 0x7f9f8c143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 -- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 msgr2=0x7f9f8c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 -- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f8c0afd60 msgr2=0x7f9f8c143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f8c0afd60 0x7f9f8c143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 -- 172.21.15.32:0/345427722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8000b180 con 0x7f9f8c0b2710 2022-08-01T23:03:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f914ef700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c156850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f9f84001250 tx=0x7f9f84007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f84004ab0 con 0x7f9f8c0b2710 2022-08-01T23:03:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f84004c10 con 0x7f9f8c0b2710 2022-08-01T23:03:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c14cd70 con 0x7f9f8c0b2710 2022-08-01T23:03:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f84007170 con 0x7f9f8c0b2710 2022-08-01T23:03:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c0b2f60 con 0x7f9f8c0b2710 2022-08-01T23:03:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.858+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c15a860 con 0x7f9f8c0b2710 2022-08-01T23:03:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.862+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f840072d0 con 0x7f9f8c0b2710 2022-08-01T23:03:11.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.862+0000 7f9f89ffb700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f6c053b00 0x7f9f6c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:11.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.862+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9f8406bd80 con 0x7f9f8c0b2710 2022-08-01T23:03:11.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.862+0000 7f9f90cee700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f6c053b00 0x7f9f6c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:11.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.862+0000 7f9f90cee700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f6c053b00 0x7f9f6c055fc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f9f8001a000 tx=0x7f9f80014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.866+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f84042b40 con 0x7f9f8c0b2710 2022-08-01T23:03:11.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.990+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9f8c0b61b0 con 0x7f9f8c0b2710 2022-08-01T23:03:11.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f89ffb700 1 -- 172.21.15.32:0/345427722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v97) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9f8406d030 con 0x7f9f8c0b2710 2022-08-01T23:03:11.997 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f6c053b00 msgr2=0x7f9f6c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f6c053b00 0x7f9f6c055fc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f9f8001a000 tx=0x7f9f80014000).stop 2022-08-01T23:03:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 msgr2=0x7f9f8c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c156850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f9f84001250 tx=0x7f9f84007b80).stop 2022-08-01T23:03:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 shutdown_connections 2022-08-01T23:03:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f6c053b00 0x7f9f6c055fc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f8c0b2710 0x7f9f8c156850 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f8c0b01a0 0x7f9f8c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 --2- 172.21.15.32:0/345427722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f8c0afd60 0x7f9f8c143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 >> 172.21.15.32:0/345427722 conn(0x7f9f8c060310 msgr2=0x7f9f8c148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 shutdown_connections 2022-08-01T23:03:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:11.994+0000 7f9f91cf0700 1 -- 172.21.15.32:0/345427722 wait complete. 2022-08-01T23:03:12.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:12.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.110+0000 7f0a2a2b0700 1 Processor -- start 2022-08-01T23:03:12.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.110+0000 7f0a2a2b0700 1 -- start start 2022-08-01T23:03:12.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a240b7500 0x7f0a240b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 0x7f0a2413e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a240b2510 con 0x7f0a2413e8a0 2022-08-01T23:03:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a241443e0 con 0x7f0a240b31f0 2022-08-01T23:03:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a240b3700 con 0x7f0a240b7500 2022-08-01T23:03:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a28aad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 0x7f0a2413e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a28aad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 0x7f0a2413e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46426/0 (socket says 172.21.15.32:46426) 2022-08-01T23:03:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a28aad700 1 -- 172.21.15.32:0/724756021 learned_addr learned my addr 172.21.15.32:0/724756021 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a29aaf700 1 --1- 172.21.15.32:0/724756021 >> v1:172.21.15.32:6789/0 conn(0x7f0a2413e8a0 0x7f0a24064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33970/0 (socket says 172.21.15.32:33970) 2022-08-01T23:03:12.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a292ae700 1 --2- 172.21.15.32:0/724756021 >> v2:172.21.15.55:3300/0 conn(0x7f0a240b7500 0x7f0a240b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a292ae700 1 -- 172.21.15.32:0/724756021 >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 msgr2=0x7f0a2413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:12.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a292ae700 1 --2- 172.21.15.32:0/724756021 >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 0x7f0a2413e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a292ae700 1 -- 172.21.15.32:0/724756021 >> v1:172.21.15.32:6789/0 conn(0x7f0a2413e8a0 legacy=0x7f0a24064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a292ae700 1 -- 172.21.15.32:0/724756021 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240b2740 con 0x7f0a240b7500 2022-08-01T23:03:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a28aad700 1 --2- 172.21.15.32:0/724756021 >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 0x7f0a2413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a292ae700 1 --2- 172.21.15.32:0/724756021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b7500 0x7f0a240b78e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f0a20002f70 tx=0x7f0a20006d90).ready entity=mon.1 client_cookie=9b0b634f251ece10 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a13fff700 1 -- 172.21.15.32:0/724756021 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a2000d4a0 con 0x7f0a240b7500 2022-08-01T23:03:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a13fff700 1 -- 172.21.15.32:0/724756021 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20000e90 con 0x7f0a240b7500 2022-08-01T23:03:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a13fff700 1 -- 172.21.15.32:0/724756021 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a2000d7f0 con 0x7f0a240b7500 2022-08-01T23:03:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/724756021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b7500 msgr2=0x7f0a240b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/724756021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b7500 0x7f0a240b78e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f0a20002f70 tx=0x7f0a20006d90).stop 2022-08-01T23:03:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/724756021 shutdown_connections 2022-08-01T23:03:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/724756021 >> v2:172.21.15.32:3300/0 conn(0x7f0a240b31f0 0x7f0a2413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/724756021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b7500 0x7f0a240b78e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/724756021 >> 172.21.15.32:0/724756021 conn(0x7f0a2405f840 msgr2=0x7f0a2405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/724756021 shutdown_connections 2022-08-01T23:03:12.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.114+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/724756021 wait complete. 2022-08-01T23:03:12.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 Processor -- start 2022-08-01T23:03:12.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 -- start start 2022-08-01T23:03:12.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b31f0 0x7f0a240aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 0x7f0a2409f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a2413e8a0 0x7f0a24153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2409eca0 con 0x7f0a240b7500 2022-08-01T23:03:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240ad330 con 0x7f0a240b31f0 2022-08-01T23:03:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a2a2b0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24150cd0 con 0x7f0a2413e8a0 2022-08-01T23:03:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a28aad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 0x7f0a2409f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a28aad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 0x7f0a2409f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46432/0 (socket says 172.21.15.32:46432) 2022-08-01T23:03:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a28aad700 1 -- 172.21.15.32:0/10501088 learned_addr learned my addr 172.21.15.32:0/10501088 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a292ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b31f0 0x7f0a240aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.118+0000 7f0a29aaf700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a2413e8a0 0x7f0a24153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a28aad700 1 -- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a2413e8a0 msgr2=0x7f0a24153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a28aad700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a2413e8a0 0x7f0a24153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a28aad700 1 -- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b31f0 msgr2=0x7f0a240aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a28aad700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b31f0 0x7f0a240aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a28aad700 1 -- 172.21.15.32:0/10501088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1400b1e0 con 0x7f0a240b7500 2022-08-01T23:03:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a292ae700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b31f0 0x7f0a240aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a28aad700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 0x7f0a2409f090 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f0a1400c000 tx=0x7f0a14006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a14012600 con 0x7f0a240b7500 2022-08-01T23:03:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14012760 con 0x7f0a240b7500 2022-08-01T23:03:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2000b120 con 0x7f0a240b7500 2022-08-01T23:03:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a14012ac0 con 0x7f0a240b7500 2022-08-01T23:03:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2414a8a0 con 0x7f0a240b7500 2022-08-01T23:03:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240574e0 con 0x7f0a240b7500 2022-08-01T23:03:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0a14022070 con 0x7f0a240b7500 2022-08-01T23:03:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.122+0000 7f0a127fc700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0a04053980 0x7f0a04055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.126+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0a1406c4c0 con 0x7f0a240b7500 2022-08-01T23:03:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.126+0000 7f0a292ae700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0a04053980 0x7f0a04055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.126+0000 7f0a292ae700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0a04053980 0x7f0a04055e40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0a2001a000 tx=0x7f0a20014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.130+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a14069e90 con 0x7f0a240b7500 2022-08-01T23:03:12.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.254+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0a24060330 con 0x7f0a240b7500 2022-08-01T23:03:12.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a127fc700 1 -- 172.21.15.32:0/10501088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a14043280 con 0x7f0a240b7500 2022-08-01T23:03:12.261 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:12.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0a04053980 msgr2=0x7f0a04055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0a04053980 0x7f0a04055e40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0a2001a000 tx=0x7f0a20014000).stop 2022-08-01T23:03:12.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 msgr2=0x7f0a2409f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 0x7f0a2409f090 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f0a1400c000 tx=0x7f0a14006eb0).stop 2022-08-01T23:03:12.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 shutdown_connections 2022-08-01T23:03:12.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0a04053980 0x7f0a04055e40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a2413e8a0 0x7f0a24153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a240b7500 0x7f0a2409f090 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 --2- 172.21.15.32:0/10501088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240b31f0 0x7f0a240aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 >> 172.21.15.32:0/10501088 conn(0x7f0a2405f840 msgr2=0x7f0a2413dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 shutdown_connections 2022-08-01T23:03:12.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.258+0000 7f0a2a2b0700 1 -- 172.21.15.32:0/10501088 wait complete. 2022-08-01T23:03:12.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:03:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.374+0000 7f603ffff700 1 Processor -- start 2022-08-01T23:03:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.374+0000 7f603ffff700 1 -- start start 2022-08-01T23:03:12.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040064b10 0x7f6040144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60400b36e0 0x7f6040141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6040064ef0 con 0x7f6040141860 2022-08-01T23:03:12.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604013ebf0 con 0x7f60400b36e0 2022-08-01T23:03:12.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6040066690 con 0x7f6040064b10 2022-08-01T23:03:12.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60400b36e0 0x7f6040141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60400b36e0 0x7f6040141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46438/0 (socket says 172.21.15.32:46438) 2022-08-01T23:03:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 -- 172.21.15.32:0/2655376892 learned_addr learned my addr 172.21.15.32:0/2655376892 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603effd700 1 --2- 172.21.15.32:0/2655376892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040064b10 0x7f6040144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 -- 172.21.15.32:0/2655376892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040064b10 msgr2=0x7f6040144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 --2- 172.21.15.32:0/2655376892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040064b10 0x7f6040144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 -- 172.21.15.32:0/2655376892 >> v1:172.21.15.55:6789/0 conn(0x7f6040141860 legacy=0x7f6040066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 -- 172.21.15.32:0/2655376892 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400b4850 con 0x7f60400b36e0 2022-08-01T23:03:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603e7fc700 1 --2- 172.21.15.32:0/2655376892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f6040141370 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f6028002f70 tx=0x7f6028006eb0).ready entity=mon.0 client_cookie=302a3f250a86e466 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603dffb700 1 -- 172.21.15.32:0/2655376892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f602800d5c0 con 0x7f60400b36e0 2022-08-01T23:03:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603dffb700 1 -- 172.21.15.32:0/2655376892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6028000e90 con 0x7f60400b36e0 2022-08-01T23:03:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603dffb700 1 -- 172.21.15.32:0/2655376892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f602800d950 con 0x7f60400b36e0 2022-08-01T23:03:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- 172.21.15.32:0/2655376892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 msgr2=0x7f6040141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 --2- 172.21.15.32:0/2655376892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f6040141370 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f6028002f70 tx=0x7f6028006eb0).stop 2022-08-01T23:03:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- 172.21.15.32:0/2655376892 shutdown_connections 2022-08-01T23:03:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 --2- 172.21.15.32:0/2655376892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f6040141370 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 --2- 172.21.15.32:0/2655376892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040064b10 0x7f6040144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- 172.21.15.32:0/2655376892 >> 172.21.15.32:0/2655376892 conn(0x7f604005f840 msgr2=0x7f604005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- 172.21.15.32:0/2655376892 shutdown_connections 2022-08-01T23:03:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.378+0000 7f603ffff700 1 -- 172.21.15.32:0/2655376892 wait complete. 2022-08-01T23:03:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 Processor -- start 2022-08-01T23:03:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 -- start start 2022-08-01T23:03:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6040064b10 0x7f60401440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f60400b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040141860 0x7f60400b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b0f60 con 0x7f60400b36e0 2022-08-01T23:03:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b0d00 con 0x7f6040064b10 2022-08-01T23:03:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b1280 con 0x7f6040141860 2022-08-01T23:03:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f60400b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f60400b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46446/0 (socket says 172.21.15.32:46446) 2022-08-01T23:03:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603e7fc700 1 -- 172.21.15.32:0/3211076511 learned_addr learned my addr 172.21.15.32:0/3211076511 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603effd700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6040064b10 0x7f60401440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.382+0000 7f603f7fe700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040141860 0x7f60400b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603e7fc700 1 -- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040141860 msgr2=0x7f60400b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603e7fc700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040141860 0x7f60400b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603e7fc700 1 -- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6040064b10 msgr2=0x7f60401440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603e7fc700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6040064b10 0x7f60401440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603e7fc700 1 -- 172.21.15.32:0/3211076511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602800b1e0 con 0x7f60400b36e0 2022-08-01T23:03:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603f7fe700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040141860 0x7f60400b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603e7fc700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f60400b5560 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f6028002f70 tx=0x7f6028015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6028011860 con 0x7f60400b36e0 2022-08-01T23:03:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60280119c0 con 0x7f60400b36e0 2022-08-01T23:03:12.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6028011d20 con 0x7f60400b36e0 2022-08-01T23:03:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60400b08d0 con 0x7f60400b36e0 2022-08-01T23:03:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60400b66e0 con 0x7f60400b36e0 2022-08-01T23:03:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60400b4af0 con 0x7f60400b36e0 2022-08-01T23:03:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6028025030 con 0x7f60400b36e0 2022-08-01T23:03:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f6037fff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6024053a70 0x7f6024055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f602806d410 con 0x7f60400b36e0 2022-08-01T23:03:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.386+0000 7f603effd700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6024053a70 0x7f6024055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.390+0000 7f603effd700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6024053a70 0x7f6024055f30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f603000c000 tx=0x7f6030008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.390+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6028044130 con 0x7f60400b36e0 2022-08-01T23:03:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.514+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60400b1450 con 0x7f60400b36e0 2022-08-01T23:03:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f6037fff700 1 -- 172.21.15.32:0/3211076511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6028023020 con 0x7f60400b36e0 2022-08-01T23:03:12.521 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6024053a70 msgr2=0x7f6024055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6024053a70 0x7f6024055f30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f603000c000 tx=0x7f6030008040).stop 2022-08-01T23:03:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 msgr2=0x7f60400b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f60400b5560 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f6028002f70 tx=0x7f6028015000).stop 2022-08-01T23:03:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 shutdown_connections 2022-08-01T23:03:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6024053a70 0x7f6024055f30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6040141860 0x7f60400b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60400b36e0 0x7f60400b5560 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 --2- 172.21.15.32:0/3211076511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6040064b10 0x7f60401440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 >> 172.21.15.32:0/3211076511 conn(0x7f604005f840 msgr2=0x7f6040136560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 shutdown_connections 2022-08-01T23:03:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.518+0000 7f603ffff700 1 -- 172.21.15.32:0/3211076511 wait complete. 2022-08-01T23:03:12.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:03:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.630+0000 7fcb67fff700 1 Processor -- start 2022-08-01T23:03:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- start start 2022-08-01T23:03:12.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb6813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb680b0e60 0x7fcb6813d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6813dc80 con 0x7fcb68064500 2022-08-01T23:03:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb68066af0 con 0x7fcb680b0e60 2022-08-01T23:03:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68066d90 con 0x7fcb680b6510 2022-08-01T23:03:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb680b0e60 0x7fcb6813d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb680b0e60 0x7fcb6813d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46452/0 (socket says 172.21.15.32:46452) 2022-08-01T23:03:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 -- 172.21.15.32:0/3798804604 learned_addr learned my addr 172.21.15.32:0/3798804604 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb677fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb68064500 0x7fcb680609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33996/0 (socket says 172.21.15.32:33996) 2022-08-01T23:03:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb66ffd700 1 --2- 172.21.15.32:0/3798804604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb6813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 -- 172.21.15.32:0/3798804604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 msgr2=0x7fcb6813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 --2- 172.21.15.32:0/3798804604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb6813d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 -- 172.21.15.32:0/3798804604 >> v1:172.21.15.32:6789/0 conn(0x7fcb68064500 legacy=0x7fcb680609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 -- 172.21.15.32:0/3798804604 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68067030 con 0x7fcb680b0e60 2022-08-01T23:03:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb667fc700 1 --2- 172.21.15.32:0/3798804604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb6813d790 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fcb600032f0 tx=0x7fcb60009290).ready entity=mon.0 client_cookie=a77bf3115738aee6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3798804604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb6000b8a0 con 0x7fcb680b0e60 2022-08-01T23:03:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3798804604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6000ba00 con 0x7fcb680b0e60 2022-08-01T23:03:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3798804604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb6000bb70 con 0x7fcb680b0e60 2022-08-01T23:03:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- 172.21.15.32:0/3798804604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 msgr2=0x7fcb6813d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 --2- 172.21.15.32:0/3798804604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb6813d790 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fcb600032f0 tx=0x7fcb60009290).stop 2022-08-01T23:03:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- 172.21.15.32:0/3798804604 shutdown_connections 2022-08-01T23:03:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 --2- 172.21.15.32:0/3798804604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb6813d790 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 --2- 172.21.15.32:0/3798804604 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb6813d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- 172.21.15.32:0/3798804604 >> 172.21.15.32:0/3798804604 conn(0x7fcb68063800 msgr2=0x7fcb68063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- 172.21.15.32:0/3798804604 shutdown_connections 2022-08-01T23:03:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.634+0000 7fcb67fff700 1 -- 172.21.15.32:0/3798804604 wait complete. 2022-08-01T23:03:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 Processor -- start 2022-08-01T23:03:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 -- start start 2022-08-01T23:03:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb68064500 0x7fcb68141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb68156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb68156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680b0730 con 0x7fcb680b0e60 2022-08-01T23:03:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68155e10 con 0x7fcb68064500 2022-08-01T23:03:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680b0c00 con 0x7fcb680b6510 2022-08-01T23:03:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb68156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb68156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46460/0 (socket says 172.21.15.32:46460) 2022-08-01T23:03:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 -- 172.21.15.32:0/965802994 learned_addr learned my addr 172.21.15.32:0/965802994 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb677fe700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb68156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb66ffd700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb68064500 0x7fcb68141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 -- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 msgr2=0x7fcb68156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb68156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 -- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb68064500 msgr2=0x7fcb68141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb68064500 0x7fcb68141000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 -- 172.21.15.32:0/965802994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009530 con 0x7fcb680b0e60 2022-08-01T23:03:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb667fc700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb68156270 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fcb600032f0 tx=0x7fcb60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb60021480 con 0x7fcb680b0e60 2022-08-01T23:03:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb600215e0 con 0x7fcb680b0e60 2022-08-01T23:03:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb680b0960 con 0x7fcb680b0e60 2022-08-01T23:03:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb60021940 con 0x7fcb680b0e60 2022-08-01T23:03:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.638+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6809ef50 con 0x7fcb680b0e60 2022-08-01T23:03:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.642+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb68155f80 con 0x7fcb680b0e60 2022-08-01T23:03:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.642+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb6002a070 con 0x7fcb680b0e60 2022-08-01T23:03:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.642+0000 7fcb5ffff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb40053950 0x7fcb40055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.642+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb6006d420 con 0x7fcb680b0e60 2022-08-01T23:03:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.642+0000 7fcb66ffd700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb40053950 0x7fcb40055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.642+0000 7fcb66ffd700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb40053950 0x7fcb40055e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcb5800c000 tx=0x7fcb58008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.650+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb60044050 con 0x7fcb680b0e60 2022-08-01T23:03:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.774+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb680574e0 con 0x7fcb680b0e60 2022-08-01T23:03:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.774+0000 7fcb5ffff700 1 -- 172.21.15.32:0/965802994 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcb60015070 con 0x7fcb680b0e60 2022-08-01T23:03:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:12.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb40053950 msgr2=0x7fcb40055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb40053950 0x7fcb40055e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcb5800c000 tx=0x7fcb58008040).stop 2022-08-01T23:03:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 msgr2=0x7fcb68156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb68156270 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fcb600032f0 tx=0x7fcb60016000).stop 2022-08-01T23:03:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 shutdown_connections 2022-08-01T23:03:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb40053950 0x7fcb40055e10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb680b6510 0x7fcb68156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb680b0e60 0x7fcb68156270 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 --2- 172.21.15.32:0/965802994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb68064500 0x7fcb68141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 >> 172.21.15.32:0/965802994 conn(0x7fcb68063800 msgr2=0x7fcb6813d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 shutdown_connections 2022-08-01T23:03:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.778+0000 7fcb67fff700 1 -- 172.21.15.32:0/965802994 wait complete. 2022-08-01T23:03:12.799 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:03:12.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:12.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.898+0000 7fae22c6e700 1 Processor -- start 2022-08-01T23:03:12.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.898+0000 7fae22c6e700 1 -- start start 2022-08-01T23:03:12.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae1c0b6710 0x7fae1c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae1c0b4550 0x7fae1c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c144240 con 0x7fae1c0b6710 2022-08-01T23:03:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c0b6fe0 con 0x7fae1c0b4550 2022-08-01T23:03:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c143d00 con 0x7fae1c061ec0 2022-08-01T23:03:12.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae1c0b6710 0x7fae1c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae1c0b6710 0x7fae1c0b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46466/0 (socket says 172.21.15.32:46466) 2022-08-01T23:03:12.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 -- 172.21.15.32:0/3710354646 learned_addr learned my addr 172.21.15.32:0/3710354646 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae1bfff700 1 --2- 172.21.15.32:0/3710354646 >> v2:172.21.15.55:3300/0 conn(0x7fae1c0b4550 0x7fae1c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae2120b700 1 --2- 172.21.15.32:0/3710354646 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 -- 172.21.15.32:0/3710354646 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 msgr2=0x7fae1c062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 --2- 172.21.15.32:0/3710354646 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 -- 172.21.15.32:0/3710354646 >> v2:172.21.15.55:3300/0 conn(0x7fae1c0b4550 msgr2=0x7fae1c0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 --2- 172.21.15.32:0/3710354646 >> v2:172.21.15.55:3300/0 conn(0x7fae1c0b4550 0x7fae1c0b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 -- 172.21.15.32:0/3710354646 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c143fa0 con 0x7fae1c0b6710 2022-08-01T23:03:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae20a0a700 1 --2- 172.21.15.32:0/3710354646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b6710 0x7fae1c0b5d80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fae1c1432b0 tx=0x7fae04006eb0).ready entity=mon.0 client_cookie=d7fc06f4666bc95 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae1b7fe700 1 -- 172.21.15.32:0/3710354646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0400e610 con 0x7fae1c0b6710 2022-08-01T23:03:12.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae1b7fe700 1 -- 172.21.15.32:0/3710354646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04000e90 con 0x7fae1c0b6710 2022-08-01T23:03:12.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae1b7fe700 1 -- 172.21.15.32:0/3710354646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0400e9a0 con 0x7fae1c0b6710 2022-08-01T23:03:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- 172.21.15.32:0/3710354646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b6710 msgr2=0x7fae1c0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- 172.21.15.32:0/3710354646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b6710 0x7fae1c0b5d80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fae1c1432b0 tx=0x7fae04006eb0).stop 2022-08-01T23:03:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- 172.21.15.32:0/3710354646 shutdown_connections 2022-08-01T23:03:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- 172.21.15.32:0/3710354646 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- 172.21.15.32:0/3710354646 >> v2:172.21.15.55:3300/0 conn(0x7fae1c0b4550 0x7fae1c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 --2- 172.21.15.32:0/3710354646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b6710 0x7fae1c0b5d80 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- 172.21.15.32:0/3710354646 >> 172.21.15.32:0/3710354646 conn(0x7fae1c05c620 msgr2=0x7fae1c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- 172.21.15.32:0/3710354646 shutdown_connections 2022-08-01T23:03:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.902+0000 7fae22c6e700 1 -- 172.21.15.32:0/3710354646 wait complete. 2022-08-01T23:03:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 Processor -- start 2022-08-01T23:03:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 -- start start 2022-08-01T23:03:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c129c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 0x7fae1c155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c0b6710 0x7fae1c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c12b880 con 0x7fae1c0b4550 2022-08-01T23:03:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0ba7e0 con 0x7fae1c0b6710 2022-08-01T23:03:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c12a3d0 con 0x7fae1c061ec0 2022-08-01T23:03:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 0x7fae1c155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae20a0a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c129c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 0x7fae1c155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46474/0 (socket says 172.21.15.32:46474) 2022-08-01T23:03:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 -- 172.21.15.32:0/1511454559 learned_addr learned my addr 172.21.15.32:0/1511454559 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae2120b700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c0b6710 0x7fae1c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 -- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 msgr2=0x7fae1c129c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c129c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 -- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c0b6710 msgr2=0x7fae1c155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c0b6710 0x7fae1c155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 -- 172.21.15.32:0/1511454559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0400b1e0 con 0x7fae1c0b4550 2022-08-01T23:03:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae1bfff700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 0x7fae1c155a30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fae0c00c000 tx=0x7fae0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0c011750 con 0x7fae1c0b4550 2022-08-01T23:03:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c0118b0 con 0x7fae1c0b4550 2022-08-01T23:03:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c1532e0 con 0x7fae1c0b4550 2022-08-01T23:03:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0c011c10 con 0x7fae1c0b4550 2022-08-01T23:03:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.906+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c12c560 con 0x7fae1c0b4550 2022-08-01T23:03:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.910+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c061780 con 0x7fae1c0b4550 2022-08-01T23:03:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.910+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fae0c011d70 con 0x7fae1c0b4550 2022-08-01T23:03:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.910+0000 7fae19ffb700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae00053b30 0x7fae00055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.910+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fae0c06c430 con 0x7fae1c0b4550 2022-08-01T23:03:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.910+0000 7fae20a0a700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae00053b30 0x7fae00055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.910+0000 7fae20a0a700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae00053b30 0x7fae00055ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fae0401c000 tx=0x7fae04015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:12.914+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fae0c043080 con 0x7fae1c0b4550 2022-08-01T23:03:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae1c0574e0 con 0x7fae1c0b4550 2022-08-01T23:03:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae19ffb700 1 -- 172.21.15.32:0/1511454559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fae0c0466a0 con 0x7fae1c0b4550 2022-08-01T23:03:13.082 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:13.083 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:09.472203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6981":{"gid":6981,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2562641804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2562641804},{"type":"v1","addr":"172.21.15.32:6835","nonce":2562641804}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae00053b30 msgr2=0x7fae00055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:13.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae00053b30 0x7fae00055ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fae0401c000 tx=0x7fae04015000).stop 2022-08-01T23:03:13.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 msgr2=0x7fae1c155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:13.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 0x7fae1c155a30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fae0c00c000 tx=0x7fae0c006df0).stop 2022-08-01T23:03:13.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 shutdown_connections 2022-08-01T23:03:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fae00053b30 0x7fae00055ff0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c0b6710 0x7fae1c155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae1c0b4550 0x7fae1c155a30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 --2- 172.21.15.32:0/1511454559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fae1c061ec0 0x7fae1c129c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.078+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 >> 172.21.15.32:0/1511454559 conn(0x7fae1c05c620 msgr2=0x7fae1c142570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.082+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 shutdown_connections 2022-08-01T23:03:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:13.082+0000 7fae22c6e700 1 -- 172.21.15.32:0/1511454559 wait complete. 2022-08-01T23:03:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2022-08-01T23:03:13.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:08.469158+0000', 'modified': '2022-08-01T23:03:09.472203+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6981}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6981': {'gid': 6981, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2562641804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2562641804}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2562641804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:13.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:13.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:03:13.516 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6981,"want_state":"up:active","state":"up:active","rank_uptime":5.0321727899999997,"mdsmap_epoch":45,"osdmap_epoch":97,"osdmap_epoch_barrier":96,"uptime":9.3656562220000001} 2022-08-01T23:03:13.516 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6981, "mdsmap_epoch": 45, "osdmap_epoch": 97, "osdmap_epoch_barrier": 96, "rank_uptime": 5.03217279, "state": "up:active", "uptime": 9.365656222, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:13.517 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:03:13.517 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:03:13.518 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:03:13.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:13.519 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:13.528 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:03:13.542 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:13.542 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:13.598 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:13.599 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:13.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:13.600 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:03:13.607 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:03:13.607 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:13.610 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:03:13.626 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:13.627 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:13.682 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:13.683 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:13.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:13.685 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:03:13.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:03:13.676+0000 7f4673d9e080 -1 init, newargv = 0x5646b6b1b7f0 newargc=9 2022-08-01T23:03:13.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23224]: starting ceph client 2022-08-01T23:03:13.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23224]: starting fuse 2022-08-01T23:03:14.792 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:03:14.810 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:14.811 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:14.866 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:14.867 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:14.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:14.869 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:03:14.874 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:03:14.874 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:03:14.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:14.875 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:03:14.876 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:03:14.876 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:03:14.877 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:03:14.877 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:03:14.878 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:14.878 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:03:14.878 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:03:14.879 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:03:14.879 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:14.880 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:03:14.880 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:03:14.880 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:03:14.881 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:03:14.881 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:14.882 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:03:14.882 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:03:14.882 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:03:14.883 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:03:14.883 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:03:14.884 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:14.884 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:03:14.884 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:03:14.959 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23224.asok 2022-08-01T23:03:14.964 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23224.asok 2022-08-01T23:03:14.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:14.965 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23224.asok status 2022-08-01T23:03:15.052 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:03:15.052 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:03:15.052 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:03:15.053 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:03:15.053 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:03:15.053 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:03:15.053 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:03:15.053 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23224", 2022-08-01T23:03:15.054 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:03:15.054 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:15.054 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:03:15.054 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:03:15.055 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 7143, 2022-08-01T23:03:15.055 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:03:15.055 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:03:15.055 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:03:15.056 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 7143 2022-08-01T23:03:15.056 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:15.056 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:03:15.056 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:03:15.057 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:03:15.057 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2437853910 2022-08-01T23:03:15.057 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:03:15.057 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:15.057 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:03:15.058 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:03:15.058 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:03:15.058 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2437853910 2022-08-01T23:03:15.058 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:15.059 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.7143 172.21.15.130:0/2437853910", 2022-08-01T23:03:15.059 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/2437853910", 2022-08-01T23:03:15.059 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:03:15.059 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 45, 2022-08-01T23:03:15.059 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 97, 2022-08-01T23:03:15.060 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:03:15.060 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:03:15.060 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:03:15.060 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:03:15.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:15.062 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:15.063 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:15.070 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:03:15.071 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:15.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:15.072 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:15.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 Processor -- start 2022-08-01T23:03:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- start start 2022-08-01T23:03:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86a00b1f40 0x7f86a00b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f86a0148530 0x7f86a00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a00b04e0 con 0x7f86a00a7250 2022-08-01T23:03:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00a9250 con 0x7f86a00b1f40 2022-08-01T23:03:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00b0780 con 0x7f86a0148530 2022-08-01T23:03:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86a00b1f40 0x7f86a00b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86a00b1f40 0x7f86a00b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46482/0 (socket says 172.21.15.32:46482) 2022-08-01T23:03:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 -- 172.21.15.32:0/586952797 learned_addr learned my addr 172.21.15.32:0/586952797 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f8696ffd700 1 --2- 172.21.15.32:0/586952797 >> v2:172.21.15.55:3300/0 conn(0x7f86a0148530 0x7f86a00b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 -- 172.21.15.32:0/586952797 >> v2:172.21.15.55:3300/0 conn(0x7f86a0148530 msgr2=0x7f86a00b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 --2- 172.21.15.32:0/586952797 >> v2:172.21.15.55:3300/0 conn(0x7f86a0148530 0x7f86a00b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 -- 172.21.15.32:0/586952797 >> v1:172.21.15.55:6789/0 conn(0x7f86a00a7250 legacy=0x7f86a01448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 -- 172.21.15.32:0/586952797 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00b0a20 con 0x7f86a00b1f40 2022-08-01T23:03:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869f7fe700 1 --2- 172.21.15.32:0/586952797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a00b1f40 0x7f86a00b1160 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f86a01478c0 tx=0x7f8690006eb0).ready entity=mon.0 client_cookie=b0640986adcd5680 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869effd700 1 -- 172.21.15.32:0/586952797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f869000d610 con 0x7f86a00b1f40 2022-08-01T23:03:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869effd700 1 -- 172.21.15.32:0/586952797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690000e90 con 0x7f86a00b1f40 2022-08-01T23:03:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f869effd700 1 -- 172.21.15.32:0/586952797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f869000d9a0 con 0x7f86a00b1f40 2022-08-01T23:03:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- 172.21.15.32:0/586952797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a00b1f40 msgr2=0x7f86a00b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 --2- 172.21.15.32:0/586952797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a00b1f40 0x7f86a00b1160 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f86a01478c0 tx=0x7f8690006eb0).stop 2022-08-01T23:03:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- 172.21.15.32:0/586952797 shutdown_connections 2022-08-01T23:03:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 --2- 172.21.15.32:0/586952797 >> v2:172.21.15.55:3300/0 conn(0x7f86a0148530 0x7f86a00b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 --2- 172.21.15.32:0/586952797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a00b1f40 0x7f86a00b1160 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- 172.21.15.32:0/586952797 >> 172.21.15.32:0/586952797 conn(0x7f86a005f840 msgr2=0x7f86a005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- 172.21.15.32:0/586952797 shutdown_connections 2022-08-01T23:03:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.238+0000 7f86a5c84700 1 -- 172.21.15.32:0/586952797 wait complete. 2022-08-01T23:03:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 Processor -- start 2022-08-01T23:03:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 -- start start 2022-08-01T23:03:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86a00a7250 0x7f86a00ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86a00b1f40 0x7f86a0157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 0x7f86a0157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0149b80 con 0x7f86a0148530 2022-08-01T23:03:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0157b00 con 0x7f86a00b1f40 2022-08-01T23:03:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00abe70 con 0x7f86a00a7250 2022-08-01T23:03:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 0x7f86a0157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86a00a7250 0x7f86a00ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86a00b1f40 0x7f86a0157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 0x7f86a0157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46490/0 (socket says 172.21.15.32:46490) 2022-08-01T23:03:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 -- 172.21.15.32:0/4249445772 learned_addr learned my addr 172.21.15.32:0/4249445772 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 -- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86a00a7250 msgr2=0x7f86a00ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86a00a7250 0x7f86a00ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 -- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86a00b1f40 msgr2=0x7f86a0157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86a00b1f40 0x7f86a0157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 -- 172.21.15.32:0/4249445772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869000b1e0 con 0x7f86a0148530 2022-08-01T23:03:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869ffff700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 0x7f86a0157610 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f86a00acff0 tx=0x7f8698007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f869800f470 con 0x7f86a0148530 2022-08-01T23:03:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869800f5d0 con 0x7f86a0148530 2022-08-01T23:03:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a00ac110 con 0x7f86a0148530 2022-08-01T23:03:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f869800f930 con 0x7f86a0148530 2022-08-01T23:03:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.242+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a0156c80 con 0x7f86a0148530 2022-08-01T23:03:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.246+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8678003410 con 0x7f86a0148530 2022-08-01T23:03:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.246+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8698019030 con 0x7f86a0148530 2022-08-01T23:03:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.246+0000 7f869d7fa700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8684053950 0x7f8684055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.246+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f869806b780 con 0x7f86a0148530 2022-08-01T23:03:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.246+0000 7f869f7fe700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8684053950 0x7f8684055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.246+0000 7f869f7fe700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8684053950 0x7f8684055e10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f869001c000 tx=0x7f8690015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.254+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86980423b0 con 0x7f86a0148530 2022-08-01T23:03:15.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.378+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86780036c0 con 0x7f86a0148530 2022-08-01T23:03:15.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.382+0000 7f869d7fa700 1 -- 172.21.15.32:0/4249445772 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f86980459d0 con 0x7f86a0148530 2022-08-01T23:03:15.385 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:15.385 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:09.472841+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:06.881558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:07.840459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:15.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8684053950 msgr2=0x7f8684055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8684053950 0x7f8684055e10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f869001c000 tx=0x7f8690015000).stop 2022-08-01T23:03:15.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 msgr2=0x7f86a0157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 0x7f86a0157610 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f86a00acff0 tx=0x7f8698007f40).stop 2022-08-01T23:03:15.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 shutdown_connections 2022-08-01T23:03:15.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8684053950 0x7f8684055e10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a0148530 0x7f86a0157610 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86a00b1f40 0x7f86a0157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 --2- 172.21.15.32:0/4249445772 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f86a00a7250 0x7f86a00ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 >> 172.21.15.32:0/4249445772 conn(0x7f86a005f840 msgr2=0x7f86a00b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:15.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 shutdown_connections 2022-08-01T23:03:15.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.386+0000 7f86a5c84700 1 -- 172.21.15.32:0/4249445772 wait complete. 2022-08-01T23:03:15.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:03:06.881558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:03:07.840459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:03:15.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:15.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.506+0000 7fb00a51d700 1 Processor -- start 2022-08-01T23:03:15.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.506+0000 7fb00a51d700 1 -- start start 2022-08-01T23:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb0040b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040ae7e0 con 0x7fb004147450 2022-08-01T23:03:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040b1960 con 0x7fb0040b1580 2022-08-01T23:03:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040b2470 con 0x7fb0040b1ef0 2022-08-01T23:03:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb008aba700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb004147450 0x7fb004143740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34040/0 (socket says 172.21.15.32:34040) 2022-08-01T23:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb008aba700 1 -- 172.21.15.32:0/3352775850 learned_addr learned my addr 172.21.15.32:0/3352775850 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7faffb7fe700 1 --2- 172.21.15.32:0/3352775850 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb0040b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290162562 0 0) 0x7fb0040ae7e0 con 0x7fb004147450 2022-08-01T23:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe4003260 con 0x7fb004147450 2022-08-01T23:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859027821 0 0) 0x7fb0040b1960 con 0x7fb0040b1580 2022-08-01T23:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0040ae7e0 con 0x7fb0040b1580 2022-08-01T23:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7faffb7fe700 1 -- 172.21.15.32:0/3352775850 >> v1:172.21.15.55:6789/0 conn(0x7fb0040b1580 legacy=0x7fb004146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7faffb7fe700 1 -- 172.21.15.32:0/3352775850 >> v1:172.21.15.32:6789/0 conn(0x7fb004147450 legacy=0x7fb004143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7faffb7fe700 1 -- 172.21.15.32:0/3352775850 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040b2710 con 0x7fb0040b1ef0 2022-08-01T23:03:15.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7faffb7fe700 1 --2- 172.21.15.32:0/3352775850 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb0040b3210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fafec002f70 tx=0x7fafec006df0).ready entity=mon.2 client_cookie=176433f2098c3399 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafec00d5c0 con 0x7fb0040b1ef0 2022-08-01T23:03:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec000e90 con 0x7fb0040b1ef0 2022-08-01T23:03:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb0037fe700 1 -- 172.21.15.32:0/3352775850 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafec00d910 con 0x7fb0040b1ef0 2022-08-01T23:03:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- 172.21.15.32:0/3352775850 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 msgr2=0x7fb0040b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 --2- 172.21.15.32:0/3352775850 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb0040b3210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fafec002f70 tx=0x7fafec006df0).stop 2022-08-01T23:03:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- 172.21.15.32:0/3352775850 shutdown_connections 2022-08-01T23:03:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 --2- 172.21.15.32:0/3352775850 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb0040b3210 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- 172.21.15.32:0/3352775850 >> 172.21.15.32:0/3352775850 conn(0x7fb004060950 msgr2=0x7fb004060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- 172.21.15.32:0/3352775850 shutdown_connections 2022-08-01T23:03:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.510+0000 7fb00a51d700 1 -- 172.21.15.32:0/3352775850 wait complete. 2022-08-01T23:03:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.514+0000 7fb00a51d700 1 Processor -- start 2022-08-01T23:03:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.514+0000 7fb00a51d700 1 -- start start 2022-08-01T23:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 0x7fb0040b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb004152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb004147450 0x7fb0040675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004065d30 con 0x7fb0040b1580 2022-08-01T23:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004156730 con 0x7fb004147450 2022-08-01T23:03:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041569d0 con 0x7fb0040b1ef0 2022-08-01T23:03:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 0x7fb0040b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 0x7fb0040b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46500/0 (socket says 172.21.15.32:46500) 2022-08-01T23:03:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 -- 172.21.15.32:0/113594506 learned_addr learned my addr 172.21.15.32:0/113594506 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7faffb7fe700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb004152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb008aba700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb004147450 0x7fb0040675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 -- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 msgr2=0x7fb004152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb004152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 -- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb004147450 msgr2=0x7fb0040675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb004147450 0x7fb0040675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 -- 172.21.15.32:0/113594506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec00b180 con 0x7fb0040b1580 2022-08-01T23:03:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb003fff700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 0x7fb0040b05c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7faff4001110 tx=0x7faff4007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff4012400 con 0x7fb0040b1580 2022-08-01T23:03:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff4012560 con 0x7fb0040b1580 2022-08-01T23:03:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff40128c0 con 0x7fb0040b1580 2022-08-01T23:03:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0040ad600 con 0x7fb0040b1580 2022-08-01T23:03:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb004064190 con 0x7fb0040b1580 2022-08-01T23:03:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faff4019030 con 0x7fb0040b1580 2022-08-01T23:03:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.518+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb004065f30 con 0x7fb0040b1580 2022-08-01T23:03:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.522+0000 7fb001ffb700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafe8053a70 0x7fafe8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.522+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faff406b540 con 0x7fb0040b1580 2022-08-01T23:03:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.522+0000 7faffb7fe700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafe8053a70 0x7fafe8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.522+0000 7faffb7fe700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafe8053a70 0x7fafe8055f30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fafec018000 tx=0x7fafec014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.526+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faff40421f0 con 0x7fb0040b1580 2022-08-01T23:03:15.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.654+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb0040673b0 con 0x7fb0040b1580 2022-08-01T23:03:15.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb001ffb700 1 -- 172.21.15.32:0/113594506 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v97) v1 ==== 110+0+10 (secure 0 0 0) 0x7faff4045810 con 0x7fb0040b1580 2022-08-01T23:03:15.661 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:15.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafe8053a70 msgr2=0x7fafe8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafe8053a70 0x7fafe8055f30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fafec018000 tx=0x7fafec014000).stop 2022-08-01T23:03:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 msgr2=0x7fb0040b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 0x7fb0040b05c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7faff4001110 tx=0x7faff4007ae0).stop 2022-08-01T23:03:15.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 shutdown_connections 2022-08-01T23:03:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafe8053a70 0x7fafe8055f30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb004147450 0x7fb0040675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0040b1ef0 0x7fb004152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 --2- 172.21.15.32:0/113594506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0040b1580 0x7fb0040b05c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 >> 172.21.15.32:0/113594506 conn(0x7fb004060950 msgr2=0x7fb0040b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 shutdown_connections 2022-08-01T23:03:15.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.658+0000 7fb00a51d700 1 -- 172.21.15.32:0/113594506 wait complete. 2022-08-01T23:03:15.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0023561700 1 Processor -- start 2022-08-01T23:03:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0023561700 1 -- start start 2022-08-01T23:03:15.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0023561700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b13f0 0x7f001c0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0023561700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c0adf10 con 0x7f001c0b2710 2022-08-01T23:03:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0023561700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c0ae1b0 con 0x7f001c0b1920 2022-08-01T23:03:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0023561700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0a9520 con 0x7f001c0b13f0 2022-08-01T23:03:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f002255f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f001c0b2710 0x7f001c0abca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34050/0 (socket says 172.21.15.32:34050) 2022-08-01T23:03:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f002255f700 1 -- 172.21.15.32:0/2558460825 learned_addr learned my addr 172.21.15.32:0/2558460825 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0021d5e700 1 --2- 172.21.15.32:0/2558460825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b13f0 0x7f001c0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660348963 0 0) 0x7f001c0adf10 con 0x7f001c0b2710 2022-08-01T23:03:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0008003260 con 0x7f001c0b2710 2022-08-01T23:03:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053581500 0 0) 0x7f001c0ae1b0 con 0x7f001c0b1920 2022-08-01T23:03:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f001c0adf10 con 0x7f001c0b1920 2022-08-01T23:03:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0021d5e700 1 -- 172.21.15.32:0/2558460825 >> v1:172.21.15.55:6789/0 conn(0x7f001c0b1920 legacy=0x7f001c14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0021d5e700 1 -- 172.21.15.32:0/2558460825 >> v1:172.21.15.32:6789/0 conn(0x7f001c0b2710 legacy=0x7f001c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0021d5e700 1 -- 172.21.15.32:0/2558460825 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0a99c0 con 0x7f001c0b13f0 2022-08-01T23:03:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.778+0000 7f0021d5e700 1 --2- 172.21.15.32:0/2558460825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b13f0 0x7f001c0afd60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f001c067bc0 tx=0x7f000c006df0).ready entity=mon.2 client_cookie=a06d46d8cad9f5d4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f000c00d5c0 con 0x7f001c0b13f0 2022-08-01T23:03:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c000e90 con 0x7f001c0b13f0 2022-08-01T23:03:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f002155d700 1 -- 172.21.15.32:0/2558460825 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f000c00d910 con 0x7f001c0b13f0 2022-08-01T23:03:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 -- 172.21.15.32:0/2558460825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b13f0 msgr2=0x7f001c0afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 --2- 172.21.15.32:0/2558460825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b13f0 0x7f001c0afd60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f001c067bc0 tx=0x7f000c006df0).stop 2022-08-01T23:03:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 -- 172.21.15.32:0/2558460825 shutdown_connections 2022-08-01T23:03:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 --2- 172.21.15.32:0/2558460825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b13f0 0x7f001c0afd60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 -- 172.21.15.32:0/2558460825 >> 172.21.15.32:0/2558460825 conn(0x7f001c060310 msgr2=0x7f001c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 -- 172.21.15.32:0/2558460825 shutdown_connections 2022-08-01T23:03:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 -- 172.21.15.32:0/2558460825 wait complete. 2022-08-01T23:03:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 Processor -- start 2022-08-01T23:03:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.782+0000 7f0023561700 1 -- start start 2022-08-01T23:03:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 0x7f001c13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f001c0b1920 0x7f001c066a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b2710 0x7f001c066ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c14f3b0 con 0x7f001c0b13f0 2022-08-01T23:03:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c063430 con 0x7f001c0b1920 2022-08-01T23:03:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0abd10 con 0x7f001c0b2710 2022-08-01T23:03:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 0x7f001c13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 0x7f001c13e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46510/0 (socket says 172.21.15.32:46510) 2022-08-01T23:03:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 -- 172.21.15.32:0/2093955984 learned_addr learned my addr 172.21.15.32:0/2093955984 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0021d5e700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f001c0b1920 0x7f001c066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0022d60700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b2710 0x7f001c066ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 -- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b2710 msgr2=0x7f001c066ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b2710 0x7f001c066ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 -- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f001c0b1920 msgr2=0x7f001c066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f001c0b1920 0x7f001c066a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 -- 172.21.15.32:0/2093955984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c00b180 con 0x7f001c0b13f0 2022-08-01T23:03:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0021d5e700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f001c0b1920 0x7f001c066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f002255f700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 0x7f001c13e1c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f00100011d0 tx=0x7f0010007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001000ce70 con 0x7f001c0b13f0 2022-08-01T23:03:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00100072e0 con 0x7f001c0b13f0 2022-08-01T23:03:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c063190 con 0x7f001c0b13f0 2022-08-01T23:03:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0010005800 con 0x7f001c0b13f0 2022-08-01T23:03:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.786+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c0673e0 con 0x7f001c0b13f0 2022-08-01T23:03:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.790+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c064120 con 0x7f001c0b13f0 2022-08-01T23:03:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.790+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f001001a060 con 0x7f001c0b13f0 2022-08-01T23:03:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.790+0000 7f001b7fe700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efffc053b00 0x7efffc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.790+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f001006b8e0 con 0x7f001c0b13f0 2022-08-01T23:03:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.790+0000 7f0021d5e700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efffc053b00 0x7efffc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.790+0000 7f0021d5e700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efffc053b00 0x7efffc055fc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f000c018000 tx=0x7f000c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.794+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00100426a0 con 0x7f001c0b13f0 2022-08-01T23:03:15.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f001c0574e0 con 0x7f001c0b13f0 2022-08-01T23:03:15.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f001b7fe700 1 -- 172.21.15.32:0/2093955984 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0010018020 con 0x7f001c0b13f0 2022-08-01T23:03:15.926 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efffc053b00 msgr2=0x7efffc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efffc053b00 0x7efffc055fc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f000c018000 tx=0x7f000c014000).stop 2022-08-01T23:03:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 msgr2=0x7f001c13e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 0x7f001c13e1c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f00100011d0 tx=0x7f0010007b50).stop 2022-08-01T23:03:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 shutdown_connections 2022-08-01T23:03:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efffc053b00 0x7efffc055fc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f001c0b2710 0x7f001c066ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f001c0b1920 0x7f001c066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 --2- 172.21.15.32:0/2093955984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f001c0b13f0 0x7f001c13e1c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 >> 172.21.15.32:0/2093955984 conn(0x7f001c060310 msgr2=0x7f001c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 shutdown_connections 2022-08-01T23:03:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:15.922+0000 7f0023561700 1 -- 172.21.15.32:0/2093955984 wait complete. 2022-08-01T23:03:15.939 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:03:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.042+0000 7fba41141700 1 Processor -- start 2022-08-01T23:03:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.042+0000 7fba41141700 1 -- start start 2022-08-01T23:03:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c1445d0 0x7fba3c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba3c0655b0 0x7fba3c144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c1454e0 con 0x7fba3c060290 2022-08-01T23:03:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c065e10 con 0x7fba3c0655b0 2022-08-01T23:03:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c060670 con 0x7fba3c1445d0 2022-08-01T23:03:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba3c0655b0 0x7fba3c144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba3c0655b0 0x7fba3c144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46522/0 (socket says 172.21.15.32:46522) 2022-08-01T23:03:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 -- 172.21.15.32:0/1369798329 learned_addr learned my addr 172.21.15.32:0/1369798329 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:16.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba33fff700 1 --2- 172.21.15.32:0/1369798329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c1445d0 0x7fba3c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 -- 172.21.15.32:0/1369798329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c1445d0 msgr2=0x7fba3c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:16.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 --2- 172.21.15.32:0/1369798329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c1445d0 0x7fba3c144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 -- 172.21.15.32:0/1369798329 >> v1:172.21.15.55:6789/0 conn(0x7fba3c060290 legacy=0x7fba3c064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 -- 172.21.15.32:0/1369798329 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c0b2540 con 0x7fba3c0655b0 2022-08-01T23:03:16.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3b59e700 1 --2- 172.21.15.32:0/1369798329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c144ff0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fba340032f0 tx=0x7fba34009290).ready entity=mon.0 client_cookie=b313cf7ee6eabc97 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3a59c700 1 -- 172.21.15.32:0/1369798329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba3400b8a0 con 0x7fba3c0655b0 2022-08-01T23:03:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3a59c700 1 -- 172.21.15.32:0/1369798329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3400ba00 con 0x7fba3c0655b0 2022-08-01T23:03:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba3a59c700 1 -- 172.21.15.32:0/1369798329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba3400bb70 con 0x7fba3c0655b0 2022-08-01T23:03:16.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- 172.21.15.32:0/1369798329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 msgr2=0x7fba3c144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 --2- 172.21.15.32:0/1369798329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c144ff0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fba340032f0 tx=0x7fba34009290).stop 2022-08-01T23:03:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- 172.21.15.32:0/1369798329 shutdown_connections 2022-08-01T23:03:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 --2- 172.21.15.32:0/1369798329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c144ff0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 --2- 172.21.15.32:0/1369798329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c1445d0 0x7fba3c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- 172.21.15.32:0/1369798329 >> 172.21.15.32:0/1369798329 conn(0x7fba3c05d0f0 msgr2=0x7fba3c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- 172.21.15.32:0/1369798329 shutdown_connections 2022-08-01T23:03:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.046+0000 7fba41141700 1 -- 172.21.15.32:0/1369798329 wait complete. 2022-08-01T23:03:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 Processor -- start 2022-08-01T23:03:16.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 -- start start 2022-08-01T23:03:16.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c060290 0x7fba3c13ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba3c13e870 0x7fba3c141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c154700 con 0x7fba3c0655b0 2022-08-01T23:03:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0ab930 con 0x7fba3c13e870 2022-08-01T23:03:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0abbd0 con 0x7fba3c060290 2022-08-01T23:03:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46526/0 (socket says 172.21.15.32:46526) 2022-08-01T23:03:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 -- 172.21.15.32:0/1988173070 learned_addr learned my addr 172.21.15.32:0/1988173070 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:16.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba3b59e700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba3c13e870 0x7fba3c141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba3ad9d700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c060290 0x7fba3c13ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 -- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c060290 msgr2=0x7fba3c13ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c060290 0x7fba3c13ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 -- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba3c13e870 msgr2=0x7fba3c141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba3c13e870 0x7fba3c141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 -- 172.21.15.32:0/1988173070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba34009530 con 0x7fba3c0655b0 2022-08-01T23:03:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba33fff700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c141860 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fba240030b0 tx=0x7fba24008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba24011780 con 0x7fba3c0655b0 2022-08-01T23:03:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba240118e0 con 0x7fba3c0655b0 2022-08-01T23:03:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c154420 con 0x7fba3c0655b0 2022-08-01T23:03:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba24011c40 con 0x7fba3c0655b0 2022-08-01T23:03:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.050+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c0aaf80 con 0x7fba3c0655b0 2022-08-01T23:03:16.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.054+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c154900 con 0x7fba3c0655b0 2022-08-01T23:03:16.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.054+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fba2401c460 con 0x7fba3c0655b0 2022-08-01T23:03:16.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.054+0000 7fba38d99700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba14053980 0x7fba14055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.054+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fba2406c170 con 0x7fba3c0655b0 2022-08-01T23:03:16.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.054+0000 7fba3ad9d700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba14053980 0x7fba14055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.058+0000 7fba3ad9d700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba14053980 0x7fba14055e40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fba2c00d000 tx=0x7fba2c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.062+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba24042dc0 con 0x7fba3c0655b0 2022-08-01T23:03:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.182+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba3c0abd90 con 0x7fba3c0655b0 2022-08-01T23:03:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba38d99700 1 -- 172.21.15.32:0/1988173070 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+10 (secure 0 0 0) 0x7fba240463e0 con 0x7fba3c0655b0 2022-08-01T23:03:16.189 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba14053980 msgr2=0x7fba14055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba14053980 0x7fba14055e40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fba2c00d000 tx=0x7fba2c009040).stop 2022-08-01T23:03:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 msgr2=0x7fba3c141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c141860 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fba240030b0 tx=0x7fba24008040).stop 2022-08-01T23:03:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 shutdown_connections 2022-08-01T23:03:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fba14053980 0x7fba14055e40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba3c13e870 0x7fba3c141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba3c0655b0 0x7fba3c141860 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 --2- 172.21.15.32:0/1988173070 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fba3c060290 0x7fba3c13ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 >> 172.21.15.32:0/1988173070 conn(0x7fba3c05d0f0 msgr2=0x7fba3c0b6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 shutdown_connections 2022-08-01T23:03:16.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.186+0000 7fba41141700 1 -- 172.21.15.32:0/1988173070 wait complete. 2022-08-01T23:03:16.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:03:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 Processor -- start 2022-08-01T23:03:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 -- start start 2022-08-01T23:03:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f53180614b0 0x7f5318061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180ad750 0x7f53180b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53180aee80 con 0x7f53181405d0 2022-08-01T23:03:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53180b70b0 con 0x7f53180614b0 2022-08-01T23:03:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5317fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53180af0b0 con 0x7f53180ad750 2022-08-01T23:03:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f53177fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53181405d0 0x7f53181466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34078/0 (socket says 172.21.15.32:34078) 2022-08-01T23:03:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f53177fe700 1 -- 172.21.15.32:0/4073908921 learned_addr learned my addr 172.21.15.32:0/4073908921 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5316ffd700 1 --2- 172.21.15.32:0/4073908921 >> v2:172.21.15.55:3300/0 conn(0x7f53180614b0 0x7f5318061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f53167fc700 1 --2- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180ad750 0x7f53180b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5315ffb700 1 -- 172.21.15.32:0/4073908921 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765992953 0 0) 0x7f53180aee80 con 0x7f53181405d0 2022-08-01T23:03:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5315ffb700 1 -- 172.21.15.32:0/4073908921 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52fc003260 con 0x7f53181405d0 2022-08-01T23:03:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5316ffd700 1 -- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180ad750 msgr2=0x7f53180b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5316ffd700 1 --2- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180ad750 0x7f53180b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5316ffd700 1 -- 172.21.15.32:0/4073908921 >> v1:172.21.15.32:6789/0 conn(0x7f53181405d0 legacy=0x7f53181466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.306+0000 7f5316ffd700 1 -- 172.21.15.32:0/4073908921 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180af350 con 0x7f53180614b0 2022-08-01T23:03:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5316ffd700 1 --2- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180614b0 0x7f5318061890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f530c002f70 tx=0x7f530c006df0).ready entity=mon.1 client_cookie=3205cff16f1766eb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5315ffb700 1 -- 172.21.15.32:0/4073908921 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f530c00d4a0 con 0x7f53180614b0 2022-08-01T23:03:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5315ffb700 1 -- 172.21.15.32:0/4073908921 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c000e90 con 0x7f53180614b0 2022-08-01T23:03:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5315ffb700 1 -- 172.21.15.32:0/4073908921 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f530c00d7f0 con 0x7f53180614b0 2022-08-01T23:03:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 -- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180614b0 msgr2=0x7f5318061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 --2- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180614b0 0x7f5318061890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f530c002f70 tx=0x7f530c006df0).stop 2022-08-01T23:03:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 -- 172.21.15.32:0/4073908921 shutdown_connections 2022-08-01T23:03:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 --2- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180ad750 0x7f53180b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 --2- 172.21.15.32:0/4073908921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180614b0 0x7f5318061890 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 -- 172.21.15.32:0/4073908921 >> 172.21.15.32:0/4073908921 conn(0x7f5318066f70 msgr2=0x7f5318067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 -- 172.21.15.32:0/4073908921 shutdown_connections 2022-08-01T23:03:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 -- 172.21.15.32:0/4073908921 wait complete. 2022-08-01T23:03:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 Processor -- start 2022-08-01T23:03:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.310+0000 7f5317fff700 1 -- start start 2022-08-01T23:03:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180614b0 0x7f53180b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180ad750 0x7f53180b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 0x7f5318147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5318153210 con 0x7f53181405d0 2022-08-01T23:03:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180b1750 con 0x7f53180ad750 2022-08-01T23:03:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5318141110 con 0x7f53180614b0 2022-08-01T23:03:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 0x7f5318147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180614b0 0x7f53180b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53167fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180ad750 0x7f53180b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180614b0 0x7f53180b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:58276/0 (socket says 172.21.15.32:58276) 2022-08-01T23:03:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 0x7f5318147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46542/0 (socket says 172.21.15.32:46542) 2022-08-01T23:03:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5316ffd700 1 -- 172.21.15.32:0/1194404453 learned_addr learned my addr 172.21.15.32:0/1194404453 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 -- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180614b0 msgr2=0x7f53180b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180614b0 0x7f53180b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 -- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180ad750 msgr2=0x7f53180b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180ad750 0x7f53180b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 -- 172.21.15.32:0/1194404453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c00b180 con 0x7f53181405d0 2022-08-01T23:03:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f53177fe700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 0x7f5318147c30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f5310001490 tx=0x7f5310007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5310005ba0 con 0x7f53181405d0 2022-08-01T23:03:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531814a680 con 0x7f53181405d0 2022-08-01T23:03:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310005d00 con 0x7f53181405d0 2022-08-01T23:03:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53100072c0 con 0x7f53181405d0 2022-08-01T23:03:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5318153610 con 0x7f53181405d0 2022-08-01T23:03:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5310010070 con 0x7f53181405d0 2022-08-01T23:03:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.314+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5318153350 con 0x7f53181405d0 2022-08-01T23:03:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.318+0000 7f5307fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f52f4053950 0x7f52f4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.318+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f531006b940 con 0x7f53181405d0 2022-08-01T23:03:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.318+0000 7f5316ffd700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f52f4053950 0x7f52f4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.318+0000 7f5316ffd700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f52f4053950 0x7f52f4055e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f530c01a000 tx=0x7f530c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.326+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5310042660 con 0x7f53181405d0 2022-08-01T23:03:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.454+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53180574e0 con 0x7f53181405d0 2022-08-01T23:03:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.454+0000 7f5307fff700 1 -- 172.21.15.32:0/1194404453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5310045c80 con 0x7f53181405d0 2022-08-01T23:03:16.459 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f52f4053950 msgr2=0x7f52f4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f52f4053950 0x7f52f4055e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f530c01a000 tx=0x7f530c014000).stop 2022-08-01T23:03:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 msgr2=0x7f5318147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 0x7f5318147c30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f5310001490 tx=0x7f5310007d40).stop 2022-08-01T23:03:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 shutdown_connections 2022-08-01T23:03:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f52f4053950 0x7f52f4055e10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53181405d0 0x7f5318147c30 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53180ad750 0x7f53180b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 --2- 172.21.15.32:0/1194404453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53180614b0 0x7f53180b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 >> 172.21.15.32:0/1194404453 conn(0x7f5318066f70 msgr2=0x7f531805c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 shutdown_connections 2022-08-01T23:03:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.458+0000 7f5317fff700 1 -- 172.21.15.32:0/1194404453 wait complete. 2022-08-01T23:03:16.473 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:03:16.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.570+0000 7fbd72843700 1 Processor -- start 2022-08-01T23:03:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- start start 2022-08-01T23:03:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd6c0b2390 0x7fbd6c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd6c141f40 0x7fbd6c0ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c0ac970 con 0x7fbd6c0b0160 2022-08-01T23:03:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c1453e0 con 0x7fbd6c141f40 2022-08-01T23:03:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c145680 con 0x7fbd6c0b2390 2022-08-01T23:03:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd6c141f40 0x7fbd6c0ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd6c141f40 0x7fbd6c0ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46550/0 (socket says 172.21.15.32:46550) 2022-08-01T23:03:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 -- 172.21.15.32:0/714432050 learned_addr learned my addr 172.21.15.32:0/714432050 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd6bfff700 1 --2- 172.21.15.32:0/714432050 >> v2:172.21.15.55:3300/0 conn(0x7fbd6c0b2390 0x7fbd6c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 -- 172.21.15.32:0/714432050 >> v2:172.21.15.55:3300/0 conn(0x7fbd6c0b2390 msgr2=0x7fbd6c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 --2- 172.21.15.32:0/714432050 >> v2:172.21.15.55:3300/0 conn(0x7fbd6c0b2390 0x7fbd6c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 -- 172.21.15.32:0/714432050 >> v1:172.21.15.55:6789/0 conn(0x7fbd6c0b0160 legacy=0x7fbd6c141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 -- 172.21.15.32:0/714432050 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c141390 con 0x7fbd6c141f40 2022-08-01T23:03:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd6bfff700 1 --2- 172.21.15.32:0/714432050 >> v2:172.21.15.55:3300/0 conn(0x7fbd6c0b2390 0x7fbd6c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd70de0700 1 --2- 172.21.15.32:0/714432050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c141f40 0x7fbd6c0ac480 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7fbd640032f0 tx=0x7fbd64009290).ready entity=mon.0 client_cookie=a668860b16d11810 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd6affd700 1 -- 172.21.15.32:0/714432050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd6400b870 con 0x7fbd6c141f40 2022-08-01T23:03:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd6affd700 1 -- 172.21.15.32:0/714432050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6400b9d0 con 0x7fbd6c141f40 2022-08-01T23:03:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd6affd700 1 -- 172.21.15.32:0/714432050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd6400bb40 con 0x7fbd6c141f40 2022-08-01T23:03:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- 172.21.15.32:0/714432050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c141f40 msgr2=0x7fbd6c0ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 --2- 172.21.15.32:0/714432050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c141f40 0x7fbd6c0ac480 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7fbd640032f0 tx=0x7fbd64009290).stop 2022-08-01T23:03:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- 172.21.15.32:0/714432050 shutdown_connections 2022-08-01T23:03:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 --2- 172.21.15.32:0/714432050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c141f40 0x7fbd6c0ac480 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 --2- 172.21.15.32:0/714432050 >> v2:172.21.15.55:3300/0 conn(0x7fbd6c0b2390 0x7fbd6c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- 172.21.15.32:0/714432050 >> 172.21.15.32:0/714432050 conn(0x7fbd6c05f840 msgr2=0x7fbd6c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- 172.21.15.32:0/714432050 shutdown_connections 2022-08-01T23:03:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.574+0000 7fbd72843700 1 -- 172.21.15.32:0/714432050 wait complete. 2022-08-01T23:03:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 Processor -- start 2022-08-01T23:03:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 -- start start 2022-08-01T23:03:16.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 0x7fbd6c1405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbd6c0b2390 0x7fbd6c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd6c141f40 0x7fbd6c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c154b00 con 0x7fbd6c0b0160 2022-08-01T23:03:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c1410c0 con 0x7fbd6c141f40 2022-08-01T23:03:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c150670 con 0x7fbd6c0b2390 2022-08-01T23:03:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 0x7fbd6c1405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 0x7fbd6c1405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46552/0 (socket says 172.21.15.32:46552) 2022-08-01T23:03:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 -- 172.21.15.32:0/1751267428 learned_addr learned my addr 172.21.15.32:0/1751267428 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd70de0700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd6c141f40 0x7fbd6c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6b7fe700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbd6c0b2390 0x7fbd6c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 -- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbd6c0b2390 msgr2=0x7fbd6c154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbd6c0b2390 0x7fbd6c154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 -- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd6c141f40 msgr2=0x7fbd6c150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd6c141f40 0x7fbd6c150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 -- 172.21.15.32:0/1751267428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009530 con 0x7fbd6c0b0160 2022-08-01T23:03:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd70de0700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd6c141f40 0x7fbd6c150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd6bfff700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 0x7fbd6c1405e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fbd6c141200 tx=0x7fbd60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd60011780 con 0x7fbd6c0b0160 2022-08-01T23:03:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd600118e0 con 0x7fbd6c0b0160 2022-08-01T23:03:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c13bdd0 con 0x7fbd6c0b0160 2022-08-01T23:03:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd60011c40 con 0x7fbd6c0b0160 2022-08-01T23:03:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.578+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c13c310 con 0x7fbd6c0b0160 2022-08-01T23:03:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.582+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c0574e0 con 0x7fbd6c0b0160 2022-08-01T23:03:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.582+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbd60012420 con 0x7fbd6c0b0160 2022-08-01T23:03:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.582+0000 7fbd697fa700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbd4c053b00 0x7fbd4c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.582+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbd6006bf30 con 0x7fbd6c0b0160 2022-08-01T23:03:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.582+0000 7fbd6b7fe700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbd4c053b00 0x7fbd4c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.582+0000 7fbd6b7fe700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbd4c053b00 0x7fbd4c055fc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fbd5c00b000 tx=0x7fbd5c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.586+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd60042c90 con 0x7fbd6c0b0160 2022-08-01T23:03:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.746+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd6c0447e0 con 0x7fbd6c0b0160 2022-08-01T23:03:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd697fa700 1 -- 172.21.15.32:0/1751267428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fbd600462b0 con 0x7fbd6c0b0160 2022-08-01T23:03:16.753 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:09.472203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6981":{"gid":6981,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2562641804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2562641804},{"type":"v1","addr":"172.21.15.32:6835","nonce":2562641804}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbd4c053b00 msgr2=0x7fbd4c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbd4c053b00 0x7fbd4c055fc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fbd5c00b000 tx=0x7fbd5c009040).stop 2022-08-01T23:03:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 msgr2=0x7fbd6c1405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 0x7fbd6c1405e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fbd6c141200 tx=0x7fbd60008040).stop 2022-08-01T23:03:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 shutdown_connections 2022-08-01T23:03:16.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbd4c053b00 0x7fbd4c055fc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd6c141f40 0x7fbd6c150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbd6c0b2390 0x7fbd6c154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 --2- 172.21.15.32:0/1751267428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6c0b0160 0x7fbd6c1405e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 >> 172.21.15.32:0/1751267428 conn(0x7fbd6c05f840 msgr2=0x7fbd6c13ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:16.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 shutdown_connections 2022-08-01T23:03:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:16.750+0000 7fbd72843700 1 -- 172.21.15.32:0/1751267428 wait complete. 2022-08-01T23:03:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2022-08-01T23:03:17.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:08.469158+0000', 'modified': '2022-08-01T23:03:09.472203+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6981}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6981': {'gid': 6981, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2562641804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2562641804}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2562641804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:17.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:17.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:03:17.187 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":6981,"want_state":"up:active","state":"up:active","rank_uptime":8.7032477270000008,"mdsmap_epoch":45,"osdmap_epoch":97,"osdmap_epoch_barrier":96,"uptime":13.036732671999999} 2022-08-01T23:03:17.188 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 6981, "mdsmap_epoch": 45, "osdmap_epoch": 97, "osdmap_epoch_barrier": 96, "rank_uptime": 8.703247727, "state": "up:active", "uptime": 13.036732672, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:17.189 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:03:17.189 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:03:17.189 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:03:17.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.190 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:17.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:03:17.213 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:17.214 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:17.270 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:17.270 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:17.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.272 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:03:17.279 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:03:17.280 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:17.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:03:17.298 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:17.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:17.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:03:17.342+0000 7f055f632080 -1 init, newargv = 0x56180c280b40 newargc=9ceph-fuse[ 2022-08-01T23:03:17.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:22759]: starting ceph client 2022-08-01T23:03:17.358 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:17.359 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:17.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.360 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:03:17.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22759]: starting fuse 2022-08-01T23:03:17.368 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:03:17.368 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:03:17.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:03:17.369 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:03:17.370 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:03:17.370 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:03:17.371 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:03:17.371 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:17.371 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:03:17.372 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:03:17.372 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:03:17.372 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:17.373 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:03:17.373 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:03:17.374 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:03:17.374 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:03:17.374 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:17.375 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:03:17.375 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:03:17.376 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:03:17.376 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:03:17.376 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:03:17.377 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:17.377 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:03:17.377 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:03:17.419 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.22759.asok 2022-08-01T23:03:17.424 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22759.asok 2022-08-01T23:03:17.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22759.asok status 2022-08-01T23:03:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:03:17.559 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:03:17.559 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:03:17.559 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:03:17.560 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:03:17.560 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:03:17.560 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:03:17.560 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "22759", 2022-08-01T23:03:17.560 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:03:17.561 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:17.561 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:03:17.561 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:03:17.561 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7185, 2022-08-01T23:03:17.561 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:03:17.562 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:03:17.562 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:03:17.562 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7185 2022-08-01T23:03:17.562 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:17.563 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:03:17.563 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:03:17.563 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:03:17.563 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 588935690 2022-08-01T23:03:17.563 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:03:17.564 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:17.564 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:03:17.564 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:03:17.564 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:03:17.564 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 588935690 2022-08-01T23:03:17.565 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:17.565 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7185 172.21.15.157:0/588935690", 2022-08-01T23:03:17.565 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/588935690", 2022-08-01T23:03:17.565 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:03:17.565 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 45, 2022-08-01T23:03:17.566 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 97, 2022-08-01T23:03:17.566 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:03:17.566 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:03:17.566 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:03:17.566 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:03:17.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.568 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:17.568 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:17.575 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:03:17.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:17.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:17.577 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:17.594 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T23:03:17.594 DEBUG:teuthology.orchestra.run.smithi130:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2022-08-01T23:03:17.623 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T23:03:17.624 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:03:17.624 DEBUG:teuthology.orchestra.run.smithi130:> import stat 2022-08-01T23:03:17.625 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:17.625 DEBUG:teuthology.orchestra.run.smithi130:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2022-08-01T23:03:17.626 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:03:17.742 INFO:teuthology.orchestra.run.smithi130.stdout:1099511627776 2022-08-01T23:03:17.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 Processor -- start 2022-08-01T23:03:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 -- start start 2022-08-01T23:03:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc368139e50 0x7fc3680acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc36813a230 0x7fc3680b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3680ad200 con 0x7fc3680b2350 2022-08-01T23:03:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3680b28c0 con 0x7fc368139e50 2022-08-01T23:03:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc370636700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc368060ed0 con 0x7fc36813a230 2022-08-01T23:03:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc368139e50 0x7fc3680acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc368139e50 0x7fc3680acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46560/0 (socket says 172.21.15.32:46560) 2022-08-01T23:03:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 -- 172.21.15.32:0/692955032 learned_addr learned my addr 172.21.15.32:0/692955032 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36ebd3700 1 --2- 172.21.15.32:0/692955032 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc36813a230 0x7fc3680b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 -- 172.21.15.32:0/692955032 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc36813a230 msgr2=0x7fc3680b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 --2- 172.21.15.32:0/692955032 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc36813a230 0x7fc3680b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 -- 172.21.15.32:0/692955032 >> v1:172.21.15.55:6789/0 conn(0x7fc3680b2350 legacy=0x7fc368143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 -- 172.21.15.32:0/692955032 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368061130 con 0x7fc368139e50 2022-08-01T23:03:18.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.046+0000 7fc36e3d2700 1 --2- 172.21.15.32:0/692955032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc368139e50 0x7fc3680acd10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7fc364002f70 tx=0x7fc364006f40).ready entity=mon.0 client_cookie=6979abbf401016a5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc36d3d0700 1 -- 172.21.15.32:0/692955032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36400da80 con 0x7fc368139e50 2022-08-01T23:03:18.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc36d3d0700 1 -- 172.21.15.32:0/692955032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364000e90 con 0x7fc368139e50 2022-08-01T23:03:18.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc36d3d0700 1 -- 172.21.15.32:0/692955032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36400de10 con 0x7fc368139e50 2022-08-01T23:03:18.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 -- 172.21.15.32:0/692955032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc368139e50 msgr2=0x7fc3680acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 --2- 172.21.15.32:0/692955032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc368139e50 0x7fc3680acd10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7fc364002f70 tx=0x7fc364006f40).stop 2022-08-01T23:03:18.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 -- 172.21.15.32:0/692955032 shutdown_connections 2022-08-01T23:03:18.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 --2- 172.21.15.32:0/692955032 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc36813a230 0x7fc3680b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 --2- 172.21.15.32:0/692955032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc368139e50 0x7fc3680acd10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 -- 172.21.15.32:0/692955032 >> 172.21.15.32:0/692955032 conn(0x7fc3680b3610 msgr2=0x7fc3680b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 -- 172.21.15.32:0/692955032 shutdown_connections 2022-08-01T23:03:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 -- 172.21.15.32:0/692955032 wait complete. 2022-08-01T23:03:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 Processor -- start 2022-08-01T23:03:18.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.050+0000 7fc370636700 1 -- start start 2022-08-01T23:03:18.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3680b2350 0x7fc3680af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc368139e50 0x7fc36813a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 0x7fc36813d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36814c340 con 0x7fc36813a230 2022-08-01T23:03:18.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36813ae60 con 0x7fc368139e50 2022-08-01T23:03:18.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681426d0 con 0x7fc3680b2350 2022-08-01T23:03:18.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 0x7fc36813d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 0x7fc36813d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46570/0 (socket says 172.21.15.32:46570) 2022-08-01T23:03:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 -- 172.21.15.32:0/999490954 learned_addr learned my addr 172.21.15.32:0/999490954 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36e3d2700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3680b2350 0x7fc3680af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36dbd1700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc368139e50 0x7fc36813a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 -- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3680b2350 msgr2=0x7fc3680af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3680b2350 0x7fc3680af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 -- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc368139e50 msgr2=0x7fc36813a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc368139e50 0x7fc36813a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 -- 172.21.15.32:0/999490954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36400b460 con 0x7fc36813a230 2022-08-01T23:03:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36e3d2700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3680b2350 0x7fc3680af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc36ebd3700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 0x7fc36813d380 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7fc360006e00 tx=0x7fc3600091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36000c7f0 con 0x7fc36813a230 2022-08-01T23:03:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000c950 con 0x7fc36813a230 2022-08-01T23:03:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368142970 con 0x7fc36813a230 2022-08-01T23:03:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36000ccb0 con 0x7fc36813a230 2022-08-01T23:03:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3680ae660 con 0x7fc36813a230 2022-08-01T23:03:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.054+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3680574e0 con 0x7fc36813a230 2022-08-01T23:03:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.058+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc360022070 con 0x7fc36813a230 2022-08-01T23:03:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.058+0000 7fc3577fe700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc348053980 0x7fc348055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.058+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc36006c690 con 0x7fc36813a230 2022-08-01T23:03:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.058+0000 7fc36e3d2700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc348053980 0x7fc348055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.058+0000 7fc36e3d2700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc348053980 0x7fc348055e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fc36401c000 tx=0x7fc364006f40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.062+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc360043450 con 0x7fc36813a230 2022-08-01T23:03:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.226+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3680447e0 con 0x7fc36813a230 2022-08-01T23:03:18.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.226+0000 7fc3577fe700 1 -- 172.21.15.32:0/999490954 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fc360016020 con 0x7fc36813a230 2022-08-01T23:03:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:18.232 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:09.472203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6981":{"gid":6981,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2562641804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2562641804},{"type":"v1","addr":"172.21.15.32:6835","nonce":2562641804}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc348053980 msgr2=0x7fc348055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc348053980 0x7fc348055e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fc36401c000 tx=0x7fc364006f40).stop 2022-08-01T23:03:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 msgr2=0x7fc36813d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 0x7fc36813d380 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7fc360006e00 tx=0x7fc3600091d0).stop 2022-08-01T23:03:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 shutdown_connections 2022-08-01T23:03:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc348053980 0x7fc348055e40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36813a230 0x7fc36813d380 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc368139e50 0x7fc36813a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 --2- 172.21.15.32:0/999490954 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc3680b2350 0x7fc3680af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 >> 172.21.15.32:0/999490954 conn(0x7fc3680b3610 msgr2=0x7fc36813e450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 shutdown_connections 2022-08-01T23:03:18.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.230+0000 7fc370636700 1 -- 172.21.15.32:0/999490954 wait complete. 2022-08-01T23:03:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2022-08-01T23:03:18.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump inode 0x10000000000 2022-08-01T23:03:18.349 INFO:teuthology.orchestra.run.smithi032.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2022-08-01T23:03:17.613296+0000","btime":"2022-08-01T23:03:17.613296+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-08-01T23:03:17.613296+0000","atime":"2022-08-01T23:03:17.613296+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":7143,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":7143,"want_loner":7143,"mds_caps_wanted":[]} 2022-08-01T23:03:18.350 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2022-08-01T23:03:17.613296+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2022-08-01T23:03:17.613296+0000", "change_attr": 0, "client_caps": [ { "client_id": 7143, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-08-01T23:03:17.613296+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 7143, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2022-08-01T23:03:17.613296+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 7143, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2022-08-01T23:03:18.350 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:03:18.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:18.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:18.386 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:18.406 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:18.428 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:03:18.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:18.429 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:18.429 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:18.437 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:03:18.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:18.439 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:03:18.439 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:03:18.440 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:18.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23224]: fuse finished with error 0 and tester_r 0 2022-08-01T23:03:18.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:18.497 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:18.497 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:18.503 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:03:18.504 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:03:18.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:03:18.622 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:18.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:18.630 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:18.631 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:18.639 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:03:18.640 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:18.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:03:18.641 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:03:18.641 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:18.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22759]: fuse finished with error 0 and tester_r 0 2022-08-01T23:03:18.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:18.702 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:18.703 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:18.710 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:03:18.710 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:03:18.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:03:18.746 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:18.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:03:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b84e6f700 1 Processor -- start 2022-08-01T23:03:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b84e6f700 1 -- start start 2022-08-01T23:03:18.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b84e6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b80066dc0 0x7f8b8014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b84e6f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800654c0 con 0x7f8b800b3190 2022-08-01T23:03:18.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b84e6f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800b3570 con 0x7f8b800b2730 2022-08-01T23:03:18.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b84e6f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b80062190 con 0x7f8b80066dc0 2022-08-01T23:03:18.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b7f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8b800b3190 0x7f8b80060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34118/0 (socket says 172.21.15.32:34118) 2022-08-01T23:03:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b7f7fe700 1 -- 172.21.15.32:0/3658939051 learned_addr learned my addr 172.21.15.32:0/3658939051 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.866+0000 7f8b7ffff700 1 --2- 172.21.15.32:0/3658939051 >> v2:172.21.15.32:3300/0 conn(0x7f8b80066dc0 0x7f8b8014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7ffff700 1 -- 172.21.15.32:0/3658939051 >> v1:172.21.15.55:6789/0 conn(0x7f8b800b2730 legacy=0x7f8b800666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7ffff700 1 -- 172.21.15.32:0/3658939051 >> v1:172.21.15.32:6789/0 conn(0x7f8b800b3190 legacy=0x7f8b80060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7effd700 1 -- 172.21.15.32:0/3658939051 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733331927 0 0) 0x7f8b800654c0 con 0x7f8b800b3190 2022-08-01T23:03:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7ffff700 1 -- 172.21.15.32:0/3658939051 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8005fbf0 con 0x7f8b80066dc0 2022-08-01T23:03:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7ffff700 1 --2- 172.21.15.32:0/3658939051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b80066dc0 0x7f8b8014eac0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f8b780032f0 tx=0x7f8b780091d0).ready entity=mon.0 client_cookie=4b4653dc3f3f2567 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7effd700 1 -- 172.21.15.32:0/3658939051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b78009ea0 con 0x7f8b80066dc0 2022-08-01T23:03:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7effd700 1 -- 172.21.15.32:0/3658939051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7800b8a0 con 0x7f8b80066dc0 2022-08-01T23:03:18.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b7effd700 1 -- 172.21.15.32:0/3658939051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b7800ba10 con 0x7f8b80066dc0 2022-08-01T23:03:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 -- 172.21.15.32:0/3658939051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b80066dc0 msgr2=0x7f8b8014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/3658939051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b80066dc0 0x7f8b8014eac0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f8b780032f0 tx=0x7f8b780091d0).stop 2022-08-01T23:03:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 -- 172.21.15.32:0/3658939051 shutdown_connections 2022-08-01T23:03:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/3658939051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b80066dc0 0x7f8b8014eac0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 -- 172.21.15.32:0/3658939051 >> 172.21.15.32:0/3658939051 conn(0x7f8b80063870 msgr2=0x7f8b80063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 -- 172.21.15.32:0/3658939051 shutdown_connections 2022-08-01T23:03:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.870+0000 7f8b84e6f700 1 -- 172.21.15.32:0/3658939051 wait complete. 2022-08-01T23:03:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 Processor -- start 2022-08-01T23:03:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 -- start start 2022-08-01T23:03:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b80066dc0 0x7f8b80140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 0x7f8b800a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8b800b3190 0x7f8b800a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8013f930 con 0x7f8b800b2730 2022-08-01T23:03:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b1c80 con 0x7f8b80066dc0 2022-08-01T23:03:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b84e6f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80145a00 con 0x7f8b800b3190 2022-08-01T23:03:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 0x7f8b800a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 0x7f8b800a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46582/0 (socket says 172.21.15.32:46582) 2022-08-01T23:03:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 -- 172.21.15.32:0/1855792501 learned_addr learned my addr 172.21.15.32:0/1855792501 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b7f7fe700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b80066dc0 0x7f8b80140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b7ffff700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8b800b3190 0x7f8b800a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 -- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8b800b3190 msgr2=0x7f8b800a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8b800b3190 0x7f8b800a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 -- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b80066dc0 msgr2=0x7f8b80140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b80066dc0 0x7f8b80140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 -- 172.21.15.32:0/1855792501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b78009470 con 0x7f8b800b2730 2022-08-01T23:03:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.874+0000 7f8b77fff700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 0x7f8b800a3f80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f8b80068010 tx=0x7f8b68007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b68012400 con 0x7f8b800b2730 2022-08-01T23:03:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68012560 con 0x7f8b800b2730 2022-08-01T23:03:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800a31d0 con 0x7f8b800b2730 2022-08-01T23:03:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b680128c0 con 0x7f8b800b2730 2022-08-01T23:03:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800a3710 con 0x7f8b800b2730 2022-08-01T23:03:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8005fe90 con 0x7f8b800b2730 2022-08-01T23:03:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8b68019030 con 0x7f8b800b2730 2022-08-01T23:03:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7d7fa700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8b640539e0 0x7f8b64055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8b6806b330 con 0x7f8b800b2730 2022-08-01T23:03:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7f7fe700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8b640539e0 0x7f8b64055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.878+0000 7f8b7f7fe700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8b640539e0 0x7f8b64055ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f8b7000d000 tx=0x7f8b70007c00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:18.886+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b68042090 con 0x7f8b800b2730 2022-08-01T23:03:19.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.014+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8b800574e0 con 0x7f8b800b2730 2022-08-01T23:03:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b7d7fa700 1 -- 172.21.15.32:0/1855792501 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v97) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f8b680456b0 con 0x7f8b800b2730 2022-08-01T23:03:19.021 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:19.022 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 97, 2022-08-01T23:03:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:03:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:03:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:03:09.472841+0000", 2022-08-01T23:03:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:03:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:03:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:03:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:03:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:03:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:03:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:03:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:03:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:03:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:03:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:03:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:03:19.026 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 18, 2022-08-01T23:03:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:03:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:03:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:03:19.027 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:03:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:03:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:03:19.028 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:03:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:03:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:19.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:03:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:03:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:03:19.031 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:03:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:03:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:19.032 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:19.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:03:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:19.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:19.035 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:19.036 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:19.037 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:19.038 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:19.039 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.040 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:19.042 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:19.043 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:03:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:19.044 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:03:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:03:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:03:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:03:19.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:19.047 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:19.048 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:19.049 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:19.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:03:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:19.052 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:19.053 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:19.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:19.055 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:19.056 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:19.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:19.059 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:19.060 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:03:19.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 17, 2022-08-01T23:03:19.062 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:03:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:03:06.881558+0000", 2022-08-01T23:03:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:19.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:19.064 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:19.065 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:19.066 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:19.067 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "96", 2022-08-01T23:03:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:19.068 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:19.069 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:19.070 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:19.071 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:19.072 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:19.073 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.074 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:19.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:19.076 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:03:19.077 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:03:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:03:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:03:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:03:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 18, 2022-08-01T23:03:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:03:19.080 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:03:07.840459+0000", 2022-08-01T23:03:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:19.081 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:19.082 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:19.083 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:19.084 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:19.085 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "97", 2022-08-01T23:03:19.086 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:19.087 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:19.088 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:19.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:19.091 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:19.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:19.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:19.094 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:19.095 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:03:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:03:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:03:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.097 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:03:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.098 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.099 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2022-08-01T23:03:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.100 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:03:19.101 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.102 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:03:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.103 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:03:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.106 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:03:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:03:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:03:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.111 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:03:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.113 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:03:19.113 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:19.113 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.114 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.114 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:03:19.114 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:03:19.115 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:03:19.115 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:03:19.115 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.115 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.115 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.116 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.116 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:19.116 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:03:19.117 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.117 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.117 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.117 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.118 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.118 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.118 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.118 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2022-08-01T23:03:19.119 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.119 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.119 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.120 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:03:19.120 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.121 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:03:19.121 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.122 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.123 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:03:19.123 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.124 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:03:19.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.125 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.125 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:03:19.126 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:03:19.127 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.128 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:03:19.129 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:03:19.131 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:19.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.132 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:03:19.132 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:03:19.132 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:03:19.132 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:03:19.132 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.133 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.133 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.134 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:19.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:03:19.134 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.134 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.135 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.135 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.136 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.136 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 95, 2022-08-01T23:03:19.136 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.136 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.136 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.137 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.137 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:03:19.137 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.138 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:03:19.139 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.140 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:03:19.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:03:19.142 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.142 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.142 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.143 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.143 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.143 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:03:19.144 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.145 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:03:19.145 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.146 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.146 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.147 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:03:19.147 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.148 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:03:19.148 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:19.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.149 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:03:19.149 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:03:19.149 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:03:19.149 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:03:19.150 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.150 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.150 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:19.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:03:19.152 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.152 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.152 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.152 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.152 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.153 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.153 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.153 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 95, 2022-08-01T23:03:19.153 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.153 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.154 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:03:19.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:03:19.156 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.157 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:03:19.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:03:19.159 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.161 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.162 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:03:19.162 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.163 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:03:19.163 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.165 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.165 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.166 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:03:19.166 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:03:19.167 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:19.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.168 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:03:19.168 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:03:19.169 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:03:19.169 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:03:19.169 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.170 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.170 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.171 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:19.171 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:03:19.172 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.172 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.172 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.172 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.172 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.173 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.173 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.173 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2022-08-01T23:03:19.174 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.174 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.174 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.175 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:03:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:03:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:03:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:03:19.181 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.182 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.182 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.183 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:03:19.183 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.184 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:03:19.184 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.185 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.186 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:03:19.187 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.187 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:03:19.188 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:19.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.189 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:03:19.189 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:03:19.189 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:03:19.189 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:03:19.190 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.190 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.190 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:19.191 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:03:19.192 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.192 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.192 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.192 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.193 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.193 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:19.194 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.194 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.194 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.194 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.195 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:03:19.195 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.196 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:03:19.196 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.197 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.197 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.197 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.198 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:03:19.198 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.199 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:03:19.200 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.201 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.201 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:03:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.203 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:03:19.203 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.204 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.204 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:03:19.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:03:19.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:19.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.207 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.207 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:03:19.207 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:03:19.207 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:03:19.208 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:03:19.208 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.208 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.208 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.209 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.209 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:19.209 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:03:19.210 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.210 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.210 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.210 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.211 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.211 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.211 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.211 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2022-08-01T23:03:19.212 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.212 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.212 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.212 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.213 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:03:19.213 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.214 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:03:19.214 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.215 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.215 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.218 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.218 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.219 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:03:19.219 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.220 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:03:19.220 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.221 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.221 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.222 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:03:19.222 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.223 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:03:19.223 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:19.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.224 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.224 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:03:19.225 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:03:19.225 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:03:19.225 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:03:19.225 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.225 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.226 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:19.227 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:03:19.227 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:19.227 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:19.227 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:19.228 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:19.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:19.235 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:19.235 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:19.235 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2022-08-01T23:03:19.235 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:19.236 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:19.236 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:19.236 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.237 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:03:19.237 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.238 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:03:19.238 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.239 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:19.239 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.240 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:03:19.240 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.241 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:03:19.241 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.242 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:19.242 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.243 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:03:19.243 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.244 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:03:19.245 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.245 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.246 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:19.246 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:19.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:19.247 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:03:19.248 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:19.249 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:03:19.249 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:19.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.249 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.250 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:03:19.250 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:03:19.250 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:03:19.251 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:03:19.251 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:19.251 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:19.252 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:19.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:19.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.253 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:19.253 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:03:19.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.254 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:19.254 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.254 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.255 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.255 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.255 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.255 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:03:19.256 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.256 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:19.257 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.257 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.257 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.257 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.258 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.258 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:03:19.258 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.259 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:19.259 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.259 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.259 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.260 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.260 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.260 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:03:19.260 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.261 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:19.261 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.261 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.261 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.262 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.262 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.262 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:03:19.262 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.263 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:19.263 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.263 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.264 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.264 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.264 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:03:19.265 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.265 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:19.266 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.266 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.266 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.266 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.266 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.267 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:03:19.267 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.268 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:19.268 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.268 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.268 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.268 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.269 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.269 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:03:19.269 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:19.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:19.270 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:19.270 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:19.271 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:19.271 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:19.271 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:19.271 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:03:19.271 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:19.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.272 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:19.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:03:19.272 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:03:19.273 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:03:19.273 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:03:19.273 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:03:19.273 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:03:19.273 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:03:19.274 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:03:19.274 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:19.274 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:03:19.274 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:03:19.275 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:19.275 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:03:19.275 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:19.275 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:19.276 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:03:19.276 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:03:19.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:03:19.277 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:19.277 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:19.277 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:03:19.277 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:19.278 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:19.278 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:03:19.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:19.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:19.279 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:03:19.279 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:03:19.279 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:03:19.280 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:03:19.280 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:03:19.280 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8b640539e0 msgr2=0x7f8b64055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8b640539e0 0x7f8b64055ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f8b7000d000 tx=0x7f8b70007c00).stop 2022-08-01T23:03:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 msgr2=0x7f8b800a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 0x7f8b800a3f80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f8b80068010 tx=0x7f8b68007c20).stop 2022-08-01T23:03:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 shutdown_connections 2022-08-01T23:03:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8b640539e0 0x7f8b64055ea0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8b800b3190 0x7f8b800a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b800b2730 0x7f8b800a3f80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 --2- 172.21.15.32:0/1855792501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b80066dc0 0x7f8b80140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 >> 172.21.15.32:0/1855792501 conn(0x7f8b80063870 msgr2=0x7f8b80066510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 shutdown_connections 2022-08-01T23:03:19.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.018+0000 7f8b84e6f700 1 -- 172.21.15.32:0/1855792501 wait complete. 2022-08-01T23:03:19.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 Processor -- start 2022-08-01T23:03:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 -- start start 2022-08-01T23:03:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5980b2710 0x7fb59814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5980a8c50 0x7fb59814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980a7250 con 0x7fb5980b01a0 2022-08-01T23:03:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5980a7480 con 0x7fb5980b2710 2022-08-01T23:03:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb59ccee700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5980a7720 con 0x7fb5980a8c50 2022-08-01T23:03:19.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5980b2710 0x7fb59814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5980b2710 0x7fb59814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46590/0 (socket says 172.21.15.32:46590) 2022-08-01T23:03:19.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 -- 172.21.15.32:0/2863570210 learned_addr learned my addr 172.21.15.32:0/2863570210 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:19.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb597fff700 1 --2- 172.21.15.32:0/2863570210 >> v2:172.21.15.55:3300/0 conn(0x7fb5980a8c50 0x7fb59814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5967fc700 1 -- 172.21.15.32:0/2863570210 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236872327 0 0) 0x7fb5980a7250 con 0x7fb5980b01a0 2022-08-01T23:03:19.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 -- 172.21.15.32:0/2863570210 >> v2:172.21.15.55:3300/0 conn(0x7fb5980a8c50 msgr2=0x7fb59814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 --2- 172.21.15.32:0/2863570210 >> v2:172.21.15.55:3300/0 conn(0x7fb5980a8c50 0x7fb59814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 -- 172.21.15.32:0/2863570210 >> v1:172.21.15.32:6789/0 conn(0x7fb5980b01a0 legacy=0x7fb5980a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.394+0000 7fb5977fe700 1 -- 172.21.15.32:0/2863570210 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980a79c0 con 0x7fb5980b2710 2022-08-01T23:03:19.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb5977fe700 1 --2- 172.21.15.32:0/2863570210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb59814afd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fb58c002f70 tx=0x7fb58c006eb0).ready entity=mon.0 client_cookie=705099fd361ab595 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb5967fc700 1 -- 172.21.15.32:0/2863570210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb58c00d610 con 0x7fb5980b2710 2022-08-01T23:03:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb5967fc700 1 -- 172.21.15.32:0/2863570210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c000e90 con 0x7fb5980b2710 2022-08-01T23:03:19.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb5967fc700 1 -- 172.21.15.32:0/2863570210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb58c00d960 con 0x7fb5980b2710 2022-08-01T23:03:19.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 -- 172.21.15.32:0/2863570210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 msgr2=0x7fb59814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 --2- 172.21.15.32:0/2863570210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb59814afd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fb58c002f70 tx=0x7fb58c006eb0).stop 2022-08-01T23:03:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 -- 172.21.15.32:0/2863570210 shutdown_connections 2022-08-01T23:03:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 --2- 172.21.15.32:0/2863570210 >> v2:172.21.15.55:3300/0 conn(0x7fb5980a8c50 0x7fb59814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 --2- 172.21.15.32:0/2863570210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb59814afd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 -- 172.21.15.32:0/2863570210 >> 172.21.15.32:0/2863570210 conn(0x7fb598060310 msgr2=0x7fb5980b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 -- 172.21.15.32:0/2863570210 shutdown_connections 2022-08-01T23:03:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.398+0000 7fb59ccee700 1 -- 172.21.15.32:0/2863570210 wait complete. 2022-08-01T23:03:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 Processor -- start 2022-08-01T23:03:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 -- start start 2022-08-01T23:03:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5980a8c50 0x7fb59814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5980b01a0 0x7fb5980aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb598156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598159a50 con 0x7fb5980b2710 2022-08-01T23:03:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598159cf0 con 0x7fb5980a8c50 2022-08-01T23:03:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb598159f90 con 0x7fb5980b01a0 2022-08-01T23:03:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb598156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb598156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46598/0 (socket says 172.21.15.32:46598) 2022-08-01T23:03:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 -- 172.21.15.32:0/4136003502 learned_addr learned my addr 172.21.15.32:0/4136003502 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb5977fe700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5980a8c50 0x7fb59814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb596ffd700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5980b01a0 0x7fb5980aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 -- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5980b01a0 msgr2=0x7fb5980aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5980b01a0 0x7fb5980aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 -- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5980a8c50 msgr2=0x7fb59814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5980a8c50 0x7fb59814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 -- 172.21.15.32:0/4136003502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c00b1e0 con 0x7fb5980b2710 2022-08-01T23:03:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb596ffd700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5980b01a0 0x7fb5980aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb597fff700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb598156370 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fb59000a000 tx=0x7fb590008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb590013620 con 0x7fb5980b2710 2022-08-01T23:03:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590013780 con 0x7fb5980b2710 2022-08-01T23:03:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59815a230 con 0x7fb5980b2710 2022-08-01T23:03:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.402+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb590013ae0 con 0x7fb5980b2710 2022-08-01T23:03:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59814d730 con 0x7fb5980b2710 2022-08-01T23:03:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59815a3c0 con 0x7fb5980b2710 2022-08-01T23:03:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb590013c40 con 0x7fb5980b2710 2022-08-01T23:03:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb594ff9700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb578053b00 0x7fb578055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb59006c200 con 0x7fb5980b2710 2022-08-01T23:03:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb5977fe700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb578053b00 0x7fb578055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.406+0000 7fb5977fe700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb578053b00 0x7fb578055fc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fb58c01c000 tx=0x7fb58c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.410+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb590042f60 con 0x7fb5980b2710 2022-08-01T23:03:19.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5980574e0 con 0x7fb5980b2710 2022-08-01T23:03:19.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb594ff9700 1 -- 172.21.15.32:0/4136003502 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fb590046580 con 0x7fb5980b2710 2022-08-01T23:03:19.574 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:19.574 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:09.472203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6981},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6981":{"gid":6981,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2562641804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2562641804},{"type":"v1","addr":"172.21.15.32:6835","nonce":2562641804}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:19.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb578053b00 msgr2=0x7fb578055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb578053b00 0x7fb578055fc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fb58c01c000 tx=0x7fb58c015000).stop 2022-08-01T23:03:19.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 msgr2=0x7fb598156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb598156370 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fb59000a000 tx=0x7fb590008040).stop 2022-08-01T23:03:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 shutdown_connections 2022-08-01T23:03:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb578053b00 0x7fb578055fc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980b2710 0x7fb598156370 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb5980b01a0 0x7fb5980aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 --2- 172.21.15.32:0/4136003502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5980a8c50 0x7fb59814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 >> 172.21.15.32:0/4136003502 conn(0x7fb598060310 msgr2=0x7fb59814a270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 shutdown_connections 2022-08-01T23:03:19.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.570+0000 7fb59ccee700 1 -- 172.21.15.32:0/4136003502 wait complete. 2022-08-01T23:03:19.579 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2022-08-01T23:03:19.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:03:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 Processor -- start 2022-08-01T23:03:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 -- start start 2022-08-01T23:03:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f32f8061a20 0x7f32f80b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32f812ba00 0x7f32f8146890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f8061e00 con 0x7f32f80b7080 2022-08-01T23:03:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f80b2e40 con 0x7f32f812ba00 2022-08-01T23:03:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.686+0000 7f32fd9af700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f812c090 con 0x7f32f8061a20 2022-08-01T23:03:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32f80b7080 0x7f32f812b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34146/0 (socket says 172.21.15.32:34146) 2022-08-01T23:03:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f67fc700 1 -- 172.21.15.32:0/2086973952 learned_addr learned my addr 172.21.15.32:0/2086973952 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f77fe700 1 --2- 172.21.15.32:0/2086973952 >> v2:172.21.15.32:3300/0 conn(0x7f32f812ba00 0x7f32f8146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f6ffd700 1 --2- 172.21.15.32:0/2086973952 >> v2:172.21.15.55:3300/0 conn(0x7f32f8061a20 0x7f32f80b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f5ffb700 1 -- 172.21.15.32:0/2086973952 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839671060 0 0) 0x7f32f8061e00 con 0x7f32f80b7080 2022-08-01T23:03:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f5ffb700 1 -- 172.21.15.32:0/2086973952 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d8003260 con 0x7f32f80b7080 2022-08-01T23:03:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f77fe700 1 -- 172.21.15.32:0/2086973952 >> v2:172.21.15.55:3300/0 conn(0x7f32f8061a20 msgr2=0x7f32f80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f77fe700 1 --2- 172.21.15.32:0/2086973952 >> v2:172.21.15.55:3300/0 conn(0x7f32f8061a20 0x7f32f80b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f77fe700 1 -- 172.21.15.32:0/2086973952 >> v1:172.21.15.32:6789/0 conn(0x7f32f80b7080 legacy=0x7f32f812b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f77fe700 1 -- 172.21.15.32:0/2086973952 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f812c330 con 0x7f32f812ba00 2022-08-01T23:03:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f6ffd700 1 --2- 172.21.15.32:0/2086973952 >> v2:172.21.15.55:3300/0 conn(0x7f32f8061a20 0x7f32f80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f77fe700 1 --2- 172.21.15.32:0/2086973952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f812ba00 0x7f32f8146890 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f32f0002f70 tx=0x7f32f0006df0).ready entity=mon.0 client_cookie=1ab61b2c688bfd6a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f5ffb700 1 -- 172.21.15.32:0/2086973952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32f000d570 con 0x7f32f812ba00 2022-08-01T23:03:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f5ffb700 1 -- 172.21.15.32:0/2086973952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0000e90 con 0x7f32f812ba00 2022-08-01T23:03:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32f5ffb700 1 -- 172.21.15.32:0/2086973952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32f000d8c0 con 0x7f32f812ba00 2022-08-01T23:03:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 -- 172.21.15.32:0/2086973952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f812ba00 msgr2=0x7f32f8146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 --2- 172.21.15.32:0/2086973952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f812ba00 0x7f32f8146890 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f32f0002f70 tx=0x7f32f0006df0).stop 2022-08-01T23:03:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 -- 172.21.15.32:0/2086973952 shutdown_connections 2022-08-01T23:03:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 --2- 172.21.15.32:0/2086973952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f812ba00 0x7f32f8146890 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 --2- 172.21.15.32:0/2086973952 >> v2:172.21.15.55:3300/0 conn(0x7f32f8061a20 0x7f32f80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 -- 172.21.15.32:0/2086973952 >> 172.21.15.32:0/2086973952 conn(0x7f32f8066f70 msgr2=0x7f32f8067380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 -- 172.21.15.32:0/2086973952 shutdown_connections 2022-08-01T23:03:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.690+0000 7f32fd9af700 1 -- 172.21.15.32:0/2086973952 wait complete. 2022-08-01T23:03:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 Processor -- start 2022-08-01T23:03:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 -- start start 2022-08-01T23:03:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32f8061a20 0x7f32f812a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 0x7f32f80aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32f812ba00 0x7f32f80af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8149510 con 0x7f32f80b7080 2022-08-01T23:03:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8142120 con 0x7f32f8061a20 2022-08-01T23:03:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80af960 con 0x7f32f812ba00 2022-08-01T23:03:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 0x7f32f80aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 0x7f32f80aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46610/0 (socket says 172.21.15.32:46610) 2022-08-01T23:03:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 -- 172.21.15.32:0/1916978295 learned_addr learned my addr 172.21.15.32:0/1916978295 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f6ffd700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32f8061a20 0x7f32f812a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f77fe700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32f812ba00 0x7f32f80af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 -- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32f812ba00 msgr2=0x7f32f80af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32f812ba00 0x7f32f80af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 -- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32f8061a20 msgr2=0x7f32f812a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32f8061a20 0x7f32f812a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 -- 172.21.15.32:0/1916978295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f000b180 con 0x7f32f80b7080 2022-08-01T23:03:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32f67fc700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 0x7f32f80aecd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f32e0001490 tx=0x7f32e0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32e0005c70 con 0x7f32f80b7080 2022-08-01T23:03:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e0005dd0 con 0x7f32f80b7080 2022-08-01T23:03:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f8149740 con 0x7f32f80b7080 2022-08-01T23:03:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32e0004a90 con 0x7f32f80b7080 2022-08-01T23:03:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.694+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f8152370 con 0x7f32f80b7080 2022-08-01T23:03:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.698+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f812c5d0 con 0x7f32f80b7080 2022-08-01T23:03:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.698+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f32e001b070 con 0x7f32f80b7080 2022-08-01T23:03:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.698+0000 7f32effff700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f32dc0539e0 0x7f32dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.698+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f32e006b9a0 con 0x7f32f80b7080 2022-08-01T23:03:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.698+0000 7f32f6ffd700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f32dc0539e0 0x7f32dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.698+0000 7f32f6ffd700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f32dc0539e0 0x7f32dc055ea0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f32e800c000 tx=0x7f32e8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.706+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f32e0042760 con 0x7f32f80b7080 2022-08-01T23:03:19.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:19.866+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f32f80574e0 con 0x7f32f80b7080 2022-08-01T23:03:20.586 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:03:20.582+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:03:20.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.598+0000 7f32effff700 1 -- 172.21.15.32:0/1916978295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f32e0045d80 con 0x7f32f80b7080 2022-08-01T23:03:20.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f32dc0539e0 msgr2=0x7f32dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f32dc0539e0 0x7f32dc055ea0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f32e800c000 tx=0x7f32e8008040).stop 2022-08-01T23:03:20.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 msgr2=0x7f32f80aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 0x7f32f80aecd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f32e0001490 tx=0x7f32e0007e00).stop 2022-08-01T23:03:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 shutdown_connections 2022-08-01T23:03:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f32dc0539e0 0x7f32dc055ea0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f32f812ba00 0x7f32f80af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f80b7080 0x7f32f80aecd0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 --2- 172.21.15.32:0/1916978295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32f8061a20 0x7f32f812a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 >> 172.21.15.32:0/1916978295 conn(0x7f32f8066f70 msgr2=0x7f32f80b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 shutdown_connections 2022-08-01T23:03:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.602+0000 7f32fd9af700 1 -- 172.21.15.32:0/1916978295 wait complete. 2022-08-01T23:03:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:03:20.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:20.629 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:03:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.722+0000 7ffbad1e2700 1 Processor -- start 2022-08-01T23:03:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.722+0000 7ffbad1e2700 1 -- start start 2022-08-01T23:03:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffba80af450 0x7ffba80af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:20.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffba80ad9e0 0x7ffba80a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:20.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80aff30 con 0x7ffba80ad4b0 2022-08-01T23:03:20.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba805c7a0 con 0x7ffba80af450 2022-08-01T23:03:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba8140990 con 0x7ffba80ad9e0 2022-08-01T23:03:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffba80af450 0x7ffba80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffba80af450 0x7ffba80af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46628/0 (socket says 172.21.15.32:46628) 2022-08-01T23:03:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 -- 172.21.15.32:0/1993393181 learned_addr learned my addr 172.21.15.32:0/1993393181 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 -- 172.21.15.32:0/1993393181 >> v2:172.21.15.55:3300/0 conn(0x7ffba80ad9e0 msgr2=0x7ffba80a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 --2- 172.21.15.32:0/1993393181 >> v2:172.21.15.55:3300/0 conn(0x7ffba80ad9e0 0x7ffba80a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T23:03:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 -- 172.21.15.32:0/1993393181 >> v1:172.21.15.55:6789/0 conn(0x7ffba80ad4b0 legacy=0x7ffba81432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 -- 172.21.15.32:0/1993393181 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba81439f0 con 0x7ffba80af450 2022-08-01T23:03:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba7fff700 1 --2- 172.21.15.32:0/1993393181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80af450 0x7ffba80af830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7ffba80ad900 tx=0x7ffb98006eb0).ready entity=mon.0 client_cookie=b949557c15f3f517 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba6ffd700 1 -- 172.21.15.32:0/1993393181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9800d5c0 con 0x7ffba80af450 2022-08-01T23:03:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba6ffd700 1 -- 172.21.15.32:0/1993393181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98000e90 con 0x7ffba80af450 2022-08-01T23:03:20.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffba6ffd700 1 -- 172.21.15.32:0/1993393181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9800d950 con 0x7ffba80af450 2022-08-01T23:03:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- 172.21.15.32:0/1993393181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80af450 msgr2=0x7ffba80af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/1993393181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80af450 0x7ffba80af830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7ffba80ad900 tx=0x7ffb98006eb0).stop 2022-08-01T23:03:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- 172.21.15.32:0/1993393181 shutdown_connections 2022-08-01T23:03:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/1993393181 >> v2:172.21.15.55:3300/0 conn(0x7ffba80ad9e0 0x7ffba80a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T23:03:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/1993393181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80af450 0x7ffba80af830 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- 172.21.15.32:0/1993393181 >> 172.21.15.32:0/1993393181 conn(0x7ffba80612d0 msgr2=0x7ffba8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- 172.21.15.32:0/1993393181 shutdown_connections 2022-08-01T23:03:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.726+0000 7ffbad1e2700 1 -- 172.21.15.32:0/1993393181 wait complete. 2022-08-01T23:03:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 Processor -- start 2022-08-01T23:03:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 -- start start 2022-08-01T23:03:20.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffba80ad4b0 0x7ffba8142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 0x7ffba8137d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffba80af450 0x7ffba8138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80abb50 con 0x7ffba80ad9e0 2022-08-01T23:03:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80abdf0 con 0x7ffba80af450 2022-08-01T23:03:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbad1e2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80ac090 con 0x7ffba80ad4b0 2022-08-01T23:03:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 0x7ffba8137d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 0x7ffba8137d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46636/0 (socket says 172.21.15.32:46636) 2022-08-01T23:03:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 -- 172.21.15.32:0/980986329 learned_addr learned my addr 172.21.15.32:0/980986329 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba7fff700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffba80ad4b0 0x7ffba8142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbac9e1700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffba80af450 0x7ffba8138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 -- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffba80ad4b0 msgr2=0x7ffba8142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffba80ad4b0 0x7ffba8142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 -- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffba80af450 msgr2=0x7ffba8138200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffba80af450 0x7ffba8138200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 -- 172.21.15.32:0/980986329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9800b1e0 con 0x7ffba80ad9e0 2022-08-01T23:03:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba7fff700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffba80ad4b0 0x7ffba8142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffbac9e1700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffba80af450 0x7ffba8138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba77fe700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 0x7ffba8137d10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7ffb9c002f90 tx=0x7ffb9c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.730+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9c013460 con 0x7ffba80ad9e0 2022-08-01T23:03:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c0135c0 con 0x7ffba80ad9e0 2022-08-01T23:03:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba80ac330 con 0x7ffba80ad9e0 2022-08-01T23:03:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9c013920 con 0x7ffba80ad9e0 2022-08-01T23:03:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80ac830 con 0x7ffba80ad9e0 2022-08-01T23:03:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba80574e0 con 0x7ffba80ad9e0 2022-08-01T23:03:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb9c01a060 con 0x7ffba80ad9e0 2022-08-01T23:03:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba57fa700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb88053b00 0x7ffb88055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ffb9c06ba10 con 0x7ffba80ad9e0 2022-08-01T23:03:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba7fff700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb88053b00 0x7ffb88055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.734+0000 7ffba7fff700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb88053b00 0x7ffb88055fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ffb9801a000 tx=0x7ffb98015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:20.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.742+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb9c042780 con 0x7ffba80ad9e0 2022-08-01T23:03:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.906+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffba8061b20 con 0x7ffba80ad9e0 2022-08-01T23:03:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.906+0000 7ffba57fa700 1 -- 172.21.15.32:0/980986329 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2132 (secure 0 0 0) 0x7ffb9c018030 con 0x7ffba80ad9e0 2022-08-01T23:03:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:20.912 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:08.469158+0000","modified":"2022-08-01T23:03:20.584170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T23:03:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb88053b00 msgr2=0x7ffb88055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb88053b00 0x7ffb88055fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ffb9801a000 tx=0x7ffb98015000).stop 2022-08-01T23:03:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 msgr2=0x7ffba8137d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 0x7ffba8137d10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7ffb9c002f90 tx=0x7ffb9c006df0).stop 2022-08-01T23:03:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 shutdown_connections 2022-08-01T23:03:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffb88053b00 0x7ffb88055fc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffba80af450 0x7ffba8138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffba80ad9e0 0x7ffba8137d10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 --2- 172.21.15.32:0/980986329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffba80ad4b0 0x7ffba8142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:20.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 >> 172.21.15.32:0/980986329 conn(0x7ffba80612d0 msgr2=0x7ffba8138d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:20.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 shutdown_connections 2022-08-01T23:03:20.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:20.910+0000 7ffbad1e2700 1 -- 172.21.15.32:0/980986329 wait complete. 2022-08-01T23:03:20.916 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2022-08-01T23:03:20.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:03:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.018+0000 7f3a946ad700 1 Processor -- start 2022-08-01T23:03:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.018+0000 7f3a946ad700 1 -- start start 2022-08-01T23:03:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a946ad700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a8c0b2050 0x7f3a8c0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a946ad700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c0b05b0 con 0x7f3a8c0b0140 2022-08-01T23:03:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a946ad700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c0a7960 con 0x7f3a8c0b2430 2022-08-01T23:03:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a946ad700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c0b07e0 con 0x7f3a8c0b2050 2022-08-01T23:03:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a8c0b0140 0x7f3a8c0a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34184/0 (socket says 172.21.15.32:34184) 2022-08-01T23:03:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 -- 172.21.15.32:0/141941976 learned_addr learned my addr 172.21.15.32:0/141941976 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a92449700 1 --2- 172.21.15.32:0/141941976 >> v2:172.21.15.55:3300/0 conn(0x7f3a8c0b2050 0x7f3a8c0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514777033 0 0) 0x7f3a8c0b05b0 con 0x7f3a8c0b0140 2022-08-01T23:03:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a74003260 con 0x7f3a8c0b0140 2022-08-01T23:03:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251122577 0 0) 0x7f3a74003260 con 0x7f3a8c0b0140 2022-08-01T23:03:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 >> v2:172.21.15.55:3300/0 conn(0x7f3a8c0b2050 msgr2=0x7f3a8c0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 --2- 172.21.15.32:0/141941976 >> v2:172.21.15.55:3300/0 conn(0x7f3a8c0b2050 0x7f3a8c0b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 >> v1:172.21.15.55:6789/0 conn(0x7f3a8c0b2430 legacy=0x7f3a8c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0b0a80 con 0x7f3a8c0b0140 2022-08-01T23:03:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f3a80001660 con 0x7f3a8c0b0140 2022-08-01T23:03:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 >> v1:172.21.15.32:6789/0 conn(0x7f3a8c0b0140 legacy=0x7f3a8c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74003750 0x7f3a74003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a74004500 0x7f3a74004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a74004e70 0x7f3a7400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0b07e0 con 0x7f3a74004500 2022-08-01T23:03:21.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7400bd20 con 0x7f3a74003750 2022-08-01T23:03:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7400bee0 con 0x7f3a74004e70 2022-08-01T23:03:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a74004500 0x7f3a74004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a92449700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74003750 0x7f3a74003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a92c4a700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a74004e70 0x7f3a7400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 -- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a74004e70 msgr2=0x7f3a7400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a74004e70 0x7f3a7400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 -- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74003750 msgr2=0x7f3a74003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74003750 0x7f3a74003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 -- 172.21.15.32:0/141941976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74003260 con 0x7f3a74004500 2022-08-01T23:03:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a92c4a700 1 -- 172.21.15.32:0/141941976 reap_dead start 2022-08-01T23:03:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91c48700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a74004500 0x7f3a74004980 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f3a80001370 tx=0x7f3a8c0b0c10).ready entity=mon.0 client_cookie=32007c5f33c9470c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a800070d0 con 0x7f3a74004500 2022-08-01T23:03:21.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a80007230 con 0x7f3a74004500 2022-08-01T23:03:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.022+0000 7f3a91447700 1 -- 172.21.15.32:0/141941976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8000c8b0 con 0x7f3a74004500 2022-08-01T23:03:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- 172.21.15.32:0/141941976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a74004500 msgr2=0x7f3a74004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a74004500 0x7f3a74004980 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f3a80001370 tx=0x7f3a8c0b0c10).stop 2022-08-01T23:03:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- 172.21.15.32:0/141941976 shutdown_connections 2022-08-01T23:03:21.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 --2- 172.21.15.32:0/141941976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a74004500 0x7f3a74004980 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- 172.21.15.32:0/141941976 >> 172.21.15.32:0/141941976 conn(0x7f3a8c05f840 msgr2=0x7f3a8c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- 172.21.15.32:0/141941976 shutdown_connections 2022-08-01T23:03:21.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- 172.21.15.32:0/141941976 wait complete. 2022-08-01T23:03:21.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 Processor -- start 2022-08-01T23:03:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- start start 2022-08-01T23:03:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74004500 0x7f3a8c148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 0x7f3a8c0aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a8c138520 0x7f3a8c1389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c148430 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c148620 con 0x7f3a74004500 2022-08-01T23:03:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a946ad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c14d790 con 0x7f3a8c138520 2022-08-01T23:03:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a91c48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 0x7f3a8c0aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a91c48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 0x7f3a8c0aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46656/0 (socket says 172.21.15.32:46656) 2022-08-01T23:03:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a91c48700 1 -- 172.21.15.32:0/1472337319 learned_addr learned my addr 172.21.15.32:0/1472337319 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a92449700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74004500 0x7f3a8c148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.026+0000 7f3a92c4a700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a8c138520 0x7f3a8c1389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a91c48700 1 -- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a8c138520 msgr2=0x7f3a8c1389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a91c48700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a8c138520 0x7f3a8c1389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a91c48700 1 -- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74004500 msgr2=0x7f3a8c148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a91c48700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74004500 0x7f3a8c148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a91c48700 1 -- 172.21.15.32:0/1472337319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0aa2a0 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a92c4a700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a8c138520 0x7f3a8c1389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:21.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a91c48700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 0x7f3a8c0aaa30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f3a80001370 tx=0x7f3a80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8000ced0 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000c6d0 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c14dc30 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a80004710 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c0aaf20 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c14d950 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a80010070 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a7f7fe700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a6c053a70 0x7f3a6c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f3a8006c9f0 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.030+0000 7f3a92449700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a6c053a70 0x7f3a6c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.034+0000 7f3a92449700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a6c053a70 0x7f3a6c055f30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3a8800b000 tx=0x7f3a88006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:21.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.038+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a8006ee60 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.202+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a8c14b380 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.598+0000 7f3a7f7fe700 1 -- 172.21.15.32:0/1472337319 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a80024070 con 0x7f3a8c0aa5b0 2022-08-01T23:03:21.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a6c053a70 msgr2=0x7f3a6c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a6c053a70 0x7f3a6c055f30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3a8800b000 tx=0x7f3a88006df0).stop 2022-08-01T23:03:21.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 msgr2=0x7f3a8c0aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 0x7f3a8c0aaa30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f3a80001370 tx=0x7f3a80015000).stop 2022-08-01T23:03:21.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 shutdown_connections 2022-08-01T23:03:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a6c053a70 0x7f3a6c055f30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a8c138520 0x7f3a8c1389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a8c0aa5b0 0x7f3a8c0aaa30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 --2- 172.21.15.32:0/1472337319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a74004500 0x7f3a8c148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 >> 172.21.15.32:0/1472337319 conn(0x7f3a8c05f840 msgr2=0x7f3a8c0a8260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 shutdown_connections 2022-08-01T23:03:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.602+0000 7f3a946ad700 1 -- 172.21.15.32:0/1472337319 wait complete. 2022-08-01T23:03:21.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:03:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.730+0000 7f9add151700 1 Processor -- start 2022-08-01T23:03:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.730+0000 7f9add151700 1 -- start start 2022-08-01T23:03:21.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9add151700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ad8060f00 0x7f9ad80b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9add151700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ad8145250 0x7f9ad80ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9add151700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80ace20 con 0x7f9ad80b6750 2022-08-01T23:03:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9add151700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80ae6f0 con 0x7f9ad8060f00 2022-08-01T23:03:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9add151700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad8144870 con 0x7f9ad8145250 2022-08-01T23:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ad8060f00 0x7f9ad80b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ad8060f00 0x7f9ad80b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46666/0 (socket says 172.21.15.32:46666) 2022-08-01T23:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 -- 172.21.15.32:0/2681888590 learned_addr learned my addr 172.21.15.32:0/2681888590 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9ad759e700 1 --2- 172.21.15.32:0/2681888590 >> v2:172.21.15.55:3300/0 conn(0x7f9ad8145250 0x7f9ad80ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 -- 172.21.15.32:0/2681888590 >> v2:172.21.15.55:3300/0 conn(0x7f9ad8145250 msgr2=0x7f9ad80ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 --2- 172.21.15.32:0/2681888590 >> v2:172.21.15.55:3300/0 conn(0x7f9ad8145250 0x7f9ad80ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 -- 172.21.15.32:0/2681888590 >> v1:172.21.15.55:6789/0 conn(0x7f9ad80b6750 legacy=0x7f9ad8141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 -- 172.21.15.32:0/2681888590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8144b10 con 0x7f9ad8060f00 2022-08-01T23:03:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9acffff700 1 --2- 172.21.15.32:0/2681888590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8060f00 0x7f9ad80b6b30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f9ac0002f70 tx=0x7f9ac0006eb0).ready entity=mon.0 client_cookie=7cb4734b632abe7c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9ad659c700 1 -- 172.21.15.32:0/2681888590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac000d5c0 con 0x7f9ad8060f00 2022-08-01T23:03:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9ad659c700 1 -- 172.21.15.32:0/2681888590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0000e90 con 0x7f9ad8060f00 2022-08-01T23:03:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.734+0000 7f9ad659c700 1 -- 172.21.15.32:0/2681888590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ac000d950 con 0x7f9ad8060f00 2022-08-01T23:03:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- 172.21.15.32:0/2681888590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8060f00 msgr2=0x7f9ad80b6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 --2- 172.21.15.32:0/2681888590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8060f00 0x7f9ad80b6b30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f9ac0002f70 tx=0x7f9ac0006eb0).stop 2022-08-01T23:03:21.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- 172.21.15.32:0/2681888590 shutdown_connections 2022-08-01T23:03:21.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 --2- 172.21.15.32:0/2681888590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad8060f00 0x7f9ad80b6b30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 --2- 172.21.15.32:0/2681888590 >> v2:172.21.15.55:3300/0 conn(0x7f9ad8145250 0x7f9ad80ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- 172.21.15.32:0/2681888590 >> 172.21.15.32:0/2681888590 conn(0x7f9ad80b3610 msgr2=0x7f9ad80b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:21.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- 172.21.15.32:0/2681888590 shutdown_connections 2022-08-01T23:03:21.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- 172.21.15.32:0/2681888590 wait complete. 2022-08-01T23:03:21.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 Processor -- start 2022-08-01T23:03:21.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- start start 2022-08-01T23:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ad8060f00 0x7f9ad8140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad80b6750 0x7f9ad80b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ad8145250 0x7f9ad80b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80af680 con 0x7f9ad80b6750 2022-08-01T23:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80afc70 con 0x7f9ad8145250 2022-08-01T23:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.738+0000 7f9add151700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80af9d0 con 0x7f9ad8060f00 2022-08-01T23:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9acffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad80b6750 0x7f9ad80b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ad8145250 0x7f9ad80b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9acffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad80b6750 0x7f9ad80b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46670/0 (socket says 172.21.15.32:46670) 2022-08-01T23:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9acffff700 1 -- 172.21.15.32:0/3397491254 learned_addr learned my addr 172.21.15.32:0/3397491254 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad6d9d700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ad8060f00 0x7f9ad8140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 -- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ad8060f00 msgr2=0x7f9ad8140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ad8060f00 0x7f9ad8140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 -- 172.21.15.32:0/3397491254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad80b6750 msgr2=0x7f9ad80b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad80b6750 0x7f9ad80b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 -- 172.21.15.32:0/3397491254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac000b1e0 con 0x7f9ad8145250 2022-08-01T23:03:21.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad759e700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ad8145250 0x7f9ad80b14a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f9ad000c000 tx=0x7f9ad00091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:21.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad0012600 con 0x7f9ad8145250 2022-08-01T23:03:21.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad80b1e30 con 0x7f9ad8145250 2022-08-01T23:03:21.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0012760 con 0x7f9ad8145250 2022-08-01T23:03:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80a9d90 con 0x7f9ad8145250 2022-08-01T23:03:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad0012ac0 con 0x7f9ad8145250 2022-08-01T23:03:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.742+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80b1c90 con 0x7f9ad8145250 2022-08-01T23:03:21.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.746+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ad0022070 con 0x7f9ad8145250 2022-08-01T23:03:21.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.746+0000 7f9ad4d99700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9abc053980 0x7f9abc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:21.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.746+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9ad006c720 con 0x7f9ad8145250 2022-08-01T23:03:21.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.746+0000 7f9ad6d9d700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9abc053980 0x7f9abc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:21.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.746+0000 7f9ad6d9d700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9abc053980 0x7f9abc055e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f9ac800d000 tx=0x7f9ac8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.754+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ad80b1c90 con 0x7f9ad8145250 2022-08-01T23:03:21.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:21.886+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ad80b1c90 con 0x7f9ad8145250 2022-08-01T23:03:22.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.610+0000 7f9ad4d99700 1 -- 172.21.15.32:0/3397491254 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 185+0+0 (secure 0 0 0) 0x7f9ad80b1c90 con 0x7f9ad8145250 2022-08-01T23:03:22.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9abc053980 msgr2=0x7f9abc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9abc053980 0x7f9abc055e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f9ac800d000 tx=0x7f9ac8009040).stop 2022-08-01T23:03:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ad8145250 msgr2=0x7f9ad80b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ad8145250 0x7f9ad80b14a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f9ad000c000 tx=0x7f9ad00091d0).stop 2022-08-01T23:03:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 shutdown_connections 2022-08-01T23:03:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9abc053980 0x7f9abc055e40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ad8145250 0x7f9ad80b14a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad80b6750 0x7f9ad80b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 --2- 172.21.15.32:0/3397491254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9ad8060f00 0x7f9ad8140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 >> 172.21.15.32:0/3397491254 conn(0x7f9ad80b3610 msgr2=0x7f9ad80b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 shutdown_connections 2022-08-01T23:03:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.614+0000 7f9add151700 1 -- 172.21.15.32:0/3397491254 wait complete. 2022-08-01T23:03:22.621 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:03:22.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:03:22.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 Processor -- start 2022-08-01T23:03:22.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 -- start start 2022-08-01T23:03:22.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa40b4550 0x7f3aa40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:22.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3aa40b6af0 0x7f3aa412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:22.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa4144240 con 0x7f3aa40b6710 2022-08-01T23:03:22.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa4143d00 con 0x7f3aa40b4550 2022-08-01T23:03:22.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aab94f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa4143fa0 con 0x7f3aa40b6af0 2022-08-01T23:03:22.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aa8eea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa40b4550 0x7f3aa40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:22.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aa8eea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa40b4550 0x7f3aa40b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46680/0 (socket says 172.21.15.32:46680) 2022-08-01T23:03:22.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aa8eea700 1 -- 172.21.15.32:0/3725075324 learned_addr learned my addr 172.21.15.32:0/3725075324 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:22.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.738+0000 7f3aa9eec700 1 --2- 172.21.15.32:0/3725075324 >> v2:172.21.15.55:3300/0 conn(0x7f3aa40b6af0 0x7f3aa412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:22.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aa8eea700 1 -- 172.21.15.32:0/3725075324 >> v2:172.21.15.55:3300/0 conn(0x7f3aa40b6af0 msgr2=0x7f3aa412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:22.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aa8eea700 1 --2- 172.21.15.32:0/3725075324 >> v2:172.21.15.55:3300/0 conn(0x7f3aa40b6af0 0x7f3aa412bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aa8eea700 1 -- 172.21.15.32:0/3725075324 >> v1:172.21.15.32:6789/0 conn(0x7f3aa40b6710 legacy=0x7f3aa4061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:22.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3a9bfff700 1 -- 172.21.15.32:0/3725075324 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196751525 0 0) 0x7f3aa4144240 con 0x7f3aa40b6710 2022-08-01T23:03:22.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aa8eea700 1 -- 172.21.15.32:0/3725075324 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4061780 con 0x7f3aa40b4550 2022-08-01T23:03:22.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aa9eec700 1 --2- 172.21.15.32:0/3725075324 >> v2:172.21.15.55:3300/0 conn(0x7f3aa40b6af0 0x7f3aa412bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:22.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aa8eea700 1 --2- 172.21.15.32:0/3725075324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa40b5d80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f3aa41432b0 tx=0x7f3aa0006eb0).ready entity=mon.0 client_cookie=5c0811fbbb8a8bd8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:22.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3a9bfff700 1 -- 172.21.15.32:0/3725075324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa000d610 con 0x7f3aa40b4550 2022-08-01T23:03:22.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3a9bfff700 1 -- 172.21.15.32:0/3725075324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0000e90 con 0x7f3aa40b4550 2022-08-01T23:03:22.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3a9bfff700 1 -- 172.21.15.32:0/3725075324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa000d960 con 0x7f3aa40b4550 2022-08-01T23:03:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- 172.21.15.32:0/3725075324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 msgr2=0x7f3aa40b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 --2- 172.21.15.32:0/3725075324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa40b5d80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f3aa41432b0 tx=0x7f3aa0006eb0).stop 2022-08-01T23:03:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- 172.21.15.32:0/3725075324 shutdown_connections 2022-08-01T23:03:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 --2- 172.21.15.32:0/3725075324 >> v2:172.21.15.55:3300/0 conn(0x7f3aa40b6af0 0x7f3aa412bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 --2- 172.21.15.32:0/3725075324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa40b5d80 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- 172.21.15.32:0/3725075324 >> 172.21.15.32:0/3725075324 conn(0x7f3aa405c620 msgr2=0x7f3aa405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- 172.21.15.32:0/3725075324 shutdown_connections 2022-08-01T23:03:22.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- 172.21.15.32:0/3725075324 wait complete. 2022-08-01T23:03:22.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 Processor -- start 2022-08-01T23:03:22.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- start start 2022-08-01T23:03:22.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa4146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3aa40b6710 0x7f3aa4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3aa40b6af0 0x7f3aa40b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4147c70 con 0x7f3aa40b4550 2022-08-01T23:03:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4146650 con 0x7f3aa40b6710 2022-08-01T23:03:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.742+0000 7f3aab94f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41474e0 con 0x7f3aa40b6af0 2022-08-01T23:03:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa4146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa4146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46684/0 (socket says 172.21.15.32:46684) 2022-08-01T23:03:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 -- 172.21.15.32:0/2137702683 learned_addr learned my addr 172.21.15.32:0/2137702683 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa9eec700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3aa40b6af0 0x7f3aa40b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa8eea700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3aa40b6710 0x7f3aa4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 -- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3aa40b6af0 msgr2=0x7f3aa40b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3aa40b6af0 0x7f3aa40b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 -- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3aa40b6710 msgr2=0x7f3aa4152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3aa40b6710 0x7f3aa4152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 -- 172.21.15.32:0/2137702683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa000b1e0 con 0x7f3aa40b4550 2022-08-01T23:03:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa9eec700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3aa40b6af0 0x7f3aa40b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa8eea700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3aa40b6710 0x7f3aa4152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa96eb700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa4146ff0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f3aa4147670 tx=0x7f3a94007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a9400ce10 con 0x7f3aa40b4550 2022-08-01T23:03:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9400f470 con 0x7f3aa40b4550 2022-08-01T23:03:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa4147ea0 con 0x7f3aa40b4550 2022-08-01T23:03:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a9400f7d0 con 0x7f3aa40b4550 2022-08-01T23:03:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa40b34d0 con 0x7f3aa40b4550 2022-08-01T23:03:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a94019030 con 0x7f3aa40b4550 2022-08-01T23:03:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa4148630 con 0x7f3aa40b4550 2022-08-01T23:03:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3a9a7fc700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a8c053b30 0x7f3a8c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f3a9406b260 con 0x7f3aa40b4550 2022-08-01T23:03:22.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.746+0000 7f3aa8eea700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a8c053b30 0x7f3a8c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:22.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.750+0000 7f3aa8eea700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a8c053b30 0x7f3a8c055ff0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f3aa0017000 tx=0x7f3aa0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:22.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.754+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a940425a0 con 0x7f3aa40b4550 2022-08-01T23:03:22.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:22.882+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3aa4061a20 con 0x7f3aa40b4550 2022-08-01T23:03:23.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.622+0000 7f3a9a7fc700 1 -- 172.21.15.32:0/2137702683 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3a940457f0 con 0x7f3aa40b4550 2022-08-01T23:03:23.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a8c053b30 msgr2=0x7f3a8c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a8c053b30 0x7f3a8c055ff0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f3aa0017000 tx=0x7f3aa0015000).stop 2022-08-01T23:03:23.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 msgr2=0x7f3aa4146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa4146ff0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f3aa4147670 tx=0x7f3a94007ac0).stop 2022-08-01T23:03:23.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 shutdown_connections 2022-08-01T23:03:23.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a8c053b30 0x7f3a8c055ff0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3aa40b6af0 0x7f3aa40b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3aa40b6710 0x7f3aa4152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 --2- 172.21.15.32:0/2137702683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa40b4550 0x7f3aa4146ff0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.630+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 >> 172.21.15.32:0/2137702683 conn(0x7f3aa405c620 msgr2=0x7f3aa4062100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:23.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.634+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 shutdown_connections 2022-08-01T23:03:23.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.634+0000 7f3aab94f700 1 -- 172.21.15.32:0/2137702683 wait complete. 2022-08-01T23:03:23.639 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:03:23.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:03:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.754+0000 7fd7eb071700 1 Processor -- start 2022-08-01T23:03:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.754+0000 7fd7eb071700 1 -- start start 2022-08-01T23:03:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7e40b3340 0x7fd7e413e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40ada80 con 0x7fd7e40b3720 2022-08-01T23:03:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40b0090 con 0x7fd7e4145220 2022-08-01T23:03:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e41456f0 con 0x7fd7e40b3340 2022-08-01T23:03:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7e8e0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7e40b3340 0x7fd7e413e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7dbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd7e40b3720 0x7fd7e4141590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34236/0 (socket says 172.21.15.32:34236) 2022-08-01T23:03:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7dbfff700 1 -- 172.21.15.32:0/187625363 learned_addr learned my addr 172.21.15.32:0/187625363 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7db7fe700 1 -- 172.21.15.32:0/187625363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880702139 0 0) 0x7fd7e40ada80 con 0x7fd7e40b3720 2022-08-01T23:03:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7db7fe700 1 -- 172.21.15.32:0/187625363 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7d0003260 con 0x7fd7e40b3720 2022-08-01T23:03:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/187625363 >> v1:172.21.15.55:6789/0 conn(0x7fd7e4145220 legacy=0x7fd7e414bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/187625363 >> v1:172.21.15.32:6789/0 conn(0x7fd7e40b3720 legacy=0x7fd7e4141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/187625363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40b02f0 con 0x7fd7e40b3340 2022-08-01T23:03:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7e8e0d700 1 --2- 172.21.15.32:0/187625363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e413e6f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7fd7e4141320 tx=0x7fd7e0006df0).ready entity=mon.0 client_cookie=62221d0752b8a5ed server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7db7fe700 1 -- 172.21.15.32:0/187625363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e000d610 con 0x7fd7e40b3340 2022-08-01T23:03:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7db7fe700 1 -- 172.21.15.32:0/187625363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e0000e90 con 0x7fd7e40b3340 2022-08-01T23:03:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7db7fe700 1 -- 172.21.15.32:0/187625363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e000d960 con 0x7fd7e40b3340 2022-08-01T23:03:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- 172.21.15.32:0/187625363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 msgr2=0x7fd7e413e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 --2- 172.21.15.32:0/187625363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e413e6f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7fd7e4141320 tx=0x7fd7e0006df0).stop 2022-08-01T23:03:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- 172.21.15.32:0/187625363 shutdown_connections 2022-08-01T23:03:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 --2- 172.21.15.32:0/187625363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e413e6f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- 172.21.15.32:0/187625363 >> 172.21.15.32:0/187625363 conn(0x7fd7e4060950 msgr2=0x7fd7e4060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- 172.21.15.32:0/187625363 shutdown_connections 2022-08-01T23:03:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.758+0000 7fd7eb071700 1 -- 172.21.15.32:0/187625363 wait complete. 2022-08-01T23:03:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 Processor -- start 2022-08-01T23:03:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 -- start start 2022-08-01T23:03:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e4143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7e40b3720 0x7fd7e40adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd7e4145220 0x7fd7e40ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40af210 con 0x7fd7e40b3340 2022-08-01T23:03:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40afa60 con 0x7fd7e40b3720 2022-08-01T23:03:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40671c0 con 0x7fd7e4145220 2022-08-01T23:03:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e4143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e4143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46698/0 (socket says 172.21.15.32:46698) 2022-08-01T23:03:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/179418943 learned_addr learned my addr 172.21.15.32:0/179418943 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7dbfff700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7e40b3720 0x7fd7e40adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e960e700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd7e4145220 0x7fd7e40ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd7e4145220 msgr2=0x7fd7e40ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd7e4145220 0x7fd7e40ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7e40b3720 msgr2=0x7fd7e40adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7e40b3720 0x7fd7e40adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 -- 172.21.15.32:0/179418943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e000b180 con 0x7fd7e40b3340 2022-08-01T23:03:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7e8e0d700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e4143940 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fd7e4141320 tx=0x7fd7e0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e0013de0 con 0x7fd7e40b3340 2022-08-01T23:03:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000d290 con 0x7fd7e40b3340 2022-08-01T23:03:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e40af470 con 0x7fd7e40b3340 2022-08-01T23:03:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7e001e5f0 con 0x7fd7e40b3340 2022-08-01T23:03:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.762+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4144040 con 0x7fd7e40b3340 2022-08-01T23:03:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.766+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e40574e0 con 0x7fd7e40b3340 2022-08-01T23:03:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.766+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd7e0025030 con 0x7fd7e40b3340 2022-08-01T23:03:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.766+0000 7fd7d9ffb700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd7c4053a70 0x7fd7c4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.766+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd7e006c970 con 0x7fd7e40b3340 2022-08-01T23:03:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.766+0000 7fd7dbfff700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd7c4053a70 0x7fd7c4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.766+0000 7fd7dbfff700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd7c4053a70 0x7fd7c4055f30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fd7d400d000 tx=0x7fd7d4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.770+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7e0043a10 con 0x7fd7e40b3340 2022-08-01T23:03:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.898+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd7e40a9260 con 0x7fd7e40b3340 2022-08-01T23:03:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7d9ffb700 1 -- 172.21.15.32:0/179418943 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7e00230a0 con 0x7fd7e40b3340 2022-08-01T23:03:23.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd7c4053a70 msgr2=0x7fd7c4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd7c4053a70 0x7fd7c4055f30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fd7d400d000 tx=0x7fd7d4009040).stop 2022-08-01T23:03:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 msgr2=0x7fd7e4143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e4143940 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fd7e4141320 tx=0x7fd7e0014000).stop 2022-08-01T23:03:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 shutdown_connections 2022-08-01T23:03:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd7c4053a70 0x7fd7c4055f30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd7e4145220 0x7fd7e40ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7e40b3720 0x7fd7e40adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 --2- 172.21.15.32:0/179418943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e40b3340 0x7fd7e4143940 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 >> 172.21.15.32:0/179418943 conn(0x7fd7e4060950 msgr2=0x7fd7e413ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 shutdown_connections 2022-08-01T23:03:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:23.906+0000 7fd7eb071700 1 -- 172.21.15.32:0/179418943 wait complete. 2022-08-01T23:03:23.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-08-01T23:03:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e359e700 1 Processor -- start 2022-08-01T23:03:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e359e700 1 -- start start 2022-08-01T23:03:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13e40a8c50 0x7f13e4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e40a7250 con 0x7f13e40b2710 2022-08-01T23:03:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e40a7480 con 0x7f13e40b01a0 2022-08-01T23:03:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e40a7720 con 0x7f13e40a8c50 2022-08-01T23:03:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f13e40b2710 0x7f13e414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34248/0 (socket says 172.21.15.32:34248) 2022-08-01T23:03:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e259c700 1 -- 172.21.15.32:0/2634151459 learned_addr learned my addr 172.21.15.32:0/2634151459 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/2634151459 >> v2:172.21.15.55:3300/0 conn(0x7f13e40a8c50 0x7f13e4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7176051 0 0) 0x7f13e40a7250 con 0x7f13e40b2710 2022-08-01T23:03:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c8003260 con 0x7f13e40b2710 2022-08-01T23:03:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615421748 0 0) 0x7f13e40a7480 con 0x7f13e40b01a0 2022-08-01T23:03:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e40a7250 con 0x7f13e40b01a0 2022-08-01T23:03:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106665806 0 0) 0x7f13c8003260 con 0x7f13e40b2710 2022-08-01T23:03:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 >> v2:172.21.15.55:3300/0 conn(0x7f13e40a8c50 msgr2=0x7f13e4152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 --2- 172.21.15.32:0/2634151459 >> v2:172.21.15.55:3300/0 conn(0x7f13e40a8c50 0x7f13e4152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 >> v1:172.21.15.55:6789/0 conn(0x7f13e40b01a0 legacy=0x7f13e40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40a79c0 con 0x7f13e40b2710 2022-08-01T23:03:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f13d8001660 con 0x7f13e40b2710 2022-08-01T23:03:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/2634151459 >> v2:172.21.15.55:3300/0 conn(0x7f13e40a8c50 0x7f13e4152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 >> v1:172.21.15.32:6789/0 conn(0x7f13e40b2710 legacy=0x7f13e414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.022+0000 7f13e159a700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13c80038e0 0x7f13c8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13c8004500 0x7f13c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13c800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40a7720 con 0x7f13c8004e70 2022-08-01T23:03:24.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c800bcf0 con 0x7f13c8004500 2022-08-01T23:03:24.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c800beb0 con 0x7f13c80038e0 2022-08-01T23:03:24.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13c800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e1d9b700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13c8004500 0x7f13c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e259c700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13c80038e0 0x7f13c8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 -- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13c80038e0 msgr2=0x7f13c8003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13c80038e0 0x7f13c8003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 -- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13c8004500 msgr2=0x7f13c8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13c8004500 0x7f13c8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 -- 172.21.15.32:0/2634151459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8003260 con 0x7f13c8004e70 2022-08-01T23:03:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e1d9b700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13c8004500 0x7f13c8004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 -- 172.21.15.32:0/2634151459 reap_dead start 2022-08-01T23:03:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13c800b750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f13dc00c000 tx=0x7f13dc006df0).ready entity=mon.0 client_cookie=8608cfc740e1398d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13dc00e800 con 0x7f13c8004e70 2022-08-01T23:03:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc00e960 con 0x7f13c8004e70 2022-08-01T23:03:24.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e159a700 1 -- 172.21.15.32:0/2634151459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13dc00ecc0 con 0x7f13c8004e70 2022-08-01T23:03:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 -- 172.21.15.32:0/2634151459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 msgr2=0x7f13c800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13c800b750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f13dc00c000 tx=0x7f13dc006df0).stop 2022-08-01T23:03:24.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 -- 172.21.15.32:0/2634151459 shutdown_connections 2022-08-01T23:03:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 --2- 172.21.15.32:0/2634151459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13c800b750 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 -- 172.21.15.32:0/2634151459 >> 172.21.15.32:0/2634151459 conn(0x7f13e4060310 msgr2=0x7f13e40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 -- 172.21.15.32:0/2634151459 shutdown_connections 2022-08-01T23:03:24.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.026+0000 7f13e359e700 1 -- 172.21.15.32:0/2634151459 wait complete. 2022-08-01T23:03:24.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 Processor -- start 2022-08-01T23:03:24.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- start start 2022-08-01T23:03:24.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13e414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13e4064c80 0x7f13e40672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13e40663b0 0x7f13e4145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40677a0 con 0x7f13c8004e70 2022-08-01T23:03:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e4063b20 con 0x7f13e4064c80 2022-08-01T23:03:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40ac420 con 0x7f13e40663b0 2022-08-01T23:03:24.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13e414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13e414abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46718/0 (socket says 172.21.15.32:46718) 2022-08-01T23:03:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 -- 172.21.15.32:0/665155898 learned_addr learned my addr 172.21.15.32:0/665155898 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e1d9b700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13e4064c80 0x7f13e40672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13e40663b0 0x7f13e4145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 -- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13e40663b0 msgr2=0x7f13e4145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13e40663b0 0x7f13e4145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 -- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13e4064c80 msgr2=0x7f13e40672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13e4064c80 0x7f13e40672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 -- 172.21.15.32:0/665155898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40ac6c0 con 0x7f13c8004e70 2022-08-01T23:03:24.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e2d9d700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13e40663b0 0x7f13e4145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:24.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e1d9b700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13e4064c80 0x7f13e40672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:24.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e259c700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13e414abd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f13d800d000 tx=0x7f13e40a7b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d80116b0 con 0x7f13c8004e70 2022-08-01T23:03:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8011810 con 0x7f13c8004e70 2022-08-01T23:03:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d8011b70 con 0x7f13c8004e70 2022-08-01T23:03:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e40ab3b0 con 0x7f13c8004e70 2022-08-01T23:03:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e40abec0 con 0x7f13c8004e70 2022-08-01T23:03:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.030+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e40574e0 con 0x7f13c8004e70 2022-08-01T23:03:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.034+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f13d8011cd0 con 0x7f13c8004e70 2022-08-01T23:03:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.034+0000 7f13d37fe700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13c0053aa0 0x7f13c0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.034+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f13d806b8a0 con 0x7f13c8004e70 2022-08-01T23:03:24.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.034+0000 7f13e1d9b700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13c0053aa0 0x7f13c0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.034+0000 7f13e1d9b700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13c0053aa0 0x7f13c0055f60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f13d4003000 tx=0x7f13d4007d80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:24.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.038+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13d8042940 con 0x7f13c8004e70 2022-08-01T23:03:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.206+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f13e40670b0 con 0x7f13c8004e70 2022-08-01T23:03:24.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:24.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:24.506 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:24.508 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.626+0000 7f13d37fe700 1 -- 172.21.15.32:0/665155898 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v335) v1 ==== 138+0+0 (secure 0 0 0) 0x7f13d8019020 con 0x7f13c8004e70 2022-08-01T23:03:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.626+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13c0053aa0 msgr2=0x7f13c0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13c0053aa0 0x7f13c0055f60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f13d4003000 tx=0x7f13d4007d80).stop 2022-08-01T23:03:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 msgr2=0x7f13e414abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13e414abd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f13d800d000 tx=0x7f13e40a7b50).stop 2022-08-01T23:03:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 shutdown_connections 2022-08-01T23:03:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f13c0053aa0 0x7f13c0055f60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13e40663b0 0x7f13e4145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13e4064c80 0x7f13e40672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 --2- 172.21.15.32:0/665155898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13c8004e70 0x7f13e414abd0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 >> 172.21.15.32:0/665155898 conn(0x7f13e4060310 msgr2=0x7f13e4149af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:24.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 shutdown_connections 2022-08-01T23:03:24.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.630+0000 7f13e359e700 1 -- 172.21.15.32:0/665155898 wait complete. 2022-08-01T23:03:24.651 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:03:24.652 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:03:24.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-08-01T23:03:24.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 Processor -- start 2022-08-01T23:03:24.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 -- start start 2022-08-01T23:03:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45b80b2aa0 0x7f45b80b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45b8140c10 0x7f45b80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b81467f0 con 0x7f45b80b6510 2022-08-01T23:03:24.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b8062140 con 0x7f45b8140c10 2022-08-01T23:03:24.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bd2ef700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80656b0 con 0x7f45b80b2aa0 2022-08-01T23:03:24.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45b80b6510 0x7f45b8140500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34270/0 (socket says 172.21.15.32:34270) 2022-08-01T23:03:24.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b7fff700 1 -- 172.21.15.32:0/1533969520 learned_addr learned my addr 172.21.15.32:0/1533969520 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:24.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45bcaee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45b8140c10 0x7f45b80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b77fe700 1 --2- 172.21.15.32:0/1533969520 >> v2:172.21.15.55:3300/0 conn(0x7f45b80b2aa0 0x7f45b80b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b6ffd700 1 -- 172.21.15.32:0/1533969520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822593628 0 0) 0x7f45b81467f0 con 0x7f45b80b6510 2022-08-01T23:03:24.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b6ffd700 1 -- 172.21.15.32:0/1533969520 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4598003260 con 0x7f45b80b6510 2022-08-01T23:03:24.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b77fe700 1 -- 172.21.15.32:0/1533969520 >> v2:172.21.15.32:3300/0 conn(0x7f45b8140c10 msgr2=0x7f45b80b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:24.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.754+0000 7f45b77fe700 1 --2- 172.21.15.32:0/1533969520 >> v2:172.21.15.32:3300/0 conn(0x7f45b8140c10 0x7f45b80b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45b77fe700 1 -- 172.21.15.32:0/1533969520 >> v1:172.21.15.32:6789/0 conn(0x7f45b80b6510 legacy=0x7f45b8140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45b77fe700 1 -- 172.21.15.32:0/1533969520 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b81476a0 con 0x7f45b80b2aa0 2022-08-01T23:03:24.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45b77fe700 1 --2- 172.21.15.32:0/1533969520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80b0e60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f45b00032f0 tx=0x7f45b00091d0).ready entity=mon.1 client_cookie=57780d1337662275 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:24.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45b6ffd700 1 -- 172.21.15.32:0/1533969520 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b000b600 con 0x7f45b80b2aa0 2022-08-01T23:03:24.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45b6ffd700 1 -- 172.21.15.32:0/1533969520 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b000b760 con 0x7f45b80b2aa0 2022-08-01T23:03:24.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45b6ffd700 1 -- 172.21.15.32:0/1533969520 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b000b8d0 con 0x7f45b80b2aa0 2022-08-01T23:03:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 -- 172.21.15.32:0/1533969520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 msgr2=0x7f45b80b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/1533969520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80b0e60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f45b00032f0 tx=0x7f45b00091d0).stop 2022-08-01T23:03:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 -- 172.21.15.32:0/1533969520 shutdown_connections 2022-08-01T23:03:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/1533969520 >> v2:172.21.15.32:3300/0 conn(0x7f45b8140c10 0x7f45b80b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/1533969520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80b0e60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 -- 172.21.15.32:0/1533969520 >> 172.21.15.32:0/1533969520 conn(0x7f45b8063800 msgr2=0x7f45b8063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 -- 172.21.15.32:0/1533969520 shutdown_connections 2022-08-01T23:03:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.758+0000 7f45bd2ef700 1 -- 172.21.15.32:0/1533969520 wait complete. 2022-08-01T23:03:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 Processor -- start 2022-08-01T23:03:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 -- start start 2022-08-01T23:03:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45b80b6510 0x7f45b80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 0x7f45b80ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80acb40 con 0x7f45b8140c10 2022-08-01T23:03:24.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8141260 con 0x7f45b80b2aa0 2022-08-01T23:03:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8141500 con 0x7f45b80b6510 2022-08-01T23:03:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 0x7f45b80ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 0x7f45b80ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46736/0 (socket says 172.21.15.32:46736) 2022-08-01T23:03:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 -- 172.21.15.32:0/3953318209 learned_addr learned my addr 172.21.15.32:0/3953318209 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45b7fff700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45b77fe700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45b80b6510 0x7f45b80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 -- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45b80b6510 msgr2=0x7f45b80adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45b80b6510 0x7f45b80adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 -- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 msgr2=0x7f45b80af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 -- 172.21.15.32:0/3953318209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0009470 con 0x7f45b8140c10 2022-08-01T23:03:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45b77fe700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45b80b6510 0x7f45b80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bcaee700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 0x7f45b80ae4b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f45a800c000 tx=0x7f45a8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a8010780 con 0x7f45b8140c10 2022-08-01T23:03:24.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a80108e0 con 0x7f45b8140c10 2022-08-01T23:03:24.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a8010c40 con 0x7f45b8140c10 2022-08-01T23:03:24.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.762+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b80afcb0 con 0x7f45b8140c10 2022-08-01T23:03:24.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b8141a40 con 0x7f45b8140c10 2022-08-01T23:03:24.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b81416c0 con 0x7f45b8140c10 2022-08-01T23:03:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f45a801c460 con 0x7f45b8140c10 2022-08-01T23:03:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45b57fa700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f459c053a70 0x7f459c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f45a806c1c0 con 0x7f45b8140c10 2022-08-01T23:03:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45b7fff700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f459c053a70 0x7f459c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.766+0000 7f45b7fff700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f459c053a70 0x7f459c055f30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f45a000d000 tx=0x7f45a0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.774+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45a8043200 con 0x7f45b8140c10 2022-08-01T23:03:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:24.942+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f45b80574e0 con 0x7f45b8140c10 2022-08-01T23:03:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.626+0000 7f45b57fa700 1 -- 172.21.15.32:0/3953318209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v336) v1 ==== 130+0+0 (secure 0 0 0) 0x7f45a8046820 con 0x7f45b8140c10 2022-08-01T23:03:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f459c053a70 msgr2=0x7f459c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f459c053a70 0x7f459c055f30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f45a000d000 tx=0x7f45a0009040).stop 2022-08-01T23:03:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 msgr2=0x7f45b80ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 0x7f45b80ae4b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f45a800c000 tx=0x7f45a8006df0).stop 2022-08-01T23:03:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 shutdown_connections 2022-08-01T23:03:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f459c053a70 0x7f459c055f30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8140c10 0x7f45b80ae4b0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f45b80b6510 0x7f45b80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 --2- 172.21.15.32:0/3953318209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45b80b2aa0 0x7f45b80af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 >> 172.21.15.32:0/3953318209 conn(0x7f45b8063800 msgr2=0x7f45b80b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 shutdown_connections 2022-08-01T23:03:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.630+0000 7f45bd2ef700 1 -- 172.21.15.32:0/3953318209 wait complete. 2022-08-01T23:03:25.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:03:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.762+0000 7fd0e826f700 1 Processor -- start 2022-08-01T23:03:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.762+0000 7fd0e826f700 1 -- start start 2022-08-01T23:03:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e826f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0e0060560 0x7fd0e00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e826f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0e013f920 0x7fd0e013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e826f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e0061b20 con 0x7fd0e00aa050 2022-08-01T23:03:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e826f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e0140280 con 0x7fd0e0060560 2022-08-01T23:03:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e826f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e0061dc0 con 0x7fd0e013f920 2022-08-01T23:03:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e600b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0e00aa050 0x7fd0e013ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34282/0 (socket says 172.21.15.32:34282) 2022-08-01T23:03:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e600b700 1 -- 172.21.15.32:0/3844882210 learned_addr learned my addr 172.21.15.32:0/3844882210 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e580a700 1 --2- 172.21.15.32:0/3844882210 >> v2:172.21.15.32:3300/0 conn(0x7fd0e0060560 0x7fd0e00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e680c700 1 --2- 172.21.15.32:0/3844882210 >> v2:172.21.15.55:3300/0 conn(0x7fd0e013f920 0x7fd0e013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e5009700 1 -- 172.21.15.32:0/3844882210 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607592378 0 0) 0x7fd0e0061b20 con 0x7fd0e00aa050 2022-08-01T23:03:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e5009700 1 -- 172.21.15.32:0/3844882210 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c8003260 con 0x7fd0e00aa050 2022-08-01T23:03:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e580a700 1 -- 172.21.15.32:0/3844882210 >> v2:172.21.15.55:3300/0 conn(0x7fd0e013f920 msgr2=0x7fd0e013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e580a700 1 --2- 172.21.15.32:0/3844882210 >> v2:172.21.15.55:3300/0 conn(0x7fd0e013f920 0x7fd0e013fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e580a700 1 -- 172.21.15.32:0/3844882210 >> v1:172.21.15.32:6789/0 conn(0x7fd0e00aa050 legacy=0x7fd0e013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e580a700 1 -- 172.21.15.32:0/3844882210 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0060d20 con 0x7fd0e0060560 2022-08-01T23:03:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e580a700 1 --2- 172.21.15.32:0/3844882210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e0060560 0x7fd0e00aa430 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fd0e00aac80 tx=0x7fd0d0006df0).ready entity=mon.0 client_cookie=87ec7acb4d33b6a5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e5009700 1 -- 172.21.15.32:0/3844882210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d0014070 con 0x7fd0e0060560 2022-08-01T23:03:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e5009700 1 -- 172.21.15.32:0/3844882210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d0000e90 con 0x7fd0e0060560 2022-08-01T23:03:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.766+0000 7fd0e5009700 1 -- 172.21.15.32:0/3844882210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d000fae0 con 0x7fd0e0060560 2022-08-01T23:03:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- 172.21.15.32:0/3844882210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e0060560 msgr2=0x7fd0e00aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 --2- 172.21.15.32:0/3844882210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e0060560 0x7fd0e00aa430 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fd0e00aac80 tx=0x7fd0d0006df0).stop 2022-08-01T23:03:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- 172.21.15.32:0/3844882210 shutdown_connections 2022-08-01T23:03:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 --2- 172.21.15.32:0/3844882210 >> v2:172.21.15.55:3300/0 conn(0x7fd0e013f920 0x7fd0e013fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 --2- 172.21.15.32:0/3844882210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e0060560 0x7fd0e00aa430 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- 172.21.15.32:0/3844882210 >> 172.21.15.32:0/3844882210 conn(0x7fd0e00b2b00 msgr2=0x7fd0e00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- 172.21.15.32:0/3844882210 shutdown_connections 2022-08-01T23:03:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- 172.21.15.32:0/3844882210 wait complete. 2022-08-01T23:03:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 Processor -- start 2022-08-01T23:03:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- start start 2022-08-01T23:03:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0e0060560 0x7fd0e00b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 0x7fd0e00af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0e013f920 0x7fd0e0143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0142ad0 con 0x7fd0e00aa050 2022-08-01T23:03:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00addc0 con 0x7fd0e013f920 2022-08-01T23:03:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e826f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0143b30 con 0x7fd0e0060560 2022-08-01T23:03:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 0x7fd0e00af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 0x7fd0e00af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46748/0 (socket says 172.21.15.32:46748) 2022-08-01T23:03:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 -- 172.21.15.32:0/1717480440 learned_addr learned my addr 172.21.15.32:0/1717480440 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e600b700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0e0060560 0x7fd0e00b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e680c700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0e013f920 0x7fd0e0143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 -- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0e0060560 msgr2=0x7fd0e00b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0e0060560 0x7fd0e00b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 -- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0e013f920 msgr2=0x7fd0e0143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0e013f920 0x7fd0e0143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.770+0000 7fd0e580a700 1 -- 172.21.15.32:0/1717480440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d000b180 con 0x7fd0e00aa050 2022-08-01T23:03:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0e580a700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 0x7fd0e00af100 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fd0e00adf00 tx=0x7fd0d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:25.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d0010d60 con 0x7fd0e00aa050 2022-08-01T23:03:25.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d0002f70 con 0x7fd0e00aa050 2022-08-01T23:03:25.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e0142360 con 0x7fd0e00aa050 2022-08-01T23:03:25.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0d001d640 con 0x7fd0e00aa050 2022-08-01T23:03:25.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e0142830 con 0x7fd0e00aa050 2022-08-01T23:03:25.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e013e830 con 0x7fd0e00aa050 2022-08-01T23:03:25.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd0d001d7a0 con 0x7fd0e00aa050 2022-08-01T23:03:25.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0cf7fe700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0c0053950 0x7fd0c0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:25.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd0d006fde0 con 0x7fd0e00aa050 2022-08-01T23:03:25.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.774+0000 7fd0e600b700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0c0053950 0x7fd0c0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:25.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.778+0000 7fd0e600b700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0c0053950 0x7fd0c0055e10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fd0dc00d000 tx=0x7fd0dc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:25.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.782+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0d0046d70 con 0x7fd0e00aa050 2022-08-01T23:03:25.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.906+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd0e00574e0 con 0x7fd0e00aa050 2022-08-01T23:03:25.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0cf7fe700 1 -- 172.21.15.32:0/1717480440 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0d004a390 con 0x7fd0e00aa050 2022-08-01T23:03:25.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0c0053950 msgr2=0x7fd0c0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0c0053950 0x7fd0c0055e10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fd0dc00d000 tx=0x7fd0dc009040).stop 2022-08-01T23:03:25.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 msgr2=0x7fd0e00af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:25.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 0x7fd0e00af100 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fd0e00adf00 tx=0x7fd0d0016000).stop 2022-08-01T23:03:25.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 shutdown_connections 2022-08-01T23:03:25.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd0c0053950 0x7fd0c0055e10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0e013f920 0x7fd0e0143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e00aa050 0x7fd0e00af100 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 --2- 172.21.15.32:0/1717480440 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd0e0060560 0x7fd0e00b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:25.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 >> 172.21.15.32:0/1717480440 conn(0x7fd0e00b2b00 msgr2=0x7fd0e005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:25.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 shutdown_connections 2022-08-01T23:03:25.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:25.914+0000 7fd0e826f700 1 -- 172.21.15.32:0/1717480440 wait complete. 2022-08-01T23:03:25.931 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:03:25.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:25.949 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:25.968 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:25.989 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:26.009 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:03:26.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:26.010 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:26.011 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:26.018 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:03:26.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:03:26.019 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:26.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:26.020 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:26.021 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:26.027 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:03:26.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:03:26.028 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:26.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:03:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.134+0000 7fd0626ad700 1 Processor -- start 2022-08-01T23:03:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- start start 2022-08-01T23:03:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd05c0b1160 0x7fd05c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c0b04e0 con 0x7fd05c0b1f40 2022-08-01T23:03:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c0b0780 con 0x7fd05c148530 2022-08-01T23:03:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05c0a9250 con 0x7fd05c0b1160 2022-08-01T23:03:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd060c4a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd05c0b1160 0x7fd05c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd05bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd05c0b1f40 0x7fd05c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34296/0 (socket says 172.21.15.32:34296) 2022-08-01T23:03:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd05bfff700 1 -- 172.21.15.32:0/1928824849 learned_addr learned my addr 172.21.15.32:0/1928824849 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd060c4a700 1 -- 172.21.15.32:0/1928824849 >> v1:172.21.15.55:6789/0 conn(0x7fd05c148530 legacy=0x7fd05c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd060c4a700 1 -- 172.21.15.32:0/1928824849 >> v1:172.21.15.32:6789/0 conn(0x7fd05c0b1f40 legacy=0x7fd05c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd060c4a700 1 -- 172.21.15.32:0/1928824849 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c0b0a20 con 0x7fd05c0b1160 2022-08-01T23:03:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd060c4a700 1 --2- 172.21.15.32:0/1928824849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1160 0x7fd05c1448b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fd05c1478c0 tx=0x7fd0540091d0).ready entity=mon.0 client_cookie=78cdfbe0d370b633 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd05affd700 1 -- 172.21.15.32:0/1928824849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd05400b860 con 0x7fd05c0b1160 2022-08-01T23:03:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd05affd700 1 -- 172.21.15.32:0/1928824849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05400b9c0 con 0x7fd05c0b1160 2022-08-01T23:03:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd05affd700 1 -- 172.21.15.32:0/1928824849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd05400bb30 con 0x7fd05c0b1160 2022-08-01T23:03:26.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- 172.21.15.32:0/1928824849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1160 msgr2=0x7fd05c1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 --2- 172.21.15.32:0/1928824849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1160 0x7fd05c1448b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fd05c1478c0 tx=0x7fd0540091d0).stop 2022-08-01T23:03:26.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- 172.21.15.32:0/1928824849 shutdown_connections 2022-08-01T23:03:26.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 --2- 172.21.15.32:0/1928824849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1160 0x7fd05c1448b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- 172.21.15.32:0/1928824849 >> 172.21.15.32:0/1928824849 conn(0x7fd05c05f840 msgr2=0x7fd05c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:26.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- 172.21.15.32:0/1928824849 shutdown_connections 2022-08-01T23:03:26.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.138+0000 7fd0626ad700 1 -- 172.21.15.32:0/1928824849 wait complete. 2022-08-01T23:03:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 Processor -- start 2022-08-01T23:03:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 -- start start 2022-08-01T23:03:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 0x7fd05c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 0x7fd05c14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd05c148530 0x7fd05c0aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c156f70 con 0x7fd05c0b1f40 2022-08-01T23:03:26.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c157210 con 0x7fd05c0b1160 2022-08-01T23:03:26.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c1574b0 con 0x7fd05c148530 2022-08-01T23:03:26.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 0x7fd05c14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 0x7fd05c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 0x7fd05c1491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:56546/0 (socket says 172.21.15.32:56546) 2022-08-01T23:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05bfff700 1 -- 172.21.15.32:0/327172314 learned_addr learned my addr 172.21.15.32:0/327172314 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 0x7fd05c14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46764/0 (socket says 172.21.15.32:46764) 2022-08-01T23:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd060c4a700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd05c148530 0x7fd05c0aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 -- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd05c148530 msgr2=0x7fd05c0aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd05c148530 0x7fd05c0aa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 -- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 msgr2=0x7fd05c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 0x7fd05c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 -- 172.21.15.32:0/327172314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054009470 con 0x7fd05c0b1f40 2022-08-01T23:03:26.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05bfff700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 0x7fd05c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:26.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd05b7fe700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 0x7fd05c14a180 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fd05c0b3300 tx=0x7fd04c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04c00cca0 con 0x7fd05c0b1f40 2022-08-01T23:03:26.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c00ce00 con 0x7fd05c0b1f40 2022-08-01T23:03:26.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05c14b6c0 con 0x7fd05c0b1f40 2022-08-01T23:03:26.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04c0125f0 con 0x7fd05c0b1f40 2022-08-01T23:03:26.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.142+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c14a670 con 0x7fd05c0b1f40 2022-08-01T23:03:26.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.146+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c157680 con 0x7fd05c0b1f40 2022-08-01T23:03:26.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.146+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd04c019030 con 0x7fd05c0b1f40 2022-08-01T23:03:26.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.146+0000 7fd0597fa700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd03c053a70 0x7fd03c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.146+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd04c06aec0 con 0x7fd05c0b1f40 2022-08-01T23:03:26.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.146+0000 7fd05bfff700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd03c053a70 0x7fd03c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.146+0000 7fd05bfff700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd03c053a70 0x7fd03c055f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fd05000d000 tx=0x7fd050007c40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.154+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd04c041f60 con 0x7fd05c0b1f40 2022-08-01T23:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.278+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd05c0b0cc0 con 0x7fd05c0b1f40 2022-08-01T23:03:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0597fa700 1 -- 172.21.15.32:0/327172314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v100) v1 ==== 81+0+29221 (secure 0 0 0) 0x7fd04c045580 con 0x7fd05c0b1f40 2022-08-01T23:03:26.287 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:26.287 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:26.287 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 100, 2022-08-01T23:03:26.288 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:03:26.288 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:03:26.288 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:03:23.609086+0000", 2022-08-01T23:03:26.288 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:26.288 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:26.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:03:26.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:03:26.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:03:26.289 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:03:26.290 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:03:26.290 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:03:26.290 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:03:26.290 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:26.290 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:03:26.291 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:03:26.291 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:03:26.291 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:03:26.291 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:03:26.291 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 18, 2022-08-01T23:03:26.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:03:26.292 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:03:26.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:03:26.292 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:03:26.292 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:03:26.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.293 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:03:26.293 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:03:26.293 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:03:26.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:26.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:26.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:26.294 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:26.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:26.295 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:26.295 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:26.295 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:26.295 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:03:26.295 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:03:26.296 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:03:26.296 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:03:26.296 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:03:26.296 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:26.296 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:26.297 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:26.297 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:26.297 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:26.297 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:26.297 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:26.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.298 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:03:26.298 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:26.298 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:26.298 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:26.299 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:26.299 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:26.299 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:26.299 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:26.299 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:26.300 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:26.300 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:26.300 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:26.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:26.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:26.301 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:26.301 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:26.301 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:26.301 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:26.301 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:26.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:26.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:26.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:26.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:26.302 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:26.303 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:26.303 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:26.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:26.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.303 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:26.304 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:26.304 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:26.304 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:26.304 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:26.304 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:26.304 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:26.305 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:26.305 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:26.305 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:26.305 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:26.306 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:26.306 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:03:26.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.306 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:26.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:03:26.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:03:26.307 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:03:26.308 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:03:26.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:26.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:26.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:26.308 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:26.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:26.309 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:26.309 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:26.309 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:26.309 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:26.310 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:26.310 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:26.310 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:26.310 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:26.310 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:26.311 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:26.311 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:26.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:26.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:26.311 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:03:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:26.312 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:26.313 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:26.313 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:26.313 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:26.313 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:26.313 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:26.314 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:26.314 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:26.314 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:26.314 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:26.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:26.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:26.315 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:26.315 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:26.315 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:26.316 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:26.316 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:26.316 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:26.316 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:26.316 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:26.317 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:26.317 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:26.317 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:26.317 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:26.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:26.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.318 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:26.318 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:26.318 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:26.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:26.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:26.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:26.319 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:26.319 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:26.320 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:26.320 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:26.320 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:26.320 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:26.321 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:26.321 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:03:26.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.321 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:26.322 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:03:26.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.322 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:26.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:03:26.322 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.323 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.323 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.323 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.323 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.324 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.324 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2022-08-01T23:03:26.324 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.324 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.324 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:03:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.326 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:03:26.326 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.327 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.327 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.328 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:03:26.328 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.329 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:03:26.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.330 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:03:26.331 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:03:26.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.333 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:03:26.334 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:03:26.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:26.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.336 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:03:26.336 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:03:26.336 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:03:26.336 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:03:26.336 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.336 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.337 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.337 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.337 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:26.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:03:26.338 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.338 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.338 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.339 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.339 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.339 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.339 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.339 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2022-08-01T23:03:26.340 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.340 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.340 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.340 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:03:26.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:03:26.342 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.343 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.344 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:03:26.344 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.345 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:03:26.345 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.346 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:03:26.346 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.347 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:03:26.348 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.348 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:03:26.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:03:26.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:03:26.351 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:03:26.352 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:03:26.352 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:03:26.352 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.352 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.352 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.353 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:26.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:03:26.354 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.354 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.354 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.354 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.355 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.355 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 95, 2022-08-01T23:03:26.355 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.356 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.356 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.356 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:03:26.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:03:26.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.358 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.359 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:03:26.360 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.360 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:03:26.361 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.361 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.362 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.362 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:03:26.363 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.364 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:03:26.364 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.365 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:03:26.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:03:26.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:26.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:03:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:03:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:03:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:03:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.369 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.369 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.370 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:26.370 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:03:26.370 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.370 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.370 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.371 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.371 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.371 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.371 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.371 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 95, 2022-08-01T23:03:26.372 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.372 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.372 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:03:26.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:03:26.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.375 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:03:26.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:03:26.377 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.378 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.378 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:03:26.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:03:26.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.381 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:03:26.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:03:26.383 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:26.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.384 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:03:26.384 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:03:26.384 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:03:26.385 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:03:26.385 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.385 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.385 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.386 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:26.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:03:26.387 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.387 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.387 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.387 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.388 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.388 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.388 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2022-08-01T23:03:26.388 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.388 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.389 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:03:26.390 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.390 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:03:26.391 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.391 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.392 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.392 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.392 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:03:26.393 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.393 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:03:26.394 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.395 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.395 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.395 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:03:26.396 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.396 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:03:26.397 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.397 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:03:26.399 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.399 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:03:26.400 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:26.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.400 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:03:26.401 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:03:26.401 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:03:26.401 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:03:26.401 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.401 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.402 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.403 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:26.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:03:26.403 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.403 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.403 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.404 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.404 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.404 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.404 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.404 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 85, 2022-08-01T23:03:26.405 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.405 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.405 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.405 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.406 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:03:26.406 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.407 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:03:26.407 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.408 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.408 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.409 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:03:26.409 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.410 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:03:26.410 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.411 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.411 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.412 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:03:26.412 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.413 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:03:26.413 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.414 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.414 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.415 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:03:26.415 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.416 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:03:26.416 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:26.416 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.416 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.417 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:03:26.417 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:03:26.417 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:03:26.417 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:03:26.418 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.418 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.418 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.419 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:26.419 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:03:26.419 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.420 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.420 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.420 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.420 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.420 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.421 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.421 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2022-08-01T23:03:26.421 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.421 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.421 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.422 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.422 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:03:26.422 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.423 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:03:26.423 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.424 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.424 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.425 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:03:26.425 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.426 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:03:26.426 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.426 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.427 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.427 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.427 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.428 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:03:26.428 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.429 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:03:26.429 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.430 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.430 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.430 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.431 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:03:26.431 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.432 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:03:26.432 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:26.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.432 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.433 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:03:26.433 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:03:26.433 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:03:26.433 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:03:26.434 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.434 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.434 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.434 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:26.435 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:03:26.435 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:26.436 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:26.436 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:26.436 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:26.436 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:26.436 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:26.437 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:26.437 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2022-08-01T23:03:26.437 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:26.437 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:26.437 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:26.438 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.438 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:03:26.438 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.439 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:03:26.440 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.440 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:26.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:03:26.441 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.442 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:03:26.442 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.443 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:26.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:03:26.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.445 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:03:26.445 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.446 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.446 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:26.446 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:26.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:26.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:03:26.447 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:26.448 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:03:26.448 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:26.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.449 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:03:26.449 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:03:26.450 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:03:26.450 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:03:26.450 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:26.450 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:26.451 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:26.451 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:26.451 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.451 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:26.451 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:03:26.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.452 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:26.452 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.452 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.452 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.453 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.453 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.453 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:03:26.453 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.454 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:26.454 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.454 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.455 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.455 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.455 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.455 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:03:26.455 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.456 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:26.456 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.456 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.457 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.457 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.457 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.457 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:03:26.457 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.465 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:26.466 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.466 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.466 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.466 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.467 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:03:26.467 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.468 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:26.468 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.468 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.469 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.469 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.469 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.469 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:03:26.470 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.470 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:26.471 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.471 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.471 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.471 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.471 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.472 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:03:26.472 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.473 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:26.473 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.473 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.473 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.474 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.474 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.474 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:03:26.474 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:26.475 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:26.475 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:26.476 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:26.476 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:26.476 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:26.476 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:26.476 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:03:26.477 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:26.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.477 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:26.477 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:03:26.478 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:03:26.478 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:03:26.478 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:03:26.478 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:03:26.479 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/2562641804": "2022-08-02T23:03:19.870097+0000", 2022-08-01T23:03:26.479 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/2562641804": "2022-08-02T23:03:19.870097+0000" 2022-08-01T23:03:26.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.479 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:03:26.480 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:03:26.480 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:03:26.480 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:26.480 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:03:26.481 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:03:26.481 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:26.481 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:03:26.481 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:26.482 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:26.482 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:03:26.482 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:03:26.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.482 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:03:26.483 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:26.483 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:26.483 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:03:26.483 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:26.484 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:26.484 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:03:26.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:26.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:26.485 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:03:26.485 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:03:26.485 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:03:26.485 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:03:26.485 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:03:26.486 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:26.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd03c053a70 msgr2=0x7fd03c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd03c053a70 0x7fd03c055f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fd05000d000 tx=0x7fd050007c40).stop 2022-08-01T23:03:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 msgr2=0x7fd05c14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 0x7fd05c14a180 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fd05c0b3300 tx=0x7fd04c007b90).stop 2022-08-01T23:03:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 shutdown_connections 2022-08-01T23:03:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd03c053a70 0x7fd03c055f30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd05c148530 0x7fd05c0aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd05c0b1f40 0x7fd05c14a180 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.282+0000 7fd0626ad700 1 --2- 172.21.15.32:0/327172314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd05c0b1160 0x7fd05c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.286+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 >> 172.21.15.32:0/327172314 conn(0x7fd05c05f840 msgr2=0x7fd05c144a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.286+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 shutdown_connections 2022-08-01T23:03:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.286+0000 7fd0626ad700 1 -- 172.21.15.32:0/327172314 wait complete. 2022-08-01T23:03:26.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.598+0000 7f82f759e700 1 Processor -- start 2022-08-01T23:03:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.598+0000 7f82f759e700 1 -- start start 2022-08-01T23:03:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82f80b2720 0x7f82f80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b1ed0 0x7f82f80b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f8147470 con 0x7f82f805c6d0 2022-08-01T23:03:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f8144150 con 0x7f82f80b2720 2022-08-01T23:03:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8143110 con 0x7f82f80b1ed0 2022-08-01T23:03:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82f80b2720 0x7f82f80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82f80b2720 0x7f82f80b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46770/0 (socket says 172.21.15.32:46770) 2022-08-01T23:03:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 -- 172.21.15.32:0/1233427460 learned_addr learned my addr 172.21.15.32:0/1233427460 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f6d9d700 1 --2- 172.21.15.32:0/1233427460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b1ed0 0x7f82f80b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f5d9b700 1 -- 172.21.15.32:0/1233427460 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969211813 0 0) 0x7f82f8147470 con 0x7f82f805c6d0 2022-08-01T23:03:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f5d9b700 1 -- 172.21.15.32:0/1233427460 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d8003260 con 0x7f82f805c6d0 2022-08-01T23:03:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 -- 172.21.15.32:0/1233427460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b1ed0 msgr2=0x7f82f80b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 --2- 172.21.15.32:0/1233427460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b1ed0 0x7f82f80b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 -- 172.21.15.32:0/1233427460 >> v1:172.21.15.55:6789/0 conn(0x7f82f805c6d0 legacy=0x7f82f8143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 -- 172.21.15.32:0/1233427460 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8143300 con 0x7f82f80b2720 2022-08-01T23:03:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f659c700 1 --2- 172.21.15.32:0/1233427460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80b2720 0x7f82f80b0910 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f82e8002f70 tx=0x7f82e8006eb0).ready entity=mon.0 client_cookie=7630138443e43347 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f5d9b700 1 -- 172.21.15.32:0/1233427460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82e800d5c0 con 0x7f82f80b2720 2022-08-01T23:03:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f5d9b700 1 -- 172.21.15.32:0/1233427460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8000e90 con 0x7f82f80b2720 2022-08-01T23:03:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f5d9b700 1 -- 172.21.15.32:0/1233427460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82e800d950 con 0x7f82f80b2720 2022-08-01T23:03:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- 172.21.15.32:0/1233427460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80b2720 msgr2=0x7f82f80b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 --2- 172.21.15.32:0/1233427460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80b2720 0x7f82f80b0910 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f82e8002f70 tx=0x7f82e8006eb0).stop 2022-08-01T23:03:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- 172.21.15.32:0/1233427460 shutdown_connections 2022-08-01T23:03:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 --2- 172.21.15.32:0/1233427460 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b1ed0 0x7f82f80b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 --2- 172.21.15.32:0/1233427460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80b2720 0x7f82f80b0910 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- 172.21.15.32:0/1233427460 >> 172.21.15.32:0/1233427460 conn(0x7f82f8061360 msgr2=0x7f82f8062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.602+0000 7f82f759e700 1 -- 172.21.15.32:0/1233427460 shutdown_connections 2022-08-01T23:03:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- 172.21.15.32:0/1233427460 wait complete. 2022-08-01T23:03:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 Processor -- start 2022-08-01T23:03:26.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- start start 2022-08-01T23:03:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f80b1ed0 0x7f82f80af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b2720 0x7f82f8144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 0x7f82f8144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80ab230 con 0x7f82f80abc10 2022-08-01T23:03:26.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80ac2a0 con 0x7f82f80b1ed0 2022-08-01T23:03:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80ab460 con 0x7f82f80b2720 2022-08-01T23:03:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 0x7f82f8144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 0x7f82f8144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46778/0 (socket says 172.21.15.32:46778) 2022-08-01T23:03:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 -- 172.21.15.32:0/1110364064 learned_addr learned my addr 172.21.15.32:0/1110364064 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82effff700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b2720 0x7f82f8144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f659c700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f80b1ed0 0x7f82f80af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 -- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b2720 msgr2=0x7f82f8144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b2720 0x7f82f8144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 -- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f80b1ed0 msgr2=0x7f82f80af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f80b1ed0 0x7f82f80af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 -- 172.21.15.32:0/1110364064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e800b1e0 con 0x7f82f80abc10 2022-08-01T23:03:26.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82effff700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b2720 0x7f82f8144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:26.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f6d9d700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 0x7f82f8144870 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f82f0006e00 tx=0x7f82f00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82f000c850 con 0x7f82f80abc10 2022-08-01T23:03:26.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f000c9b0 con 0x7f82f80abc10 2022-08-01T23:03:26.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82f000cd10 con 0x7f82f80abc10 2022-08-01T23:03:26.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f80ac540 con 0x7f82f80abc10 2022-08-01T23:03:26.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.606+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f80ac840 con 0x7f82f80abc10 2022-08-01T23:03:26.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.610+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f80574e0 con 0x7f82f80abc10 2022-08-01T23:03:26.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.610+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f82f00123f0 con 0x7f82f80abc10 2022-08-01T23:03:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.610+0000 7f82ef7fe700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82dc053a70 0x7f82dc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.610+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f82f006c150 con 0x7f82f80abc10 2022-08-01T23:03:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.610+0000 7f82f659c700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82dc053a70 0x7f82dc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.610+0000 7f82f659c700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82dc053a70 0x7f82dc055f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f82e801a000 tx=0x7f82e8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.618+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f82f0043190 con 0x7f82f80abc10 2022-08-01T23:03:26.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82f80ab890 con 0x7f82f80abc10 2022-08-01T23:03:26.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82ef7fe700 1 -- 172.21.15.32:0/1110364064 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f82f00467b0 con 0x7f82f80abc10 2022-08-01T23:03:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:26.791 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":7126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2962899903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2962899903},{"type":"v1","addr":"172.21.15.32:6835","nonce":2962899903}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[]} 2022-08-01T23:03:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82dc053a70 msgr2=0x7f82dc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82dc053a70 0x7f82dc055f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f82e801a000 tx=0x7f82e8015000).stop 2022-08-01T23:03:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 msgr2=0x7f82f8144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 0x7f82f8144870 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f82f0006e00 tx=0x7f82f00091d0).stop 2022-08-01T23:03:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 shutdown_connections 2022-08-01T23:03:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f82dc053a70 0x7f82dc055f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f80abc10 0x7f82f8144870 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f82f80b2720 0x7f82f8144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 --2- 172.21.15.32:0/1110364064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f80b1ed0 0x7f82f80af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 >> 172.21.15.32:0/1110364064 conn(0x7f82f8061360 msgr2=0x7f82f813fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 shutdown_connections 2022-08-01T23:03:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.786+0000 7f82f759e700 1 -- 172.21.15.32:0/1110364064 wait complete. 2022-08-01T23:03:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2022-08-01T23:03:26.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 Processor -- start 2022-08-01T23:03:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 -- start start 2022-08-01T23:03:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd4fc0b4640 0x7fd4fc0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4fc144d20 0x7fd4fc14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc061bd0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc0a9fd0 con 0x7fd4fc144d20 2022-08-01T23:03:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fb59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc1451a0 con 0x7fd4fc0b4640 2022-08-01T23:03:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fa59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4fc0617f0 0x7fd4fc144610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34324/0 (socket says 172.21.15.32:34324) 2022-08-01T23:03:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fa59c700 1 -- 172.21.15.32:0/2063506364 learned_addr learned my addr 172.21.15.32:0/2063506364 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fad9d700 1 --2- 172.21.15.32:0/2063506364 >> v2:172.21.15.32:3300/0 conn(0x7fd4fc144d20 0x7fd4fc14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4f9d9b700 1 --2- 172.21.15.32:0/2063506364 >> v2:172.21.15.55:3300/0 conn(0x7fd4fc0b4640 0x7fd4fc0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4f959a700 1 -- 172.21.15.32:0/2063506364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662753969 0 0) 0x7fd4fc061bd0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4f959a700 1 -- 172.21.15.32:0/2063506364 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e0003260 con 0x7fd4fc0617f0 2022-08-01T23:03:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fad9d700 1 -- 172.21.15.32:0/2063506364 >> v2:172.21.15.55:3300/0 conn(0x7fd4fc0b4640 msgr2=0x7fd4fc0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:26.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fad9d700 1 --2- 172.21.15.32:0/2063506364 >> v2:172.21.15.55:3300/0 conn(0x7fd4fc0b4640 0x7fd4fc0b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fad9d700 1 -- 172.21.15.32:0/2063506364 >> v1:172.21.15.32:6789/0 conn(0x7fd4fc0617f0 legacy=0x7fd4fc144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fad9d700 1 -- 172.21.15.32:0/2063506364 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0a88e0 con 0x7fd4fc144d20 2022-08-01T23:03:26.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4fad9d700 1 --2- 172.21.15.32:0/2063506364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc144d20 0x7fd4fc14aaa0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7fd4f40032f0 tx=0x7fd4f40091d0).ready entity=mon.0 client_cookie=4e17bef43995653b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4f959a700 1 -- 172.21.15.32:0/2063506364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f400b600 con 0x7fd4fc144d20 2022-08-01T23:03:26.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4f959a700 1 -- 172.21.15.32:0/2063506364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f400b760 con 0x7fd4fc144d20 2022-08-01T23:03:26.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.906+0000 7fd4f959a700 1 -- 172.21.15.32:0/2063506364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f400b910 con 0x7fd4fc144d20 2022-08-01T23:03:26.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- 172.21.15.32:0/2063506364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc144d20 msgr2=0x7fd4fc14aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/2063506364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc144d20 0x7fd4fc14aaa0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7fd4f40032f0 tx=0x7fd4f40091d0).stop 2022-08-01T23:03:26.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- 172.21.15.32:0/2063506364 shutdown_connections 2022-08-01T23:03:26.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/2063506364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc144d20 0x7fd4fc14aaa0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/2063506364 >> v2:172.21.15.55:3300/0 conn(0x7fd4fc0b4640 0x7fd4fc0b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- 172.21.15.32:0/2063506364 >> 172.21.15.32:0/2063506364 conn(0x7fd4fc066f70 msgr2=0x7fd4fc067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- 172.21.15.32:0/2063506364 shutdown_connections 2022-08-01T23:03:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- 172.21.15.32:0/2063506364 wait complete. 2022-08-01T23:03:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 Processor -- start 2022-08-01T23:03:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- start start 2022-08-01T23:03:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 0x7fd4fc0b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4fc0b4640 0x7fd4fc143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4fc144d20 0x7fd4fc143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0630d0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0635a0 con 0x7fd4fc0b4640 2022-08-01T23:03:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc144480 con 0x7fd4fc144d20 2022-08-01T23:03:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 0x7fd4fc0b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 0x7fd4fc0b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46788/0 (socket says 172.21.15.32:46788) 2022-08-01T23:03:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 -- 172.21.15.32:0/3768630967 learned_addr learned my addr 172.21.15.32:0/3768630967 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fad9d700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4fc144d20 0x7fd4fc143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4f9d9b700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4fc0b4640 0x7fd4fc143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 -- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4fc144d20 msgr2=0x7fd4fc143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4fc144d20 0x7fd4fc143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 -- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4fc0b4640 msgr2=0x7fd4fc143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4fc0b4640 0x7fd4fc143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4fa59c700 1 -- 172.21.15.32:0/3768630967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4009470 con 0x7fd4fc0617f0 2022-08-01T23:03:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.910+0000 7fd4f9d9b700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4fc0b4640 0x7fd4fc143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4fa59c700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 0x7fd4fc0b1e10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7fd4ec001110 tx=0x7fd4ec007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4ec0070d0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec007230 con 0x7fd4fc0617f0 2022-08-01T23:03:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc0b3ed0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4ec005610 con 0x7fd4fc0617f0 2022-08-01T23:03:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc066bc0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc0a8b80 con 0x7fd4fc0617f0 2022-08-01T23:03:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd4ec010070 con 0x7fd4fc0617f0 2022-08-01T23:03:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4eb7fe700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4d8053980 0x7fd4d8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd4ec06b4e0 con 0x7fd4fc0617f0 2022-08-01T23:03:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.914+0000 7fd4f9d9b700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4d8053980 0x7fd4d8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.918+0000 7fd4f9d9b700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4d8053980 0x7fd4d8055e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fd4f000b000 tx=0x7fd4f0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:26.922+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd4ec0423f0 con 0x7fd4fc0617f0 2022-08-01T23:03:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.082+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4fc159e70 con 0x7fd4fc0617f0 2022-08-01T23:03:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4eb7fe700 1 -- 172.21.15.32:0/3768630967 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fd4ec045a10 con 0x7fd4fc0617f0 2022-08-01T23:03:27.089 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:27.090 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1375091029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1375091029},{"type":"v1","addr":"172.21.15.55:6835","nonce":1375091029}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6978,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/1484980030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1484980030},{"type":"v1","addr":"172.21.15.32:6833","nonce":1484980030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":7126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2962899903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2962899903},{"type":"v1","addr":"172.21.15.32:6835","nonce":2962899903}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[]} 2022-08-01T23:03:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4d8053980 msgr2=0x7fd4d8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4d8053980 0x7fd4d8055e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fd4f000b000 tx=0x7fd4f0008040).stop 2022-08-01T23:03:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 msgr2=0x7fd4fc0b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 0x7fd4fc0b1e10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7fd4ec001110 tx=0x7fd4ec007ad0).stop 2022-08-01T23:03:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 shutdown_connections 2022-08-01T23:03:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd4d8053980 0x7fd4d8055e40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4fc144d20 0x7fd4fc143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4fc0b4640 0x7fd4fc143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 --2- 172.21.15.32:0/3768630967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4fc0617f0 0x7fd4fc0b1e10 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 >> 172.21.15.32:0/3768630967 conn(0x7fd4fc066f70 msgr2=0x7fd4fc0b8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:27.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 shutdown_connections 2022-08-01T23:03:27.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.086+0000 7fd4fb59e700 1 -- 172.21.15.32:0/3768630967 wait complete. 2022-08-01T23:03:27.094 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2022-08-01T23:03:27.103 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:03:27.103 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:03:27.103 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:03:27.104 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:27.104 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:03:27.104 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:03:27.105 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:03:27.105 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:27.105 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:03:27.106 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:03:27.106 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:03:27.106 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:27.117 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:03:27.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:03:27.118 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:03:27.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:03:27.123 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:03:27.124 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:03:27.126 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:03:27.126 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:27.162 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:03:27.163 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:27.163 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:03:27.164 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:27.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:03:27.196 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:03:27.227 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:03:27.236 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:03:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3759e700 1 Processor -- start 2022-08-01T23:03:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3759e700 1 -- start start 2022-08-01T23:03:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5a380b4ee0 0x7f5a38067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a38065af0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a380650a0 con 0x7f5a380aaf10 2022-08-01T23:03:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a380b3e70 con 0x7f5a380b4ee0 2022-08-01T23:03:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a35d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5a380ab2f0 0x7f5a380b76b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34364/0 (socket says 172.21.15.32:34364) 2022-08-01T23:03:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a35d9b700 1 -- 172.21.15.32:0/3242839989 learned_addr learned my addr 172.21.15.32:0/3242839989 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a36d9d700 1 --2- 172.21.15.32:0/3242839989 >> v2:172.21.15.55:3300/0 conn(0x7f5a380b4ee0 0x7f5a38067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143363627 0 0) 0x7f5a38065af0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1c003260 con 0x7f5a380ab2f0 2022-08-01T23:03:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404503238 0 0) 0x7f5a380650a0 con 0x7f5a380aaf10 2022-08-01T23:03:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a38065af0 con 0x7f5a380aaf10 2022-08-01T23:03:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273959107 0 0) 0x7f5a1c003260 con 0x7f5a380ab2f0 2022-08-01T23:03:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a36d9d700 1 -- 172.21.15.32:0/3242839989 >> v1:172.21.15.55:6789/0 conn(0x7f5a380aaf10 legacy=0x7f5a38064990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a36d9d700 1 -- 172.21.15.32:0/3242839989 >> v1:172.21.15.32:6789/0 conn(0x7f5a380ab2f0 legacy=0x7f5a380b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a36d9d700 1 -- 172.21.15.32:0/3242839989 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38066ff0 con 0x7f5a380b4ee0 2022-08-01T23:03:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a36d9d700 1 --2- 172.21.15.32:0/3242839989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380b4ee0 0x7f5a38067490 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f5a2c0019a0 tx=0x7f5a300091d0).ready entity=mon.1 client_cookie=15fd85231669bd53 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2c001800 con 0x7f5a380b4ee0 2022-08-01T23:03:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3000f510 con 0x7f5a380b4ee0 2022-08-01T23:03:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.266+0000 7f5a3559a700 1 -- 172.21.15.32:0/3242839989 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a3000f680 con 0x7f5a380b4ee0 2022-08-01T23:03:27.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- 172.21.15.32:0/3242839989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380b4ee0 msgr2=0x7f5a38067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 --2- 172.21.15.32:0/3242839989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380b4ee0 0x7f5a38067490 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f5a2c0019a0 tx=0x7f5a300091d0).stop 2022-08-01T23:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- 172.21.15.32:0/3242839989 shutdown_connections 2022-08-01T23:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 --2- 172.21.15.32:0/3242839989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380b4ee0 0x7f5a38067490 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- 172.21.15.32:0/3242839989 >> 172.21.15.32:0/3242839989 conn(0x7f5a38062d70 msgr2=0x7f5a3805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- 172.21.15.32:0/3242839989 shutdown_connections 2022-08-01T23:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- 172.21.15.32:0/3242839989 wait complete. 2022-08-01T23:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 Processor -- start 2022-08-01T23:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- start start 2022-08-01T23:03:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380aaf10 0x7f5a3805c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 0x7f5a38150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a380b4ee0 0x7f5a38150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a38150130 con 0x7f5a380ab2f0 2022-08-01T23:03:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380ac7a0 con 0x7f5a380aaf10 2022-08-01T23:03:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380aca40 con 0x7f5a380b4ee0 2022-08-01T23:03:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a35d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 0x7f5a38150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a35d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 0x7f5a38150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46828/0 (socket says 172.21.15.32:46828) 2022-08-01T23:03:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a35d9b700 1 -- 172.21.15.32:0/1343461170 learned_addr learned my addr 172.21.15.32:0/1343461170 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a3659c700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380aaf10 0x7f5a3805c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.270+0000 7f5a36d9d700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a380b4ee0 0x7f5a38150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a35d9b700 1 -- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a380b4ee0 msgr2=0x7f5a38150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a35d9b700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a380b4ee0 0x7f5a38150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a35d9b700 1 -- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380aaf10 msgr2=0x7f5a3805c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a35d9b700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380aaf10 0x7f5a3805c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a35d9b700 1 -- 172.21.15.32:0/1343461170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30009470 con 0x7f5a380ab2f0 2022-08-01T23:03:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a36d9d700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a380b4ee0 0x7f5a38150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a3659c700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380aaf10 0x7f5a3805c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a35d9b700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 0x7f5a38150690 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f5a2c0015d0 tx=0x7f5a2c002b60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2c0072e0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0054e0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a380acce0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2c005840 con 0x7f5a380ab2f0 2022-08-01T23:03:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a380a00f0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a380574e0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a2c010070 con 0x7f5a380ab2f0 2022-08-01T23:03:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a277fe700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a14053a70 0x7f5a14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5a2c06b750 con 0x7f5a380ab2f0 2022-08-01T23:03:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.274+0000 7f5a3659c700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a14053a70 0x7f5a14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:27.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.278+0000 7f5a3659c700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a14053a70 0x7f5a14055f30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f5a2800b000 tx=0x7f5a28009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:27.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.282+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a2c0427f0 con 0x7f5a380ab2f0 2022-08-01T23:03:27.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:27.418+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5a380ace70 con 0x7f5a380ab2f0 2022-08-01T23:03:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.253+0000 7f5a277fe700 1 -- 172.21.15.32:0/1343461170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5a2c018070 con 0x7f5a380ab2f0 2022-08-01T23:03:28.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a14053a70 msgr2=0x7f5a14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a14053a70 0x7f5a14055f30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f5a2800b000 tx=0x7f5a28009040).stop 2022-08-01T23:03:28.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 msgr2=0x7f5a38150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 0x7f5a38150690 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f5a2c0015d0 tx=0x7f5a2c002b60).stop 2022-08-01T23:03:28.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 shutdown_connections 2022-08-01T23:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a14053a70 0x7f5a14055f30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a380b4ee0 0x7f5a38150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a380ab2f0 0x7f5a38150690 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 --2- 172.21.15.32:0/1343461170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a380aaf10 0x7f5a3805c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 >> 172.21.15.32:0/1343461170 conn(0x7f5a38062d70 msgr2=0x7f5a38144cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 shutdown_connections 2022-08-01T23:03:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.257+0000 7f5a3759e700 1 -- 172.21.15.32:0/1343461170 wait complete. 2022-08-01T23:03:28.267 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:03:28.277 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:03:28.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:03:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 Processor -- start 2022-08-01T23:03:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 -- start start 2022-08-01T23:03:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f258c0b7840 0x7f258c060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f258c143400 0x7f258c143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c0671e0 con 0x7f258c0b2930 2022-08-01T23:03:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c065540 con 0x7f258c143400 2022-08-01T23:03:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25929b6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c0b7c20 con 0x7f258c0b7840 2022-08-01T23:03:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25921b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f258c143400 0x7f258c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25921b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f258c143400 0x7f258c143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46840/0 (socket says 172.21.15.32:46840) 2022-08-01T23:03:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25921b5700 1 -- 172.21.15.32:0/2264020397 learned_addr learned my addr 172.21.15.32:0/2264020397 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25911b3700 1 --1- 172.21.15.32:0/2264020397 >> v1:172.21.15.32:6789/0 conn(0x7f258c0b2930 0x7f258c0610d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34378/0 (socket says 172.21.15.32:34378) 2022-08-01T23:03:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.381+0000 7f25919b4700 1 --2- 172.21.15.32:0/2264020397 >> v2:172.21.15.55:3300/0 conn(0x7f258c0b7840 0x7f258c060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25921b5700 1 -- 172.21.15.32:0/2264020397 >> v2:172.21.15.55:3300/0 conn(0x7f258c0b7840 msgr2=0x7f258c060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25921b5700 1 --2- 172.21.15.32:0/2264020397 >> v2:172.21.15.55:3300/0 conn(0x7f258c0b7840 0x7f258c060be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25921b5700 1 -- 172.21.15.32:0/2264020397 >> v1:172.21.15.32:6789/0 conn(0x7f258c0b2930 legacy=0x7f258c0610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25921b5700 1 -- 172.21.15.32:0/2264020397 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0617e0 con 0x7f258c143400 2022-08-01T23:03:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25909b2700 1 -- 172.21.15.32:0/2264020397 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908997190 0 0) 0x7f258c0671e0 con 0x7f258c0b2930 2022-08-01T23:03:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25921b5700 1 --2- 172.21.15.32:0/2264020397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c143400 0x7f258c143890 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f2584002f70 tx=0x7f2584006eb0).ready entity=mon.0 client_cookie=f0c14eb3832a637a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25909b2700 1 -- 172.21.15.32:0/2264020397 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258400d610 con 0x7f258c143400 2022-08-01T23:03:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25909b2700 1 -- 172.21.15.32:0/2264020397 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584000e90 con 0x7f258c143400 2022-08-01T23:03:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25909b2700 1 -- 172.21.15.32:0/2264020397 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258400d960 con 0x7f258c143400 2022-08-01T23:03:28.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 -- 172.21.15.32:0/2264020397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c143400 msgr2=0x7f258c143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 --2- 172.21.15.32:0/2264020397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c143400 0x7f258c143890 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f2584002f70 tx=0x7f2584006eb0).stop 2022-08-01T23:03:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 -- 172.21.15.32:0/2264020397 shutdown_connections 2022-08-01T23:03:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 --2- 172.21.15.32:0/2264020397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c143400 0x7f258c143890 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 --2- 172.21.15.32:0/2264020397 >> v2:172.21.15.55:3300/0 conn(0x7f258c0b7840 0x7f258c060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 -- 172.21.15.32:0/2264020397 >> 172.21.15.32:0/2264020397 conn(0x7f258c05f840 msgr2=0x7f258c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 -- 172.21.15.32:0/2264020397 shutdown_connections 2022-08-01T23:03:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 -- 172.21.15.32:0/2264020397 wait complete. 2022-08-01T23:03:28.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 Processor -- start 2022-08-01T23:03:28.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.385+0000 7f25929b6700 1 -- start start 2022-08-01T23:03:28.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 0x7f258c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f258c0b7840 0x7f258c0ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f258c143400 0x7f258c0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0aba00 con 0x7f258c0b2930 2022-08-01T23:03:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0af900 con 0x7f258c0b7840 2022-08-01T23:03:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0ad550 con 0x7f258c143400 2022-08-01T23:03:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 0x7f258c0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 0x7f258c0acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46842/0 (socket says 172.21.15.32:46842) 2022-08-01T23:03:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 -- 172.21.15.32:0/2768016531 learned_addr learned my addr 172.21.15.32:0/2768016531 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25911b3700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f258c0b7840 0x7f258c0ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25921b5700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f258c143400 0x7f258c0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 -- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f258c143400 msgr2=0x7f258c0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f258c143400 0x7f258c0b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 -- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f258c0b7840 msgr2=0x7f258c0ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f258c0b7840 0x7f258c0ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 -- 172.21.15.32:0/2768016531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258400b1e0 con 0x7f258c0b2930 2022-08-01T23:03:28.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25921b5700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f258c143400 0x7f258c0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:28.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25919b4700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 0x7f258c0acf20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f258800a000 tx=0x7f2588008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:28.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2588005a70 con 0x7f258c0b2930 2022-08-01T23:03:28.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588005bd0 con 0x7f258c0b2930 2022-08-01T23:03:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c0b6890 con 0x7f258c0b2930 2022-08-01T23:03:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2588006030 con 0x7f258c0b2930 2022-08-01T23:03:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c0b0e10 con 0x7f258c0b2930 2022-08-01T23:03:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2588006190 con 0x7f258c0b2930 2022-08-01T23:03:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.389+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c0b0130 con 0x7f258c0b2930 2022-08-01T23:03:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.393+0000 7f257affd700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f256c053950 0x7f256c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.393+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f258806bcf0 con 0x7f258c0b2930 2022-08-01T23:03:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.393+0000 7f25911b3700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f256c053950 0x7f256c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.393+0000 7f25911b3700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f256c053950 0x7f256c055e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f257c00b000 tx=0x7f257c007160).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:28.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.397+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2588042d90 con 0x7f258c0b2930 2022-08-01T23:03:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.561+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f258c0adfa0 con 0x7f258c0b2930 2022-08-01T23:03:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f257affd700 1 -- 172.21.15.32:0/2768016531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f258c0adfa0 con 0x7f258c0b2930 2022-08-01T23:03:28.569 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:28.569 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:28.570 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:03:28.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.570 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:03:28.570 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:03:28.571 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.571 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:28.571 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.571 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.571 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.572 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:03:28.573 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:03:28.573 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.573 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:28.573 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.573 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.574 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.575 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:03:28.575 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:03:28.575 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.575 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:28.575 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.576 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.576 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.577 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:03:28.577 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:03:28.577 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.577 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.578 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.578 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.579 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:03:28.579 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:03:28.579 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.579 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.580 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.580 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.581 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:03:28.581 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:03:28.581 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.581 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.582 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.583 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:03:28.583 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:03:28.584 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.584 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.584 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.584 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.585 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:03:28.585 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:03:28.586 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.586 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.586 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.586 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.587 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:03:28.587 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:03:28.588 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.588 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.588 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.589 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:03:28.589 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:03:28.589 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.590 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.591 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:03:28.591 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:03:28.592 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.592 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.592 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.592 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.593 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:03:28.593 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:03:28.594 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.594 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:28.594 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:03:28.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:03:28.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.595 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:03:28.595 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:03:28.595 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.596 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:28.596 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:03:28.596 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:03:28.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.597 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:03:28.597 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:03:28.597 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.598 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:28.598 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:28.598 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:28.598 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.599 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:03:28.599 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:03:28.600 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.600 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:28.600 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:03:28.600 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:28.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:28.601 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:03:28.601 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:03:28.602 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:28.602 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:28.602 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:03:28.602 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:28.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:28.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:28.603 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f256c053950 msgr2=0x7f256c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f256c053950 0x7f256c055e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f257c00b000 tx=0x7f257c007160).stop 2022-08-01T23:03:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 msgr2=0x7f258c0acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 0x7f258c0acf20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f258800a000 tx=0x7f2588008040).stop 2022-08-01T23:03:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 shutdown_connections 2022-08-01T23:03:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f256c053950 0x7f256c055e10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f258c143400 0x7f258c0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f258c0b7840 0x7f258c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 --2- 172.21.15.32:0/2768016531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f258c0b2930 0x7f258c0acf20 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 >> 172.21.15.32:0/2768016531 conn(0x7f258c05f840 msgr2=0x7f258c145a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 shutdown_connections 2022-08-01T23:03:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.565+0000 7f25929b6700 1 -- 172.21.15.32:0/2768016531 wait complete. 2022-08-01T23:03:28.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4ffff700 1 Processor -- start 2022-08-01T23:03:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4ffff700 1 -- start start 2022-08-01T23:03:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8c500b2710 0x7f8c5014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500a7250 con 0x7f8c500b01a0 2022-08-01T23:03:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500a7480 con 0x7f8c500a8c50 2022-08-01T23:03:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500a7720 con 0x7f8c500b2710 2022-08-01T23:03:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c500b01a0 0x7f8c500a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34392/0 (socket says 172.21.15.32:34392) 2022-08-01T23:03:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.709+0000 7f8c4e7fc700 1 -- 172.21.15.32:0/4079308665 learned_addr learned my addr 172.21.15.32:0/4079308665 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 --2- 172.21.15.32:0/4079308665 >> v2:172.21.15.55:3300/0 conn(0x7f8c500b2710 0x7f8c5014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089331073 0 0) 0x7f8c500a7250 con 0x7f8c500b01a0 2022-08-01T23:03:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c34003260 con 0x7f8c500b01a0 2022-08-01T23:03:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186536856 0 0) 0x7f8c500a7480 con 0x7f8c500a8c50 2022-08-01T23:03:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c500a7250 con 0x7f8c500a8c50 2022-08-01T23:03:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3489327243 0 0) 0x7f8c34003260 con 0x7f8c500b01a0 2022-08-01T23:03:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 >> v2:172.21.15.55:3300/0 conn(0x7f8c500b2710 msgr2=0x7f8c5014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 --2- 172.21.15.32:0/4079308665 >> v2:172.21.15.55:3300/0 conn(0x7f8c500b2710 0x7f8c5014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 >> v1:172.21.15.55:6789/0 conn(0x7f8c500a8c50 legacy=0x7f8c50151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500a79c0 con 0x7f8c500b01a0 2022-08-01T23:03:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f8c400019e0 con 0x7f8c500b01a0 2022-08-01T23:03:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 >> v1:172.21.15.32:6789/0 conn(0x7f8c500b01a0 legacy=0x7f8c500a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340038e0 0x7f8c34003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c34004500 0x7f8c34004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c34004e70 0x7f8c3400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500a7720 con 0x7f8c340038e0 2022-08-01T23:03:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3400bcf0 con 0x7f8c34004e70 2022-08-01T23:03:28.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3400beb0 con 0x7f8c34004500 2022-08-01T23:03:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340038e0 0x7f8c34003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c34004e70 0x7f8c3400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4e7fc700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c34004500 0x7f8c34004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 -- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c34004500 msgr2=0x7f8c34004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c34004500 0x7f8c34004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 -- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c34004e70 msgr2=0x7f8c3400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c34004e70 0x7f8c3400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4f7fe700 1 -- 172.21.15.32:0/4079308665 reap_dead start 2022-08-01T23:03:28.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 -- 172.21.15.32:0/4079308665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34003260 con 0x7f8c340038e0 2022-08-01T23:03:28.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4e7fc700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c34004500 0x7f8c34004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:28.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4effd700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340038e0 0x7f8c34003ce0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f8c4400c000 tx=0x7f8c44006df0).ready entity=mon.0 client_cookie=50eca0f3fac4bc23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:28.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4400e800 con 0x7f8c340038e0 2022-08-01T23:03:28.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400e960 con 0x7f8c340038e0 2022-08-01T23:03:28.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4dffb700 1 -- 172.21.15.32:0/4079308665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c4400ecc0 con 0x7f8c340038e0 2022-08-01T23:03:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 -- 172.21.15.32:0/4079308665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340038e0 msgr2=0x7f8c34003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340038e0 0x7f8c34003ce0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f8c4400c000 tx=0x7f8c44006df0).stop 2022-08-01T23:03:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 -- 172.21.15.32:0/4079308665 shutdown_connections 2022-08-01T23:03:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/4079308665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340038e0 0x7f8c34003ce0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 -- 172.21.15.32:0/4079308665 >> 172.21.15.32:0/4079308665 conn(0x7f8c50060310 msgr2=0x7f8c500b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:28.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 -- 172.21.15.32:0/4079308665 shutdown_connections 2022-08-01T23:03:28.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.713+0000 7f8c4ffff700 1 -- 172.21.15.32:0/4079308665 wait complete. 2022-08-01T23:03:28.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 Processor -- start 2022-08-01T23:03:28.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 -- start start 2022-08-01T23:03:28.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c340038e0 0x7f8c5014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c50063190 0x7f8c50064980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 0x7f8c500aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50066150 con 0x7f8c50064e70 2022-08-01T23:03:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50064470 con 0x7f8c340038e0 2022-08-01T23:03:28.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500ac360 con 0x7f8c50063190 2022-08-01T23:03:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 0x7f8c500aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 0x7f8c500aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46866/0 (socket says 172.21.15.32:46866) 2022-08-01T23:03:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 -- 172.21.15.32:0/2422135161 learned_addr learned my addr 172.21.15.32:0/2422135161 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:28.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4e7fc700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c50063190 0x7f8c50064980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4effd700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c340038e0 0x7f8c5014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 -- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c50063190 msgr2=0x7f8c50064980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c50063190 0x7f8c50064980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 -- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c340038e0 msgr2=0x7f8c5014ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c340038e0 0x7f8c5014ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.717+0000 7f8c4f7fe700 1 -- 172.21.15.32:0/2422135161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500641d0 con 0x7f8c50064e70 2022-08-01T23:03:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c4f7fe700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 0x7f8c500aa260 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f8c50061c10 tx=0x7f8c48007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:28.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c48007240 con 0x7f8c50064e70 2022-08-01T23:03:28.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c480073a0 con 0x7f8c50064e70 2022-08-01T23:03:28.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c48006700 con 0x7f8c50064e70 2022-08-01T23:03:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c500ab690 con 0x7f8c50064e70 2022-08-01T23:03:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c50065300 con 0x7f8c50064e70 2022-08-01T23:03:28.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50064670 con 0x7f8c50064e70 2022-08-01T23:03:28.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c48006860 con 0x7f8c50064e70 2022-08-01T23:03:28.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c3ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c2c053b00 0x7f8c2c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:28.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8c4806b400 con 0x7f8c50064e70 2022-08-01T23:03:28.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c4effd700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c2c053b00 0x7f8c2c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:28.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.721+0000 7f8c4effd700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c2c053b00 0x7f8c2c055fc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8c4401d000 tx=0x7f8c44019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:28.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.725+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c48042460 con 0x7f8c50064e70 2022-08-01T23:03:28.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.853+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c500ac520 con 0x7f8c50064e70 2022-08-01T23:03:28.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.857+0000 7f8c3ffff700 1 -- 172.21.15.32:0/2422135161 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13416 (secure 0 0 0) 0x7f8c48045a80 con 0x7f8c50064e70 2022-08-01T23:03:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:28.863 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":103,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:28.248973+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:28.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c2c053b00 msgr2=0x7f8c2c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c2c053b00 0x7f8c2c055fc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8c4401d000 tx=0x7f8c44019000).stop 2022-08-01T23:03:28.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 msgr2=0x7f8c500aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 0x7f8c500aa260 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f8c50061c10 tx=0x7f8c48007a00).stop 2022-08-01T23:03:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 shutdown_connections 2022-08-01T23:03:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8c2c053b00 0x7f8c2c055fc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c50064e70 0x7f8c500aa260 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8c50063190 0x7f8c50064980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 --2- 172.21.15.32:0/2422135161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c340038e0 0x7f8c5014ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:28.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 >> 172.21.15.32:0/2422135161 conn(0x7f8c50060310 msgr2=0x7f8c500ad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:28.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 shutdown_connections 2022-08-01T23:03:28.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.861+0000 7f8c4ffff700 1 -- 172.21.15.32:0/2422135161 wait complete. 2022-08-01T23:03:28.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:03:28.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:28.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.989+0000 7f7f1ef67700 1 Processor -- start 2022-08-01T23:03:28.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.989+0000 7f7f1ef67700 1 -- start start 2022-08-01T23:03:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f181445d0 0x7f7f18144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f180655b0 0x7f7f18144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f181454e0 con 0x7f7f18060290 2022-08-01T23:03:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f18065e10 con 0x7f7f180655b0 2022-08-01T23:03:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f18060670 con 0x7f7f181445d0 2022-08-01T23:03:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f180655b0 0x7f7f18144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f180655b0 0x7f7f18144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46872/0 (socket says 172.21.15.32:46872) 2022-08-01T23:03:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 -- 172.21.15.32:0/2250714643 learned_addr learned my addr 172.21.15.32:0/2250714643 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1cd03700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f18060290 0x7f7f18064ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34412/0 (socket says 172.21.15.32:34412) 2022-08-01T23:03:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f17fff700 1 --2- 172.21.15.32:0/2250714643 >> v2:172.21.15.55:3300/0 conn(0x7f7f181445d0 0x7f7f18144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 -- 172.21.15.32:0/2250714643 >> v2:172.21.15.55:3300/0 conn(0x7f7f181445d0 msgr2=0x7f7f18144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:29.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 --2- 172.21.15.32:0/2250714643 >> v2:172.21.15.55:3300/0 conn(0x7f7f181445d0 0x7f7f18144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 -- 172.21.15.32:0/2250714643 >> v1:172.21.15.32:6789/0 conn(0x7f7f18060290 legacy=0x7f7f18064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f177fe700 1 -- 172.21.15.32:0/2250714643 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1662286819 0 0) 0x7f7f181454e0 con 0x7f7f18060290 2022-08-01T23:03:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 -- 172.21.15.32:0/2250714643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180b2540 con 0x7f7f180655b0 2022-08-01T23:03:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1d504700 1 --2- 172.21.15.32:0/2250714643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f180655b0 0x7f7f18144ff0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f7f100032f0 tx=0x7f7f10009290).ready entity=mon.0 client_cookie=c3d0e2edfe84bc25 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f177fe700 1 -- 172.21.15.32:0/2250714643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f1000b870 con 0x7f7f180655b0 2022-08-01T23:03:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f177fe700 1 -- 172.21.15.32:0/2250714643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1000b9d0 con 0x7f7f180655b0 2022-08-01T23:03:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f177fe700 1 -- 172.21.15.32:0/2250714643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f1000bb40 con 0x7f7f180655b0 2022-08-01T23:03:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- 172.21.15.32:0/2250714643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f180655b0 msgr2=0x7f7f18144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/2250714643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f180655b0 0x7f7f18144ff0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f7f100032f0 tx=0x7f7f10009290).stop 2022-08-01T23:03:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- 172.21.15.32:0/2250714643 shutdown_connections 2022-08-01T23:03:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/2250714643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f180655b0 0x7f7f18144ff0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/2250714643 >> v2:172.21.15.55:3300/0 conn(0x7f7f181445d0 0x7f7f18144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- 172.21.15.32:0/2250714643 >> 172.21.15.32:0/2250714643 conn(0x7f7f1805d0f0 msgr2=0x7f7f1805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- 172.21.15.32:0/2250714643 shutdown_connections 2022-08-01T23:03:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- 172.21.15.32:0/2250714643 wait complete. 2022-08-01T23:03:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 Processor -- start 2022-08-01T23:03:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.993+0000 7f7f1ef67700 1 -- start start 2022-08-01T23:03:29.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 0x7f7f1813f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f180655b0 0x7f7f180af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7f180afe10 0x7f7f180b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180aeff0 con 0x7f7f18060290 2022-08-01T23:03:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1813dc50 con 0x7f7f180655b0 2022-08-01T23:03:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180b0270 con 0x7f7f180afe10 2022-08-01T23:03:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 0x7f7f1813f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 0x7f7f1813f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46878/0 (socket says 172.21.15.32:46878) 2022-08-01T23:03:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 -- 172.21.15.32:0/3562706209 learned_addr learned my addr 172.21.15.32:0/3562706209 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f17fff700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f180655b0 0x7f7f180af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1d504700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7f180afe10 0x7f7f180b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 -- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7f180afe10 msgr2=0x7f7f180b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7f180afe10 0x7f7f180b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 -- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f180655b0 msgr2=0x7f7f180af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f180655b0 0x7f7f180af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 -- 172.21.15.32:0/3562706209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10009530 con 0x7f7f18060290 2022-08-01T23:03:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1cd03700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 0x7f7f1813f490 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f7f0c000ed0 tx=0x7f7f0c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f0c0070d0 con 0x7f7f18060290 2022-08-01T23:03:29.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c007230 con 0x7f7f18060290 2022-08-01T23:03:29.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1813de60 con 0x7f7f18060290 2022-08-01T23:03:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f0c005740 con 0x7f7f18060290 2022-08-01T23:03:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1813b740 con 0x7f7f18060290 2022-08-01T23:03:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:28.997+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f180574e0 con 0x7f7f18060290 2022-08-01T23:03:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.001+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f0c01a060 con 0x7f7f18060290 2022-08-01T23:03:29.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.001+0000 7f7f15ffb700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ef8053950 0x7f7ef8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.001+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7f0c06b0c0 con 0x7f7f18060290 2022-08-01T23:03:29.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.001+0000 7f7f17fff700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ef8053950 0x7f7ef8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.001+0000 7f7f17fff700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ef8053950 0x7f7ef8055e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f7f0800b000 tx=0x7f7f08006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.005+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f0c042140 con 0x7f7f18060290 2022-08-01T23:03:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.133+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7f18069a70 con 0x7f7f18060290 2022-08-01T23:03:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f15ffb700 1 -- 172.21.15.32:0/3562706209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7f0c018070 con 0x7f7f18060290 2022-08-01T23:03:29.143 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ef8053950 msgr2=0x7f7ef8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ef8053950 0x7f7ef8055e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f7f0800b000 tx=0x7f7f08006df0).stop 2022-08-01T23:03:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 msgr2=0x7f7f1813f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 0x7f7f1813f490 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f7f0c000ed0 tx=0x7f7f0c007ac0).stop 2022-08-01T23:03:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 shutdown_connections 2022-08-01T23:03:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7ef8053950 0x7f7ef8055e10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7f180afe10 0x7f7f180b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f180655b0 0x7f7f180af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 --2- 172.21.15.32:0/3562706209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f18060290 0x7f7f1813f490 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 >> 172.21.15.32:0/3562706209 conn(0x7f7f1805d0f0 msgr2=0x7f7f18140360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 shutdown_connections 2022-08-01T23:03:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.137+0000 7f7f1ef67700 1 -- 172.21.15.32:0/3562706209 wait complete. 2022-08-01T23:03:29.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 Processor -- start 2022-08-01T23:03:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 -- start start 2022-08-01T23:03:29.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20bc0b41f0 0x7f20bc061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc142ff0 0x7f20bc1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20bc0b60e0 con 0x7f20bc0b6420 2022-08-01T23:03:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20bc0b2100 con 0x7f20bc0b41f0 2022-08-01T23:03:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c1035700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc0b23a0 con 0x7f20bc142ff0 2022-08-01T23:03:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c0834700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc142ff0 0x7f20bc1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20bbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f20bc0b6420 0x7f20bc061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:55332/0 (socket says 172.21.15.32:55332) 2022-08-01T23:03:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20bbfff700 1 -- 172.21.15.32:0/2226177184 learned_addr learned my addr 172.21.15.32:0/2226177184 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20bb7fe700 1 --2- 172.21.15.32:0/2226177184 >> v2:172.21.15.55:3300/0 conn(0x7f20bc0b41f0 0x7f20bc061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20baffd700 1 -- 172.21.15.32:0/2226177184 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180725135 0 0) 0x7f20ac001110 con 0x7f20bc0b6420 2022-08-01T23:03:29.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20baffd700 1 -- 172.21.15.32:0/2226177184 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209c003260 con 0x7f20bc0b6420 2022-08-01T23:03:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c0834700 1 -- 172.21.15.32:0/2226177184 >> v2:172.21.15.55:3300/0 conn(0x7f20bc0b41f0 msgr2=0x7f20bc061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c0834700 1 --2- 172.21.15.32:0/2226177184 >> v2:172.21.15.55:3300/0 conn(0x7f20bc0b41f0 0x7f20bc061d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c0834700 1 -- 172.21.15.32:0/2226177184 >> v1:172.21.15.55:6789/0 conn(0x7f20bc0b6420 legacy=0x7f20bc061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c0834700 1 -- 172.21.15.32:0/2226177184 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc0b2640 con 0x7f20bc142ff0 2022-08-01T23:03:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20bb7fe700 1 --2- 172.21.15.32:0/2226177184 >> v2:172.21.15.55:3300/0 conn(0x7f20bc0b41f0 0x7f20bc061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20c0834700 1 --2- 172.21.15.32:0/2226177184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc142ff0 0x7f20bc1434a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f20b40032f0 tx=0x7f20b40091d0).ready entity=mon.2 client_cookie=360cb9c022bf436 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20baffd700 1 -- 172.21.15.32:0/2226177184 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20b400b600 con 0x7f20bc142ff0 2022-08-01T23:03:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20baffd700 1 -- 172.21.15.32:0/2226177184 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b400b760 con 0x7f20bc142ff0 2022-08-01T23:03:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.261+0000 7f20baffd700 1 -- 172.21.15.32:0/2226177184 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20b400b8d0 con 0x7f20bc142ff0 2022-08-01T23:03:29.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- 172.21.15.32:0/2226177184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc142ff0 msgr2=0x7f20bc1434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 --2- 172.21.15.32:0/2226177184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc142ff0 0x7f20bc1434a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f20b40032f0 tx=0x7f20b40091d0).stop 2022-08-01T23:03:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- 172.21.15.32:0/2226177184 shutdown_connections 2022-08-01T23:03:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 --2- 172.21.15.32:0/2226177184 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc142ff0 0x7f20bc1434a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 --2- 172.21.15.32:0/2226177184 >> v2:172.21.15.55:3300/0 conn(0x7f20bc0b41f0 0x7f20bc061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- 172.21.15.32:0/2226177184 >> 172.21.15.32:0/2226177184 conn(0x7f20bc05c620 msgr2=0x7f20bc05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- 172.21.15.32:0/2226177184 shutdown_connections 2022-08-01T23:03:29.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- 172.21.15.32:0/2226177184 wait complete. 2022-08-01T23:03:29.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 Processor -- start 2022-08-01T23:03:29.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- start start 2022-08-01T23:03:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20bc0b41f0 0x7f20bc146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc0b6420 0x7f20bc12af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 0x7f20bc12b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc0b08c0 con 0x7f20bc142ff0 2022-08-01T23:03:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc12b920 con 0x7f20bc0b41f0 2022-08-01T23:03:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c1035700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc0afee0 con 0x7f20bc0b6420 2022-08-01T23:03:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c0834700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 0x7f20bc12b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c0834700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 0x7f20bc12b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46894/0 (socket says 172.21.15.32:46894) 2022-08-01T23:03:29.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c0834700 1 -- 172.21.15.32:0/1864748369 learned_addr learned my addr 172.21.15.32:0/1864748369 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:29.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20bb7fe700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc0b6420 0x7f20bc12af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20bbfff700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20bc0b41f0 0x7f20bc146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.265+0000 7f20c0834700 1 -- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc0b6420 msgr2=0x7f20bc12af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c0834700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc0b6420 0x7f20bc12af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c0834700 1 -- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20bc0b41f0 msgr2=0x7f20bc146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c0834700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20bc0b41f0 0x7f20bc146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c0834700 1 -- 172.21.15.32:0/1864748369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a400b180 con 0x7f20bc142ff0 2022-08-01T23:03:29.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c0834700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 0x7f20bc12b430 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f20b40032f0 tx=0x7f20b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20b400b8d0 con 0x7f20bc142ff0 2022-08-01T23:03:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b40107a0 con 0x7f20bc142ff0 2022-08-01T23:03:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b4009470 con 0x7f20bc142ff0 2022-08-01T23:03:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20b400b600 con 0x7f20bc142ff0 2022-08-01T23:03:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc0b0420 con 0x7f20bc142ff0 2022-08-01T23:03:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20bc0b28e0 con 0x7f20bc142ff0 2022-08-01T23:03:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f20b4027030 con 0x7f20bc142ff0 2022-08-01T23:03:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20b97fa700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f20a0053a70 0x7f20a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f20b406cc70 con 0x7f20bc142ff0 2022-08-01T23:03:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.269+0000 7f20bbfff700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f20a0053a70 0x7f20a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.273+0000 7f20bbfff700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f20a0053a70 0x7f20a0055f30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f20ac00b000 tx=0x7f20ac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.277+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f20b40476a0 con 0x7f20bc142ff0 2022-08-01T23:03:29.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.401+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20bc0574e0 con 0x7f20bc142ff0 2022-08-01T23:03:29.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20b97fa700 1 -- 172.21.15.32:0/1864748369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20b4015020 con 0x7f20bc142ff0 2022-08-01T23:03:29.411 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:29.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f20a0053a70 msgr2=0x7f20a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f20a0053a70 0x7f20a0055f30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f20ac00b000 tx=0x7f20ac009040).stop 2022-08-01T23:03:29.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 msgr2=0x7f20bc12b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 0x7f20bc12b430 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f20b40032f0 tx=0x7f20b4016000).stop 2022-08-01T23:03:29.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 shutdown_connections 2022-08-01T23:03:29.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f20a0053a70 0x7f20a0055f30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20bc142ff0 0x7f20bc12b430 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f20bc0b6420 0x7f20bc12af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 --2- 172.21.15.32:0/1864748369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20bc0b41f0 0x7f20bc146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 >> 172.21.15.32:0/1864748369 conn(0x7f20bc05c620 msgr2=0x7f20bc0b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 shutdown_connections 2022-08-01T23:03:29.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.405+0000 7f20c1035700 1 -- 172.21.15.32:0/1864748369 wait complete. 2022-08-01T23:03:29.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.521+0000 7f409ad3d700 1 Processor -- start 2022-08-01T23:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.521+0000 7f409ad3d700 1 -- start start 2022-08-01T23:03:29.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40940b15f0 0x7f40941411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40941416c0 0x7f4094141b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940af0b0 con 0x7f40940b5490 2022-08-01T23:03:29.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40940ac490 con 0x7f40941416c0 2022-08-01T23:03:29.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40940af2e0 con 0x7f40940b15f0 2022-08-01T23:03:29.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40941416c0 0x7f4094141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f4099d3b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40940b5490 0x7f409413e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34440/0 (socket says 172.21.15.32:34440) 2022-08-01T23:03:29.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40941416c0 0x7f4094141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46902/0 (socket says 172.21.15.32:46902) 2022-08-01T23:03:29.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f4099d3b700 1 -- 172.21.15.32:0/2720779460 learned_addr learned my addr 172.21.15.32:0/2720779460 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:29.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409953a700 1 --2- 172.21.15.32:0/2720779460 >> v2:172.21.15.55:3300/0 conn(0x7f40940b15f0 0x7f40941411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 -- 172.21.15.32:0/2720779460 >> v2:172.21.15.55:3300/0 conn(0x7f40940b15f0 msgr2=0x7f40941411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:29.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 --2- 172.21.15.32:0/2720779460 >> v2:172.21.15.55:3300/0 conn(0x7f40940b15f0 0x7f40941411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 -- 172.21.15.32:0/2720779460 >> v1:172.21.15.32:6789/0 conn(0x7f40940b5490 legacy=0x7f409413e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 -- 172.21.15.32:0/2720779460 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940af580 con 0x7f40941416c0 2022-08-01T23:03:29.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409a53c700 1 --2- 172.21.15.32:0/2720779460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40941416c0 0x7f4094141b10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f4088002f70 tx=0x7f4088006df0).ready entity=mon.0 client_cookie=e86408fd5d87d0a4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f4098d39700 1 -- 172.21.15.32:0/2720779460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408800d570 con 0x7f40941416c0 2022-08-01T23:03:29.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f4098d39700 1 -- 172.21.15.32:0/2720779460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4088000e90 con 0x7f40941416c0 2022-08-01T23:03:29.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f4098d39700 1 -- 172.21.15.32:0/2720779460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408800d8c0 con 0x7f40941416c0 2022-08-01T23:03:29.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- 172.21.15.32:0/2720779460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40941416c0 msgr2=0x7f4094141b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 --2- 172.21.15.32:0/2720779460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40941416c0 0x7f4094141b10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f4088002f70 tx=0x7f4088006df0).stop 2022-08-01T23:03:29.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- 172.21.15.32:0/2720779460 shutdown_connections 2022-08-01T23:03:29.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 --2- 172.21.15.32:0/2720779460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40941416c0 0x7f4094141b10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 --2- 172.21.15.32:0/2720779460 >> v2:172.21.15.55:3300/0 conn(0x7f40940b15f0 0x7f40941411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- 172.21.15.32:0/2720779460 >> 172.21.15.32:0/2720779460 conn(0x7f40940626f0 msgr2=0x7f4094062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- 172.21.15.32:0/2720779460 shutdown_connections 2022-08-01T23:03:29.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.525+0000 7f409ad3d700 1 -- 172.21.15.32:0/2720779460 wait complete. 2022-08-01T23:03:29.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 Processor -- start 2022-08-01T23:03:29.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 -- start start 2022-08-01T23:03:29.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40940b15f0 0x7f40941505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 0x7f4094154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40941416c0 0x7f4094139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094153bc0 con 0x7f40940b5490 2022-08-01T23:03:29.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409413d200 con 0x7f40940b15f0 2022-08-01T23:03:29.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409ad3d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4094153df0 con 0x7f40941416c0 2022-08-01T23:03:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 0x7f4094154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 0x7f4094154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46906/0 (socket says 172.21.15.32:46906) 2022-08-01T23:03:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 -- 172.21.15.32:0/395783813 learned_addr learned my addr 172.21.15.32:0/395783813 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:29.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f4099d3b700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40940b15f0 0x7f40941505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409a53c700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40941416c0 0x7f4094139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 -- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40941416c0 msgr2=0x7f4094139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40941416c0 0x7f4094139450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 -- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40940b15f0 msgr2=0x7f40941505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40940b15f0 0x7f40941505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 -- 172.21.15.32:0/395783813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408800b180 con 0x7f40940b5490 2022-08-01T23:03:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f409953a700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 0x7f4094154290 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f408c00c000 tx=0x7f408c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.529+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c012600 con 0x7f40940b5490 2022-08-01T23:03:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c012760 con 0x7f40940b5490 2022-08-01T23:03:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409413d4a0 con 0x7f40940b5490 2022-08-01T23:03:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c012ac0 con 0x7f40940b5490 2022-08-01T23:03:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409413a960 con 0x7f40940b5490 2022-08-01T23:03:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40940afd80 con 0x7f40940b5490 2022-08-01T23:03:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f408c022070 con 0x7f40940b5490 2022-08-01T23:03:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4086ffd700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4074053a70 0x7f4074055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f408c06c380 con 0x7f40940b5490 2022-08-01T23:03:29.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4099d3b700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4074053a70 0x7f4074055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.533+0000 7f4099d3b700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4074053a70 0x7f4074055f30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f409000d000 tx=0x7f4090007be0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.537+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f408c0432f0 con 0x7f40940b5490 2022-08-01T23:03:29.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.673+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4094153fc0 con 0x7f40940b5490 2022-08-01T23:03:29.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.673+0000 7f4086ffd700 1 -- 172.21.15.32:0/395783813 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+143 (secure 0 0 0) 0x7f408c046910 con 0x7f40940b5490 2022-08-01T23:03:29.681 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:29.682 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:03:29.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:29.683 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:03:29.683 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:03:29.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:29.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:29.684 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:03:29.685 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:03:29.685 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:29.686 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:03:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4074053a70 msgr2=0x7f4074055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4074053a70 0x7f4074055f30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f409000d000 tx=0x7f4090007be0).stop 2022-08-01T23:03:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 msgr2=0x7f4094154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 0x7f4094154290 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f408c00c000 tx=0x7f408c0091d0).stop 2022-08-01T23:03:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 shutdown_connections 2022-08-01T23:03:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4074053a70 0x7f4074055f30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f40941416c0 0x7f4094139450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40940b5490 0x7f4094154290 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 --2- 172.21.15.32:0/395783813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40940b15f0 0x7f40941505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 >> 172.21.15.32:0/395783813 conn(0x7f40940626f0 msgr2=0x7f40940b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 shutdown_connections 2022-08-01T23:03:29.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.677+0000 7f409ad3d700 1 -- 172.21.15.32:0/395783813 wait complete. 2022-08-01T23:03:29.695 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:03:29.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:03:29.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f6283fff700 1 Processor -- start 2022-08-01T23:03:29.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f6283fff700 1 -- start start 2022-08-01T23:03:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f6283fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6284140b00 0x7f6284064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f6283fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840b4f60 con 0x7f62840b38a0 2022-08-01T23:03:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f6283fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840b4a60 con 0x7f62840647d0 2022-08-01T23:03:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f6283fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62840b4cc0 con 0x7f6284140b00 2022-08-01T23:03:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f627a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f62840b38a0 0x7f62841403f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34454/0 (socket says 172.21.15.32:34454) 2022-08-01T23:03:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f627a7fc700 1 -- 172.21.15.32:0/3999850622 learned_addr learned my addr 172.21.15.32:0/3999850622 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f62837fe700 1 --2- 172.21.15.32:0/3999850622 >> v2:172.21.15.55:3300/0 conn(0x7f6284140b00 0x7f6284064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991207078 0 0) 0x7f62840b4f60 con 0x7f62840b38a0 2022-08-01T23:03:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.801+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6264003260 con 0x7f62840b38a0 2022-08-01T23:03:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296693883 0 0) 0x7f6264003260 con 0x7f62840b38a0 2022-08-01T23:03:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 >> v2:172.21.15.55:3300/0 conn(0x7f6284140b00 msgr2=0x7f6284064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 --2- 172.21.15.32:0/3999850622 >> v2:172.21.15.55:3300/0 conn(0x7f6284140b00 0x7f6284064bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 >> v1:172.21.15.55:6789/0 conn(0x7f62840647d0 legacy=0x7f62841436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284143250 con 0x7f62840b38a0 2022-08-01T23:03:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f626c001660 con 0x7f62840b38a0 2022-08-01T23:03:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 >> v1:172.21.15.32:6789/0 conn(0x7f62840b38a0 legacy=0x7f62841403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6264003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6264004500 0x7f6264004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6264004e70 0x7f626400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840b4cc0 con 0x7f6264003750 2022-08-01T23:03:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626400bd20 con 0x7f6264004500 2022-08-01T23:03:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626400bee0 con 0x7f6264004e70 2022-08-01T23:03:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6264003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f627a7fc700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6264004500 0x7f6264004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62837fe700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6264004e70 0x7f626400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 -- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6264004e70 msgr2=0x7f626400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6264004e70 0x7f626400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 -- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6264004500 msgr2=0x7f6264004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6264004500 0x7f6264004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62837fe700 1 -- 172.21.15.32:0/3999850622 reap_dead start 2022-08-01T23:03:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 -- 172.21.15.32:0/3999850622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264003260 con 0x7f6264003750 2022-08-01T23:03:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f627a7fc700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6264004500 0x7f6264004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6282ffd700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6264003b70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f6284063dd0 tx=0x7f62740079e0).ready entity=mon.0 client_cookie=8787ffb0c5299115 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6274001ea0 con 0x7f6264003750 2022-08-01T23:03:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6274014410 con 0x7f6264003750 2022-08-01T23:03:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f62827fc700 1 -- 172.21.15.32:0/3999850622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6274014770 con 0x7f6264003750 2022-08-01T23:03:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 -- 172.21.15.32:0/3999850622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 msgr2=0x7f6264003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6264003b70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f6284063dd0 tx=0x7f62740079e0).stop 2022-08-01T23:03:29.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 -- 172.21.15.32:0/3999850622 shutdown_connections 2022-08-01T23:03:29.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 --2- 172.21.15.32:0/3999850622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6264003b70 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 -- 172.21.15.32:0/3999850622 >> 172.21.15.32:0/3999850622 conn(0x7f628405f840 msgr2=0x7f628405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 -- 172.21.15.32:0/3999850622 shutdown_connections 2022-08-01T23:03:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 -- 172.21.15.32:0/3999850622 wait complete. 2022-08-01T23:03:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 Processor -- start 2022-08-01T23:03:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.805+0000 7f6283fff700 1 -- start start 2022-08-01T23:03:29.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6284152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62841442e0 0x7f62840b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f62841497b0 0x7f6284148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284149c30 con 0x7f6264003750 2022-08-01T23:03:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840b1690 con 0x7f62841442e0 2022-08-01T23:03:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6284147c20 con 0x7f62841497b0 2022-08-01T23:03:29.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6284152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6284152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46924/0 (socket says 172.21.15.32:46924) 2022-08-01T23:03:29.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 -- 172.21.15.32:0/1362007644 learned_addr learned my addr 172.21.15.32:0/1362007644 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:29.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f627a7fc700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62841442e0 0x7f62840b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 -- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f62841497b0 msgr2=0x7f6284148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f62837fe700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f62841497b0 0x7f6284148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f62841497b0 0x7f6284148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 -- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62841442e0 msgr2=0x7f62840b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:29.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62841442e0 0x7f62840b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:29.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 -- 172.21.15.32:0/1362007644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284147e50 con 0x7f6264003750 2022-08-01T23:03:29.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f627a7fc700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62841442e0 0x7f62840b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:29.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f62837fe700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f62841497b0 0x7f6284148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:29.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6282ffd700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6284152fc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f62840b17d0 tx=0x7f6274015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6274001ea0 con 0x7f6264003750 2022-08-01T23:03:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6274018070 con 0x7f6264003750 2022-08-01T23:03:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62841480f0 con 0x7f6264003750 2022-08-01T23:03:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6274007cd0 con 0x7f6264003750 2022-08-01T23:03:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6284148d20 con 0x7f6264003750 2022-08-01T23:03:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.809+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62840574e0 con 0x7f6264003750 2022-08-01T23:03:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.813+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f627401d070 con 0x7f6264003750 2022-08-01T23:03:29.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.813+0000 7f6280ff9700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6268053a10 0x7f6268055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:29.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.813+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f627406c990 con 0x7f6264003750 2022-08-01T23:03:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.813+0000 7f627a7fc700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6268053a10 0x7f6268055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.813+0000 7f627a7fc700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6268053a10 0x7f6268055ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f626c00d000 tx=0x7f626c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:29.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.817+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6274043ad0 con 0x7f6264003750 2022-08-01T23:03:29.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:29.945+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6284069d90 con 0x7f6264003750 2022-08-01T23:03:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.649+0000 7f6280ff9700 1 -- 172.21.15.32:0/1362007644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v104) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6274026030 con 0x7f6264003750 2022-08-01T23:03:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6268053a10 msgr2=0x7f6268055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6268053a10 0x7f6268055ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f626c00d000 tx=0x7f626c009040).stop 2022-08-01T23:03:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 msgr2=0x7f6284152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6284152fc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f62840b17d0 tx=0x7f6274015000).stop 2022-08-01T23:03:30.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 shutdown_connections 2022-08-01T23:03:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6268053a10 0x7f6268055ed0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f62841497b0 0x7f6284148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62841442e0 0x7f62840b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 --2- 172.21.15.32:0/1362007644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6264003750 0x7f6284152fc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 >> 172.21.15.32:0/1362007644 conn(0x7f628405f840 msgr2=0x7f628405ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:30.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 shutdown_connections 2022-08-01T23:03:30.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.653+0000 7f6283fff700 1 -- 172.21.15.32:0/1362007644 wait complete. 2022-08-01T23:03:30.664 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:03:30.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:03:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.781+0000 7f73a3ff5700 1 Processor -- start 2022-08-01T23:03:30.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.781+0000 7f73a3ff5700 1 -- start start 2022-08-01T23:03:30.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f739c0b28a0 0x7f739c1432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:30.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c061360 0x7f739c143790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:30.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c064aa0 con 0x7f739c0600f0 2022-08-01T23:03:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739c064d40 con 0x7f739c0b28a0 2022-08-01T23:03:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c064fe0 con 0x7f739c061360 2022-08-01T23:03:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a1d91700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f739c0600f0 0x7f739c1405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34474/0 (socket says 172.21.15.32:34474) 2022-08-01T23:03:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a1d91700 1 -- 172.21.15.32:0/3261857306 learned_addr learned my addr 172.21.15.32:0/3261857306 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a2592700 1 --2- 172.21.15.32:0/3261857306 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c061360 0x7f739c143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a1590700 1 --2- 172.21.15.32:0/3261857306 >> v2:172.21.15.55:3300/0 conn(0x7f739c0b28a0 0x7f739c1432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a0d8f700 1 -- 172.21.15.32:0/3261857306 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618270420 0 0) 0x7f739c064aa0 con 0x7f739c0600f0 2022-08-01T23:03:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a0d8f700 1 -- 172.21.15.32:0/3261857306 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7384003260 con 0x7f739c0600f0 2022-08-01T23:03:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a2592700 1 -- 172.21.15.32:0/3261857306 >> v2:172.21.15.55:3300/0 conn(0x7f739c0b28a0 msgr2=0x7f739c1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a2592700 1 --2- 172.21.15.32:0/3261857306 >> v2:172.21.15.55:3300/0 conn(0x7f739c0b28a0 0x7f739c1432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a2592700 1 -- 172.21.15.32:0/3261857306 >> v1:172.21.15.32:6789/0 conn(0x7f739c0600f0 legacy=0x7f739c1405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a2592700 1 -- 172.21.15.32:0/3261857306 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c065280 con 0x7f739c061360 2022-08-01T23:03:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a1590700 1 --2- 172.21.15.32:0/3261857306 >> v2:172.21.15.55:3300/0 conn(0x7f739c0b28a0 0x7f739c1432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a2592700 1 --2- 172.21.15.32:0/3261857306 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c061360 0x7f739c143790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f73940032f0 tx=0x7f73940091d0).ready entity=mon.2 client_cookie=7b6b44f6114d97f3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a0d8f700 1 -- 172.21.15.32:0/3261857306 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f739400b600 con 0x7f739c061360 2022-08-01T23:03:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a0d8f700 1 -- 172.21.15.32:0/3261857306 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739400b760 con 0x7f739c061360 2022-08-01T23:03:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a0d8f700 1 -- 172.21.15.32:0/3261857306 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f739400b8d0 con 0x7f739c061360 2022-08-01T23:03:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- 172.21.15.32:0/3261857306 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c061360 msgr2=0x7f739c143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/3261857306 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c061360 0x7f739c143790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f73940032f0 tx=0x7f73940091d0).stop 2022-08-01T23:03:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- 172.21.15.32:0/3261857306 shutdown_connections 2022-08-01T23:03:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/3261857306 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c061360 0x7f739c143790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/3261857306 >> v2:172.21.15.55:3300/0 conn(0x7f739c0b28a0 0x7f739c1432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- 172.21.15.32:0/3261857306 >> 172.21.15.32:0/3261857306 conn(0x7f739c05d0f0 msgr2=0x7f739c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- 172.21.15.32:0/3261857306 shutdown_connections 2022-08-01T23:03:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.785+0000 7f73a3ff5700 1 -- 172.21.15.32:0/3261857306 wait complete. 2022-08-01T23:03:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 Processor -- start 2022-08-01T23:03:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 -- start start 2022-08-01T23:03:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c0600f0 0x7f739c146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 0x7f739c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f739c0b28a0 0x7f739c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c147360 con 0x7f739c061360 2022-08-01T23:03:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0b10f0 con 0x7f739c0b28a0 2022-08-01T23:03:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a3ff5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0b1320 con 0x7f739c0600f0 2022-08-01T23:03:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 0x7f739c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 0x7f739c0af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46940/0 (socket says 172.21.15.32:46940) 2022-08-01T23:03:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 -- 172.21.15.32:0/2835265238 learned_addr learned my addr 172.21.15.32:0/2835265238 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a2592700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f739c0b28a0 0x7f739c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1d91700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c0600f0 0x7f739c146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 -- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c0600f0 msgr2=0x7f739c146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c0600f0 0x7f739c146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 -- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f739c0b28a0 msgr2=0x7f739c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f739c0b28a0 0x7f739c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 -- 172.21.15.32:0/2835265238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739800b180 con 0x7f739c061360 2022-08-01T23:03:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.789+0000 7f73a1590700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 0x7f739c0af9e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f739800c000 tx=0x7f7398006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7398010780 con 0x7f739c061360 2022-08-01T23:03:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980108e0 con 0x7f739c061360 2022-08-01T23:03:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7394009470 con 0x7f739c061360 2022-08-01T23:03:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7398010c40 con 0x7f739c061360 2022-08-01T23:03:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c148e70 con 0x7f739c061360 2022-08-01T23:03:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c149270 con 0x7f739c061360 2022-08-01T23:03:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f739801c460 con 0x7f739c061360 2022-08-01T23:03:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.793+0000 7f738effd700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f737c053a70 0x7f737c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.797+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f739806c480 con 0x7f739c061360 2022-08-01T23:03:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.797+0000 7f73a1d91700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f737c053a70 0x7f737c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.797+0000 7f73a1d91700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f737c053a70 0x7f737c055f30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f739000d000 tx=0x7f7390009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.801+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7398043220 con 0x7f739c061360 2022-08-01T23:03:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:30.929+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f739c0604d0 con 0x7f739c061360 2022-08-01T23:03:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.237+0000 7f738effd700 1 -- 172.21.15.32:0/2835265238 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v105) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7398046840 con 0x7f739c061360 2022-08-01T23:03:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f737c053a70 msgr2=0x7f737c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f737c053a70 0x7f737c055f30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f739000d000 tx=0x7f7390009040).stop 2022-08-01T23:03:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 msgr2=0x7f739c0af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 0x7f739c0af9e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f739800c000 tx=0x7f7398006df0).stop 2022-08-01T23:03:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 shutdown_connections 2022-08-01T23:03:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f737c053a70 0x7f737c055f30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f739c0b28a0 0x7f739c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f739c061360 0x7f739c0af9e0 secure :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0x7f739800c000 tx=0x7f7398006df0).stop 2022-08-01T23:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 --2- 172.21.15.32:0/2835265238 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f739c0600f0 0x7f739c146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 >> 172.21.15.32:0/2835265238 conn(0x7f739c05d0f0 msgr2=0x7f739c13fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 shutdown_connections 2022-08-01T23:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.245+0000 7f73a3ff5700 1 -- 172.21.15.32:0/2835265238 wait complete. 2022-08-01T23:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:03:31.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:03:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.365+0000 7f7cb994f700 1 Processor -- start 2022-08-01T23:03:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.365+0000 7f7cb994f700 1 -- start start 2022-08-01T23:03:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cb40614b0 0x7f7cb4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb40ad750 0x7f7cb40b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7cb41405d0 0x7f7cb4140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40aee80 con 0x7f7cb40614b0 2022-08-01T23:03:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40b70b0 con 0x7f7cb41405d0 2022-08-01T23:03:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40af0b0 con 0x7f7cb40ad750 2022-08-01T23:03:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cb40614b0 0x7f7cb4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cb40614b0 0x7f7cb4061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46948/0 (socket says 172.21.15.32:46948) 2022-08-01T23:03:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 -- 172.21.15.32:0/2963709028 learned_addr learned my addr 172.21.15.32:0/2963709028 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb914e700 1 --2- 172.21.15.32:0/2963709028 >> v2:172.21.15.55:3300/0 conn(0x7f7cb41405d0 0x7f7cb4140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/2963709028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb40ad750 0x7f7cb40b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 -- 172.21.15.32:0/2963709028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb40ad750 msgr2=0x7f7cb40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 --2- 172.21.15.32:0/2963709028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb40ad750 0x7f7cb40b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 -- 172.21.15.32:0/2963709028 >> v2:172.21.15.55:3300/0 conn(0x7f7cb41405d0 msgr2=0x7f7cb4140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 --2- 172.21.15.32:0/2963709028 >> v2:172.21.15.55:3300/0 conn(0x7f7cb41405d0 0x7f7cb4140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 -- 172.21.15.32:0/2963709028 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40af350 con 0x7f7cb40614b0 2022-08-01T23:03:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb914e700 1 --2- 172.21.15.32:0/2963709028 >> v2:172.21.15.55:3300/0 conn(0x7f7cb41405d0 0x7f7cb4140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb894d700 1 --2- 172.21.15.32:0/2963709028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4061890 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f7c9c002f70 tx=0x7f7c9c006eb0).ready entity=mon.0 client_cookie=39cf76e5dfd1527 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb37fe700 1 -- 172.21.15.32:0/2963709028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c9c00d570 con 0x7f7cb40614b0 2022-08-01T23:03:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb37fe700 1 -- 172.21.15.32:0/2963709028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c000e90 con 0x7f7cb40614b0 2022-08-01T23:03:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb37fe700 1 -- 172.21.15.32:0/2963709028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c9c00d900 con 0x7f7cb40614b0 2022-08-01T23:03:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- 172.21.15.32:0/2963709028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 msgr2=0x7f7cb4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- 172.21.15.32:0/2963709028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4061890 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f7c9c002f70 tx=0x7f7c9c006eb0).stop 2022-08-01T23:03:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- 172.21.15.32:0/2963709028 shutdown_connections 2022-08-01T23:03:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- 172.21.15.32:0/2963709028 >> v2:172.21.15.55:3300/0 conn(0x7f7cb41405d0 0x7f7cb4140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- 172.21.15.32:0/2963709028 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb40ad750 0x7f7cb40b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 --2- 172.21.15.32:0/2963709028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4061890 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- 172.21.15.32:0/2963709028 >> 172.21.15.32:0/2963709028 conn(0x7f7cb4066f70 msgr2=0x7f7cb4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- 172.21.15.32:0/2963709028 shutdown_connections 2022-08-01T23:03:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.369+0000 7f7cb994f700 1 -- 172.21.15.32:0/2963709028 wait complete. 2022-08-01T23:03:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 Processor -- start 2022-08-01T23:03:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 -- start start 2022-08-01T23:03:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cb40ad750 0x7f7cb40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb41405d0 0x7f7cb40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40b0580 con 0x7f7cb40614b0 2022-08-01T23:03:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40b0db0 con 0x7f7cb40ad750 2022-08-01T23:03:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40b1050 con 0x7f7cb41405d0 2022-08-01T23:03:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4146090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46952/0 (socket says 172.21.15.32:46952) 2022-08-01T23:03:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 -- 172.21.15.32:0/3613377921 learned_addr learned my addr 172.21.15.32:0/3613377921 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cb40ad750 0x7f7cb40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb914e700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb41405d0 0x7f7cb40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 -- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb41405d0 msgr2=0x7f7cb40b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb41405d0 0x7f7cb40b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 -- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cb40ad750 msgr2=0x7f7cb40b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cb40ad750 0x7f7cb40b8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 -- 172.21.15.32:0/3613377921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac009470 con 0x7f7cb40614b0 2022-08-01T23:03:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb894d700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4146090 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f7c9c002f70 tx=0x7f7c9c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c9c011950 con 0x7f7cb40614b0 2022-08-01T23:03:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c00b1e0 con 0x7f7cb40614b0 2022-08-01T23:03:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c011ab0 con 0x7f7cb40614b0 2022-08-01T23:03:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c9c011e10 con 0x7f7cb40614b0 2022-08-01T23:03:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.373+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40b3a40 con 0x7f7cb40614b0 2022-08-01T23:03:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.377+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb40af5f0 con 0x7f7cb40614b0 2022-08-01T23:03:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.377+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7c9c01a040 con 0x7f7cb40614b0 2022-08-01T23:03:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.377+0000 7f7cb1ffb700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c98053950 0x7f7c98055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.377+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f7c9c06d2e0 con 0x7f7cb40614b0 2022-08-01T23:03:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.377+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c98053950 0x7f7c98055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.377+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c98053950 0x7f7c98055e10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f7ca400b000 tx=0x7f7ca4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.385+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c9c044010 con 0x7f7cb40614b0 2022-08-01T23:03:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.545+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f7cb40574e0 con 0x7f7cb40614b0 2022-08-01T23:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb1ffb700 1 -- 172.21.15.32:0/3613377921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v49) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7c9c023070 con 0x7f7cb40614b0 2022-08-01T23:03:31.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c98053950 msgr2=0x7f7c98055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c98053950 0x7f7c98055e10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f7ca400b000 tx=0x7f7ca4006df0).stop 2022-08-01T23:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 msgr2=0x7f7cb4146090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4146090 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f7c9c002f70 tx=0x7f7c9c015000).stop 2022-08-01T23:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 shutdown_connections 2022-08-01T23:03:31.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7c98053950 0x7f7c98055e10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7cb41405d0 0x7f7cb40b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cb40ad750 0x7f7cb40b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 --2- 172.21.15.32:0/3613377921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40614b0 0x7f7cb4146090 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 >> 172.21.15.32:0/3613377921 conn(0x7f7cb4066f70 msgr2=0x7f7cb4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 shutdown_connections 2022-08-01T23:03:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.561+0000 7f7cb994f700 1 -- 172.21.15.32:0/3613377921 wait complete. 2022-08-01T23:03:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 19 and data pool 20 2022-08-01T23:03:31.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:31.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.665+0000 7f4379538700 1 Processor -- start 2022-08-01T23:03:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.665+0000 7f4379538700 1 -- start start 2022-08-01T23:03:31.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43740ad750 0x7f4374061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43740aee80 con 0x7f43741405d0 2022-08-01T23:03:31.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43740b70b0 con 0x7f43740614b0 2022-08-01T23:03:31.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43740af0b0 con 0x7f43740ad750 2022-08-01T23:03:31.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4378d37700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43741405d0 0x7f43741466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34520/0 (socket says 172.21.15.32:34520) 2022-08-01T23:03:31.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4378d37700 1 -- 172.21.15.32:0/1012738616 learned_addr learned my addr 172.21.15.32:0/1012738616 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:31.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f43737fe700 1 --2- 172.21.15.32:0/1012738616 >> v2:172.21.15.32:3300/0 conn(0x7f43740ad750 0x7f4374061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4372ffd700 1 -- 172.21.15.32:0/1012738616 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060812860 0 0) 0x7f43740aee80 con 0x7f43741405d0 2022-08-01T23:03:31.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4372ffd700 1 -- 172.21.15.32:0/1012738616 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4360003260 con 0x7f43741405d0 2022-08-01T23:03:31.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4372ffd700 1 -- 172.21.15.32:0/1012738616 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560463895 0 0) 0x7f43740b70b0 con 0x7f43740614b0 2022-08-01T23:03:31.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f43737fe700 1 -- 172.21.15.32:0/1012738616 >> v1:172.21.15.55:6789/0 conn(0x7f43740614b0 legacy=0x7f43740b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f43737fe700 1 -- 172.21.15.32:0/1012738616 >> v1:172.21.15.32:6789/0 conn(0x7f43741405d0 legacy=0x7f43741466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f43737fe700 1 -- 172.21.15.32:0/1012738616 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43740af350 con 0x7f43740ad750 2022-08-01T23:03:31.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f43737fe700 1 --2- 172.21.15.32:0/1012738616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f4374061890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f4368002f70 tx=0x7f4368006df0).ready entity=mon.0 client_cookie=6e08ac17be937e67 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4372ffd700 1 -- 172.21.15.32:0/1012738616 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436800d610 con 0x7f43740ad750 2022-08-01T23:03:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4372ffd700 1 -- 172.21.15.32:0/1012738616 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4368000e90 con 0x7f43740ad750 2022-08-01T23:03:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4372ffd700 1 -- 172.21.15.32:0/1012738616 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436800d960 con 0x7f43740ad750 2022-08-01T23:03:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- 172.21.15.32:0/1012738616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 msgr2=0x7f4374061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 --2- 172.21.15.32:0/1012738616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f4374061890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f4368002f70 tx=0x7f4368006df0).stop 2022-08-01T23:03:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- 172.21.15.32:0/1012738616 shutdown_connections 2022-08-01T23:03:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 --2- 172.21.15.32:0/1012738616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f4374061890 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- 172.21.15.32:0/1012738616 >> 172.21.15.32:0/1012738616 conn(0x7f4374066f70 msgr2=0x7f4374067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- 172.21.15.32:0/1012738616 shutdown_connections 2022-08-01T23:03:31.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.669+0000 7f4379538700 1 -- 172.21.15.32:0/1012738616 wait complete. 2022-08-01T23:03:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 Processor -- start 2022-08-01T23:03:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 -- start start 2022-08-01T23:03:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43740614b0 0x7f43741566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f43740abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43741405d0 0x7f43740b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43740abc30 con 0x7f43740ad750 2022-08-01T23:03:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4374155cf0 con 0x7f43741405d0 2022-08-01T23:03:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4374155f90 con 0x7f43740614b0 2022-08-01T23:03:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f43740abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f43740abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46982/0 (socket says 172.21.15.32:46982) 2022-08-01T23:03:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 -- 172.21.15.32:0/3153075557 learned_addr learned my addr 172.21.15.32:0/3153075557 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4378d37700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43741405d0 0x7f43740b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4373fff700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43740614b0 0x7f43741566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 -- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43740614b0 msgr2=0x7f43741566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43740614b0 0x7f43741566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 -- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43741405d0 msgr2=0x7f43740b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43741405d0 0x7f43740b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 -- 172.21.15.32:0/3153075557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436800b180 con 0x7f43740ad750 2022-08-01T23:03:31.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43737fe700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f43740abe80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f4368002f70 tx=0x7f4368014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4368013de0 con 0x7f43740ad750 2022-08-01T23:03:31.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800d290 con 0x7f43740ad750 2022-08-01T23:03:31.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4374156230 con 0x7f43740ad750 2022-08-01T23:03:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436801e5f0 con 0x7f43740ad750 2022-08-01T23:03:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.673+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43740ac370 con 0x7f43740ad750 2022-08-01T23:03:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.677+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43741563c0 con 0x7f43740ad750 2022-08-01T23:03:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.677+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4368025030 con 0x7f43740ad750 2022-08-01T23:03:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.677+0000 7f43717fa700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4354053950 0x7f4354055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.677+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f436806cde0 con 0x7f43740ad750 2022-08-01T23:03:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.677+0000 7f4373fff700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4354053950 0x7f4354055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.677+0000 7f4373fff700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4354053950 0x7f4354055e10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f436400d000 tx=0x7f4364007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.685+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4368043be0 con 0x7f43740ad750 2022-08-01T23:03:31.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.813+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43740af5f0 con 0x7f43740ad750 2022-08-01T23:03:31.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.813+0000 7f43717fa700 1 -- 172.21.15.32:0/3153075557 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16311 (secure 0 0 0) 0x7f4368023020 con 0x7f43740ad750 2022-08-01T23:03:31.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:31.821 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":106,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:31.549316+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:30.001802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:30.985153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:31.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4354053950 msgr2=0x7f4354055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4354053950 0x7f4354055e10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f436400d000 tx=0x7f4364007a00).stop 2022-08-01T23:03:31.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 msgr2=0x7f43740abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f43740abe80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f4368002f70 tx=0x7f4368014000).stop 2022-08-01T23:03:31.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 shutdown_connections 2022-08-01T23:03:31.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4354053950 0x7f4354055e10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43741405d0 0x7f43740b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43740ad750 0x7f43740abe80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 --2- 172.21.15.32:0/3153075557 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43740614b0 0x7f43741566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 >> 172.21.15.32:0/3153075557 conn(0x7f4374066f70 msgr2=0x7f43740b86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:31.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 shutdown_connections 2022-08-01T23:03:31.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.817+0000 7f4379538700 1 -- 172.21.15.32:0/3153075557 wait complete. 2022-08-01T23:03:31.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.925+0000 7f3ae329c700 1 Processor -- start 2022-08-01T23:03:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.925+0000 7f3ae329c700 1 -- start start 2022-08-01T23:03:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3adc0b1160 0x7f3adc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc0b04e0 con 0x7f3adc0b1f40 2022-08-01T23:03:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc0b0780 con 0x7f3adc148530 2022-08-01T23:03:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc0a9250 con 0x7f3adc0b1160 2022-08-01T23:03:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1038700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3adc0b1f40 0x7f3adc0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34532/0 (socket says 172.21.15.32:34532) 2022-08-01T23:03:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1038700 1 -- 172.21.15.32:0/2886776522 learned_addr learned my addr 172.21.15.32:0/2886776522 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:31.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1839700 1 --2- 172.21.15.32:0/2886776522 >> v2:172.21.15.55:3300/0 conn(0x7f3adc0b1160 0x7f3adc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210008238 0 0) 0x7f3adc0b04e0 con 0x7f3adc0b1f40 2022-08-01T23:03:31.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac0003260 con 0x7f3adc0b1f40 2022-08-01T23:03:31.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146901935 0 0) 0x7f3adc0b0780 con 0x7f3adc148530 2022-08-01T23:03:31.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3adc0b04e0 con 0x7f3adc148530 2022-08-01T23:03:31.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1839700 1 -- 172.21.15.32:0/2886776522 >> v1:172.21.15.55:6789/0 conn(0x7f3adc148530 legacy=0x7f3adc0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1839700 1 -- 172.21.15.32:0/2886776522 >> v1:172.21.15.32:6789/0 conn(0x7f3adc0b1f40 legacy=0x7f3adc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1839700 1 -- 172.21.15.32:0/2886776522 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc0b0a20 con 0x7f3adc0b1160 2022-08-01T23:03:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae1839700 1 --2- 172.21.15.32:0/2886776522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1160 0x7f3adc1448b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f3adc1478c0 tx=0x7f3ad4006df0).ready entity=mon.1 client_cookie=f034ac09e346a16b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ad400d610 con 0x7f3adc0b1160 2022-08-01T23:03:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad4000e90 con 0x7f3adc0b1160 2022-08-01T23:03:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ad3fff700 1 -- 172.21.15.32:0/2886776522 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ad400d960 con 0x7f3adc0b1160 2022-08-01T23:03:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- 172.21.15.32:0/2886776522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1160 msgr2=0x7f3adc1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 --2- 172.21.15.32:0/2886776522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1160 0x7f3adc1448b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f3adc1478c0 tx=0x7f3ad4006df0).stop 2022-08-01T23:03:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- 172.21.15.32:0/2886776522 shutdown_connections 2022-08-01T23:03:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 --2- 172.21.15.32:0/2886776522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1160 0x7f3adc1448b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- 172.21.15.32:0/2886776522 >> 172.21.15.32:0/2886776522 conn(0x7f3adc05f840 msgr2=0x7f3adc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- 172.21.15.32:0/2886776522 shutdown_connections 2022-08-01T23:03:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- 172.21.15.32:0/2886776522 wait complete. 2022-08-01T23:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 Processor -- start 2022-08-01T23:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.929+0000 7f3ae329c700 1 -- start start 2022-08-01T23:03:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 0x7f3adc0aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1f40 0x7f3adc0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3adc148530 0x7f3adc0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc14aff0 con 0x7f3adc0b1160 2022-08-01T23:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc14b170 con 0x7f3adc0b1f40 2022-08-01T23:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc14b410 con 0x7f3adc148530 2022-08-01T23:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 0x7f3adc0aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 0x7f3adc0aa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46994/0 (socket says 172.21.15.32:46994) 2022-08-01T23:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 -- 172.21.15.32:0/1549689003 learned_addr learned my addr 172.21.15.32:0/1549689003 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1839700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3adc148530 0x7f3adc0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae0837700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1f40 0x7f3adc0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 -- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3adc148530 msgr2=0x7f3adc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3adc148530 0x7f3adc0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 -- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1f40 msgr2=0x7f3adc0aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1f40 0x7f3adc0aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 -- 172.21.15.32:0/1549689003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad400b180 con 0x7f3adc0b1160 2022-08-01T23:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae0837700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1f40 0x7f3adc0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae1038700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 0x7f3adc0aa8a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f3acc0011d0 tx=0x7f3acc007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3acc0070d0 con 0x7f3adc0b1160 2022-08-01T23:03:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc007230 con 0x7f3adc0b1160 2022-08-01T23:03:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc14b6b0 con 0x7f3adc0b1160 2022-08-01T23:03:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3acc005610 con 0x7f3adc0b1160 2022-08-01T23:03:31.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.933+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc0ae470 con 0x7f3adc0b1160 2022-08-01T23:03:31.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.937+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc0afb30 con 0x7f3adc0b1160 2022-08-01T23:03:31.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.937+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3acc01a060 con 0x7f3adc0b1160 2022-08-01T23:03:31.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.937+0000 7f3ad27fc700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ac4053a70 0x7f3ac4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:31.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.937+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3acc06b930 con 0x7f3adc0b1160 2022-08-01T23:03:31.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.937+0000 7f3ae0837700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ac4053a70 0x7f3ac4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:31.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.937+0000 7f3ae0837700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ac4053a70 0x7f3ac4055f30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3ad800b000 tx=0x7f3ad8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:31.945+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3acc0425e0 con 0x7f3adc0b1160 2022-08-01T23:03:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.069+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3adc0b0cc0 con 0x7f3adc0b1160 2022-08-01T23:03:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.069+0000 7f3ad27fc700 1 -- 172.21.15.32:0/1549689003 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16311 (secure 0 0 0) 0x7f3acc018020 con 0x7f3adc0b1160 2022-08-01T23:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":106,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:31.549316+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:30.001802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:30.985153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ac4053a70 msgr2=0x7f3ac4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ac4053a70 0x7f3ac4055f30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3ad800b000 tx=0x7f3ad8009040).stop 2022-08-01T23:03:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 msgr2=0x7f3adc0aa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 0x7f3adc0aa8a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f3acc0011d0 tx=0x7f3acc007b50).stop 2022-08-01T23:03:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 shutdown_connections 2022-08-01T23:03:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ac4053a70 0x7f3ac4055f30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3adc148530 0x7f3adc0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3adc0b1f40 0x7f3adc0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 --2- 172.21.15.32:0/1549689003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3adc0b1160 0x7f3adc0aa8a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 >> 172.21.15.32:0/1549689003 conn(0x7f3adc05f840 msgr2=0x7f3adc0a7250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 shutdown_connections 2022-08-01T23:03:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.073+0000 7f3ae329c700 1 -- 172.21.15.32:0/1549689003 wait complete. 2022-08-01T23:03:32.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:03:32.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.193+0000 7fb469d60700 1 Processor -- start 2022-08-01T23:03:32.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.193+0000 7fb469d60700 1 -- start start 2022-08-01T23:03:32.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4640b1460 0x7fb4640b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4641413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640a7960 con 0x7fb4640b2050 2022-08-01T23:03:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640b18d0 con 0x7fb4640b1460 2022-08-01T23:03:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640b07e0 con 0x7fb4640b2430 2022-08-01T23:03:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4640b1460 0x7fb4640b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4640b1460 0x7fb4640b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47002/0 (socket says 172.21.15.32:47002) 2022-08-01T23:03:32.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 -- 172.21.15.32:0/253190979 learned_addr learned my addr 172.21.15.32:0/253190979 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:32.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb4637fe700 1 --1- 172.21.15.32:0/253190979 >> v1:172.21.15.32:6789/0 conn(0x7fb4640b2050 0x7fb4640a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34544/0 (socket says 172.21.15.32:34544) 2022-08-01T23:03:32.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb463fff700 1 --2- 172.21.15.32:0/253190979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4641413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 -- 172.21.15.32:0/253190979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 msgr2=0x7fb4641413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:32.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 --2- 172.21.15.32:0/253190979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4641413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 -- 172.21.15.32:0/253190979 >> v1:172.21.15.32:6789/0 conn(0x7fb4640b2050 legacy=0x7fb4640a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb4627fc700 1 -- 172.21.15.32:0/253190979 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063914977 0 0) 0x7fb4640a7960 con 0x7fb4640b2050 2022-08-01T23:03:32.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 -- 172.21.15.32:0/253190979 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640b0a80 con 0x7fb4640b1460 2022-08-01T23:03:32.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb462ffd700 1 --2- 172.21.15.32:0/253190979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b1460 0x7fb4640b0140 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fb454002f70 tx=0x7fb454006eb0).ready entity=mon.0 client_cookie=fe95a20f29329e42 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb4627fc700 1 -- 172.21.15.32:0/253190979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45400d570 con 0x7fb4640b1460 2022-08-01T23:03:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb4627fc700 1 -- 172.21.15.32:0/253190979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454000e90 con 0x7fb4640b1460 2022-08-01T23:03:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb4627fc700 1 -- 172.21.15.32:0/253190979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45400d8c0 con 0x7fb4640b1460 2022-08-01T23:03:32.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- 172.21.15.32:0/253190979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b1460 msgr2=0x7fb4640b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 --2- 172.21.15.32:0/253190979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b1460 0x7fb4640b0140 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fb454002f70 tx=0x7fb454006eb0).stop 2022-08-01T23:03:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- 172.21.15.32:0/253190979 shutdown_connections 2022-08-01T23:03:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 --2- 172.21.15.32:0/253190979 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4641413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 --2- 172.21.15.32:0/253190979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b1460 0x7fb4640b0140 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- 172.21.15.32:0/253190979 >> 172.21.15.32:0/253190979 conn(0x7fb46405f840 msgr2=0x7fb46405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- 172.21.15.32:0/253190979 shutdown_connections 2022-08-01T23:03:32.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.197+0000 7fb469d60700 1 -- 172.21.15.32:0/253190979 wait complete. 2022-08-01T23:03:32.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 Processor -- start 2022-08-01T23:03:32.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 -- start start 2022-08-01T23:03:32.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640b1460 0x7fb4640a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 0x7fb464148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4640b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46414b740 con 0x7fb4640b2050 2022-08-01T23:03:32.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641484d0 con 0x7fb4640b1460 2022-08-01T23:03:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464148700 con 0x7fb4640b2430 2022-08-01T23:03:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 0x7fb464148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 0x7fb464148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47008/0 (socket says 172.21.15.32:47008) 2022-08-01T23:03:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 -- 172.21.15.32:0/1616969827 learned_addr learned my addr 172.21.15.32:0/1616969827 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:32.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb4637fe700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640b1460 0x7fb4640a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb463fff700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4640b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 -- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 msgr2=0x7fb4640b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4640b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 -- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640b1460 msgr2=0x7fb4640a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640b1460 0x7fb4640a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 -- 172.21.15.32:0/1616969827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45400b1e0 con 0x7fb4640b2050 2022-08-01T23:03:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb462ffd700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 0x7fb464148cc0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fb46414ab20 tx=0x7fb454015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb454011760 con 0x7fb4640b2050 2022-08-01T23:03:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4540118c0 con 0x7fb4640b2050 2022-08-01T23:03:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46414aea0 con 0x7fb4640b2050 2022-08-01T23:03:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb454011c20 con 0x7fb4640b2050 2022-08-01T23:03:32.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.201+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464138850 con 0x7fb4640b2050 2022-08-01T23:03:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.205+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb454025030 con 0x7fb4640b2050 2022-08-01T23:03:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.205+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464149350 con 0x7fb4640b2050 2022-08-01T23:03:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.205+0000 7fb460ff9700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4440539e0 0x7fb444055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.205+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb45406d110 con 0x7fb4640b2050 2022-08-01T23:03:32.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.205+0000 7fb4637fe700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4440539e0 0x7fb444055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.205+0000 7fb4637fe700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4440539e0 0x7fb444055ea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fb45800d000 tx=0x7fb458007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.213+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb454043ed0 con 0x7fb4640b2050 2022-08-01T23:03:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.377+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb4640574e0 con 0x7fb4640b2050 2022-08-01T23:03:32.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.557+0000 7fb460ff9700 1 -- 172.21.15.32:0/1616969827 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb454023020 con 0x7fb4640b2050 2022-08-01T23:03:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4440539e0 msgr2=0x7fb444055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4440539e0 0x7fb444055ea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fb45800d000 tx=0x7fb458007a00).stop 2022-08-01T23:03:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 msgr2=0x7fb464148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 0x7fb464148cc0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fb46414ab20 tx=0x7fb454015000).stop 2022-08-01T23:03:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 shutdown_connections 2022-08-01T23:03:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4440539e0 0x7fb444055ea0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4640b2430 0x7fb4640b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4640b2050 0x7fb464148cc0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 --2- 172.21.15.32:0/1616969827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640b1460 0x7fb4640a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 >> 172.21.15.32:0/1616969827 conn(0x7fb46405f840 msgr2=0x7fb46413b700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 shutdown_connections 2022-08-01T23:03:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.561+0000 7fb469d60700 1 -- 172.21.15.32:0/1616969827 wait complete. 2022-08-01T23:03:32.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 Processor -- start 2022-08-01T23:03:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 -- start start 2022-08-01T23:03:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46a80614b0 0x7f46a8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a81405d0 0x7f46a8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80aee80 con 0x7f46a80ad750 2022-08-01T23:03:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80b70b0 con 0x7f46a81405d0 2022-08-01T23:03:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46aeccf700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80af0b0 con 0x7f46a80614b0 2022-08-01T23:03:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a81405d0 0x7f46a8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a81405d0 0x7f46a8140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47020/0 (socket says 172.21.15.32:47020) 2022-08-01T23:03:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46adccd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46a80614b0 0x7f46a8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 -- 172.21.15.32:0/2457640150 learned_addr learned my addr 172.21.15.32:0/2457640150 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 -- 172.21.15.32:0/2457640150 >> v2:172.21.15.55:3300/0 conn(0x7f46a80614b0 msgr2=0x7f46a8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 --2- 172.21.15.32:0/2457640150 >> v2:172.21.15.55:3300/0 conn(0x7f46a80614b0 0x7f46a8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 -- 172.21.15.32:0/2457640150 >> v1:172.21.15.55:6789/0 conn(0x7f46a80ad750 legacy=0x7f46a80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 -- 172.21.15.32:0/2457640150 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80af350 con 0x7f46a81405d0 2022-08-01T23:03:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46ae4ce700 1 --2- 172.21.15.32:0/2457640150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a8140a60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f46a80aed30 tx=0x7f46a0009290).ready entity=mon.0 client_cookie=3d031dca3b40ec server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46acccb700 1 -- 172.21.15.32:0/2457640150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a000b860 con 0x7f46a81405d0 2022-08-01T23:03:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46acccb700 1 -- 172.21.15.32:0/2457640150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a000b9c0 con 0x7f46a81405d0 2022-08-01T23:03:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.685+0000 7f46acccb700 1 -- 172.21.15.32:0/2457640150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a000bb30 con 0x7f46a81405d0 2022-08-01T23:03:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- 172.21.15.32:0/2457640150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 msgr2=0x7f46a8140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 --2- 172.21.15.32:0/2457640150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a8140a60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f46a80aed30 tx=0x7f46a0009290).stop 2022-08-01T23:03:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- 172.21.15.32:0/2457640150 shutdown_connections 2022-08-01T23:03:32.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 --2- 172.21.15.32:0/2457640150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a8140a60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 --2- 172.21.15.32:0/2457640150 >> v2:172.21.15.55:3300/0 conn(0x7f46a80614b0 0x7f46a8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- 172.21.15.32:0/2457640150 >> 172.21.15.32:0/2457640150 conn(0x7f46a8066f70 msgr2=0x7f46a8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- 172.21.15.32:0/2457640150 shutdown_connections 2022-08-01T23:03:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- 172.21.15.32:0/2457640150 wait complete. 2022-08-01T23:03:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 Processor -- start 2022-08-01T23:03:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- start start 2022-08-01T23:03:32.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a80614b0 0x7f46a80b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a80ad750 0x7f46a80b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a80b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8152870 con 0x7f46a81405d0 2022-08-01T23:03:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80b40e0 con 0x7f46a80614b0 2022-08-01T23:03:32.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46aeccf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80b3ba0 con 0x7f46a80ad750 2022-08-01T23:03:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46ae4ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a80b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46ad4cc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a80ad750 0x7f46a80b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46ae4ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a80b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47026/0 (socket says 172.21.15.32:47026) 2022-08-01T23:03:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46adccd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a80614b0 0x7f46a80b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.689+0000 7f46ae4ce700 1 -- 172.21.15.32:0/4276579450 learned_addr learned my addr 172.21.15.32:0/4276579450 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46ae4ce700 1 -- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a80ad750 msgr2=0x7f46a80b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46ae4ce700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a80ad750 0x7f46a80b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46ae4ce700 1 -- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a80614b0 msgr2=0x7f46a80b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46ae4ce700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a80614b0 0x7f46a80b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46ae4ce700 1 -- 172.21.15.32:0/4276579450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a0009530 con 0x7f46a81405d0 2022-08-01T23:03:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46ae4ce700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a80b2880 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f46a80aed30 tx=0x7f46a0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a00032f0 con 0x7f46a81405d0 2022-08-01T23:03:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a0021440 con 0x7f46a81405d0 2022-08-01T23:03:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a80b3e40 con 0x7f46a81405d0 2022-08-01T23:03:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a00217a0 con 0x7f46a81405d0 2022-08-01T23:03:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a80a2950 con 0x7f46a81405d0 2022-08-01T23:03:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a80af5f0 con 0x7f46a81405d0 2022-08-01T23:03:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.693+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f46a0029070 con 0x7f46a81405d0 2022-08-01T23:03:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.697+0000 7f4696ffd700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4688053a70 0x7f4688055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.697+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f46a006cfd0 con 0x7f46a81405d0 2022-08-01T23:03:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.697+0000 7f46adccd700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4688053a70 0x7f4688055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.697+0000 7f46adccd700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4688053a70 0x7f4688055f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f46a400b000 tx=0x7f46a4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.701+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46a0043c80 con 0x7f46a81405d0 2022-08-01T23:03:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.861+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46a80b20b0 con 0x7f46a81405d0 2022-08-01T23:03:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f4696ffd700 1 -- 172.21.15.32:0/4276579450 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f46a0015020 con 0x7f46a81405d0 2022-08-01T23:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:31.549702+0000","modified":"2022-08-01T23:03:32.552769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/550523700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":550523700},{"type":"v1","addr":"172.21.15.32:6835","nonce":550523700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T23:03:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4688053a70 msgr2=0x7f4688055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4688053a70 0x7f4688055f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f46a400b000 tx=0x7f46a4006df0).stop 2022-08-01T23:03:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 msgr2=0x7f46a80b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a80b2880 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f46a80aed30 tx=0x7f46a0016000).stop 2022-08-01T23:03:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 shutdown_connections 2022-08-01T23:03:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4688053a70 0x7f4688055f30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a81405d0 0x7f46a80b2880 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a80ad750 0x7f46a80b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 --2- 172.21.15.32:0/4276579450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a80614b0 0x7f46a80b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 >> 172.21.15.32:0/4276579450 conn(0x7f46a8066f70 msgr2=0x7f46a8143840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:32.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 shutdown_connections 2022-08-01T23:03:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.865+0000 7f46aeccf700 1 -- 172.21.15.32:0/4276579450 wait complete. 2022-08-01T23:03:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2022-08-01T23:03:32.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:32.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 Processor -- start 2022-08-01T23:03:32.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 -- start start 2022-08-01T23:03:32.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00b1f40 0x7ff4b00b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00b04e0 con 0x7ff4b00a7250 2022-08-01T23:03:32.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00b0780 con 0x7ff4b0148530 2022-08-01T23:03:32.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00a9250 con 0x7ff4b00b1f40 2022-08-01T23:03:32.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4aed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4b00a7250 0x7ff4b01448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34574/0 (socket says 172.21.15.32:34574) 2022-08-01T23:03:32.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4aed9d700 1 -- 172.21.15.32:0/1943818371 learned_addr learned my addr 172.21.15.32:0/1943818371 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:32.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ae59c700 1 --2- 172.21.15.32:0/1943818371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00b1f40 0x7ff4b00b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227229347 0 0) 0x7ff4b00b04e0 con 0x7ff4b00a7250 2022-08-01T23:03:32.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff490003260 con 0x7ff4b00a7250 2022-08-01T23:03:32.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180582222 0 0) 0x7ff4b00b0780 con 0x7ff4b0148530 2022-08-01T23:03:32.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b00b04e0 con 0x7ff4b0148530 2022-08-01T23:03:32.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 169052842 0 0) 0x7ff490003260 con 0x7ff4b00a7250 2022-08-01T23:03:32.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ae59c700 1 -- 172.21.15.32:0/1943818371 >> v1:172.21.15.55:6789/0 conn(0x7ff4b0148530 legacy=0x7ff4b00b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ae59c700 1 -- 172.21.15.32:0/1943818371 >> v1:172.21.15.32:6789/0 conn(0x7ff4b00a7250 legacy=0x7ff4b01448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ae59c700 1 -- 172.21.15.32:0/1943818371 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00b0a20 con 0x7ff4b00b1f40 2022-08-01T23:03:32.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ae59c700 1 --2- 172.21.15.32:0/1943818371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00b1f40 0x7ff4b00b1160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff4a8001b80 tx=0x7ff4a0006df0).ready entity=mon.2 client_cookie=67a20cb3e15938ee server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:32.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4a80019e0 con 0x7ff4b00b1f40 2022-08-01T23:03:32.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0000e90 con 0x7ff4b00b1f40 2022-08-01T23:03:32.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4ad59a700 1 -- 172.21.15.32:0/1943818371 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4a0012460 con 0x7ff4b00b1f40 2022-08-01T23:03:32.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 -- 172.21.15.32:0/1943818371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00b1f40 msgr2=0x7ff4b00b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:32.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.981+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1943818371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00b1f40 0x7ff4b00b1160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff4a8001b80 tx=0x7ff4a0006df0).stop 2022-08-01T23:03:32.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- 172.21.15.32:0/1943818371 shutdown_connections 2022-08-01T23:03:32.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1943818371 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00b1f40 0x7ff4b00b1160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:32.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- 172.21.15.32:0/1943818371 >> 172.21.15.32:0/1943818371 conn(0x7ff4b005f840 msgr2=0x7ff4b005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:32.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- 172.21.15.32:0/1943818371 shutdown_connections 2022-08-01T23:03:32.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- 172.21.15.32:0/1943818371 wait complete. 2022-08-01T23:03:32.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 Processor -- start 2022-08-01T23:03:32.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- start start 2022-08-01T23:03:32.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00a7250 0x7ff4b014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 0x7ff4b00abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4b0148530 0x7ff4b00ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:32.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00aaec0 con 0x7ff4b00b1f40 2022-08-01T23:03:32.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00aa980 con 0x7ff4b0148530 2022-08-01T23:03:32.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4b48de700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00aac20 con 0x7ff4b00a7250 2022-08-01T23:03:32.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 0x7ff4b00abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:32.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 0x7ff4b00abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47038/0 (socket says 172.21.15.32:47038) 2022-08-01T23:03:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4ae59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00a7250 0x7ff4b014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4b0148530 0x7ff4b00ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 -- 172.21.15.32:0/1279337574 learned_addr learned my addr 172.21.15.32:0/1279337574 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 -- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00a7250 msgr2=0x7ff4b014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00a7250 0x7ff4b014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 -- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4b0148530 msgr2=0x7ff4b00ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4b0148530 0x7ff4b00ac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.985+0000 7ff4add9b700 1 -- 172.21.15.32:0/1279337574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a000b180 con 0x7ff4b00b1f40 2022-08-01T23:03:33.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4add9b700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 0x7ff4b00abb80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7ff498001110 tx=0x7ff498007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49800cde0 con 0x7ff4b00b1f40 2022-08-01T23:03:33.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498012400 con 0x7ff4b00b1f40 2022-08-01T23:03:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b014ac50 con 0x7ff4b00b1f40 2022-08-01T23:03:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff498012760 con 0x7ff4b00b1f40 2022-08-01T23:03:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b0148c60 con 0x7ff4b00b1f40 2022-08-01T23:03:33.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b00574e0 con 0x7ff4b00b1f40 2022-08-01T23:03:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.989+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff498019030 con 0x7ff4b00b1f40 2022-08-01T23:03:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.993+0000 7ff4a77fe700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff494053980 0x7ff494055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.993+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff49806b3c0 con 0x7ff4b00b1f40 2022-08-01T23:03:33.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.993+0000 7ff4ae59c700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff494053980 0x7ff494055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.993+0000 7ff4ae59c700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff494053980 0x7ff494055e40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7ff4a001b000 tx=0x7ff4a0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:32.997+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff498042070 con 0x7ff4b00b1f40 2022-08-01T23:03:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.125+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4b00447e0 con 0x7ff4b00b1f40 2022-08-01T23:03:33.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4a77fe700 1 -- 172.21.15.32:0/1279337574 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ff498045690 con 0x7ff4b00b1f40 2022-08-01T23:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":107,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:32.553487+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:30.001802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:30.985153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff494053980 msgr2=0x7ff494055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff494053980 0x7ff494055e40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7ff4a001b000 tx=0x7ff4a0014000).stop 2022-08-01T23:03:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 msgr2=0x7ff4b00abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 0x7ff4b00abb80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7ff498001110 tx=0x7ff498007a60).stop 2022-08-01T23:03:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 shutdown_connections 2022-08-01T23:03:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff494053980 0x7ff494055e40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4b0148530 0x7ff4b00ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4b00b1f40 0x7ff4b00abb80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 --2- 172.21.15.32:0/1279337574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4b00a7250 0x7ff4b014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 >> 172.21.15.32:0/1279337574 conn(0x7ff4b005f840 msgr2=0x7ff4b00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 shutdown_connections 2022-08-01T23:03:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.129+0000 7ff4b48de700 1 -- 172.21.15.32:0/1279337574 wait complete. 2022-08-01T23:03:33.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:03:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.253+0000 7f0170e0a700 1 Processor -- start 2022-08-01T23:03:33.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.253+0000 7f0170e0a700 1 -- start start 2022-08-01T23:03:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.253+0000 7f0170e0a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f016c0a9320 0x7f016c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.253+0000 7f0170e0a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.253+0000 7f0170e0a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c0b12d0 con 0x7f016c0b0d10 2022-08-01T23:03:33.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.253+0000 7f0170e0a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0a9700 con 0x7f016c0a9320 2022-08-01T23:03:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0b1500 con 0x7f016c0b0560 2022-08-01T23:03:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f016c0a9320 0x7f016c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f016c0a9320 0x7f016c1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47044/0 (socket says 172.21.15.32:47044) 2022-08-01T23:03:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 -- 172.21.15.32:0/1894835249 learned_addr learned my addr 172.21.15.32:0/1894835249 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016bfff700 1 --2- 172.21.15.32:0/1894835249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 -- 172.21.15.32:0/1894835249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 msgr2=0x7f016c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 --2- 172.21.15.32:0/1894835249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 -- 172.21.15.32:0/1894835249 >> v1:172.21.15.55:6789/0 conn(0x7f016c0b0d10 legacy=0x7f016c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 -- 172.21.15.32:0/1894835249 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0b17a0 con 0x7f016c0a9320 2022-08-01T23:03:33.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016b7fe700 1 --2- 172.21.15.32:0/1894835249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c1466e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f015c002f70 tx=0x7f015c006eb0).ready entity=mon.0 client_cookie=f847ace67235bef0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016a7fc700 1 -- 172.21.15.32:0/1894835249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c00d5c0 con 0x7f016c0a9320 2022-08-01T23:03:33.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016a7fc700 1 -- 172.21.15.32:0/1894835249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c000e90 con 0x7f016c0a9320 2022-08-01T23:03:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f016a7fc700 1 -- 172.21.15.32:0/1894835249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c00d950 con 0x7f016c0a9320 2022-08-01T23:03:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 -- 172.21.15.32:0/1894835249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 msgr2=0x7f016c1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 --2- 172.21.15.32:0/1894835249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c1466e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f015c002f70 tx=0x7f015c006eb0).stop 2022-08-01T23:03:33.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 -- 172.21.15.32:0/1894835249 shutdown_connections 2022-08-01T23:03:33.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 --2- 172.21.15.32:0/1894835249 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 --2- 172.21.15.32:0/1894835249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c1466e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 -- 172.21.15.32:0/1894835249 >> 172.21.15.32:0/1894835249 conn(0x7f016c0612d0 msgr2=0x7f016c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:33.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 -- 172.21.15.32:0/1894835249 shutdown_connections 2022-08-01T23:03:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.257+0000 7f0170e0a700 1 -- 172.21.15.32:0/1894835249 wait complete. 2022-08-01T23:03:33.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 Processor -- start 2022-08-01T23:03:33.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 -- start start 2022-08-01T23:03:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c067a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b0d10 0x7f016c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c063cf0 con 0x7f016c0a9320 2022-08-01T23:03:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c065d80 con 0x7f016c0b0d10 2022-08-01T23:03:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c063f20 con 0x7f016c0b0560 2022-08-01T23:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c067a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47050/0 (socket says 172.21.15.32:47050) 2022-08-01T23:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 -- 172.21.15.32:0/2681512377 learned_addr learned my addr 172.21.15.32:0/2681512377 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016bfff700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b0d10 0x7f016c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016affd700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 -- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 msgr2=0x7f016c14f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c14f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 -- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b0d10 msgr2=0x7f016c14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b0d10 0x7f016c14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 -- 172.21.15.32:0/2681512377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c00b1e0 con 0x7f016c0a9320 2022-08-01T23:03:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f016b7fe700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c067a10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f015c002f70 tx=0x7f015c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c011950 con 0x7f016c0a9320 2022-08-01T23:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c011ab0 con 0x7f016c0a9320 2022-08-01T23:03:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c065fb0 con 0x7f016c0a9320 2022-08-01T23:03:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c011e10 con 0x7f016c0a9320 2022-08-01T23:03:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.261+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c063960 con 0x7f016c0a9320 2022-08-01T23:03:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.265+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c0574e0 con 0x7f016c0a9320 2022-08-01T23:03:33.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.265+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f015c025030 con 0x7f016c0a9320 2022-08-01T23:03:33.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.265+0000 7f0168ff9700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f014c053980 0x7f014c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.265+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f015c06d330 con 0x7f016c0a9320 2022-08-01T23:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.265+0000 7f016affd700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f014c053980 0x7f014c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.265+0000 7f016affd700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f014c053980 0x7f014c055e40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f016000b000 tx=0x7f0160009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.273+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f015c0440f0 con 0x7f016c0a9320 2022-08-01T23:03:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.437+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f016c13dbe0 con 0x7f016c0a9320 2022-08-01T23:03:33.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0168ff9700 1 -- 172.21.15.32:0/2681512377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7f015c0230e0 con 0x7f016c0a9320 2022-08-01T23:03:33.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f014c053980 msgr2=0x7f014c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f014c053980 0x7f014c055e40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f016000b000 tx=0x7f0160009040).stop 2022-08-01T23:03:33.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 msgr2=0x7f016c067a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c067a10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f015c002f70 tx=0x7f015c015000).stop 2022-08-01T23:03:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 shutdown_connections 2022-08-01T23:03:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f014c053980 0x7f014c055e40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b0d10 0x7f016c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f016c0b0560 0x7f016c14f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 --2- 172.21.15.32:0/2681512377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f016c0a9320 0x7f016c067a10 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 >> 172.21.15.32:0/2681512377 conn(0x7f016c0612d0 msgr2=0x7f016c13c060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 shutdown_connections 2022-08-01T23:03:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.445+0000 7f0170e0a700 1 -- 172.21.15.32:0/2681512377 wait complete. 2022-08-01T23:03:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:03:33.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:03:33.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 Processor -- start 2022-08-01T23:03:33.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 -- start start 2022-08-01T23:03:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b080b6710 0x7f0b080b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b4550 0x7f0b080b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b08144240 con 0x7f0b08061ec0 2022-08-01T23:03:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080b6fe0 con 0x7f0b080b6710 2022-08-01T23:03:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b0d36c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08143d00 con 0x7f0b080b4550 2022-08-01T23:03:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b080b6710 0x7f0b080b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b080b6710 0x7f0b080b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47058/0 (socket says 172.21.15.32:47058) 2022-08-01T23:03:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 -- 172.21.15.32:0/2915093192 learned_addr learned my addr 172.21.15.32:0/2915093192 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b067fc700 1 --2- 172.21.15.32:0/2915093192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b4550 0x7f0b080b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 -- 172.21.15.32:0/2915093192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b4550 msgr2=0x7f0b080b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 --2- 172.21.15.32:0/2915093192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b4550 0x7f0b080b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 -- 172.21.15.32:0/2915093192 >> v1:172.21.15.32:6789/0 conn(0x7f0b08061ec0 legacy=0x7f0b08146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 -- 172.21.15.32:0/2915093192 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08143fa0 con 0x7f0b080b6710 2022-08-01T23:03:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b05ffb700 1 -- 172.21.15.32:0/2915093192 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669498430 0 0) 0x7f0b08144240 con 0x7f0b08061ec0 2022-08-01T23:03:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b06ffd700 1 --2- 172.21.15.32:0/2915093192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b6710 0x7f0b080b5d80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f0af8002f70 tx=0x7f0af8006eb0).ready entity=mon.0 client_cookie=c4736a1ed8ada250 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b05ffb700 1 -- 172.21.15.32:0/2915093192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0af800d570 con 0x7f0b080b6710 2022-08-01T23:03:33.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b05ffb700 1 -- 172.21.15.32:0/2915093192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af8000e90 con 0x7f0b080b6710 2022-08-01T23:03:33.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.573+0000 7f0b05ffb700 1 -- 172.21.15.32:0/2915093192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0af800d8c0 con 0x7f0b080b6710 2022-08-01T23:03:33.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2915093192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b6710 msgr2=0x7f0b080b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2915093192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b6710 0x7f0b080b5d80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f0af8002f70 tx=0x7f0af8006eb0).stop 2022-08-01T23:03:33.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2915093192 shutdown_connections 2022-08-01T23:03:33.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2915093192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b4550 0x7f0b080b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2915093192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b6710 0x7f0b080b5d80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2915093192 >> 172.21.15.32:0/2915093192 conn(0x7f0b0805c620 msgr2=0x7f0b0805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:33.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2915093192 shutdown_connections 2022-08-01T23:03:33.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2915093192 wait complete. 2022-08-01T23:03:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 Processor -- start 2022-08-01T23:03:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- start start 2022-08-01T23:03:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b08061ec0 0x7f0b080b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 0x7f0b080adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b6710 0x7f0b0812b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0812b3f0 con 0x7f0b080b4550 2022-08-01T23:03:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0812b190 con 0x7f0b08061ec0 2022-08-01T23:03:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b0d36c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080ad850 con 0x7f0b080b6710 2022-08-01T23:03:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.577+0000 7f0b067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 0x7f0b080adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 0x7f0b080adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47066/0 (socket says 172.21.15.32:47066) 2022-08-01T23:03:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 -- 172.21.15.32:0/2653490981 learned_addr learned my addr 172.21.15.32:0/2653490981 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b06ffd700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b08061ec0 0x7f0b080b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b077fe700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b6710 0x7f0b0812b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 -- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b6710 msgr2=0x7f0b0812b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b6710 0x7f0b0812b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 -- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b08061ec0 msgr2=0x7f0b080b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b08061ec0 0x7f0b080b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 -- 172.21.15.32:0/2653490981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af800b1e0 con 0x7f0b080b4550 2022-08-01T23:03:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b06ffd700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b08061ec0 0x7f0b080b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b077fe700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b6710 0x7f0b0812b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b067fc700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 0x7f0b080adf90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f0afc00c000 tx=0x7f0afc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0afc00c830 con 0x7f0b080b4550 2022-08-01T23:03:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc00c990 con 0x7f0b080b4550 2022-08-01T23:03:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b080adaf0 con 0x7f0b080b4550 2022-08-01T23:03:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0afc00ccf0 con 0x7f0b080b4550 2022-08-01T23:03:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b080b02d0 con 0x7f0b080b4550 2022-08-01T23:03:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b080adc80 con 0x7f0b080b4550 2022-08-01T23:03:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.581+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0afc01c460 con 0x7f0b080b4550 2022-08-01T23:03:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.585+0000 7f0af7fff700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0ae4053b00 0x7f0ae4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.585+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0afc06c520 con 0x7f0b080b4550 2022-08-01T23:03:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.585+0000 7f0b06ffd700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0ae4053b00 0x7f0ae4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.585+0000 7f0b06ffd700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0ae4053b00 0x7f0ae4055fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f0af801a000 tx=0x7f0af8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.589+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0afc069e90 con 0x7f0b080b4550 2022-08-01T23:03:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.757+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0b08061780 con 0x7f0b080b4550 2022-08-01T23:03:33.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.765+0000 7f0af7fff700 1 -- 172.21.15.32:0/2653490981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0afc043280 con 0x7f0b080b4550 2022-08-01T23:03:33.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0ae4053b00 msgr2=0x7f0ae4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0ae4053b00 0x7f0ae4055fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f0af801a000 tx=0x7f0af8015000).stop 2022-08-01T23:03:33.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 msgr2=0x7f0b080adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 0x7f0b080adf90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f0afc00c000 tx=0x7f0afc006df0).stop 2022-08-01T23:03:33.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 shutdown_connections 2022-08-01T23:03:33.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0ae4053b00 0x7f0ae4055fc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b080b6710 0x7f0b0812b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b080b4550 0x7f0b080adf90 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 --2- 172.21.15.32:0/2653490981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b08061ec0 0x7f0b080b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 >> 172.21.15.32:0/2653490981 conn(0x7f0b0805c620 msgr2=0x7f0b08128490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 shutdown_connections 2022-08-01T23:03:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.769+0000 7f0b0d36c700 1 -- 172.21.15.32:0/2653490981 wait complete. 2022-08-01T23:03:33.781 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:03:33.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 Processor -- start 2022-08-01T23:03:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 -- start start 2022-08-01T23:03:33.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cb80614b0 0x7f4cb8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb80aee80 con 0x7f4cb80ad750 2022-08-01T23:03:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb80b70b0 con 0x7f4cb80614b0 2022-08-01T23:03:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.885+0000 7f4cbede8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80af0b0 con 0x7f4cb81405d0 2022-08-01T23:03:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cb80614b0 0x7f4cb8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cb80614b0 0x7f4cb8061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47072/0 (socket says 172.21.15.32:47072) 2022-08-01T23:03:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 -- 172.21.15.32:0/1904397737 learned_addr learned my addr 172.21.15.32:0/1904397737 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbe5e7700 1 --2- 172.21.15.32:0/1904397737 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 -- 172.21.15.32:0/1904397737 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 msgr2=0x7f4cb8140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 --2- 172.21.15.32:0/1904397737 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb8140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 -- 172.21.15.32:0/1904397737 >> v1:172.21.15.55:6789/0 conn(0x7f4cb80ad750 legacy=0x7f4cb80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 -- 172.21.15.32:0/1904397737 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb80af350 con 0x7f4cb80614b0 2022-08-01T23:03:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbcde4700 1 -- 172.21.15.32:0/1904397737 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920402097 0 0) 0x7f4cb80aee80 con 0x7f4cb80ad750 2022-08-01T23:03:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbdde6700 1 --2- 172.21.15.32:0/1904397737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80614b0 0x7f4cb8061890 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f4cb80aed30 tx=0x7f4ca8006eb0).ready entity=mon.0 client_cookie=c8f9af335cd75e07 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbcde4700 1 -- 172.21.15.32:0/1904397737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ca800d5c0 con 0x7f4cb80614b0 2022-08-01T23:03:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbcde4700 1 -- 172.21.15.32:0/1904397737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca8000e90 con 0x7f4cb80614b0 2022-08-01T23:03:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbcde4700 1 -- 172.21.15.32:0/1904397737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ca800d910 con 0x7f4cb80614b0 2022-08-01T23:03:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 -- 172.21.15.32:0/1904397737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80614b0 msgr2=0x7f4cb8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 --2- 172.21.15.32:0/1904397737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80614b0 0x7f4cb8061890 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f4cb80aed30 tx=0x7f4ca8006eb0).stop 2022-08-01T23:03:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 -- 172.21.15.32:0/1904397737 shutdown_connections 2022-08-01T23:03:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 --2- 172.21.15.32:0/1904397737 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb8140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 --2- 172.21.15.32:0/1904397737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80614b0 0x7f4cb8061890 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 -- 172.21.15.32:0/1904397737 >> 172.21.15.32:0/1904397737 conn(0x7f4cb8066f70 msgr2=0x7f4cb8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 -- 172.21.15.32:0/1904397737 shutdown_connections 2022-08-01T23:03:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 -- 172.21.15.32:0/1904397737 wait complete. 2022-08-01T23:03:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.889+0000 7f4cbede8700 1 Processor -- start 2022-08-01T23:03:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 -- start start 2022-08-01T23:03:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb80614b0 0x7f4cb8142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 0x7f4cb80abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb80ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80a9f90 con 0x7f4cb80ad750 2022-08-01T23:03:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80a8e00 con 0x7f4cb80614b0 2022-08-01T23:03:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80ace80 con 0x7f4cb81405d0 2022-08-01T23:03:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 0x7f4cb80abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 0x7f4cb80abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47080/0 (socket says 172.21.15.32:47080) 2022-08-01T23:03:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 -- 172.21.15.32:0/2356485198 learned_addr learned my addr 172.21.15.32:0/2356485198 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbe5e7700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb80ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbdde6700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb80614b0 0x7f4cb8142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 -- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 msgr2=0x7f4cb80ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb80ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 -- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb80614b0 msgr2=0x7f4cb8142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb80614b0 0x7f4cb8142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 -- 172.21.15.32:0/2356485198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca800b1e0 con 0x7f4cb80ad750 2022-08-01T23:03:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbd5e5700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 0x7f4cb80abe40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f4cb80ab2a0 tx=0x7f4cb4007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cb4012400 con 0x7f4cb80ad750 2022-08-01T23:03:33.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4012560 con 0x7f4cb80ad750 2022-08-01T23:03:33.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb8142fa0 con 0x7f4cb80ad750 2022-08-01T23:03:33.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cb40128c0 con 0x7f4cb80ad750 2022-08-01T23:03:33.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.893+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb80acbe0 con 0x7f4cb80ad750 2022-08-01T23:03:33.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.897+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb80af5f0 con 0x7f4cb80ad750 2022-08-01T23:03:33.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.897+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4cb4019030 con 0x7f4cb80ad750 2022-08-01T23:03:33.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.897+0000 7f4ca6ffd700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c98053980 0x7f4c98055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:33.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.897+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4cb406b220 con 0x7f4cb80ad750 2022-08-01T23:03:33.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.897+0000 7f4cbdde6700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c98053980 0x7f4c98055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:33.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.897+0000 7f4cbdde6700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c98053980 0x7f4c98055e40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f4ca801c000 tx=0x7f4ca8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:33.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:33.905+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4cb4042090 con 0x7f4cb80ad750 2022-08-01T23:03:34.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.065+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cb80574e0 con 0x7f4cb80ad750 2022-08-01T23:03:34.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4ca6ffd700 1 -- 172.21.15.32:0/2356485198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f4cb40456b0 con 0x7f4cb80ad750 2022-08-01T23:03:34.074 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:34.075 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:31.549702+0000","modified":"2022-08-01T23:03:32.552769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/550523700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":550523700},{"type":"v1","addr":"172.21.15.32:6835","nonce":550523700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T23:03:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c98053980 msgr2=0x7f4c98055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c98053980 0x7f4c98055e40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f4ca801c000 tx=0x7f4ca8015000).stop 2022-08-01T23:03:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 msgr2=0x7f4cb80abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 0x7f4cb80abe40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f4cb80ab2a0 tx=0x7f4cb4007a60).stop 2022-08-01T23:03:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 shutdown_connections 2022-08-01T23:03:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c98053980 0x7f4c98055e40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cb81405d0 0x7f4cb80ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb80ad750 0x7f4cb80abe40 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 --2- 172.21.15.32:0/2356485198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb80614b0 0x7f4cb8142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 >> 172.21.15.32:0/2356485198 conn(0x7f4cb8066f70 msgr2=0x7f4cb80b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 shutdown_connections 2022-08-01T23:03:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.069+0000 7f4cbede8700 1 -- 172.21.15.32:0/2356485198 wait complete. 2022-08-01T23:03:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2022-08-01T23:03:34.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:31.549702+0000', 'modified': '2022-08-01T23:03:32.552769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7332': {'gid': 7332, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/550523700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 550523700}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 550523700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:34.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:34.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:03:34.512 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":7332,"want_state":"up:active","state":"up:active","rank_uptime":2.9465608570000001,"mdsmap_epoch":50,"osdmap_epoch":107,"osdmap_epoch_barrier":106,"uptime":7.2639121900000001} 2022-08-01T23:03:34.513 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 7332, "mdsmap_epoch": 50, "osdmap_epoch": 107, "osdmap_epoch_barrier": 106, "rank_uptime": 2.946560857, "state": "up:active", "uptime": 7.26391219, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:34.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.613+0000 7fe25359e700 1 Processor -- start 2022-08-01T23:03:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.613+0000 7fe25359e700 1 -- start start 2022-08-01T23:03:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540a9da0 0x7fe254060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2541402d0 0x7fe254140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe254067030 con 0x7fe2540b31d0 2022-08-01T23:03:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe254064a70 con 0x7fe2541402d0 2022-08-01T23:03:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe254061f60 con 0x7fe2540a9da0 2022-08-01T23:03:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2541402d0 0x7fe254140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2541402d0 0x7fe254140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47090/0 (socket says 172.21.15.32:47090) 2022-08-01T23:03:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 -- 172.21.15.32:0/4004650898 learned_addr learned my addr 172.21.15.32:0/4004650898 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe24bfff700 1 --2- 172.21.15.32:0/4004650898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540a9da0 0x7fe254060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 -- 172.21.15.32:0/4004650898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540a9da0 msgr2=0x7fe254060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 --2- 172.21.15.32:0/4004650898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540a9da0 0x7fe254060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe251d9b700 1 -- 172.21.15.32:0/4004650898 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686294246 0 0) 0x7fe254067030 con 0x7fe2540b31d0 2022-08-01T23:03:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 -- 172.21.15.32:0/4004650898 >> v1:172.21.15.55:6789/0 conn(0x7fe2540b31d0 legacy=0x7fe254066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 -- 172.21.15.32:0/4004650898 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2540b35b0 con 0x7fe2541402d0 2022-08-01T23:03:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe252d9d700 1 --2- 172.21.15.32:0/4004650898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe254140760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fe24c0032f0 tx=0x7fe24c009290).ready entity=mon.0 client_cookie=40485126966f712a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe251d9b700 1 -- 172.21.15.32:0/4004650898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c00b870 con 0x7fe2541402d0 2022-08-01T23:03:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe251d9b700 1 -- 172.21.15.32:0/4004650898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c00b9d0 con 0x7fe2541402d0 2022-08-01T23:03:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe251d9b700 1 -- 172.21.15.32:0/4004650898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c00bb40 con 0x7fe2541402d0 2022-08-01T23:03:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- 172.21.15.32:0/4004650898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 msgr2=0x7fe254140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 --2- 172.21.15.32:0/4004650898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe254140760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fe24c0032f0 tx=0x7fe24c009290).stop 2022-08-01T23:03:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- 172.21.15.32:0/4004650898 shutdown_connections 2022-08-01T23:03:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 --2- 172.21.15.32:0/4004650898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe254140760 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 --2- 172.21.15.32:0/4004650898 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540a9da0 0x7fe254060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- 172.21.15.32:0/4004650898 >> 172.21.15.32:0/4004650898 conn(0x7fe254063870 msgr2=0x7fe254063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- 172.21.15.32:0/4004650898 shutdown_connections 2022-08-01T23:03:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.617+0000 7fe25359e700 1 -- 172.21.15.32:0/4004650898 wait complete. 2022-08-01T23:03:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 Processor -- start 2022-08-01T23:03:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 -- start start 2022-08-01T23:03:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2540a9da0 0x7fe254133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540b31d0 0x7fe2541259c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe2540ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254142180 con 0x7fe2541402d0 2022-08-01T23:03:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254143bb0 con 0x7fe2540a9da0 2022-08-01T23:03:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe254143e50 con 0x7fe2540b31d0 2022-08-01T23:03:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe252d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe2540ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe252d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe2540ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47096/0 (socket says 172.21.15.32:47096) 2022-08-01T23:03:34.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe252d9d700 1 -- 172.21.15.32:0/4235340243 learned_addr learned my addr 172.21.15.32:0/4235340243 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:34.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe25259c700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2540a9da0 0x7fe254133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.621+0000 7fe24bfff700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540b31d0 0x7fe2541259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe252d9d700 1 -- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540b31d0 msgr2=0x7fe2541259c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe252d9d700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540b31d0 0x7fe2541259c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe252d9d700 1 -- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2540a9da0 msgr2=0x7fe254133880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe252d9d700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2540a9da0 0x7fe254133880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe252d9d700 1 -- 172.21.15.32:0/4235340243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c009530 con 0x7fe2541402d0 2022-08-01T23:03:34.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24bfff700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540b31d0 0x7fe2541259c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe252d9d700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe2540ac130 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fe24c0032f0 tx=0x7fe24c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c00bb40 con 0x7fe2541402d0 2022-08-01T23:03:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c021620 con 0x7fe2541402d0 2022-08-01T23:03:34.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe254125280 con 0x7fe2541402d0 2022-08-01T23:03:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c00b870 con 0x7fe2541402d0 2022-08-01T23:03:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe254133d70 con 0x7fe2541402d0 2022-08-01T23:03:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe254134960 con 0x7fe2541402d0 2022-08-01T23:03:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe24c028070 con 0x7fe2541402d0 2022-08-01T23:03:34.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24b7fe700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe238053980 0x7fe238055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe24c06cd40 con 0x7fe2541402d0 2022-08-01T23:03:34.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.625+0000 7fe25259c700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe238053980 0x7fe238055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.629+0000 7fe25259c700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe238053980 0x7fe238055e40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fe23c00d000 tx=0x7fe23c007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:34.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.633+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe24c043b00 con 0x7fe2541402d0 2022-08-01T23:03:34.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.761+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe254066af0 con 0x7fe2541402d0 2022-08-01T23:03:34.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe24b7fe700 1 -- 172.21.15.32:0/4235340243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fe24c015030 con 0x7fe2541402d0 2022-08-01T23:03:34.769 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:34.770 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":107,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:32.553487+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:30.001802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:30.985153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe238053980 msgr2=0x7fe238055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe238053980 0x7fe238055e40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fe23c00d000 tx=0x7fe23c007a00).stop 2022-08-01T23:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 msgr2=0x7fe2540ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe2540ac130 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fe24c0032f0 tx=0x7fe24c016000).stop 2022-08-01T23:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 shutdown_connections 2022-08-01T23:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe238053980 0x7fe238055e40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2541402d0 0x7fe2540ac130 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe2540b31d0 0x7fe2541259c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 --2- 172.21.15.32:0/4235340243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2540a9da0 0x7fe254133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 >> 172.21.15.32:0/4235340243 conn(0x7fe254063870 msgr2=0x7fe254063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:34.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 shutdown_connections 2022-08-01T23:03:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.765+0000 7fe25359e700 1 -- 172.21.15.32:0/4235340243 wait complete. 2022-08-01T23:03:34.782 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:03:30.001802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:03:30.985153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:03:34.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.885+0000 7f8210c12700 1 Processor -- start 2022-08-01T23:03:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.885+0000 7f8210c12700 1 -- start start 2022-08-01T23:03:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f820c05c960 0x7f820c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f820c05d240 0x7f820c13f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c13fa10 con 0x7f820c0acfc0 2022-08-01T23:03:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c067f00 con 0x7f820c05c960 2022-08-01T23:03:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c05cd40 con 0x7f820c05d240 2022-08-01T23:03:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f820c05c960 0x7f820c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f820c05c960 0x7f820c0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47100/0 (socket says 172.21.15.32:47100) 2022-08-01T23:03:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 -- 172.21.15.32:0/3009624769 learned_addr learned my addr 172.21.15.32:0/3009624769 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820affd700 1 --2- 172.21.15.32:0/3009624769 >> v2:172.21.15.55:3300/0 conn(0x7f820c05d240 0x7f820c13f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 -- 172.21.15.32:0/3009624769 >> v2:172.21.15.55:3300/0 conn(0x7f820c05d240 msgr2=0x7f820c13f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 --2- 172.21.15.32:0/3009624769 >> v2:172.21.15.55:3300/0 conn(0x7f820c05d240 0x7f820c13f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 -- 172.21.15.32:0/3009624769 >> v1:172.21.15.55:6789/0 conn(0x7f820c0acfc0 legacy=0x7f820c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 -- 172.21.15.32:0/3009624769 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0b1920 con 0x7f820c05c960 2022-08-01T23:03:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820b7fe700 1 --2- 172.21.15.32:0/3009624769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b1280 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f81fc002f70 tx=0x7f81fc006eb0).ready entity=mon.0 client_cookie=6732777da7bd74e0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820a7fc700 1 -- 172.21.15.32:0/3009624769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81fc00d5c0 con 0x7f820c05c960 2022-08-01T23:03:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820a7fc700 1 -- 172.21.15.32:0/3009624769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc000e90 con 0x7f820c05c960 2022-08-01T23:03:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f820a7fc700 1 -- 172.21.15.32:0/3009624769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81fc00d950 con 0x7f820c05c960 2022-08-01T23:03:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- 172.21.15.32:0/3009624769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 msgr2=0x7f820c0b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 --2- 172.21.15.32:0/3009624769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b1280 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f81fc002f70 tx=0x7f81fc006eb0).stop 2022-08-01T23:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- 172.21.15.32:0/3009624769 shutdown_connections 2022-08-01T23:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 --2- 172.21.15.32:0/3009624769 >> v2:172.21.15.55:3300/0 conn(0x7f820c05d240 0x7f820c13f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 --2- 172.21.15.32:0/3009624769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b1280 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- 172.21.15.32:0/3009624769 >> 172.21.15.32:0/3009624769 conn(0x7f820c060310 msgr2=0x7f820c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- 172.21.15.32:0/3009624769 shutdown_connections 2022-08-01T23:03:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.889+0000 7f8210c12700 1 -- 172.21.15.32:0/3009624769 wait complete. 2022-08-01T23:03:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 Processor -- start 2022-08-01T23:03:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 -- start start 2022-08-01T23:03:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f820c05d240 0x7f820c1397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f820c0acfc0 0x7f820c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b6160 con 0x7f820c05c960 2022-08-01T23:03:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b5ec0 con 0x7f820c0acfc0 2022-08-01T23:03:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0ab2f0 con 0x7f820c05d240 2022-08-01T23:03:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47106/0 (socket says 172.21.15.32:47106) 2022-08-01T23:03:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 -- 172.21.15.32:0/2357931219 learned_addr learned my addr 172.21.15.32:0/2357931219 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820bfff700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f820c0acfc0 0x7f820c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820affd700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f820c05d240 0x7f820c1397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 -- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f820c05d240 msgr2=0x7f820c1397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f820c05d240 0x7f820c1397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 -- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f820c0acfc0 msgr2=0x7f820c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f820c0acfc0 0x7f820c0b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:34.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 -- 172.21.15.32:0/2357931219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc00b1e0 con 0x7f820c05c960 2022-08-01T23:03:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820bfff700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f820c0acfc0 0x7f820c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820affd700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f820c05d240 0x7f820c1397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f820b7fe700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b5890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f81fc002f70 tx=0x7f81fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81fc011990 con 0x7f820c05c960 2022-08-01T23:03:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc011af0 con 0x7f820c05c960 2022-08-01T23:03:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c0ab590 con 0x7f820c05c960 2022-08-01T23:03:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81fc011e50 con 0x7f820c05c960 2022-08-01T23:03:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.893+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c0b4500 con 0x7f820c05c960 2022-08-01T23:03:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.897+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c0ab750 con 0x7f820c05c960 2022-08-01T23:03:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.897+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f81fc025030 con 0x7f820c05c960 2022-08-01T23:03:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.897+0000 7f8208ff9700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81ec053a10 0x7f81ec055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.897+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f81fc06d2a0 con 0x7f820c05c960 2022-08-01T23:03:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.897+0000 7f820affd700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81ec053a10 0x7f81ec055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.897+0000 7f820affd700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81ec053a10 0x7f81ec055ed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f8200006000 tx=0x7f8200007e90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:34.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:34.905+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f81fc044060 con 0x7f820c05c960 2022-08-01T23:03:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.033+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f820c0b1bc0 con 0x7f820c05c960 2022-08-01T23:03:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.033+0000 7f8208ff9700 1 -- 172.21.15.32:0/2357931219 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7f81fc0230b0 con 0x7f820c05c960 2022-08-01T23:03:35.041 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81ec053a10 msgr2=0x7f81ec055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81ec053a10 0x7f81ec055ed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f8200006000 tx=0x7f8200007e90).stop 2022-08-01T23:03:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 msgr2=0x7f820c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b5890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f81fc002f70 tx=0x7f81fc015000).stop 2022-08-01T23:03:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 shutdown_connections 2022-08-01T23:03:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f81ec053a10 0x7f81ec055ed0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f820c0acfc0 0x7f820c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f820c05d240 0x7f820c1397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 --2- 172.21.15.32:0/2357931219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820c05c960 0x7f820c0b5890 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 >> 172.21.15.32:0/2357931219 conn(0x7f820c060310 msgr2=0x7f820c13ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 shutdown_connections 2022-08-01T23:03:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.037+0000 7f8210c12700 1 -- 172.21.15.32:0/2357931219 wait complete. 2022-08-01T23:03:35.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 Processor -- start 2022-08-01T23:03:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 -- start start 2022-08-01T23:03:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72340614b0 0x7f7234061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72340ad750 0x7f72340b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72341405d0 0x7f7234140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340aee80 con 0x7f72340614b0 2022-08-01T23:03:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340b70b0 con 0x7f72340ad750 2022-08-01T23:03:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a9ee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72340af0b0 con 0x7f72341405d0 2022-08-01T23:03:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72340614b0 0x7f7234061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72340614b0 0x7f7234061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47114/0 (socket says 172.21.15.32:47114) 2022-08-01T23:03:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 -- 172.21.15.32:0/2109198634 learned_addr learned my addr 172.21.15.32:0/2109198634 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72391eb700 1 --2- 172.21.15.32:0/2109198634 >> v2:172.21.15.55:3300/0 conn(0x7f72340ad750 0x7f72340b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f723a1ed700 1 --2- 172.21.15.32:0/2109198634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72341405d0 0x7f7234140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 -- 172.21.15.32:0/2109198634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72341405d0 msgr2=0x7f7234140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 --2- 172.21.15.32:0/2109198634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72341405d0 0x7f7234140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 -- 172.21.15.32:0/2109198634 >> v2:172.21.15.55:3300/0 conn(0x7f72340ad750 msgr2=0x7f72340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 --2- 172.21.15.32:0/2109198634 >> v2:172.21.15.55:3300/0 conn(0x7f72340ad750 0x7f72340b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 -- 172.21.15.32:0/2109198634 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340af350 con 0x7f72340614b0 2022-08-01T23:03:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72391eb700 1 --2- 172.21.15.32:0/2109198634 >> v2:172.21.15.55:3300/0 conn(0x7f72340ad750 0x7f72340b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72399ec700 1 --2- 172.21.15.32:0/2109198634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340614b0 0x7f7234061890 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f72340aed30 tx=0x7f7228006eb0).ready entity=mon.0 client_cookie=4145fcd3ae9c9930 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72389ea700 1 -- 172.21.15.32:0/2109198634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f722800d570 con 0x7f72340614b0 2022-08-01T23:03:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72389ea700 1 -- 172.21.15.32:0/2109198634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7228000e90 con 0x7f72340614b0 2022-08-01T23:03:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.161+0000 7f72389ea700 1 -- 172.21.15.32:0/2109198634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f722800d8c0 con 0x7f72340614b0 2022-08-01T23:03:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- 172.21.15.32:0/2109198634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340614b0 msgr2=0x7f7234061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- 172.21.15.32:0/2109198634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340614b0 0x7f7234061890 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f72340aed30 tx=0x7f7228006eb0).stop 2022-08-01T23:03:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- 172.21.15.32:0/2109198634 shutdown_connections 2022-08-01T23:03:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- 172.21.15.32:0/2109198634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72341405d0 0x7f7234140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- 172.21.15.32:0/2109198634 >> v2:172.21.15.55:3300/0 conn(0x7f72340ad750 0x7f72340b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- 172.21.15.32:0/2109198634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340614b0 0x7f7234061890 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- 172.21.15.32:0/2109198634 >> 172.21.15.32:0/2109198634 conn(0x7f7234066f70 msgr2=0x7f7234067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- 172.21.15.32:0/2109198634 shutdown_connections 2022-08-01T23:03:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- 172.21.15.32:0/2109198634 wait complete. 2022-08-01T23:03:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 Processor -- start 2022-08-01T23:03:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- start start 2022-08-01T23:03:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340614b0 0x7f72340b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 0x7f72340aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72341405d0 0x7f72340ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723409eae0 con 0x7f72340ad750 2022-08-01T23:03:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723409ed80 con 0x7f72341405d0 2022-08-01T23:03:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a9ee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b1f30 con 0x7f72340614b0 2022-08-01T23:03:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 0x7f72340aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 0x7f72340aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47122/0 (socket says 172.21.15.32:47122) 2022-08-01T23:03:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 -- 172.21.15.32:0/1999220716 learned_addr learned my addr 172.21.15.32:0/1999220716 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f723a1ed700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72341405d0 0x7f72340ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72399ec700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340614b0 0x7f72340b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 -- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340614b0 msgr2=0x7f72340b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340614b0 0x7f72340b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 -- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72341405d0 msgr2=0x7f72340ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72341405d0 0x7f72340ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.165+0000 7f72391eb700 1 -- 172.21.15.32:0/1999220716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722800b1e0 con 0x7f72340ad750 2022-08-01T23:03:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f72391eb700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 0x7f72340aff00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f723000a000 tx=0x7f7230008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7230011780 con 0x7f72340ad750 2022-08-01T23:03:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72300118e0 con 0x7f72340ad750 2022-08-01T23:03:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340b0960 con 0x7f72340ad750 2022-08-01T23:03:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7230011c40 con 0x7f72340ad750 2022-08-01T23:03:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72340b3e70 con 0x7f72340ad750 2022-08-01T23:03:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723409ef80 con 0x7f72340ad750 2022-08-01T23:03:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7230012420 con 0x7f72340ad750 2022-08-01T23:03:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f7226ffd700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7214053a70 0x7f7214055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f723006c2c0 con 0x7f72340ad750 2022-08-01T23:03:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.169+0000 7f72399ec700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7214053a70 0x7f7214055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.173+0000 7f72399ec700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7214053a70 0x7f7214055f30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f722801c000 tx=0x7f7228015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.177+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f723409ef80 con 0x7f72340ad750 2022-08-01T23:03:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.301+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72340b2130 con 0x7f72340ad750 2022-08-01T23:03:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.301+0000 7f7226ffd700 1 -- 172.21.15.32:0/1999220716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f72340b2130 con 0x7f72340ad750 2022-08-01T23:03:35.308 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7214053a70 msgr2=0x7f7214055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7214053a70 0x7f7214055f30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f722801c000 tx=0x7f7228015000).stop 2022-08-01T23:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 msgr2=0x7f72340aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 0x7f72340aff00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f723000a000 tx=0x7f7230008040).stop 2022-08-01T23:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 shutdown_connections 2022-08-01T23:03:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7214053a70 0x7f7214055f30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72341405d0 0x7f72340ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72340ad750 0x7f72340aff00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 --2- 172.21.15.32:0/1999220716 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72340614b0 0x7f72340b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 >> 172.21.15.32:0/1999220716 conn(0x7f7234066f70 msgr2=0x7f7234144970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 shutdown_connections 2022-08-01T23:03:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.305+0000 7f723a9ee700 1 -- 172.21.15.32:0/1999220716 wait complete. 2022-08-01T23:03:35.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:03:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.429+0000 7f163a22c700 1 Processor -- start 2022-08-01T23:03:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.429+0000 7f163a22c700 1 -- start start 2022-08-01T23:03:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f16340b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16340b2430 0x7f16341413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340a7960 con 0x7f16340b0140 2022-08-01T23:03:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340b0580 con 0x7f16340b2430 2022-08-01T23:03:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340b07e0 con 0x7f16340b2050 2022-08-01T23:03:35.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1633fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16340b2430 0x7f16341413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1632ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16340b0140 0x7f16340a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34672/0 (socket says 172.21.15.32:34672) 2022-08-01T23:03:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1632ffd700 1 -- 172.21.15.32:0/2890291939 learned_addr learned my addr 172.21.15.32:0/2890291939 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16337fe700 1 --2- 172.21.15.32:0/2890291939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f16340b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16327fc700 1 -- 172.21.15.32:0/2890291939 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373214678 0 0) 0x7f16340a7960 con 0x7f16340b0140 2022-08-01T23:03:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16327fc700 1 -- 172.21.15.32:0/2890291939 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1620003260 con 0x7f16340b0140 2022-08-01T23:03:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1633fff700 1 -- 172.21.15.32:0/2890291939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 msgr2=0x7f16340b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16327fc700 1 -- 172.21.15.32:0/2890291939 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198853903 0 0) 0x7f1620003260 con 0x7f16340b0140 2022-08-01T23:03:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1633fff700 1 --2- 172.21.15.32:0/2890291939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f16340b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1633fff700 1 -- 172.21.15.32:0/2890291939 >> v1:172.21.15.32:6789/0 conn(0x7f16340b0140 legacy=0x7f16340a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1633fff700 1 -- 172.21.15.32:0/2890291939 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340b0a80 con 0x7f16340b2430 2022-08-01T23:03:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f1633fff700 1 --2- 172.21.15.32:0/2890291939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b2430 0x7f16341413e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f1624001800 tx=0x7f162c0091d0).ready entity=mon.0 client_cookie=c6d9dae2f5fecb14 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16327fc700 1 -- 172.21.15.32:0/2890291939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1624001660 con 0x7f16340b2430 2022-08-01T23:03:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16327fc700 1 -- 172.21.15.32:0/2890291939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c00b580 con 0x7f16340b2430 2022-08-01T23:03:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f16327fc700 1 -- 172.21.15.32:0/2890291939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f162c00b6f0 con 0x7f16340b2430 2022-08-01T23:03:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- 172.21.15.32:0/2890291939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b2430 msgr2=0x7f16341413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 --2- 172.21.15.32:0/2890291939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b2430 0x7f16341413e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f1624001800 tx=0x7f162c0091d0).stop 2022-08-01T23:03:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- 172.21.15.32:0/2890291939 shutdown_connections 2022-08-01T23:03:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 --2- 172.21.15.32:0/2890291939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b2430 0x7f16341413e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 --2- 172.21.15.32:0/2890291939 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f16340b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- 172.21.15.32:0/2890291939 >> 172.21.15.32:0/2890291939 conn(0x7f163405f840 msgr2=0x7f163405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- 172.21.15.32:0/2890291939 shutdown_connections 2022-08-01T23:03:35.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- 172.21.15.32:0/2890291939 wait complete. 2022-08-01T23:03:35.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 Processor -- start 2022-08-01T23:03:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.433+0000 7f163a22c700 1 -- start start 2022-08-01T23:03:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 0x7f16340b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f1634141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16340b2430 0x7f163415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340b33a0 con 0x7f16340b0140 2022-08-01T23:03:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163414bba0 con 0x7f16340b2430 2022-08-01T23:03:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163414ac20 con 0x7f16340b2050 2022-08-01T23:03:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 0x7f16340b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 0x7f16340b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47134/0 (socket says 172.21.15.32:47134) 2022-08-01T23:03:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 -- 172.21.15.32:0/1426322904 learned_addr learned my addr 172.21.15.32:0/1426322904 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1632ffd700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f1634141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1633fff700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16340b2430 0x7f163415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 -- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 msgr2=0x7f1634141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f1634141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 -- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16340b2430 msgr2=0x7f163415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16340b2430 0x7f163415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 -- 172.21.15.32:0/1426322904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c009470 con 0x7f16340b0140 2022-08-01T23:03:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1633fff700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16340b2430 0x7f163415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1632ffd700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f1634141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f16337fe700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 0x7f16340b3790 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f16280030b0 tx=0x7f1628008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1628011780 con 0x7f16340b0140 2022-08-01T23:03:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16280118e0 con 0x7f16340b0140 2022-08-01T23:03:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163414b5b0 con 0x7f16340b0140 2022-08-01T23:03:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1628011c40 con 0x7f16340b0140 2022-08-01T23:03:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16341578c0 con 0x7f16340b0140 2022-08-01T23:03:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.437+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1634157a80 con 0x7f16340b0140 2022-08-01T23:03:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.441+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f162801c460 con 0x7f16340b0140 2022-08-01T23:03:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.441+0000 7f1630ff9700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1614053a10 0x7f1614055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.441+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f162806c390 con 0x7f16340b0140 2022-08-01T23:03:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.441+0000 7f1632ffd700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1614053a10 0x7f1614055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.441+0000 7f1632ffd700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1614053a10 0x7f1614055ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f162400b000 tx=0x7f1624001140).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.449+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1628042fe0 con 0x7f16340b0140 2022-08-01T23:03:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.569+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16340b0d20 con 0x7f16340b0140 2022-08-01T23:03:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.569+0000 7f1630ff9700 1 -- 172.21.15.32:0/1426322904 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1628046600 con 0x7f16340b0140 2022-08-01T23:03:35.576 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1614053a10 msgr2=0x7f1614055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1614053a10 0x7f1614055ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f162400b000 tx=0x7f1624001140).stop 2022-08-01T23:03:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 msgr2=0x7f16340b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 0x7f16340b3790 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f16280030b0 tx=0x7f1628008040).stop 2022-08-01T23:03:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 shutdown_connections 2022-08-01T23:03:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1614053a10 0x7f1614055ed0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16340b2430 0x7f163415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16340b2050 0x7f1634141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 --2- 172.21.15.32:0/1426322904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16340b0140 0x7f16340b3790 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 >> 172.21.15.32:0/1426322904 conn(0x7f163405f840 msgr2=0x7f16340af800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 shutdown_connections 2022-08-01T23:03:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.573+0000 7f163a22c700 1 -- 172.21.15.32:0/1426322904 wait complete. 2022-08-01T23:03:35.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:03:35.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 Processor -- start 2022-08-01T23:03:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 -- start start 2022-08-01T23:03:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4d40b7500 0x7ff4d40b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4d40b31f0 0x7ff4d413e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d40b2510 con 0x7ff4d413e8a0 2022-08-01T23:03:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d41443e0 con 0x7ff4d40b7500 2022-08-01T23:03:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d9d59700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d40b3700 con 0x7ff4d40b31f0 2022-08-01T23:03:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.701+0000 7ff4d8d57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4d40b7500 0x7ff4d40b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9558700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4d413e8a0 0x7ff4d4064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34688/0 (socket says 172.21.15.32:34688) 2022-08-01T23:03:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d8d57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff4d40b7500 0x7ff4d40b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47144/0 (socket says 172.21.15.32:47144) 2022-08-01T23:03:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9558700 1 -- 172.21.15.32:0/3136438102 learned_addr learned my addr 172.21.15.32:0/3136438102 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d3fff700 1 --2- 172.21.15.32:0/3136438102 >> v2:172.21.15.55:3300/0 conn(0x7ff4d40b31f0 0x7ff4d413e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d8d57700 1 -- 172.21.15.32:0/3136438102 >> v2:172.21.15.55:3300/0 conn(0x7ff4d40b31f0 msgr2=0x7ff4d413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d8d57700 1 --2- 172.21.15.32:0/3136438102 >> v2:172.21.15.55:3300/0 conn(0x7ff4d40b31f0 0x7ff4d413e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d8d57700 1 -- 172.21.15.32:0/3136438102 >> v1:172.21.15.32:6789/0 conn(0x7ff4d413e8a0 legacy=0x7ff4d4064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d8d57700 1 -- 172.21.15.32:0/3136438102 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d40b2740 con 0x7ff4d40b7500 2022-08-01T23:03:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d8d57700 1 --2- 172.21.15.32:0/3136438102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b78e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7ff4c4002f70 tx=0x7ff4c4006d90).ready entity=mon.0 client_cookie=4ec4a104633d3032 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d37fe700 1 -- 172.21.15.32:0/3136438102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4c400d4a0 con 0x7ff4d40b7500 2022-08-01T23:03:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d37fe700 1 -- 172.21.15.32:0/3136438102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c4000e90 con 0x7ff4d40b7500 2022-08-01T23:03:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d37fe700 1 -- 172.21.15.32:0/3136438102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4c400d7f0 con 0x7ff4d40b7500 2022-08-01T23:03:35.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 -- 172.21.15.32:0/3136438102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d40b7500 msgr2=0x7ff4d40b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/3136438102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b78e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7ff4c4002f70 tx=0x7ff4c4006d90).stop 2022-08-01T23:03:35.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 -- 172.21.15.32:0/3136438102 shutdown_connections 2022-08-01T23:03:35.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/3136438102 >> v2:172.21.15.55:3300/0 conn(0x7ff4d40b31f0 0x7ff4d413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/3136438102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b78e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 -- 172.21.15.32:0/3136438102 >> 172.21.15.32:0/3136438102 conn(0x7ff4d405f840 msgr2=0x7ff4d405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 -- 172.21.15.32:0/3136438102 shutdown_connections 2022-08-01T23:03:35.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.705+0000 7ff4d9d59700 1 -- 172.21.15.32:0/3136438102 wait complete. 2022-08-01T23:03:35.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 Processor -- start 2022-08-01T23:03:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 -- start start 2022-08-01T23:03:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4d40b31f0 0x7ff4d409f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 0x7ff4d4152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b0e20 con 0x7ff4d413e8a0 2022-08-01T23:03:35.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d409ef90 con 0x7ff4d40b7500 2022-08-01T23:03:35.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d4062510 con 0x7ff4d40b31f0 2022-08-01T23:03:35.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 0x7ff4d4152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 0x7ff4d4152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47150/0 (socket says 172.21.15.32:47150) 2022-08-01T23:03:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 -- 172.21.15.32:0/2780626134 learned_addr learned my addr 172.21.15.32:0/2780626134 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d8d57700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4d40b31f0 0x7ff4d409f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d3fff700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 -- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4d40b31f0 msgr2=0x7ff4d409f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4d40b31f0 0x7ff4d409f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 -- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4d40b7500 msgr2=0x7ff4d40b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 -- 172.21.15.32:0/2780626134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c400b120 con 0x7ff4d413e8a0 2022-08-01T23:03:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9558700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 0x7ff4d4152780 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7ff4d409eeb0 tx=0x7ff4cc0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4cc005ba0 con 0x7ff4d413e8a0 2022-08-01T23:03:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc005d00 con 0x7ff4d413e8a0 2022-08-01T23:03:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d4062270 con 0x7ff4d413e8a0 2022-08-01T23:03:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4cc0072c0 con 0x7ff4d413e8a0 2022-08-01T23:03:35.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.709+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d40b1e90 con 0x7ff4d413e8a0 2022-08-01T23:03:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.713+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d40b2920 con 0x7ff4d413e8a0 2022-08-01T23:03:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.713+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff4cc01a060 con 0x7ff4d413e8a0 2022-08-01T23:03:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.713+0000 7ff4d1ffb700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff4b0053b00 0x7ff4b0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.713+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff4cc06ba70 con 0x7ff4d413e8a0 2022-08-01T23:03:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.713+0000 7ff4d8d57700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff4b0053b00 0x7ff4b0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.713+0000 7ff4d8d57700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff4b0053b00 0x7ff4b0055fc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7ff4c4018000 tx=0x7ff4c4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.717+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4cc042830 con 0x7ff4d413e8a0 2022-08-01T23:03:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.841+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4d40574e0 con 0x7ff4d413e8a0 2022-08-01T23:03:35.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.841+0000 7ff4d1ffb700 1 -- 172.21.15.32:0/2780626134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff4cc06be60 con 0x7ff4d413e8a0 2022-08-01T23:03:35.847 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:35.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff4b0053b00 msgr2=0x7ff4b0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff4b0053b00 0x7ff4b0055fc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7ff4c4018000 tx=0x7ff4c4014000).stop 2022-08-01T23:03:35.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 msgr2=0x7ff4d4152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 0x7ff4d4152780 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7ff4d409eeb0 tx=0x7ff4cc0013a0).stop 2022-08-01T23:03:35.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 shutdown_connections 2022-08-01T23:03:35.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff4b0053b00 0x7ff4b0055fc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d413e8a0 0x7ff4d4152780 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4d40b7500 0x7ff4d40b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 --2- 172.21.15.32:0/2780626134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff4d40b31f0 0x7ff4d409f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 >> 172.21.15.32:0/2780626134 conn(0x7ff4d405f840 msgr2=0x7ff4d4143a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 shutdown_connections 2022-08-01T23:03:35.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.845+0000 7ff4d9d59700 1 -- 172.21.15.32:0/2780626134 wait complete. 2022-08-01T23:03:35.861 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:03:35.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:35.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f5853fff700 1 Processor -- start 2022-08-01T23:03:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f5853fff700 1 -- start start 2022-08-01T23:03:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f5853fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f585414b750 0x7f5854065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f5853fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58540b2c20 con 0x7f58540b3070 2022-08-01T23:03:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f5853fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58540b7c20 con 0x7f58540b7840 2022-08-01T23:03:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f5853fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5854061830 con 0x7f585414b750 2022-08-01T23:03:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f58527fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58540b3070 0x7f585414b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34698/0 (socket says 172.21.15.32:34698) 2022-08-01T23:03:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.961+0000 7f58527fc700 1 -- 172.21.15.32:0/966556416 learned_addr learned my addr 172.21.15.32:0/966556416 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58537fe700 1 --2- 172.21.15.32:0/966556416 >> v2:172.21.15.55:3300/0 conn(0x7f585414b750 0x7f5854065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616611810 0 0) 0x7f58540b2c20 con 0x7f58540b3070 2022-08-01T23:03:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5834003260 con 0x7f58540b3070 2022-08-01T23:03:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581924440 0 0) 0x7f5834003260 con 0x7f58540b3070 2022-08-01T23:03:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 >> v2:172.21.15.55:3300/0 conn(0x7f585414b750 msgr2=0x7f5854065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 --2- 172.21.15.32:0/966556416 >> v2:172.21.15.55:3300/0 conn(0x7f585414b750 0x7f5854065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 >> v1:172.21.15.55:6789/0 conn(0x7f58540b7840 legacy=0x7f5854061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540b2980 con 0x7f58540b3070 2022-08-01T23:03:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f583c0019e0 con 0x7f58540b3070 2022-08-01T23:03:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 >> v1:172.21.15.32:6789/0 conn(0x7f58540b3070 legacy=0x7f585414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5834003750 0x7f5834003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5834004500 0x7f5834004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004e70 0x7f583400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854061830 con 0x7f5834004500 2022-08-01T23:03:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583400bd20 con 0x7f5834003750 2022-08-01T23:03:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5834004500 0x7f5834004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583400bee0 con 0x7f5834004e70 2022-08-01T23:03:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5852ffd700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5834003750 0x7f5834003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58537fe700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004e70 0x7f583400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 -- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004e70 msgr2=0x7f583400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004e70 0x7f583400b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 -- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5834003750 msgr2=0x7f5834003b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:35.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5834003750 0x7f5834003b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 -- 172.21.15.32:0/966556416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834003260 con 0x7f5834004500 2022-08-01T23:03:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58537fe700 1 -- 172.21.15.32:0/966556416 reap_dead start 2022-08-01T23:03:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f58527fc700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5834004500 0x7f5834004980 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f583c0016f0 tx=0x7f5854061db0).ready entity=mon.0 client_cookie=8b0269476a945f4d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f583c00cc20 con 0x7f5834004500 2022-08-01T23:03:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00cd80 con 0x7f5834004500 2022-08-01T23:03:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5851ffb700 1 -- 172.21.15.32:0/966556416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f583c005b80 con 0x7f5834004500 2022-08-01T23:03:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 -- 172.21.15.32:0/966556416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5834004500 msgr2=0x7f5834004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5834004500 0x7f5834004980 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f583c0016f0 tx=0x7f5854061db0).stop 2022-08-01T23:03:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 -- 172.21.15.32:0/966556416 shutdown_connections 2022-08-01T23:03:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 --2- 172.21.15.32:0/966556416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5834004500 0x7f5834004980 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 -- 172.21.15.32:0/966556416 >> 172.21.15.32:0/966556416 conn(0x7f585405f840 msgr2=0x7f585405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 -- 172.21.15.32:0/966556416 shutdown_connections 2022-08-01T23:03:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.965+0000 7f5853fff700 1 -- 172.21.15.32:0/966556416 wait complete. 2022-08-01T23:03:35.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 Processor -- start 2022-08-01T23:03:35.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 -- start start 2022-08-01T23:03:35.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004500 0x7f58540a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 0x7f58540a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540ae9a0 0x7f58540aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b1320 con 0x7f58541575d0 2022-08-01T23:03:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b1550 con 0x7f58540ae9a0 2022-08-01T23:03:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58540adb90 con 0x7f5834004500 2022-08-01T23:03:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 0x7f58540a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 0x7f58540a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47168/0 (socket says 172.21.15.32:47168) 2022-08-01T23:03:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 -- 172.21.15.32:0/2636369095 learned_addr learned my addr 172.21.15.32:0/2636369095 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5852ffd700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004500 0x7f58540a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58537fe700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540ae9a0 0x7f58540aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 -- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004500 msgr2=0x7f58540a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004500 0x7f58540a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 -- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540ae9a0 msgr2=0x7f58540aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:35.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540ae9a0 0x7f58540aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:35.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 -- 172.21.15.32:0/2636369095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540ade30 con 0x7f58541575d0 2022-08-01T23:03:35.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f58527fc700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 0x7f58540a1f50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f583c0016f0 tx=0x7f583c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f583c001f50 con 0x7f58541575d0 2022-08-01T23:03:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c0070d0 con 0x7f58541575d0 2022-08-01T23:03:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5854154420 con 0x7f58541575d0 2022-08-01T23:03:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f583c007430 con 0x7f58541575d0 2022-08-01T23:03:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.969+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58540b0f20 con 0x7f58541575d0 2022-08-01T23:03:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.973+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58540609e0 con 0x7f58541575d0 2022-08-01T23:03:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.973+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f583c025020 con 0x7f58541575d0 2022-08-01T23:03:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.973+0000 7f584bfff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053b00 0x7f5838055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.973+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f583c06d060 con 0x7f58541575d0 2022-08-01T23:03:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.973+0000 7f5852ffd700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053b00 0x7f5838055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.973+0000 7f5852ffd700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053b00 0x7f5838055fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f584400d000 tx=0x7f5844009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:35.981+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f583c043e20 con 0x7f58541575d0 2022-08-01T23:03:36.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.141+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58540574e0 con 0x7f58541575d0 2022-08-01T23:03:36.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.141+0000 7f584bfff700 1 -- 172.21.15.32:0/2636369095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f583c047440 con 0x7f58541575d0 2022-08-01T23:03:36.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:36.148 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:31.549702+0000","modified":"2022-08-01T23:03:32.552769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/550523700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":550523700},{"type":"v1","addr":"172.21.15.32:6835","nonce":550523700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T23:03:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053b00 msgr2=0x7f5838055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053b00 0x7f5838055fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f584400d000 tx=0x7f5844009040).stop 2022-08-01T23:03:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 msgr2=0x7f58540a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 0x7f58540a1f50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f583c0016f0 tx=0x7f583c015000).stop 2022-08-01T23:03:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 shutdown_connections 2022-08-01T23:03:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5838053b00 0x7f5838055fc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540ae9a0 0x7f58540aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58541575d0 0x7f58540a1f50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 --2- 172.21.15.32:0/2636369095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5834004500 0x7f58540a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 >> 172.21.15.32:0/2636369095 conn(0x7f585405f840 msgr2=0x7f58540b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 shutdown_connections 2022-08-01T23:03:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:36.145+0000 7f5853fff700 1 -- 172.21.15.32:0/2636369095 wait complete. 2022-08-01T23:03:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2022-08-01T23:03:36.463 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:31.549702+0000', 'modified': '2022-08-01T23:03:32.552769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7332': {'gid': 7332, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/550523700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 550523700}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 550523700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:36.463 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:36.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:03:36.581 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":7332,"want_state":"up:active","state":"up:active","rank_uptime":5.0161123700000001,"mdsmap_epoch":50,"osdmap_epoch":107,"osdmap_epoch_barrier":106,"uptime":9.3334616060000002} 2022-08-01T23:03:36.581 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 7332, "mdsmap_epoch": 50, "osdmap_epoch": 107, "osdmap_epoch_barrier": 106, "rank_uptime": 5.01611237, "state": "up:active", "uptime": 9.333461606, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:36.582 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:03:36.582 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:03:36.583 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:03:36.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:36.584 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:36.592 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:03:36.610 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:36.610 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:36.620 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:36.621 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:36.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:36.622 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:03:36.671 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:03:36.672 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:36.675 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:03:36.696 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:36.697 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:36.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:03:36.743+0000 7feab1f58080 -1 init, newargv = 0x5560fd8af940 newargc=9ceph-fuse[23316]: starting ceph client 2022-08-01T23:03:36.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr: 2022-08-01T23:03:36.754 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:36.755 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:36.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:36.756 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:03:36.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23316]: starting fuse 2022-08-01T23:03:37.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:03:37.880 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:37.881 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:37.939 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:37.940 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:37.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:37.941 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:03:37.946 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:03:37.947 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:03:37.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:37.948 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:03:37.948 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:03:37.949 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:03:37.949 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:03:37.950 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:03:37.950 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:37.951 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:03:37.951 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:03:37.951 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:03:37.952 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:37.952 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:03:37.953 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:03:37.953 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:03:37.954 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:03:37.954 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:37.955 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:03:37.955 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:03:37.956 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:03:37.956 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:03:37.957 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:03:37.957 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:03:37.957 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:03:37.958 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:03:37.996 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23316.asok 2022-08-01T23:03:38.001 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23316.asok 2022-08-01T23:03:38.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:38.002 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23316.asok status 2022-08-01T23:03:38.137 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:03:38.137 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:03:38.138 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:03:38.138 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:03:38.138 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:03:38.138 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:03:38.139 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:03:38.139 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23316", 2022-08-01T23:03:38.139 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:03:38.139 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:38.140 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:03:38.140 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:03:38.140 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 7369, 2022-08-01T23:03:38.140 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:03:38.141 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:03:38.141 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:03:38.141 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 7369 2022-08-01T23:03:38.141 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:38.142 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:03:38.142 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:03:38.142 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:03:38.142 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1647559837 2022-08-01T23:03:38.143 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:03:38.143 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:38.143 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:03:38.143 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:03:38.143 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:03:38.144 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 1647559837 2022-08-01T23:03:38.144 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:03:38.144 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.7369 172.21.15.130:0/1647559837", 2022-08-01T23:03:38.144 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/1647559837", 2022-08-01T23:03:38.145 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:03:38.145 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 50, 2022-08-01T23:03:38.145 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 107, 2022-08-01T23:03:38.145 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:03:38.146 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:03:38.146 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:03:38.146 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:03:38.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:38.148 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:38.148 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:38.155 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:03:38.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:38.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:38.158 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:38.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:38.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e96b9e700 1 Processor -- start 2022-08-01T23:03:38.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e96b9e700 1 -- start start 2022-08-01T23:03:38.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e96b9e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e90152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e96b9e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e900a7250 con 0x7f6e900b01a0 2022-08-01T23:03:38.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e96b9e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e900a7480 con 0x7f6e900b2710 2022-08-01T23:03:38.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e96b9e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900a7720 con 0x7f6e900a8c50 2022-08-01T23:03:38.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6e900b01a0 0x7f6e900a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34716/0 (socket says 172.21.15.32:34716) 2022-08-01T23:03:38.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 -- 172.21.15.32:0/393779555 learned_addr learned my addr 172.21.15.32:0/393779555 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:38.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9639d700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e90152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583287401 0 0) 0x7f6e900a7250 con 0x7f6e900b01a0 2022-08-01T23:03:38.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c003260 con 0x7f6e900b01a0 2022-08-01T23:03:38.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590900679 0 0) 0x7f6e900a7480 con 0x7f6e900b2710 2022-08-01T23:03:38.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e900a7250 con 0x7f6e900b2710 2022-08-01T23:03:38.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2241682704 0 0) 0x7f6e7c003260 con 0x7f6e900b01a0 2022-08-01T23:03:38.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 msgr2=0x7f6e90152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:38.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e90152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 >> v1:172.21.15.55:6789/0 conn(0x7f6e900b2710 legacy=0x7f6e9014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900a79c0 con 0x7f6e900b01a0 2022-08-01T23:03:38.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f6e800019e0 con 0x7f6e900b01a0 2022-08-01T23:03:38.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9639d700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e90152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:38.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 >> v1:172.21.15.32:6789/0 conn(0x7f6e900b01a0 legacy=0x7f6e900a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e7c003910 0x7f6e7c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c004500 0x7f6e7c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e7c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900a7720 con 0x7f6e7c004500 2022-08-01T23:03:38.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00b750 con 0x7f6e7c003910 2022-08-01T23:03:38.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00b910 con 0x7f6e900a8c50 2022-08-01T23:03:38.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c004500 0x7f6e7c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e95b9c700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e7c003910 0x7f6e7c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9639d700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e7c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 -- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 msgr2=0x7f6e7c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:38.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e900a8c50 0x7f6e7c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 -- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e7c003910 msgr2=0x7f6e7c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:38.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e7c003910 0x7f6e7c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e9539b700 1 -- 172.21.15.32:0/393779555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8800b180 con 0x7f6e7c004500 2022-08-01T23:03:38.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.329+0000 7f6e95b9c700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e7c003910 0x7f6e7c003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:38.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e9539b700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c004500 0x7f6e7c004980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f6e800017b0 tx=0x7f6e900b0640).ready entity=mon.0 client_cookie=e162533d7707fff4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e8000ccb0 con 0x7f6e7c004500 2022-08-01T23:03:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8000ce10 con 0x7f6e7c004500 2022-08-01T23:03:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e94b9a700 1 -- 172.21.15.32:0/393779555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e80005b80 con 0x7f6e7c004500 2022-08-01T23:03:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- 172.21.15.32:0/393779555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c004500 msgr2=0x7f6e7c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/393779555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c004500 0x7f6e7c004980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f6e800017b0 tx=0x7f6e900b0640).stop 2022-08-01T23:03:38.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e9639d700 1 -- 172.21.15.32:0/393779555 reap_dead start 2022-08-01T23:03:38.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- 172.21.15.32:0/393779555 shutdown_connections 2022-08-01T23:03:38.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- 172.21.15.32:0/393779555 >> 172.21.15.32:0/393779555 conn(0x7f6e90060310 msgr2=0x7f6e900b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:38.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- 172.21.15.32:0/393779555 shutdown_connections 2022-08-01T23:03:38.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- 172.21.15.32:0/393779555 wait complete. 2022-08-01T23:03:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 Processor -- start 2022-08-01T23:03:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- start start 2022-08-01T23:03:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e7c004500 0x7f6e90156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e9014aef0 0x7f6e90143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 0x7f6e900abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900b3310 con 0x7f6e900b5a10 2022-08-01T23:03:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900a9450 con 0x7f6e9014aef0 2022-08-01T23:03:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e96b9e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900b44b0 con 0x7f6e7c004500 2022-08-01T23:03:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e9639d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 0x7f6e900abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e9539b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e9014aef0 0x7f6e90143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e9639d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 0x7f6e900abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47190/0 (socket says 172.21.15.32:47190) 2022-08-01T23:03:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e95b9c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e7c004500 0x7f6e90156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.333+0000 7f6e9639d700 1 -- 172.21.15.32:0/2138807545 learned_addr learned my addr 172.21.15.32:0/2138807545 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9539b700 1 -- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e7c004500 msgr2=0x7f6e90156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9539b700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e7c004500 0x7f6e90156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9539b700 1 -- 172.21.15.32:0/2138807545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 msgr2=0x7f6e900abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9539b700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 0x7f6e900abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9539b700 1 -- 172.21.15.32:0/2138807545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900b5e90 con 0x7f6e9014aef0 2022-08-01T23:03:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9639d700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 0x7f6e900abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e9539b700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e9014aef0 0x7f6e90143e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f6e800017b0 tx=0x7f6e80013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e80005e90 con 0x7f6e9014aef0 2022-08-01T23:03:38.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80001dd0 con 0x7f6e9014aef0 2022-08-01T23:03:38.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9014b400 con 0x7f6e9014aef0 2022-08-01T23:03:38.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e800074d0 con 0x7f6e9014aef0 2022-08-01T23:03:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9014ac90 con 0x7f6e9014aef0 2022-08-01T23:03:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e900574e0 con 0x7f6e9014aef0 2022-08-01T23:03:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6e80027070 con 0x7f6e9014aef0 2022-08-01T23:03:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e86ffd700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6e70053b00 0x7f6e70055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6e8006c9d0 con 0x7f6e9014aef0 2022-08-01T23:03:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.337+0000 7f6e95b9c700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6e70053b00 0x7f6e70055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.341+0000 7f6e95b9c700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6e70053b00 0x7f6e70055fc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f6e8c003000 tx=0x7f6e8c007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.349+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6e800436f0 con 0x7f6e9014aef0 2022-08-01T23:03:38.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.477+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e900a9590 con 0x7f6e9014aef0 2022-08-01T23:03:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.477+0000 7f6e86ffd700 1 -- 172.21.15.32:0/2138807545 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f6e80046d10 con 0x7f6e9014aef0 2022-08-01T23:03:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:38.485 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":107,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:32.553487+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:30.001802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:30.985153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:38.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6e70053b00 msgr2=0x7f6e70055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6e70053b00 0x7f6e70055fc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f6e8c003000 tx=0x7f6e8c007a00).stop 2022-08-01T23:03:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e9014aef0 msgr2=0x7f6e90143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e9014aef0 0x7f6e90143e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f6e800017b0 tx=0x7f6e80013000).stop 2022-08-01T23:03:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 shutdown_connections 2022-08-01T23:03:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6e70053b00 0x7f6e70055fc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e900b5a10 0x7f6e900abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e9014aef0 0x7f6e90143e30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 --2- 172.21.15.32:0/2138807545 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6e7c004500 0x7f6e90156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 >> 172.21.15.32:0/2138807545 conn(0x7f6e90060310 msgr2=0x7f6e9005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 shutdown_connections 2022-08-01T23:03:38.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.481+0000 7f6e96b9e700 1 -- 172.21.15.32:0/2138807545 wait complete. 2022-08-01T23:03:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:03:30.001802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:03:30.985153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:03:38.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21f27c700 1 Processor -- start 2022-08-01T23:03:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21f27c700 1 -- start start 2022-08-01T23:03:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21f27c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2180655b0 0x7fb21814c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21f27c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb218060670 con 0x7fb218060290 2022-08-01T23:03:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21f27c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2180b2540 con 0x7fb2181445d0 2022-08-01T23:03:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21f27c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb218065e10 con 0x7fb2180655b0 2022-08-01T23:03:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d018700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb218060290 0x7fb218064ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34736/0 (socket says 172.21.15.32:34736) 2022-08-01T23:03:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d018700 1 -- 172.21.15.32:0/226166760 learned_addr learned my addr 172.21.15.32:0/226166760 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d819700 1 --2- 172.21.15.32:0/226166760 >> v2:172.21.15.55:3300/0 conn(0x7fb2180655b0 0x7fb21814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb20ffff700 1 -- 172.21.15.32:0/226166760 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009771976 0 0) 0x7fb218060670 con 0x7fb218060290 2022-08-01T23:03:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb20ffff700 1 -- 172.21.15.32:0/226166760 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc003260 con 0x7fb218060290 2022-08-01T23:03:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d819700 1 -- 172.21.15.32:0/226166760 >> v1:172.21.15.55:6789/0 conn(0x7fb2181445d0 legacy=0x7fb218144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d819700 1 -- 172.21.15.32:0/226166760 >> v1:172.21.15.32:6789/0 conn(0x7fb218060290 legacy=0x7fb218064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb20ffff700 1 -- 172.21.15.32:0/226166760 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186198171 0 0) 0x7fb1fc003260 con 0x7fb218060290 2022-08-01T23:03:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d819700 1 -- 172.21.15.32:0/226166760 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180b27e0 con 0x7fb2180655b0 2022-08-01T23:03:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb21d819700 1 --2- 172.21.15.32:0/226166760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2180655b0 0x7fb21814c980 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fb214001820 tx=0x7fb2100091d0).ready entity=mon.1 client_cookie=fe6a3c07677107c5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb20ffff700 1 -- 172.21.15.32:0/226166760 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb214001680 con 0x7fb2180655b0 2022-08-01T23:03:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb20ffff700 1 -- 172.21.15.32:0/226166760 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21000b580 con 0x7fb2180655b0 2022-08-01T23:03:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.601+0000 7fb20ffff700 1 -- 172.21.15.32:0/226166760 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb21000b6f0 con 0x7fb2180655b0 2022-08-01T23:03:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- 172.21.15.32:0/226166760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2180655b0 msgr2=0x7fb21814c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 --2- 172.21.15.32:0/226166760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2180655b0 0x7fb21814c980 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fb214001820 tx=0x7fb2100091d0).stop 2022-08-01T23:03:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- 172.21.15.32:0/226166760 shutdown_connections 2022-08-01T23:03:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 --2- 172.21.15.32:0/226166760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2180655b0 0x7fb21814c980 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- 172.21.15.32:0/226166760 >> 172.21.15.32:0/226166760 conn(0x7fb21805d0f0 msgr2=0x7fb21805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- 172.21.15.32:0/226166760 shutdown_connections 2022-08-01T23:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- 172.21.15.32:0/226166760 wait complete. 2022-08-01T23:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 Processor -- start 2022-08-01T23:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- start start 2022-08-01T23:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb218060290 0x7fb218142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 0x7fb2180b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb218141bc0 0x7fb2180b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181414b0 con 0x7fb2180655b0 2022-08-01T23:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb218067420 con 0x7fb218060290 2022-08-01T23:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21f27c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb218137070 con 0x7fb218141bc0 2022-08-01T23:03:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 0x7fb2180b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21d018700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb218060290 0x7fb218142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 0x7fb2180b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47202/0 (socket says 172.21.15.32:47202) 2022-08-01T23:03:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 -- 172.21.15.32:0/1678430584 learned_addr learned my addr 172.21.15.32:0/1678430584 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21d819700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb218141bc0 0x7fb2180b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 -- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb218141bc0 msgr2=0x7fb2180b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb218141bc0 0x7fb2180b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 -- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb218060290 msgr2=0x7fb218142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb218060290 0x7fb218142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 -- 172.21.15.32:0/1678430584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210009470 con 0x7fb2180655b0 2022-08-01T23:03:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.605+0000 7fb21c817700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 0x7fb2180b0de0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fb218067590 tx=0x7fb208007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb208012400 con 0x7fb2180655b0 2022-08-01T23:03:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208012560 con 0x7fb2180655b0 2022-08-01T23:03:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2181416e0 con 0x7fb2180655b0 2022-08-01T23:03:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2080128c0 con 0x7fb2180655b0 2022-08-01T23:03:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2180b12d0 con 0x7fb2180655b0 2022-08-01T23:03:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb218064a00 con 0x7fb2180655b0 2022-08-01T23:03:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb208010070 con 0x7fb2180655b0 2022-08-01T23:03:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb20e7fc700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb2000539e0 0x7fb200055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb20806af90 con 0x7fb2180655b0 2022-08-01T23:03:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.609+0000 7fb21d018700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb2000539e0 0x7fb200055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.613+0000 7fb21d018700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb2000539e0 0x7fb200055ea0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fb21400d000 tx=0x7fb214000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.617+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb208041cb0 con 0x7fb2180655b0 2022-08-01T23:03:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb2180574e0 con 0x7fb2180655b0 2022-08-01T23:03:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb20e7fc700 1 -- 172.21.15.32:0/1678430584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb2080452d0 con 0x7fb2180655b0 2022-08-01T23:03:38.746 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb2000539e0 msgr2=0x7fb200055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb2000539e0 0x7fb200055ea0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fb21400d000 tx=0x7fb214000bc0).stop 2022-08-01T23:03:38.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 msgr2=0x7fb2180b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 0x7fb2180b0de0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fb218067590 tx=0x7fb208007c50).stop 2022-08-01T23:03:38.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 shutdown_connections 2022-08-01T23:03:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb2000539e0 0x7fb200055ea0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb218141bc0 0x7fb2180b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2180655b0 0x7fb2180b0de0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 --2- 172.21.15.32:0/1678430584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb218060290 0x7fb218142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 >> 172.21.15.32:0/1678430584 conn(0x7fb21805d0f0 msgr2=0x7fb21813acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:38.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 shutdown_connections 2022-08-01T23:03:38.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.741+0000 7fb21f27c700 1 -- 172.21.15.32:0/1678430584 wait complete. 2022-08-01T23:03:38.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 Processor -- start 2022-08-01T23:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 -- start start 2022-08-01T23:03:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3580a9af0 0x7fe3580b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe3580ac3f0 0x7fe3580ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3580b1df0 con 0x7fe3580b0850 2022-08-01T23:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580a9ed0 con 0x7fe3580a9af0 2022-08-01T23:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35ca6b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580b2ad0 con 0x7fe3580ac3f0 2022-08-01T23:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe35659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3580a9af0 0x7fe3580b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe356d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3580b0850 0x7fe3580aedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34754/0 (socket says 172.21.15.32:34754) 2022-08-01T23:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe356d9d700 1 -- 172.21.15.32:0/2294507969 learned_addr learned my addr 172.21.15.32:0/2294507969 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.853+0000 7fe355d9b700 1 --2- 172.21.15.32:0/2294507969 >> v2:172.21.15.55:3300/0 conn(0x7fe3580ac3f0 0x7fe3580ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe355d9b700 1 -- 172.21.15.32:0/2294507969 >> v2:172.21.15.32:3300/0 conn(0x7fe3580a9af0 msgr2=0x7fe3580b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe355d9b700 1 --2- 172.21.15.32:0/2294507969 >> v2:172.21.15.32:3300/0 conn(0x7fe3580a9af0 0x7fe3580b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe355d9b700 1 -- 172.21.15.32:0/2294507969 >> v1:172.21.15.32:6789/0 conn(0x7fe3580b0850 legacy=0x7fe3580aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe355d9b700 1 -- 172.21.15.32:0/2294507969 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580b0250 con 0x7fe3580ac3f0 2022-08-01T23:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe355d9b700 1 --2- 172.21.15.32:0/2294507969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe3580ab130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fe3480032f0 tx=0x7fe3480091d0).ready entity=mon.1 client_cookie=2faeeeaf4cb58171 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35559a700 1 -- 172.21.15.32:0/2294507969 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34800b680 con 0x7fe3580ac3f0 2022-08-01T23:03:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35559a700 1 -- 172.21.15.32:0/2294507969 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34800b7e0 con 0x7fe3580ac3f0 2022-08-01T23:03:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35559a700 1 -- 172.21.15.32:0/2294507969 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34800b950 con 0x7fe3580ac3f0 2022-08-01T23:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 -- 172.21.15.32:0/2294507969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 msgr2=0x7fe3580ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/2294507969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe3580ab130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fe3480032f0 tx=0x7fe3480091d0).stop 2022-08-01T23:03:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 -- 172.21.15.32:0/2294507969 shutdown_connections 2022-08-01T23:03:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/2294507969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe3580ab130 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/2294507969 >> v2:172.21.15.32:3300/0 conn(0x7fe3580a9af0 0x7fe3580b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 -- 172.21.15.32:0/2294507969 >> 172.21.15.32:0/2294507969 conn(0x7fe35805f840 msgr2=0x7fe35805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 -- 172.21.15.32:0/2294507969 shutdown_connections 2022-08-01T23:03:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.857+0000 7fe35ca6b700 1 -- 172.21.15.32:0/2294507969 wait complete. 2022-08-01T23:03:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 Processor -- start 2022-08-01T23:03:38.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 -- start start 2022-08-01T23:03:38.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3580a9af0 0x7fe3581408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe358151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 0x7fe35814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581379d0 con 0x7fe3580b0850 2022-08-01T23:03:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe358137c70 con 0x7fe3580ac3f0 2022-08-01T23:03:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35ca6b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe358137f10 con 0x7fe3580a9af0 2022-08-01T23:03:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 0x7fe35814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 0x7fe35814de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47218/0 (socket says 172.21.15.32:47218) 2022-08-01T23:03:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 -- 172.21.15.32:0/3568440046 learned_addr learned my addr 172.21.15.32:0/3568440046 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe35659c700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3580a9af0 0x7fe3581408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe355d9b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe358151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 -- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3580a9af0 msgr2=0x7fe3581408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3580a9af0 0x7fe3581408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 -- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 msgr2=0x7fe358151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe358151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 -- 172.21.15.32:0/3568440046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348009470 con 0x7fe3580b0850 2022-08-01T23:03:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.861+0000 7fe356d9d700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 0x7fe35814de30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7fe3580ac9c0 tx=0x7fe350007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe350001d70 con 0x7fe3580b0850 2022-08-01T23:03:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3500070d0 con 0x7fe3580b0850 2022-08-01T23:03:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3581381b0 con 0x7fe3580b0850 2022-08-01T23:03:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe350007430 con 0x7fe3580b0850 2022-08-01T23:03:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe358064e00 con 0x7fe3580b0850 2022-08-01T23:03:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3580574e0 con 0x7fe3580b0850 2022-08-01T23:03:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe350010070 con 0x7fe3580b0850 2022-08-01T23:03:38.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe34f7fe700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe33c053aa0 0x7fe33c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:38.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.865+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe35006b460 con 0x7fe3580b0850 2022-08-01T23:03:38.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.869+0000 7fe35659c700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe33c053aa0 0x7fe33c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:38.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.869+0000 7fe35659c700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe33c053aa0 0x7fe33c055f60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fe34000b000 tx=0x7fe340006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.873+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe350042220 con 0x7fe3580b0850 2022-08-01T23:03:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.997+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3580b1df0 con 0x7fe3580b0850 2022-08-01T23:03:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:38.997+0000 7fe34f7fe700 1 -- 172.21.15.32:0/3568440046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3580b1df0 con 0x7fe3580b0850 2022-08-01T23:03:39.004 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe33c053aa0 msgr2=0x7fe33c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe33c053aa0 0x7fe33c055f60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fe34000b000 tx=0x7fe340006df0).stop 2022-08-01T23:03:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 msgr2=0x7fe35814de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 0x7fe35814de30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7fe3580ac9c0 tx=0x7fe350007c00).stop 2022-08-01T23:03:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 shutdown_connections 2022-08-01T23:03:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe33c053aa0 0x7fe33c055f60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3580b0850 0x7fe35814de30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3580ac3f0 0x7fe358151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 --2- 172.21.15.32:0/3568440046 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3580a9af0 0x7fe3581408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 >> 172.21.15.32:0/3568440046 conn(0x7fe35805f840 msgr2=0x7fe35813da80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 shutdown_connections 2022-08-01T23:03:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.001+0000 7fe35ca6b700 1 -- 172.21.15.32:0/3568440046 wait complete. 2022-08-01T23:03:39.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 Processor -- start 2022-08-01T23:03:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 -- start start 2022-08-01T23:03:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f39dc0b6420 0x7f39dc061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc142ff0 0x7f39dc1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39dc0b60e0 con 0x7f39dc0b41f0 2022-08-01T23:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39dc0b2100 con 0x7f39dc0b6420 2022-08-01T23:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e31e0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc0b23a0 con 0x7f39dc142ff0 2022-08-01T23:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e19dd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39dc0b41f0 0x7f39dc061af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34766/0 (socket says 172.21.15.32:34766) 2022-08-01T23:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e19dd700 1 -- 172.21.15.32:0/4182603507 learned_addr learned my addr 172.21.15.32:0/4182603507 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e21de700 1 --2- 172.21.15.32:0/4182603507 >> v2:172.21.15.55:3300/0 conn(0x7f39dc0b6420 0x7f39dc061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e29df700 1 --2- 172.21.15.32:0/4182603507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc142ff0 0x7f39dc1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e11dc700 1 -- 172.21.15.32:0/4182603507 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849868599 0 0) 0x7f39d0001110 con 0x7f39dc0b41f0 2022-08-01T23:03:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.121+0000 7f39e11dc700 1 -- 172.21.15.32:0/4182603507 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c4003260 con 0x7f39dc0b41f0 2022-08-01T23:03:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e29df700 1 -- 172.21.15.32:0/4182603507 >> v2:172.21.15.55:3300/0 conn(0x7f39dc0b6420 msgr2=0x7f39dc061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e29df700 1 --2- 172.21.15.32:0/4182603507 >> v2:172.21.15.55:3300/0 conn(0x7f39dc0b6420 0x7f39dc061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e29df700 1 -- 172.21.15.32:0/4182603507 >> v1:172.21.15.32:6789/0 conn(0x7f39dc0b41f0 legacy=0x7f39dc061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e11dc700 1 -- 172.21.15.32:0/4182603507 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944967232 0 0) 0x7f39c4003260 con 0x7f39dc0b41f0 2022-08-01T23:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e29df700 1 -- 172.21.15.32:0/4182603507 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc0b2640 con 0x7f39dc142ff0 2022-08-01T23:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e29df700 1 --2- 172.21.15.32:0/4182603507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc142ff0 0x7f39dc1434a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f39d0001990 tx=0x7f39d40091d0).ready entity=mon.2 client_cookie=3369331d3191556a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e11dc700 1 -- 172.21.15.32:0/4182603507 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d00017f0 con 0x7f39dc142ff0 2022-08-01T23:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e11dc700 1 -- 172.21.15.32:0/4182603507 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d400f510 con 0x7f39dc142ff0 2022-08-01T23:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e11dc700 1 -- 172.21.15.32:0/4182603507 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d400f680 con 0x7f39dc142ff0 2022-08-01T23:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 -- 172.21.15.32:0/4182603507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc142ff0 msgr2=0x7f39dc1434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 --2- 172.21.15.32:0/4182603507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc142ff0 0x7f39dc1434a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f39d0001990 tx=0x7f39d40091d0).stop 2022-08-01T23:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 -- 172.21.15.32:0/4182603507 shutdown_connections 2022-08-01T23:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 --2- 172.21.15.32:0/4182603507 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc142ff0 0x7f39dc1434a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 --2- 172.21.15.32:0/4182603507 >> v2:172.21.15.55:3300/0 conn(0x7f39dc0b6420 0x7f39dc061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 -- 172.21.15.32:0/4182603507 >> 172.21.15.32:0/4182603507 conn(0x7f39dc05c620 msgr2=0x7f39dc05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 -- 172.21.15.32:0/4182603507 shutdown_connections 2022-08-01T23:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.125+0000 7f39e31e0700 1 -- 172.21.15.32:0/4182603507 wait complete. 2022-08-01T23:03:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 Processor -- start 2022-08-01T23:03:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 -- start start 2022-08-01T23:03:39.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc0b41f0 0x7f39dc0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39dc0b6420 0x7f39dc0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 0x7f39dc13fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc0af830 con 0x7f39dc142ff0 2022-08-01T23:03:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc0afad0 con 0x7f39dc0b6420 2022-08-01T23:03:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e31e0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc13e1b0 con 0x7f39dc0b41f0 2022-08-01T23:03:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e29df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 0x7f39dc13fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e19dd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39dc0b6420 0x7f39dc0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e21de700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc0b41f0 0x7f39dc0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e29df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 0x7f39dc13fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47232/0 (socket says 172.21.15.32:47232) 2022-08-01T23:03:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e29df700 1 -- 172.21.15.32:0/2692933659 learned_addr learned my addr 172.21.15.32:0/2692933659 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e29df700 1 -- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc0b41f0 msgr2=0x7f39dc0b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.129+0000 7f39e29df700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc0b41f0 0x7f39dc0b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e29df700 1 -- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39dc0b6420 msgr2=0x7f39dc0b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e29df700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39dc0b6420 0x7f39dc0b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e29df700 1 -- 172.21.15.32:0/2692933659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4009470 con 0x7f39dc142ff0 2022-08-01T23:03:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e29df700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 0x7f39dc13fb10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f39d0001990 tx=0x7f39d4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d40032d0 con 0x7f39dc142ff0 2022-08-01T23:03:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d4011c30 con 0x7f39dc142ff0 2022-08-01T23:03:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc13e450 con 0x7f39dc142ff0 2022-08-01T23:03:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d40203f0 con 0x7f39dc142ff0 2022-08-01T23:03:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc0af470 con 0x7f39dc142ff0 2022-08-01T23:03:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc13e610 con 0x7f39dc142ff0 2022-08-01T23:03:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.133+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f39d401e070 con 0x7f39dc142ff0 2022-08-01T23:03:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.137+0000 7f39cf7fe700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39bc0539e0 0x7f39bc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.137+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f39d406cc00 con 0x7f39dc142ff0 2022-08-01T23:03:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.137+0000 7f39e21de700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39bc0539e0 0x7f39bc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.137+0000 7f39e21de700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39bc0539e0 0x7f39bc055ea0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f39d800b000 tx=0x7f39d8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.141+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39d4072020 con 0x7f39dc142ff0 2022-08-01T23:03:39.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.265+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39dc0b28e0 con 0x7f39dc142ff0 2022-08-01T23:03:39.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.265+0000 7f39cf7fe700 1 -- 172.21.15.32:0/2692933659 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7f39d4015020 con 0x7f39dc142ff0 2022-08-01T23:03:39.272 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39bc0539e0 msgr2=0x7f39bc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39bc0539e0 0x7f39bc055ea0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f39d800b000 tx=0x7f39d8006df0).stop 2022-08-01T23:03:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 msgr2=0x7f39dc13fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 0x7f39dc13fb10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f39d0001990 tx=0x7f39d4016000).stop 2022-08-01T23:03:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 shutdown_connections 2022-08-01T23:03:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39bc0539e0 0x7f39bc055ea0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc142ff0 0x7f39dc13fb10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39dc0b6420 0x7f39dc0b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 --2- 172.21.15.32:0/2692933659 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39dc0b41f0 0x7f39dc0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 >> 172.21.15.32:0/2692933659 conn(0x7f39dc05c620 msgr2=0x7f39dc0b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 shutdown_connections 2022-08-01T23:03:39.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.269+0000 7f39e31e0700 1 -- 172.21.15.32:0/2692933659 wait complete. 2022-08-01T23:03:39.285 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:03:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 Processor -- start 2022-08-01T23:03:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 -- start start 2022-08-01T23:03:39.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c01432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12c0061360 0x7f12c0143790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c0064aa0 con 0x7f12c00600f0 2022-08-01T23:03:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c0064d40 con 0x7f12c0061360 2022-08-01T23:03:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.385+0000 7f12c58a9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c0064fe0 con 0x7f12c00b28a0 2022-08-01T23:03:39.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12be7fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c01432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12beffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f12c00600f0 0x7f12c01405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:55684/0 (socket says 172.21.15.32:55684) 2022-08-01T23:03:39.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12beffd700 1 -- 172.21.15.32:0/1967190949 learned_addr learned my addr 172.21.15.32:0/1967190949 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:39.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12bf7fe700 1 --2- 172.21.15.32:0/1967190949 >> v2:172.21.15.55:3300/0 conn(0x7f12c0061360 0x7f12c0143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12be7fc700 1 -- 172.21.15.32:0/1967190949 >> v2:172.21.15.55:3300/0 conn(0x7f12c0061360 msgr2=0x7f12c0143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12be7fc700 1 --2- 172.21.15.32:0/1967190949 >> v2:172.21.15.55:3300/0 conn(0x7f12c0061360 0x7f12c0143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12be7fc700 1 -- 172.21.15.32:0/1967190949 >> v1:172.21.15.55:6789/0 conn(0x7f12c00600f0 legacy=0x7f12c01405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12be7fc700 1 -- 172.21.15.32:0/1967190949 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0065280 con 0x7f12c00b28a0 2022-08-01T23:03:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12be7fc700 1 --2- 172.21.15.32:0/1967190949 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c01432a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f12b0002f70 tx=0x7f12b0006df0).ready entity=mon.2 client_cookie=fe34d44d83ecddca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12bdffb700 1 -- 172.21.15.32:0/1967190949 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b000d570 con 0x7f12c00b28a0 2022-08-01T23:03:39.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12bdffb700 1 -- 172.21.15.32:0/1967190949 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0000e90 con 0x7f12c00b28a0 2022-08-01T23:03:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12bdffb700 1 -- 172.21.15.32:0/1967190949 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b000d8c0 con 0x7f12c00b28a0 2022-08-01T23:03:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 -- 172.21.15.32:0/1967190949 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 msgr2=0x7f12c01432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 --2- 172.21.15.32:0/1967190949 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c01432a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f12b0002f70 tx=0x7f12b0006df0).stop 2022-08-01T23:03:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 -- 172.21.15.32:0/1967190949 shutdown_connections 2022-08-01T23:03:39.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 --2- 172.21.15.32:0/1967190949 >> v2:172.21.15.55:3300/0 conn(0x7f12c0061360 0x7f12c0143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 --2- 172.21.15.32:0/1967190949 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c01432a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 -- 172.21.15.32:0/1967190949 >> 172.21.15.32:0/1967190949 conn(0x7f12c005d0f0 msgr2=0x7f12c005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 -- 172.21.15.32:0/1967190949 shutdown_connections 2022-08-01T23:03:39.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.389+0000 7f12c58a9700 1 -- 172.21.15.32:0/1967190949 wait complete. 2022-08-01T23:03:39.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 Processor -- start 2022-08-01T23:03:39.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- start start 2022-08-01T23:03:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c00600f0 0x7f12c01413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 0x7f12c0152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c0153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c0143730 con 0x7f12c0061360 2022-08-01T23:03:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b1400 con 0x7f12c00600f0 2022-08-01T23:03:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c0143ac0 con 0x7f12c00b28a0 2022-08-01T23:03:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 0x7f12c0152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 0x7f12c0152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47244/0 (socket says 172.21.15.32:47244) 2022-08-01T23:03:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12bf7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c0153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 -- 172.21.15.32:0/702415129 learned_addr learned my addr 172.21.15.32:0/702415129 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12beffd700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c00600f0 0x7f12c01413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 -- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 msgr2=0x7f12c0153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c0153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 -- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c00600f0 msgr2=0x7f12c01413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c00600f0 0x7f12c01413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 -- 172.21.15.32:0/702415129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b000b180 con 0x7f12c0061360 2022-08-01T23:03:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12be7fc700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 0x7f12c0152c80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f12b0002f70 tx=0x7f12b0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b000dea0 con 0x7f12c0061360 2022-08-01T23:03:39.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c0143d60 con 0x7f12c0061360 2022-08-01T23:03:39.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0013de0 con 0x7f12c0061360 2022-08-01T23:03:39.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b001e5f0 con 0x7f12c0061360 2022-08-01T23:03:39.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c00b0a50 con 0x7f12c0061360 2022-08-01T23:03:39.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.393+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c00afb30 con 0x7f12c0061360 2022-08-01T23:03:39.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.397+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12b001c070 con 0x7f12c0061360 2022-08-01T23:03:39.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.397+0000 7f12b7fff700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12a4053b00 0x7f12a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.397+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f12b006cf70 con 0x7f12c0061360 2022-08-01T23:03:39.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.397+0000 7f12beffd700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12a4053b00 0x7f12a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.401+0000 7f12beffd700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12a4053b00 0x7f12a4055fc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f12a800d000 tx=0x7f12a8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.405+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12b0043c20 con 0x7f12c0061360 2022-08-01T23:03:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12c0143920 con 0x7f12c0061360 2022-08-01T23:03:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12b7fff700 1 -- 172.21.15.32:0/702415129 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7f12c0143920 con 0x7f12c0061360 2022-08-01T23:03:39.534 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12a4053b00 msgr2=0x7f12a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12a4053b00 0x7f12a4055fc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f12a800d000 tx=0x7f12a8009040).stop 2022-08-01T23:03:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 msgr2=0x7f12c0152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 0x7f12c0152c80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f12b0002f70 tx=0x7f12b0014000).stop 2022-08-01T23:03:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 shutdown_connections 2022-08-01T23:03:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12a4053b00 0x7f12a4055fc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12c00b28a0 0x7f12c0153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12c0061360 0x7f12c0152c80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 --2- 172.21.15.32:0/702415129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c00600f0 0x7f12c01413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 >> 172.21.15.32:0/702415129 conn(0x7f12c005d0f0 msgr2=0x7f12c0066f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 shutdown_connections 2022-08-01T23:03:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.529+0000 7f12c58a9700 1 -- 172.21.15.32:0/702415129 wait complete. 2022-08-01T23:03:39.547 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:03:39.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 Processor -- start 2022-08-01T23:03:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- start start 2022-08-01T23:03:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f53280b01a0 0x7f53280a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53280afd60 0x7f53280a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53280a7250 con 0x7f53280b2710 2022-08-01T23:03:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280a7480 con 0x7f53280afd60 2022-08-01T23:03:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280a7720 con 0x7f53280b01a0 2022-08-01T23:03:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532d064700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53280b2710 0x7f532814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34792/0 (socket says 172.21.15.32:34792) 2022-08-01T23:03:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532d064700 1 -- 172.21.15.32:0/535365416 learned_addr learned my addr 172.21.15.32:0/535365416 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532d865700 1 --2- 172.21.15.32:0/535365416 >> v2:172.21.15.32:3300/0 conn(0x7f53280afd60 0x7f53280a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532c863700 1 --2- 172.21.15.32:0/535365416 >> v2:172.21.15.55:3300/0 conn(0x7f53280b01a0 0x7f53280a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f5327fff700 1 -- 172.21.15.32:0/535365416 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054269562 0 0) 0x7f53280a7250 con 0x7f53280b2710 2022-08-01T23:03:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f5327fff700 1 -- 172.21.15.32:0/535365416 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5314003260 con 0x7f53280b2710 2022-08-01T23:03:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532c863700 1 -- 172.21.15.32:0/535365416 >> v2:172.21.15.32:3300/0 conn(0x7f53280afd60 msgr2=0x7f53280a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532c863700 1 --2- 172.21.15.32:0/535365416 >> v2:172.21.15.32:3300/0 conn(0x7f53280afd60 0x7f53280a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532c863700 1 -- 172.21.15.32:0/535365416 >> v1:172.21.15.32:6789/0 conn(0x7f53280b2710 legacy=0x7f532814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532c863700 1 -- 172.21.15.32:0/535365416 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280a79c0 con 0x7f53280b01a0 2022-08-01T23:03:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532c863700 1 --2- 172.21.15.32:0/535365416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280b01a0 0x7f53280a8540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f531c002f70 tx=0x7f531c006df0).ready entity=mon.1 client_cookie=9575a04de50aa566 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f5327fff700 1 -- 172.21.15.32:0/535365416 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f531c00d4f0 con 0x7f53280b01a0 2022-08-01T23:03:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f5327fff700 1 -- 172.21.15.32:0/535365416 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c000e90 con 0x7f53280b01a0 2022-08-01T23:03:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f5327fff700 1 -- 172.21.15.32:0/535365416 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f531c00d840 con 0x7f53280b01a0 2022-08-01T23:03:39.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- 172.21.15.32:0/535365416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280b01a0 msgr2=0x7f53280a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 --2- 172.21.15.32:0/535365416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280b01a0 0x7f53280a8540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f531c002f70 tx=0x7f531c006df0).stop 2022-08-01T23:03:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- 172.21.15.32:0/535365416 shutdown_connections 2022-08-01T23:03:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 --2- 172.21.15.32:0/535365416 >> v2:172.21.15.32:3300/0 conn(0x7f53280afd60 0x7f53280a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 --2- 172.21.15.32:0/535365416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280b01a0 0x7f53280a8540 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- 172.21.15.32:0/535365416 >> 172.21.15.32:0/535365416 conn(0x7f5328060310 msgr2=0x7f53280b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- 172.21.15.32:0/535365416 shutdown_connections 2022-08-01T23:03:39.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.649+0000 7f532e066700 1 -- 172.21.15.32:0/535365416 wait complete. 2022-08-01T23:03:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 Processor -- start 2022-08-01T23:03:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- start start 2022-08-01T23:03:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280afd60 0x7f53280b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53280b01a0 0x7f53281411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53280b2710 0x7f53281416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280aa680 con 0x7f53280b01a0 2022-08-01T23:03:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280b3010 con 0x7f53280afd60 2022-08-01T23:03:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532814da40 con 0x7f53280b2710 2022-08-01T23:03:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532c863700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53280b01a0 0x7f53281411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532c863700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53280b01a0 0x7f53281411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47258/0 (socket says 172.21.15.32:47258) 2022-08-01T23:03:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532c863700 1 -- 172.21.15.32:0/2298416291 learned_addr learned my addr 172.21.15.32:0/2298416291 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280afd60 0x7f53280b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d865700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53280b2710 0x7f53281416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 -- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53280b2710 msgr2=0x7f53281416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53280b2710 0x7f53281416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 -- 172.21.15.32:0/2298416291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53280b01a0 msgr2=0x7f53281411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53280b01a0 0x7f53281411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 -- 172.21.15.32:0/2298416291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c00b180 con 0x7f53280afd60 2022-08-01T23:03:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532d064700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280afd60 0x7f53280b4d10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f5318001490 tx=0x7f5318007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5318005c70 con 0x7f53280afd60 2022-08-01T23:03:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53280b3e00 con 0x7f53280afd60 2022-08-01T23:03:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318005dd0 con 0x7f53280afd60 2022-08-01T23:03:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53180072c0 con 0x7f53280afd60 2022-08-01T23:03:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53280b5790 con 0x7f53280afd60 2022-08-01T23:03:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.653+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53280b3150 con 0x7f53280afd60 2022-08-01T23:03:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.657+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5318010070 con 0x7f53280afd60 2022-08-01T23:03:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.657+0000 7f53267fc700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5308053b30 0x7f5308055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.657+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f531806c5d0 con 0x7f53280afd60 2022-08-01T23:03:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.657+0000 7f532c863700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5308053b30 0x7f5308055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.657+0000 7f532c863700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5308053b30 0x7f5308055ff0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f531c01a000 tx=0x7f531c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:39.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.665+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5318043330 con 0x7f53280afd60 2022-08-01T23:03:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.829+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53280574e0 con 0x7f53280afd60 2022-08-01T23:03:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.829+0000 7f53267fc700 1 -- 172.21.15.32:0/2298416291 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f5318018070 con 0x7f53280afd60 2022-08-01T23:03:39.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:39.836 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:31.549702+0000","modified":"2022-08-01T23:03:32.552769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/550523700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":550523700},{"type":"v1","addr":"172.21.15.32:6835","nonce":550523700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T23:03:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5308053b30 msgr2=0x7f5308055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5308053b30 0x7f5308055ff0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f531c01a000 tx=0x7f531c014000).stop 2022-08-01T23:03:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280afd60 msgr2=0x7f53280b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280afd60 0x7f53280b4d10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f5318001490 tx=0x7f5318007da0).stop 2022-08-01T23:03:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 shutdown_connections 2022-08-01T23:03:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5308053b30 0x7f5308055ff0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f53280b2710 0x7f53281416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53280b01a0 0x7f53281411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 --2- 172.21.15.32:0/2298416291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53280afd60 0x7f53280b4d10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 >> 172.21.15.32:0/2298416291 conn(0x7f5328060310 msgr2=0x7f532805cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 shutdown_connections 2022-08-01T23:03:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:39.833+0000 7f532e066700 1 -- 172.21.15.32:0/2298416291 wait complete. 2022-08-01T23:03:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2022-08-01T23:03:40.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:31.549702+0000', 'modified': '2022-08-01T23:03:32.552769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7332': {'gid': 7332, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/550523700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 550523700}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 550523700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:40.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:40.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:03:40.273 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":7332,"want_state":"up:active","state":"up:active","rank_uptime":8.708606928,"mdsmap_epoch":50,"osdmap_epoch":107,"osdmap_epoch_barrier":106,"uptime":13.025947488} 2022-08-01T23:03:40.274 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 7332, "mdsmap_epoch": 50, "osdmap_epoch": 107, "osdmap_epoch_barrier": 106, "rank_uptime": 8.708606928, "state": "up:active", "uptime": 13.025947488, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:40.274 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:03:40.275 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:03:40.275 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:03:40.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:40.276 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:40.283 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:03:40.300 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:40.300 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:40.311 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:40.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:40.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:40.314 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:03:40.319 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:03:40.319 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:40.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:03:40.378 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:40.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:40.389 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:40.390 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:40.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:40.391 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:03:40.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:03:40.433+0000 7fd5175e5080 -1 init, newargv = 0x55d399664800 newargc=9ceph-fuse[22842]: starting ceph client 2022-08-01T23:03:40.438 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr: 2022-08-01T23:03:40.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22842]: starting fuse 2022-08-01T23:03:41.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:03:41.516 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:03:41.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:03:41.527 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:03:41.528 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:03:41.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:41.530 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:03:41.579 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:03:41.579 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:03:41.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:41.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:03:41.581 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:03:41.581 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:03:41.582 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:03:41.582 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:03:41.583 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:41.583 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:03:41.584 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:03:41.584 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:03:41.584 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:41.585 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:03:41.585 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:03:41.586 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:03:41.586 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:03:41.586 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:41.587 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:03:41.587 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:03:41.588 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:03:41.588 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:03:41.588 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:03:41.589 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:03:41.589 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:03:41.590 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:03:41.635 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.22842.asok 2022-08-01T23:03:41.640 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22842.asok 2022-08-01T23:03:41.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:41.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22842.asok status 2022-08-01T23:03:41.727 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:03:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:03:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:03:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:03:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:03:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:03:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:03:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "22842", 2022-08-01T23:03:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:03:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:03:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:03:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7536, 2022-08-01T23:03:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:03:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:03:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:03:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7536 2022-08-01T23:03:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:03:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:03:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:03:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3820190801 2022-08-01T23:03:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:03:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:03:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:03:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:03:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3820190801 2022-08-01T23:03:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:03:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7536 172.21.15.157:0/3820190801", 2022-08-01T23:03:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/3820190801", 2022-08-01T23:03:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:03:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 50, 2022-08-01T23:03:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 107, 2022-08-01T23:03:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:03:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:03:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:03:41.737 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:03:41.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:41.738 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:41.739 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:41.746 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:03:41.747 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:41.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:41.748 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:41.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-08-01T23:03:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.869+0000 7fbfcf566700 1 Processor -- start 2022-08-01T23:03:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.869+0000 7fbfcf566700 1 -- start start 2022-08-01T23:03:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80b6a90 0x7fbfc80acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfc80aa760 0x7fbfc813c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc80b6e70 con 0x7fbfc8060aa0 2022-08-01T23:03:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc8061370 con 0x7fbfc80aa760 2022-08-01T23:03:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8061610 con 0x7fbfc80b6a90 2022-08-01T23:03:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfc80aa760 0x7fbfc813c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfc80aa760 0x7fbfc813c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47268/0 (socket says 172.21.15.32:47268) 2022-08-01T23:03:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 -- 172.21.15.32:0/513029433 learned_addr learned my addr 172.21.15.32:0/513029433 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcd302700 1 --2- 172.21.15.32:0/513029433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80b6a90 0x7fbfc80acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 -- 172.21.15.32:0/513029433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80b6a90 msgr2=0x7fbfc80acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 --2- 172.21.15.32:0/513029433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80b6a90 0x7fbfc80acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 -- 172.21.15.32:0/513029433 >> v1:172.21.15.55:6789/0 conn(0x7fbfc8060aa0 legacy=0x7fbfc80aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 -- 172.21.15.32:0/513029433 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80618b0 con 0x7fbfc80aa760 2022-08-01T23:03:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcdb03700 1 --2- 172.21.15.32:0/513029433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80aa760 0x7fbfc813c600 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fbfc00032f0 tx=0x7fbfc000a430).ready entity=mon.0 client_cookie=49c5eb4f9aa105c0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfb7fff700 1 -- 172.21.15.32:0/513029433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc000cc70 con 0x7fbfc80aa760 2022-08-01T23:03:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfb7fff700 1 -- 172.21.15.32:0/513029433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000cdd0 con 0x7fbfc80aa760 2022-08-01T23:03:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfb7fff700 1 -- 172.21.15.32:0/513029433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc000f660 con 0x7fbfc80aa760 2022-08-01T23:03:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- 172.21.15.32:0/513029433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80aa760 msgr2=0x7fbfc813c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 --2- 172.21.15.32:0/513029433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80aa760 0x7fbfc813c600 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fbfc00032f0 tx=0x7fbfc000a430).stop 2022-08-01T23:03:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- 172.21.15.32:0/513029433 shutdown_connections 2022-08-01T23:03:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 --2- 172.21.15.32:0/513029433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80aa760 0x7fbfc813c600 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 --2- 172.21.15.32:0/513029433 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80b6a90 0x7fbfc80acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- 172.21.15.32:0/513029433 >> 172.21.15.32:0/513029433 conn(0x7fbfc80b3610 msgr2=0x7fbfc80b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- 172.21.15.32:0/513029433 shutdown_connections 2022-08-01T23:03:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.873+0000 7fbfcf566700 1 -- 172.21.15.32:0/513029433 wait complete. 2022-08-01T23:03:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 Processor -- start 2022-08-01T23:03:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 -- start start 2022-08-01T23:03:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc8060aa0 0x7fbfc80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80aa760 0x7fbfc80ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 0x7fbfc8142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80af500 con 0x7fbfc80b6a90 2022-08-01T23:03:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80af7a0 con 0x7fbfc8060aa0 2022-08-01T23:03:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80afa40 con 0x7fbfc80aa760 2022-08-01T23:03:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 0x7fbfc8142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcd302700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc8060aa0 0x7fbfc80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 0x7fbfc8142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47274/0 (socket says 172.21.15.32:47274) 2022-08-01T23:03:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 -- 172.21.15.32:0/3138433516 learned_addr learned my addr 172.21.15.32:0/3138433516 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfccb01700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80aa760 0x7fbfc80ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 -- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80aa760 msgr2=0x7fbfc80ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80aa760 0x7fbfc80ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 -- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc8060aa0 msgr2=0x7fbfc80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc8060aa0 0x7fbfc80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 -- 172.21.15.32:0/3138433516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc000a6d0 con 0x7fbfc80b6a90 2022-08-01T23:03:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcd302700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc8060aa0 0x7fbfc80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcdb03700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 0x7fbfc8142050 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fbfc00032f0 tx=0x7fbfc0018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc0017800 con 0x7fbfc80b6a90 2022-08-01T23:03:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc0017960 con 0x7fbfc80b6a90 2022-08-01T23:03:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc80adde0 con 0x7fbfc80b6a90 2022-08-01T23:03:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc0017cc0 con 0x7fbfc80b6a90 2022-08-01T23:03:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.877+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80afee0 con 0x7fbfc80b6a90 2022-08-01T23:03:41.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.881+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbfc000e030 con 0x7fbfc80b6a90 2022-08-01T23:03:41.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.881+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc80afc00 con 0x7fbfc80b6a90 2022-08-01T23:03:41.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.881+0000 7fbfb67fc700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfa8053a70 0x7fbfa8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:41.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.881+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbfc006dd00 con 0x7fbfc80b6a90 2022-08-01T23:03:41.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.881+0000 7fbfcd302700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfa8053a70 0x7fbfa8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:41.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.881+0000 7fbfcd302700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfa8053a70 0x7fbfa8055f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fbfc400b000 tx=0x7fbfc4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:41.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:41.889+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfc00449b0 con 0x7fbfc80b6a90 2022-08-01T23:03:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.057+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fbfc8061b50 con 0x7fbfc80b6a90 2022-08-01T23:03:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.057+0000 7fbfb67fc700 1 -- 172.21.15.32:0/3138433516 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7fbfc0047fd0 con 0x7fbfc80b6a90 2022-08-01T23:03:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:42.063 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:42.063 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:03:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2022-08-01T23:03:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764493332480, 2022-08-01T23:03:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 10846208, 2022-08-01T23:03:42.064 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8600780800, 2022-08-01T23:03:42.065 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011125140823423862, 2022-08-01T23:03:42.065 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2022-08-01T23:03:42.065 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2022-08-01T23:03:42.065 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2022-08-01T23:03:42.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.066 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2022-08-01T23:03:42.066 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2022-08-01T23:03:42.066 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2022-08-01T23:03:42.066 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764493332480, 2022-08-01T23:03:42.066 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 10846208, 2022-08-01T23:03:42.067 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8600780800, 2022-08-01T23:03:42.067 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011125140823423862 2022-08-01T23:03:42.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:03:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2022-08-01T23:03:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2022-08-01T23:03:42.068 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:03:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:03:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2022-08-01T23:03:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:03:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:03:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:03:42.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919264256 2022-08-01T23:03:42.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.071 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2022-08-01T23:03:42.071 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2022-08-01T23:03:42.071 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:03:42.071 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:03:42.071 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:03:42.072 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:03:42.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:03:42.072 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:03:42.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919264256 2022-08-01T23:03:42.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2022-08-01T23:03:42.073 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 19, 2022-08-01T23:03:42.073 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:03:42.074 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 2742, 2022-08-01T23:03:42.074 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 22, 2022-08-01T23:03:42.074 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 64, 2022-08-01T23:03:42.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 65536, 2022-08-01T23:03:42.075 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 9.0290043885943305e-08, 2022-08-01T23:03:42.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919264256 2022-08-01T23:03:42.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.076 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2022-08-01T23:03:42.076 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 20, 2022-08-01T23:03:42.076 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:03:42.076 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:03:42.076 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:03:42.077 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:03:42.077 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:03:42.077 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:03:42.077 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362919264256 2022-08-01T23:03:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:42.078 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.057+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfa8053a70 msgr2=0x7fbfa8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.057+0000 7fbfcf566700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfa8053a70 0x7fbfa8055f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fbfc400b000 tx=0x7fbfc4006df0).stop 2022-08-01T23:03:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 msgr2=0x7fbfc8142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 0x7fbfc8142050 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fbfc00032f0 tx=0x7fbfc0018000).stop 2022-08-01T23:03:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 shutdown_connections 2022-08-01T23:03:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbfa8053a70 0x7fbfa8055f30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfc80b6a90 0x7fbfc8142050 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbfc80aa760 0x7fbfc80ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 --2- 172.21.15.32:0/3138433516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc8060aa0 0x7fbfc80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 >> 172.21.15.32:0/3138433516 conn(0x7fbfc80b3610 msgr2=0x7fbfc813c570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 shutdown_connections 2022-08-01T23:03:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.061+0000 7fbfcf566700 1 -- 172.21.15.32:0/3138433516 wait complete. 2022-08-01T23:03:42.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2022-08-01T23:03:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.189+0000 7f10b4f93700 1 Processor -- start 2022-08-01T23:03:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.189+0000 7f10b4f93700 1 -- start start 2022-08-01T23:03:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10b4f93700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10b0146ae0 0x7f10b01403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10b4f93700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 0x7f10b01408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10b4f93700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b0143cf0 con 0x7f10b00b4820 2022-08-01T23:03:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10b4f93700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b01476a0 con 0x7f10b0146ae0 2022-08-01T23:03:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10b4f93700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0143f90 con 0x7f10b00b4310 2022-08-01T23:03:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 0x7f10b01408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10ae59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10b0146ae0 0x7f10b01403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 0x7f10b01408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59020/0 (socket says 172.21.15.32:59020) 2022-08-01T23:03:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 -- 172.21.15.32:0/1770345591 learned_addr learned my addr 172.21.15.32:0/1770345591 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10aed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f10b00b4820 0x7f10b00b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:55728/0 (socket says 172.21.15.32:55728) 2022-08-01T23:03:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 -- 172.21.15.32:0/1770345591 >> v2:172.21.15.55:3300/0 conn(0x7f10b0146ae0 msgr2=0x7f10b01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 --2- 172.21.15.32:0/1770345591 >> v2:172.21.15.55:3300/0 conn(0x7f10b0146ae0 0x7f10b01403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 -- 172.21.15.32:0/1770345591 >> v1:172.21.15.55:6789/0 conn(0x7f10b00b4820 legacy=0x7f10b00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 -- 172.21.15.32:0/1770345591 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00b0e20 con 0x7f10b00b4310 2022-08-01T23:03:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10a5d9b700 1 --2- 172.21.15.32:0/1770345591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 0x7f10b01408b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f1098002f70 tx=0x7f1098006eb0).ready entity=mon.2 client_cookie=1d4f72892093d7f7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10add9b700 1 -- 172.21.15.32:0/1770345591 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109800d570 con 0x7f10b00b4310 2022-08-01T23:03:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10add9b700 1 -- 172.21.15.32:0/1770345591 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098000e90 con 0x7f10b00b4310 2022-08-01T23:03:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.193+0000 7f10add9b700 1 -- 172.21.15.32:0/1770345591 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109800d900 con 0x7f10b00b4310 2022-08-01T23:03:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- 172.21.15.32:0/1770345591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 msgr2=0x7f10b01408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 --2- 172.21.15.32:0/1770345591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 0x7f10b01408b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f1098002f70 tx=0x7f1098006eb0).stop 2022-08-01T23:03:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- 172.21.15.32:0/1770345591 shutdown_connections 2022-08-01T23:03:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 --2- 172.21.15.32:0/1770345591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b00b4310 0x7f10b01408b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 --2- 172.21.15.32:0/1770345591 >> v2:172.21.15.55:3300/0 conn(0x7f10b0146ae0 0x7f10b01403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- 172.21.15.32:0/1770345591 >> 172.21.15.32:0/1770345591 conn(0x7f10b005c620 msgr2=0x7f10b005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- 172.21.15.32:0/1770345591 shutdown_connections 2022-08-01T23:03:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- 172.21.15.32:0/1770345591 wait complete. 2022-08-01T23:03:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 Processor -- start 2022-08-01T23:03:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- start start 2022-08-01T23:03:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b00b4310 0x7f10b0142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 0x7f10b0152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b0146ae0 0x7f10b01148f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0113e70 con 0x7f10b00b4820 2022-08-01T23:03:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00b0220 con 0x7f10b00b4310 2022-08-01T23:03:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10b4f93700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0143260 con 0x7f10b0146ae0 2022-08-01T23:03:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10a5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 0x7f10b0152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10a5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 0x7f10b0152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47284/0 (socket says 172.21.15.32:47284) 2022-08-01T23:03:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10a5d9b700 1 -- 172.21.15.32:0/799836672 learned_addr learned my addr 172.21.15.32:0/799836672 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10ae59c700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b00b4310 0x7f10b0142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.197+0000 7f10aed9d700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b0146ae0 0x7f10b01148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a5d9b700 1 -- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b0146ae0 msgr2=0x7f10b01148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a5d9b700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b0146ae0 0x7f10b01148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a5d9b700 1 -- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b00b4310 msgr2=0x7f10b0142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a5d9b700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b00b4310 0x7f10b0142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a5d9b700 1 -- 172.21.15.32:0/799836672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109800b1e0 con 0x7f10b00b4820 2022-08-01T23:03:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a5d9b700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 0x7f10b0152d40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f1098002f70 tx=0x7f1098015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1098011760 con 0x7f10b00b4820 2022-08-01T23:03:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10980118c0 con 0x7f10b00b4820 2022-08-01T23:03:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1098011c20 con 0x7f10b00b4820 2022-08-01T23:03:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b00af440 con 0x7f10b00b4820 2022-08-01T23:03:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b00b06f0 con 0x7f10b00b4820 2022-08-01T23:03:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b00aeac0 con 0x7f10b00b4820 2022-08-01T23:03:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f109801a040 con 0x7f10b00b4820 2022-08-01T23:03:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a7fff700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10940539e0 0x7f1094055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.201+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f109806cdd0 con 0x7f10b00b4820 2022-08-01T23:03:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.205+0000 7f10ae59c700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10940539e0 0x7f1094055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.205+0000 7f10ae59c700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10940539e0 0x7f1094055ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f10a000b000 tx=0x7f10a0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.209+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1098043c90 con 0x7f10b00b4820 2022-08-01T23:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.333+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7f10b00574e0 con 0x7f10b00b4820 2022-08-01T23:03:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.333+0000 7f10a7fff700 1 -- 172.21.15.32:0/799836672 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v107) v1 ==== 123+0+64 (secure 0 0 0) 0x7f1098023020 con 0x7f10b00b4820 2022-08-01T23:03:42.339 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:42.339 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:42.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": "cephfs_data", 2022-08-01T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_id": 20, 2022-08-01T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2 2022-08-01T23:03:42.340 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10940539e0 msgr2=0x7f1094055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10940539e0 0x7f1094055ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f10a000b000 tx=0x7f10a0006df0).stop 2022-08-01T23:03:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 msgr2=0x7f10b0152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 0x7f10b0152d40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f1098002f70 tx=0x7f1098015000).stop 2022-08-01T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 shutdown_connections 2022-08-01T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10940539e0 0x7f1094055ea0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f10b0146ae0 0x7f10b01148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10b00b4820 0x7f10b0152d40 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 --2- 172.21.15.32:0/799836672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b00b4310 0x7f10b0142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 >> 172.21.15.32:0/799836672 conn(0x7f10b005c620 msgr2=0x7f10b0145ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 shutdown_connections 2022-08-01T23:03:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.337+0000 7f10b4f93700 1 -- 172.21.15.32:0/799836672 wait complete. 2022-08-01T23:03:42.352 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T23:03:42.353 DEBUG:teuthology.orchestra.run.smithi130:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2022-08-01T23:03:42.379 INFO:teuthology.orchestra.run.smithi130.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2022-08-01T23:03:42.379 INFO:teuthology.orchestra.run.smithi130.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:42.380 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:03:42.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:42.401 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:42.421 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:42.441 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:42.462 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:03:42.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:42.463 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:42.463 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:42.471 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:03:42.472 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:42.473 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:03:42.473 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:03:42.473 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:42.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23316]: fuse finished with error 0 and tester_r 0 2022-08-01T23:03:42.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:42.490 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:42.490 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:42.538 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:03:42.539 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:03:42.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:03:42.551 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:42.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:42.559 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:42.559 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:42.567 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:03:42.568 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:42.569 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:03:42.569 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:03:42.570 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:42.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22842]: fuse finished with error 0 and tester_r 0 2022-08-01T23:03:42.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:42.586 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:42.587 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:42.639 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:03:42.640 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:03:42.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:03:42.641 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:42.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:03:42.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.753+0000 7fd177f0f700 1 Processor -- start 2022-08-01T23:03:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.753+0000 7fd177f0f700 1 -- start start 2022-08-01T23:03:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1700b1f40 0x7fd1700b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1700a7250 0x7fd1701448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700b04e0 con 0x7fd170148530 2022-08-01T23:03:42.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700a9250 con 0x7fd1700a7250 2022-08-01T23:03:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700b0780 con 0x7fd1700b1f40 2022-08-01T23:03:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1700a7250 0x7fd1701448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1700a7250 0x7fd1701448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47296/0 (socket says 172.21.15.32:47296) 2022-08-01T23:03:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 -- 172.21.15.32:0/1811733840 learned_addr learned my addr 172.21.15.32:0/1811733840 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd175cab700 1 --2- 172.21.15.32:0/1811733840 >> v2:172.21.15.55:3300/0 conn(0x7fd1700b1f40 0x7fd1700b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 -- 172.21.15.32:0/1811733840 >> v2:172.21.15.55:3300/0 conn(0x7fd1700b1f40 msgr2=0x7fd1700b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 --2- 172.21.15.32:0/1811733840 >> v2:172.21.15.55:3300/0 conn(0x7fd1700b1f40 0x7fd1700b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 -- 172.21.15.32:0/1811733840 >> v1:172.21.15.55:6789/0 conn(0x7fd170148530 legacy=0x7fd1700b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 -- 172.21.15.32:0/1811733840 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700b0a20 con 0x7fd1700a7250 2022-08-01T23:03:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd1764ac700 1 --2- 172.21.15.32:0/1811733840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700a7250 0x7fd1701448b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fd1701478c0 tx=0x7fd168009290).ready entity=mon.0 client_cookie=d085afc8af9d182 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd174ca9700 1 -- 172.21.15.32:0/1811733840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd16800b860 con 0x7fd1700a7250 2022-08-01T23:03:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd174ca9700 1 -- 172.21.15.32:0/1811733840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16800b9c0 con 0x7fd1700a7250 2022-08-01T23:03:42.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd174ca9700 1 -- 172.21.15.32:0/1811733840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd16800bb30 con 0x7fd1700a7250 2022-08-01T23:03:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- 172.21.15.32:0/1811733840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700a7250 msgr2=0x7fd1701448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 --2- 172.21.15.32:0/1811733840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700a7250 0x7fd1701448b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fd1701478c0 tx=0x7fd168009290).stop 2022-08-01T23:03:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- 172.21.15.32:0/1811733840 shutdown_connections 2022-08-01T23:03:42.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 --2- 172.21.15.32:0/1811733840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700a7250 0x7fd1701448b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 --2- 172.21.15.32:0/1811733840 >> v2:172.21.15.55:3300/0 conn(0x7fd1700b1f40 0x7fd1700b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- 172.21.15.32:0/1811733840 >> 172.21.15.32:0/1811733840 conn(0x7fd17005f840 msgr2=0x7fd17005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- 172.21.15.32:0/1811733840 shutdown_connections 2022-08-01T23:03:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.757+0000 7fd177f0f700 1 -- 172.21.15.32:0/1811733840 wait complete. 2022-08-01T23:03:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 Processor -- start 2022-08-01T23:03:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 -- start start 2022-08-01T23:03:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1700a7250 0x7fd1700a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1700b1f40 0x7fd170149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 0x7fd17014aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700ab990 con 0x7fd170148530 2022-08-01T23:03:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700abcd0 con 0x7fd1700b1f40 2022-08-01T23:03:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd177f0f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700abf70 con 0x7fd1700a7250 2022-08-01T23:03:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd1764ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 0x7fd17014aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd1764ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 0x7fd17014aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47302/0 (socket says 172.21.15.32:47302) 2022-08-01T23:03:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.761+0000 7fd1764ac700 1 -- 172.21.15.32:0/2992806513 learned_addr learned my addr 172.21.15.32:0/2992806513 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd175cab700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1700a7250 0x7fd1700a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1754aa700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1700b1f40 0x7fd170149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1764ac700 1 -- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1700a7250 msgr2=0x7fd1700a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1764ac700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1700a7250 0x7fd1700a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1764ac700 1 -- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1700b1f40 msgr2=0x7fd170149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1764ac700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1700b1f40 0x7fd170149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1764ac700 1 -- 172.21.15.32:0/2992806513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009530 con 0x7fd170148530 2022-08-01T23:03:42.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd1764ac700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 0x7fd17014aee0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fd1701478c0 tx=0x7fd168016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:42.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1680032f0 con 0x7fd170148530 2022-08-01T23:03:42.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168021440 con 0x7fd170148530 2022-08-01T23:03:42.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1700ac210 con 0x7fd170148530 2022-08-01T23:03:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1680217a0 con 0x7fd170148530 2022-08-01T23:03:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17014a730 con 0x7fd170148530 2022-08-01T23:03:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.765+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1700b0cc0 con 0x7fd170148530 2022-08-01T23:03:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.769+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd168027020 con 0x7fd170148530 2022-08-01T23:03:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.769+0000 7fd15effd700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd150053b30 0x7fd150055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.769+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd16806d310 con 0x7fd170148530 2022-08-01T23:03:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.769+0000 7fd175cab700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd150053b30 0x7fd150055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.769+0000 7fd175cab700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd150053b30 0x7fd150055ff0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fd16000c000 tx=0x7fd160008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.777+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd168044180 con 0x7fd170148530 2022-08-01T23:03:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.905+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd1700574e0 con 0x7fd170148530 2022-08-01T23:03:42.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd15effd700 1 -- 172.21.15.32:0/2992806513 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) v1 ==== 81+0+34051 (secure 0 0 0) 0x7fd168015020 con 0x7fd170148530 2022-08-01T23:03:42.915 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:42.915 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:42.915 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 107, 2022-08-01T23:03:42.915 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:03:42.916 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:03:42.916 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:03:32.553487+0000", 2022-08-01T23:03:42.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:42.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:42.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:03:42.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:03:42.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:03:42.917 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:03:42.917 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:03:42.918 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:03:42.918 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:03:42.918 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:42.918 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:03:42.919 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:03:42.919 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:03:42.919 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:03:42.919 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:03:42.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 20, 2022-08-01T23:03:42.920 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:03:42.920 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:03:42.920 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:03:42.920 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:03:42.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:03:42.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:03:42.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:03:42.922 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:03:42.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:42.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:42.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:42.922 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:42.923 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:42.923 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:42.923 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:42.923 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:42.924 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:03:42.924 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:03:42.924 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:03:42.924 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:03:42.925 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:03:42.925 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:42.925 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:42.925 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:42.925 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:42.926 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:42.926 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:42.926 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:42.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:03:42.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:42.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:42.927 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:42.927 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:42.928 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:42.928 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:42.928 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:42.928 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:42.929 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:42.929 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:42.929 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:42.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:42.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:42.930 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:42.930 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:42.930 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:42.930 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:42.931 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:42.931 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:42.931 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:42.931 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:42.931 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:42.932 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:42.932 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:42.932 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:42.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:42.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.933 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:42.933 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:42.933 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:42.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:42.934 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:42.934 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:42.934 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:42.934 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:42.935 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:42.935 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:42.935 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:42.935 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:42.936 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:03:42.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.936 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:42.936 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:03:42.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:03:42.937 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:03:42.937 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:03:42.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:42.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:42.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:42.938 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:42.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:42.939 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:42.939 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:42.939 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:42.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:42.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:42.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:42.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:42.940 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:42.941 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:42.941 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:42.941 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:42.941 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:42.942 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:42.942 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:42.942 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:42.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.943 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:03:42.943 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:42.943 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:42.943 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:42.943 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:42.944 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:42.944 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:42.944 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:42.944 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:42.945 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:42.945 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:42.945 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:42.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:42.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:42.946 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:42.946 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:42.946 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:42.947 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:42.947 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:42.947 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:42.947 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:42.947 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:42.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:42.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:42.948 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:42.948 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:42.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:42.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:42.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:42.949 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:42.950 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:42.950 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:42.950 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:42.950 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:42.951 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:42.951 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:42.951 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:42.951 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:42.952 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:42.952 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:42.952 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:03:42.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.953 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 19, 2022-08-01T23:03:42.953 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:03:42.953 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:03:30.001802+0000", 2022-08-01T23:03:42.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:42.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:42.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:42.954 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:42.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:42.955 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:42.955 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:42.955 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:42.955 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:42.956 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:42.956 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:42.956 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:42.956 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:42.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:42.957 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:42.957 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:42.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:42.957 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:42.958 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:42.958 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:42.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "107", 2022-08-01T23:03:42.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:42.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:42.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:42.959 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:42.959 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:42.960 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:42.960 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:42.960 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:42.960 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:42.961 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:42.961 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:42.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:42.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:42.961 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:42.962 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:42.962 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:42.962 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:42.962 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:42.963 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.964 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:42.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:42.966 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:42.966 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:42.966 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:42.966 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:42.966 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:42.967 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:42.967 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:42.967 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:03:42.967 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:03:42.968 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.969 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 20, 2022-08-01T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:03:30.985153+0000", 2022-08-01T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:42.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:42.971 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:42.972 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:42.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:42.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:42.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:42.972 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:42.973 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:42.973 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:42.973 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:42.973 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:42.973 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:42.974 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:42.974 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:42.974 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:42.974 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:42.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "107", 2022-08-01T23:03:42.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:42.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:42.975 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:42.976 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:42.976 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:42.976 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:42.976 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:42.976 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:42.977 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:42.977 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:42.977 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:42.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:42.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:42.978 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:42.978 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:42.978 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:42.978 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:42.979 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:42.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:42.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:42.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:42.979 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:42.980 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:42.980 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:42.980 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:42.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:42.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.981 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:42.981 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:42.981 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:42.981 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:42.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:42.982 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:42.982 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:42.982 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:42.983 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:42.984 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:42.984 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:03:42.984 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:03:42.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.985 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:42.985 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:03:42.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.985 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:42.986 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:03:42.986 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:42.986 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:42.986 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:42.987 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:42.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:42.987 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:42.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:42.987 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:42.988 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:42.988 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:42.988 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:42.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:42.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:42.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:03:42.989 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:42.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:03:42.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:42.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.991 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:42.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:42.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:42.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:03:42.992 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:42.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:03:42.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:42.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.994 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:42.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:42.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:42.995 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:03:42.995 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:42.996 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:03:42.997 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:42.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:42.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.997 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:42.998 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:42.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:42.998 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:03:42.999 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:42.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:42.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:42.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:42.999 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:03:43.000 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:43.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.001 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:03:43.001 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:03:43.001 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:03:43.001 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:03:43.002 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.002 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.002 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.003 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:43.003 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:03:43.003 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.004 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.004 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.004 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.004 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.004 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.005 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.005 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:43.005 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.005 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.006 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.006 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:03:43.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:03:43.008 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.008 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.008 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.009 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:03:43.010 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:03:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.012 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:03:43.013 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.014 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:03:43.014 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.015 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.016 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:03:43.017 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:03:43.018 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:43.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.018 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.019 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:03:43.019 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:03:43.019 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:03:43.019 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:03:43.020 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.020 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.020 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.021 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:43.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:03:43.022 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.022 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.022 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.022 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 104, 2022-08-01T23:03:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:03:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:03:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:03:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:03:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:03:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:03:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:03:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.036 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:03:43.036 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:43.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.037 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.037 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:03:43.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:03:43.037 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:03:43.038 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:03:43.038 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.038 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.038 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.039 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:43.039 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:03:43.040 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.040 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.040 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.040 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.041 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.041 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.041 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.041 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:43.041 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.042 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.042 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.042 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.043 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:03:43.043 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:03:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:03:43.046 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:03:43.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:03:43.049 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:03:43.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.051 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:03:43.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:03:43.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:43.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.054 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:03:43.054 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:03:43.055 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:03:43.055 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:03:43.055 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.055 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.056 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.056 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:43.057 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:03:43.057 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.057 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.057 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.058 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.058 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.058 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.058 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.058 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 104, 2022-08-01T23:03:43.059 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.059 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.059 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:03:43.060 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:03:43.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.062 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.062 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:03:43.063 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.064 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:03:43.064 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.065 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:03:43.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:03:43.068 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.069 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:03:43.070 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.071 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:03:43.071 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:43.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.072 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:03:43.072 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:03:43.072 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:03:43.072 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:03:43.073 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.073 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.073 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.074 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:43.074 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:03:43.074 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.075 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.075 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.075 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.075 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.075 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 104, 2022-08-01T23:03:43.076 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.076 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.077 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:03:43.078 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:03:43.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.080 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:03:43.081 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.081 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:03:43.082 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.082 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.083 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:03:43.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:03:43.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.086 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.087 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:03:43.087 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.088 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:03:43.088 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:43.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.089 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.090 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:03:43.090 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:03:43.090 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:03:43.090 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:03:43.091 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.091 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.091 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.091 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.092 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:43.092 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:03:43.092 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.093 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.093 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.093 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.093 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.093 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.094 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.094 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:43.094 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.094 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.095 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.095 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.096 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:03:43.096 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.097 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:03:43.097 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.098 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.098 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.100 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:03:43.100 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.101 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:03:43.101 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.102 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.103 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.103 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:03:43.103 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.104 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:03:43.105 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.105 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.105 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.106 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.106 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:03:43.107 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.107 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:03:43.108 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:43.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.108 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.108 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:03:43.109 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:03:43.109 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:03:43.109 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:03:43.109 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.109 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.110 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.110 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.111 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:43.111 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:03:43.111 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:43.111 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:43.111 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:43.112 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:43.112 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:43.112 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:43.112 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:43.112 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:43.113 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:43.113 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:43.113 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:43.113 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.114 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:03:43.114 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.115 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:03:43.115 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.116 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.116 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:43.116 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.117 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:03:43.117 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.118 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:03:43.118 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.119 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.119 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:43.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.120 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:03:43.120 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.121 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:03:43.121 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.122 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:43.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:43.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:43.123 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:03:43.123 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:43.124 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:03:43.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:43.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.125 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:03:43.125 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:03:43.126 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:03:43.126 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:03:43.126 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:43.126 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:43.127 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:43.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:43.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.127 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:43.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:03:43.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.128 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:43.128 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.128 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.128 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.129 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.129 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.129 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:03:43.129 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.130 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:43.130 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.130 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.131 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.131 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.131 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.131 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:03:43.131 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.132 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:43.132 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.132 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.133 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.133 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.133 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.133 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:03:43.133 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.134 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:43.134 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.134 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.135 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.135 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.135 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.135 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:03:43.135 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.136 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:43.136 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.136 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.137 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.137 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.137 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.137 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:03:43.138 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.138 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:43.138 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.139 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.139 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.139 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.139 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.139 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:03:43.140 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.140 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:43.141 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.141 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.141 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.141 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.141 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.142 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:03:43.142 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:43.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:43.143 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:43.143 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:43.143 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:43.143 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:43.144 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:43.144 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:03:43.144 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:43.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.144 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:43.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:03:43.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:03:43.145 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:03:43.145 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:03:43.145 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:03:43.146 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:03:43.146 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:03:43.146 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:03:43.146 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:43.147 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:03:43.147 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:03:43.147 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:43.147 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:03:43.147 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:03:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:03:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:03:43.149 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:43.149 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:43.149 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:03:43.149 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:43.149 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:43.150 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:03:43.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:43.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:43.150 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:03:43.150 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:03:43.151 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:03:43.151 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:03:43.151 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:03:43.151 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd150053b30 msgr2=0x7fd150055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd150053b30 0x7fd150055ff0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fd16000c000 tx=0x7fd160008040).stop 2022-08-01T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 msgr2=0x7fd17014aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 0x7fd17014aee0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fd1701478c0 tx=0x7fd168016000).stop 2022-08-01T23:03:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 shutdown_connections 2022-08-01T23:03:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd150053b30 0x7fd150055ff0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd170148530 0x7fd17014aee0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1700b1f40 0x7fd170149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 --2- 172.21.15.32:0/2992806513 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd1700a7250 0x7fd1700a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 >> 172.21.15.32:0/2992806513 conn(0x7fd17005f840 msgr2=0x7fd170144b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 shutdown_connections 2022-08-01T23:03:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:42.909+0000 7fd177f0f700 1 -- 172.21.15.32:0/2992806513 wait complete. 2022-08-01T23:03:43.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab552c700 1 Processor -- start 2022-08-01T23:03:43.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab552c700 1 -- start start 2022-08-01T23:03:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab552c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ab005c6f0 0x7f6ab00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab552c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab00aa630 con 0x7f6ab00af550 2022-08-01T23:03:43.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab552c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab005fd10 con 0x7f6ab0060330 2022-08-01T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab552c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab00b15e0 con 0x7f6ab005c6f0 2022-08-01T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6ab4d2b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ab005c6f0 0x7f6ab00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6aaffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ab00af550 0x7f6ab01445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34850/0 (socket says 172.21.15.32:34850) 2022-08-01T23:03:43.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.261+0000 7f6aaffff700 1 -- 172.21.15.32:0/2309892175 learned_addr learned my addr 172.21.15.32:0/2309892175 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab4d2b700 1 -- 172.21.15.32:0/2309892175 >> v1:172.21.15.55:6789/0 conn(0x7f6ab0060330 legacy=0x7f6ab00a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab4d2b700 1 -- 172.21.15.32:0/2309892175 >> v1:172.21.15.32:6789/0 conn(0x7f6ab00af550 legacy=0x7f6ab01445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6aaeffd700 1 -- 172.21.15.32:0/2309892175 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858874483 0 0) 0x7f6ab005fd10 con 0x7f6ab0060330 2022-08-01T23:03:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab4d2b700 1 -- 172.21.15.32:0/2309892175 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab005f840 con 0x7f6ab005c6f0 2022-08-01T23:03:43.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab4d2b700 1 --2- 172.21.15.32:0/2309892175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab005c6f0 0x7f6ab00b0e40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f6aa4001110 tx=0x7f6aa80091d0).ready entity=mon.0 client_cookie=a11c5bb0952084f8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:43.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6aaeffd700 1 -- 172.21.15.32:0/2309892175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa8009ea0 con 0x7f6ab005c6f0 2022-08-01T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6aaeffd700 1 -- 172.21.15.32:0/2309892175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa800b730 con 0x7f6ab005c6f0 2022-08-01T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6aaeffd700 1 -- 172.21.15.32:0/2309892175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa800b8a0 con 0x7f6ab005c6f0 2022-08-01T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 -- 172.21.15.32:0/2309892175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab005c6f0 msgr2=0x7f6ab00b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 --2- 172.21.15.32:0/2309892175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab005c6f0 0x7f6ab00b0e40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f6aa4001110 tx=0x7f6aa80091d0).stop 2022-08-01T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 -- 172.21.15.32:0/2309892175 shutdown_connections 2022-08-01T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 --2- 172.21.15.32:0/2309892175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab005c6f0 0x7f6ab00b0e40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 -- 172.21.15.32:0/2309892175 >> 172.21.15.32:0/2309892175 conn(0x7f6ab00612d0 msgr2=0x7f6ab0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 -- 172.21.15.32:0/2309892175 shutdown_connections 2022-08-01T23:03:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 -- 172.21.15.32:0/2309892175 wait complete. 2022-08-01T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 Processor -- start 2022-08-01T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.265+0000 7f6ab552c700 1 -- start start 2022-08-01T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab005c6f0 0x7f6ab0146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 0x7f6ab0066220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ab00af550 0x7f6ab0150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0140890 con 0x7f6ab0060330 2022-08-01T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0150060 con 0x7f6ab005c6f0 2022-08-01T23:03:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0066710 con 0x7f6ab00af550 2022-08-01T23:03:43.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 0x7f6ab0066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 0x7f6ab0066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47314/0 (socket says 172.21.15.32:47314) 2022-08-01T23:03:43.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 -- 172.21.15.32:0/3993986973 learned_addr learned my addr 172.21.15.32:0/3993986973 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaffff700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab005c6f0 0x7f6ab0146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab4d2b700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ab00af550 0x7f6ab0150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 -- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ab00af550 msgr2=0x7f6ab0150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ab00af550 0x7f6ab0150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 -- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab005c6f0 msgr2=0x7f6ab0146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab005c6f0 0x7f6ab0146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 -- 172.21.15.32:0/3993986973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa8009470 con 0x7f6ab0060330 2022-08-01T23:03:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aaf7fe700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 0x7f6ab0066220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f6aa0000ed0 tx=0x7f6aa0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa000ce20 con 0x7f6ab0060330 2022-08-01T23:03:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa000f470 con 0x7f6ab0060330 2022-08-01T23:03:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab0153fc0 con 0x7f6ab0060330 2022-08-01T23:03:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa000f7d0 con 0x7f6ab0060330 2022-08-01T23:03:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab0150250 con 0x7f6ab0060330 2022-08-01T23:03:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab0150690 con 0x7f6ab0060330 2022-08-01T23:03:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.269+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6aa0019030 con 0x7f6ab0060330 2022-08-01T23:03:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.273+0000 7f6aad7fa700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6a90053a10 0x7f6a90055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.273+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6aa006b370 con 0x7f6ab0060330 2022-08-01T23:03:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.273+0000 7f6aaffff700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6a90053a10 0x7f6a90055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.273+0000 7f6aaffff700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6a90053a10 0x7f6a90055ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f6aa400b000 tx=0x7f6aa4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.277+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6aa0042180 con 0x7f6ab0060330 2022-08-01T23:03:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.441+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ab00574e0 con 0x7f6ab0060330 2022-08-01T23:03:43.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6aad7fa700 1 -- 172.21.15.32:0/3993986973 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f6aa00457a0 con 0x7f6ab0060330 2022-08-01T23:03:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:43.449 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:31.549702+0000","modified":"2022-08-01T23:03:32.552769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/550523700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":550523700},{"type":"v1","addr":"172.21.15.32:6835","nonce":550523700}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T23:03:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6a90053a10 msgr2=0x7f6a90055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6a90053a10 0x7f6a90055ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f6aa400b000 tx=0x7f6aa4009040).stop 2022-08-01T23:03:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 msgr2=0x7f6ab0066220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 0x7f6ab0066220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f6aa0000ed0 tx=0x7f6aa0007ac0).stop 2022-08-01T23:03:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 shutdown_connections 2022-08-01T23:03:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6a90053a10 0x7f6a90055ed0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6ab00af550 0x7f6ab0150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab0060330 0x7f6ab0066220 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 --2- 172.21.15.32:0/3993986973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ab005c6f0 0x7f6ab0146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 >> 172.21.15.32:0/3993986973 conn(0x7f6ab00612d0 msgr2=0x7f6ab00a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 shutdown_connections 2022-08-01T23:03:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.445+0000 7f6ab552c700 1 -- 172.21.15.32:0/3993986973 wait complete. 2022-08-01T23:03:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2022-08-01T23:03:43.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:03:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 Processor -- start 2022-08-01T23:03:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 -- start start 2022-08-01T23:03:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f37640b6ee0 0x7f37640b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f37640b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3764061d90 con 0x7f37640619b0 2022-08-01T23:03:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3764143fa0 con 0x7f37640b6ee0 2022-08-01T23:03:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3764142f60 con 0x7f37640b5ed0 2022-08-01T23:03:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37640619b0 0x7f3764143890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34864/0 (socket says 172.21.15.32:34864) 2022-08-01T23:03:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376259c700 1 -- 172.21.15.32:0/2982436426 learned_addr learned my addr 172.21.15.32:0/2982436426 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3761d9b700 1 --2- 172.21.15.32:0/2982436426 >> v2:172.21.15.55:3300/0 conn(0x7f37640b6ee0 0x7f37640b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3762d9d700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f37640b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376159a700 1 -- 172.21.15.32:0/2982436426 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403864219 0 0) 0x7f3764061d90 con 0x7f37640619b0 2022-08-01T23:03:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376159a700 1 -- 172.21.15.32:0/2982436426 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3744003260 con 0x7f37640619b0 2022-08-01T23:03:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3761d9b700 1 -- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 msgr2=0x7f37640b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3761d9b700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f37640b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3761d9b700 1 -- 172.21.15.32:0/2982436426 >> v1:172.21.15.32:6789/0 conn(0x7f37640619b0 legacy=0x7f3764143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3761d9b700 1 -- 172.21.15.32:0/2982436426 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640ba5a0 con 0x7f37640b6ee0 2022-08-01T23:03:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3762d9d700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f37640b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f3761d9b700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f37640b48a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f37640b6880 tx=0x7f374c006df0).ready entity=mon.1 client_cookie=d203e7c2b4bceda2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376159a700 1 -- 172.21.15.32:0/2982436426 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374c00d570 con 0x7f37640b6ee0 2022-08-01T23:03:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376159a700 1 -- 172.21.15.32:0/2982436426 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c000e90 con 0x7f37640b6ee0 2022-08-01T23:03:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.561+0000 7f376159a700 1 -- 172.21.15.32:0/2982436426 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374c00d8c0 con 0x7f37640b6ee0 2022-08-01T23:03:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 msgr2=0x7f37640b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f37640b48a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f37640b6880 tx=0x7f374c006df0).stop 2022-08-01T23:03:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- 172.21.15.32:0/2982436426 shutdown_connections 2022-08-01T23:03:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f37640b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 --2- 172.21.15.32:0/2982436426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f37640b48a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- 172.21.15.32:0/2982436426 >> 172.21.15.32:0/2982436426 conn(0x7f376406af80 msgr2=0x7f376405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- 172.21.15.32:0/2982436426 shutdown_connections 2022-08-01T23:03:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- 172.21.15.32:0/2982436426 wait complete. 2022-08-01T23:03:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 Processor -- start 2022-08-01T23:03:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- start start 2022-08-01T23:03:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 0x7f37640b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f3764146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f376413a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3764146380 con 0x7f37640619b0 2022-08-01T23:03:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37640b8f30 con 0x7f37640b6ee0 2022-08-01T23:03:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37640b9160 con 0x7f37640b5ed0 2022-08-01T23:03:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 0x7f37640b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 0x7f37640b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47326/0 (socket says 172.21.15.32:47326) 2022-08-01T23:03:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f376259c700 1 -- 172.21.15.32:0/1131014164 learned_addr learned my addr 172.21.15.32:0/1131014164 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f3761d9b700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f3764146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.565+0000 7f3762d9d700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f376413a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376259c700 1 -- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 msgr2=0x7f3764146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376259c700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f3764146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376259c700 1 -- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 msgr2=0x7f376413a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376259c700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f376413a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376259c700 1 -- 172.21.15.32:0/1131014164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c009470 con 0x7f37640619b0 2022-08-01T23:03:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376259c700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 0x7f37640b85a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f3764145ca0 tx=0x7f3754007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3754007110 con 0x7f37640619b0 2022-08-01T23:03:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3754007270 con 0x7f37640619b0 2022-08-01T23:03:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c00b180 con 0x7f37640619b0 2022-08-01T23:03:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3754005030 con 0x7f37640619b0 2022-08-01T23:03:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37641453b0 con 0x7f37640619b0 2022-08-01T23:03:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3754005190 con 0x7f37640619b0 2022-08-01T23:03:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f375b7fe700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37480539e0 0x7f3748055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37640b8410 con 0x7f37640619b0 2022-08-01T23:03:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f375406e3f0 con 0x7f37640619b0 2022-08-01T23:03:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.569+0000 7f3761d9b700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37480539e0 0x7f3748055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.573+0000 7f3761d9b700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37480539e0 0x7f3748055ea0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f374c018000 tx=0x7f374c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.581+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3754041830 con 0x7f37640619b0 2022-08-01T23:03:43.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:43.745+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f37640574e0 con 0x7f37640619b0 2022-08-01T23:03:44.456 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:03:44.449+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:03:44.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.465+0000 7f375b7fe700 1 -- 172.21.15.32:0/1131014164 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f375403d090 con 0x7f37640619b0 2022-08-01T23:03:44.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37480539e0 msgr2=0x7f3748055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37480539e0 0x7f3748055ea0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f374c018000 tx=0x7f374c014000).stop 2022-08-01T23:03:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 msgr2=0x7f37640b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 0x7f37640b85a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f3764145ca0 tx=0x7f3754007a80).stop 2022-08-01T23:03:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 shutdown_connections 2022-08-01T23:03:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37480539e0 0x7f3748055ea0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37640b6ee0 0x7f376413a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37640b5ed0 0x7f3764146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 --2- 172.21.15.32:0/1131014164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37640619b0 0x7f37640b85a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 >> 172.21.15.32:0/1131014164 conn(0x7f376406af80 msgr2=0x7f376405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:44.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 shutdown_connections 2022-08-01T23:03:44.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.469+0000 7f376359e700 1 -- 172.21.15.32:0/1131014164 wait complete. 2022-08-01T23:03:44.479 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:03:44.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:44.496 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:03:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.589+0000 7eff11711700 1 Processor -- start 2022-08-01T23:03:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.589+0000 7eff11711700 1 -- start start 2022-08-01T23:03:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff11711700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b08a0 0x7eff0c142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff11711700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c143ab0 con 0x7eff0c0b3260 2022-08-01T23:03:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff11711700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c0b0c80 con 0x7eff0c0ad000 2022-08-01T23:03:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff11711700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0ae810 con 0x7eff0c0b08a0 2022-08-01T23:03:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff03fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7eff0c0b3260 0x7eff0c1433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34890/0 (socket says 172.21.15.32:34890) 2022-08-01T23:03:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff03fff700 1 -- 172.21.15.32:0/2398459189 learned_addr learned my addr 172.21.15.32:0/2398459189 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0bfff700 1 --2- 172.21.15.32:0/2398459189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b08a0 0x7eff0c142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810366657 0 0) 0x7eff0c143ab0 con 0x7eff0c0b3260 2022-08-01T23:03:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeec003260 con 0x7eff0c0b3260 2022-08-01T23:03:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718232882 0 0) 0x7eff0c0b0c80 con 0x7eff0c0ad000 2022-08-01T23:03:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff0c143ab0 con 0x7eff0c0ad000 2022-08-01T23:03:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0bfff700 1 -- 172.21.15.32:0/2398459189 >> v1:172.21.15.55:6789/0 conn(0x7eff0c0ad000 legacy=0x7eff0c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0bfff700 1 -- 172.21.15.32:0/2398459189 >> v1:172.21.15.32:6789/0 conn(0x7eff0c0b3260 legacy=0x7eff0c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0bfff700 1 -- 172.21.15.32:0/2398459189 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0aeab0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0bfff700 1 --2- 172.21.15.32:0/2398459189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b08a0 0x7eff0c142eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7efefc002f70 tx=0x7efefc006e80).ready entity=mon.2 client_cookie=257c21de5b370deb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.593+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efefc014070 con 0x7eff0c0b08a0 2022-08-01T23:03:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc000e90 con 0x7eff0c0b08a0 2022-08-01T23:03:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff0b7fe700 1 -- 172.21.15.32:0/2398459189 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efefc0107f0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 -- 172.21.15.32:0/2398459189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b08a0 msgr2=0x7eff0c142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 --2- 172.21.15.32:0/2398459189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b08a0 0x7eff0c142eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7efefc002f70 tx=0x7efefc006e80).stop 2022-08-01T23:03:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 -- 172.21.15.32:0/2398459189 shutdown_connections 2022-08-01T23:03:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 --2- 172.21.15.32:0/2398459189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b08a0 0x7eff0c142eb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 -- 172.21.15.32:0/2398459189 >> 172.21.15.32:0/2398459189 conn(0x7eff0c0612d0 msgr2=0x7eff0c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 -- 172.21.15.32:0/2398459189 shutdown_connections 2022-08-01T23:03:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 -- 172.21.15.32:0/2398459189 wait complete. 2022-08-01T23:03:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 Processor -- start 2022-08-01T23:03:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 -- start start 2022-08-01T23:03:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.597+0000 7eff11711700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff0c0ad000 0x7eff0c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 0x7eff0c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b3260 0x7eff0c0a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0a54b0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0a2e50 con 0x7eff0c0ad000 2022-08-01T23:03:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0a5210 con 0x7eff0c0b3260 2022-08-01T23:03:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 0x7eff0c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 0x7eff0c152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47354/0 (socket says 172.21.15.32:47354) 2022-08-01T23:03:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 -- 172.21.15.32:0/2070871208 learned_addr learned my addr 172.21.15.32:0/2070871208 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff10f10700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b3260 0x7eff0c0a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff0bfff700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff0c0ad000 0x7eff0c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 -- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b3260 msgr2=0x7eff0c0a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b3260 0x7eff0c0a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 -- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff0c0ad000 msgr2=0x7eff0c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff0c0ad000 0x7eff0c152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 -- 172.21.15.32:0/2070871208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc00c310 con 0x7eff0c0b08a0 2022-08-01T23:03:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff03fff700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 0x7eff0c152c50 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7efef4001110 tx=0x7efef4007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efef400cdd0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef40070d0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efef4007430 con 0x7eff0c0b08a0 2022-08-01T23:03:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c0a3040 con 0x7eff0c0b08a0 2022-08-01T23:03:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c0a4e30 con 0x7eff0c0b08a0 2022-08-01T23:03:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.601+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c0574e0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.605+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efef401a070 con 0x7eff0c0b08a0 2022-08-01T23:03:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.605+0000 7eff09ffb700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efef00539e0 0x7efef0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.605+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6506+0+0 (secure 0 0 0) 0x7efef406b220 con 0x7eff0c0b08a0 2022-08-01T23:03:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.605+0000 7eff0bfff700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efef00539e0 0x7efef0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.605+0000 7eff0bfff700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efef00539e0 0x7efef0055ea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7efefc01a000 tx=0x7eff0c0a95b0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.609+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efef4041ef0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.773+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff0c0a36b0 con 0x7eff0c0b08a0 2022-08-01T23:03:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.773+0000 7eff09ffb700 1 -- 172.21.15.32:0/2070871208 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2130 (secure 0 0 0) 0x7efef4045510 con 0x7eff0c0b08a0 2022-08-01T23:03:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:31.549702+0000","modified":"2022-08-01T23:03:44.454008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T23:03:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efef00539e0 msgr2=0x7efef0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efef00539e0 0x7efef0055ea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7efefc01a000 tx=0x7eff0c0a95b0).stop 2022-08-01T23:03:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 msgr2=0x7eff0c152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 0x7eff0c152c50 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7efef4001110 tx=0x7efef4007aa0).stop 2022-08-01T23:03:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 shutdown_connections 2022-08-01T23:03:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efef00539e0 0x7efef0055ea0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7eff0c0b3260 0x7eff0c0a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c0b08a0 0x7eff0c152c50 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 --2- 172.21.15.32:0/2070871208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff0c0ad000 0x7eff0c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 >> 172.21.15.32:0/2070871208 conn(0x7eff0c0612d0 msgr2=0x7eff0c143000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 shutdown_connections 2022-08-01T23:03:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.777+0000 7eff11711700 1 -- 172.21.15.32:0/2070871208 wait complete. 2022-08-01T23:03:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 51 2022-08-01T23:03:44.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:03:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 Processor -- start 2022-08-01T23:03:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 -- start start 2022-08-01T23:03:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fde900ad670 0x7fde900b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde9013fa40 0x7fde9013fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde900b0c40 con 0x7fde900af120 2022-08-01T23:03:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900b0ee0 con 0x7fde900ad670 2022-08-01T23:03:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde95395700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900ae9e0 con 0x7fde9013fa40 2022-08-01T23:03:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde94b94700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde9013fa40 0x7fde9013fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde8ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fde900ad670 0x7fde900b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde8f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fde900af120 0x7fde900b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:55808/0 (socket says 172.21.15.32:55808) 2022-08-01T23:03:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.897+0000 7fde8f7fe700 1 -- 172.21.15.32:0/3715027901 learned_addr learned my addr 172.21.15.32:0/3715027901 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:44.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8ffff700 1 -- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde9013fa40 msgr2=0x7fde9013fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:44.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8ffff700 1 --2- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde9013fa40 0x7fde9013fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8ffff700 1 -- 172.21.15.32:0/3715027901 >> v1:172.21.15.55:6789/0 conn(0x7fde900af120 legacy=0x7fde900b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8ffff700 1 -- 172.21.15.32:0/3715027901 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900aec80 con 0x7fde900ad670 2022-08-01T23:03:44.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8ffff700 1 --2- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900ad670 0x7fde900b0570 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fde80002f70 tx=0x7fde80006e80).ready entity=mon.1 client_cookie=de97219f2de7cb6b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:44.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8effd700 1 -- 172.21.15.32:0/3715027901 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde80014070 con 0x7fde900ad670 2022-08-01T23:03:44.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8effd700 1 -- 172.21.15.32:0/3715027901 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80000e90 con 0x7fde900ad670 2022-08-01T23:03:44.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde8effd700 1 -- 172.21.15.32:0/3715027901 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde800107f0 con 0x7fde900ad670 2022-08-01T23:03:44.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 -- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900ad670 msgr2=0x7fde900b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 --2- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900ad670 0x7fde900b0570 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fde80002f70 tx=0x7fde80006e80).stop 2022-08-01T23:03:44.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 -- 172.21.15.32:0/3715027901 shutdown_connections 2022-08-01T23:03:44.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 --2- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde9013fa40 0x7fde9013fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 --2- 172.21.15.32:0/3715027901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900ad670 0x7fde900b0570 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 -- 172.21.15.32:0/3715027901 >> 172.21.15.32:0/3715027901 conn(0x7fde90061360 msgr2=0x7fde90062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:44.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 -- 172.21.15.32:0/3715027901 shutdown_connections 2022-08-01T23:03:44.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.901+0000 7fde95395700 1 -- 172.21.15.32:0/3715027901 wait complete. 2022-08-01T23:03:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 Processor -- start 2022-08-01T23:03:44.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 -- start start 2022-08-01T23:03:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde900ad670 0x7fde901404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900af120 0x7fde901514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 0x7fde901519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900a91f0 con 0x7fde9013fa40 2022-08-01T23:03:44.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde90066ae0 con 0x7fde900af120 2022-08-01T23:03:44.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde95395700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900a8f50 con 0x7fde900ad670 2022-08-01T23:03:44.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 0x7fde901519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 0x7fde901519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47370/0 (socket says 172.21.15.32:47370) 2022-08-01T23:03:44.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900af120 0x7fde901514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 -- 172.21.15.32:0/3422725461 learned_addr learned my addr 172.21.15.32:0/3422725461 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:44.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde8ffff700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde900ad670 0x7fde901404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 -- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde900ad670 msgr2=0x7fde901404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde900ad670 0x7fde901404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 -- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900af120 msgr2=0x7fde901514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:44.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900af120 0x7fde901514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:44.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 -- 172.21.15.32:0/3422725461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8000c310 con 0x7fde9013fa40 2022-08-01T23:03:44.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde8f7fe700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900af120 0x7fde901514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.905+0000 7fde94b94700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 0x7fde901519a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fde88006e00 tx=0x7fde880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde88011920 con 0x7fde9013fa40 2022-08-01T23:03:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde88011a80 con 0x7fde9013fa40 2022-08-01T23:03:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde90066d40 con 0x7fde9013fa40 2022-08-01T23:03:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde88011de0 con 0x7fde9013fa40 2022-08-01T23:03:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde90063c40 con 0x7fde9013fa40 2022-08-01T23:03:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde90066ed0 con 0x7fde9013fa40 2022-08-01T23:03:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fde8801a470 con 0x7fde9013fa40 2022-08-01T23:03:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8d7fa700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde70053a70 0x7fde70055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fde8806e690 con 0x7fde9013fa40 2022-08-01T23:03:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8ffff700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde70053a70 0x7fde70055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:44.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.909+0000 7fde8ffff700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde70053a70 0x7fde70055f30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fde8001d000 tx=0x7fde900a9770).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:44.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:44.917+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde880453a0 con 0x7fde9013fa40 2022-08-01T23:03:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.081+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fde900af590 con 0x7fde9013fa40 2022-08-01T23:03:45.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.757+0000 7fde8d7fa700 1 -- 172.21.15.32:0/3422725461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde880489c0 con 0x7fde9013fa40 2022-08-01T23:03:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde70053a70 msgr2=0x7fde70055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde70053a70 0x7fde70055f30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fde8001d000 tx=0x7fde900a9770).stop 2022-08-01T23:03:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 msgr2=0x7fde901519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 0x7fde901519a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fde88006e00 tx=0x7fde880091d0).stop 2022-08-01T23:03:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 shutdown_connections 2022-08-01T23:03:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fde70053a70 0x7fde70055f30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9013fa40 0x7fde901519a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde900af120 0x7fde901514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 --2- 172.21.15.32:0/3422725461 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fde900ad670 0x7fde901404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 >> 172.21.15.32:0/3422725461 conn(0x7fde90061360 msgr2=0x7fde9013cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:45.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 shutdown_connections 2022-08-01T23:03:45.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.761+0000 7fde95395700 1 -- 172.21.15.32:0/3422725461 wait complete. 2022-08-01T23:03:45.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:03:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.901+0000 7fa93bfff700 1 Processor -- start 2022-08-01T23:03:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.901+0000 7fa93bfff700 1 -- start start 2022-08-01T23:03:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa93c0a9320 0x7fa93c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c0b12d0 con 0x7fa93c0b0d10 2022-08-01T23:03:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c0a9700 con 0x7fa93c0b0560 2022-08-01T23:03:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c0b1500 con 0x7fa93c0a9320 2022-08-01T23:03:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa9327fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa93c0b0d10 0x7fa93c146bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34918/0 (socket says 172.21.15.32:34918) 2022-08-01T23:03:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa9327fc700 1 -- 172.21.15.32:0/1952911541 learned_addr learned my addr 172.21.15.32:0/1952911541 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93affd700 1 --2- 172.21.15.32:0/1952911541 >> v2:172.21.15.55:3300/0 conn(0x7fa93c0a9320 0x7fa93c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654627034 0 0) 0x7fa93c0b12d0 con 0x7fa93c0b0d10 2022-08-01T23:03:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003260 con 0x7fa93c0b0d10 2022-08-01T23:03:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395525508 0 0) 0x7fa93c0a9700 con 0x7fa93c0b0560 2022-08-01T23:03:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93c0b12d0 con 0x7fa93c0b0560 2022-08-01T23:03:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93affd700 1 -- 172.21.15.32:0/1952911541 >> v1:172.21.15.55:6789/0 conn(0x7fa93c0b0560 legacy=0x7fa93c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93affd700 1 -- 172.21.15.32:0/1952911541 >> v1:172.21.15.32:6789/0 conn(0x7fa93c0b0d10 legacy=0x7fa93c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93affd700 1 -- 172.21.15.32:0/1952911541 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0b17a0 con 0x7fa93c0a9320 2022-08-01T23:03:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93affd700 1 --2- 172.21.15.32:0/1952911541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c1466e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fa92c002f70 tx=0x7fa92c006df0).ready entity=mon.1 client_cookie=9e7f730908b9155 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa92c00d540 con 0x7fa93c0a9320 2022-08-01T23:03:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c000e90 con 0x7fa93c0a9320 2022-08-01T23:03:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93a7fc700 1 -- 172.21.15.32:0/1952911541 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa92c00d890 con 0x7fa93c0a9320 2022-08-01T23:03:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- 172.21.15.32:0/1952911541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 msgr2=0x7fa93c1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 --2- 172.21.15.32:0/1952911541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c1466e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fa92c002f70 tx=0x7fa92c006df0).stop 2022-08-01T23:03:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- 172.21.15.32:0/1952911541 shutdown_connections 2022-08-01T23:03:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 --2- 172.21.15.32:0/1952911541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c1466e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- 172.21.15.32:0/1952911541 >> 172.21.15.32:0/1952911541 conn(0x7fa93c0612d0 msgr2=0x7fa93c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- 172.21.15.32:0/1952911541 shutdown_connections 2022-08-01T23:03:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.905+0000 7fa93bfff700 1 -- 172.21.15.32:0/1952911541 wait complete. 2022-08-01T23:03:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 Processor -- start 2022-08-01T23:03:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 -- start start 2022-08-01T23:03:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa93c0b0560 0x7fa93c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa93c0b0d10 0x7fa93c13b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0aa740 con 0x7fa93c0b0560 2022-08-01T23:03:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0aa9e0 con 0x7fa93c0a9320 2022-08-01T23:03:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0aac80 con 0x7fa93c0b0d10 2022-08-01T23:03:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa9327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa93c0b0560 0x7fa93c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57166/0 (socket says 172.21.15.32:57166) 2022-08-01T23:03:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa9327fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa93c0b0560 0x7fa93c156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47382/0 (socket says 172.21.15.32:47382) 2022-08-01T23:03:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 -- 172.21.15.32:0/3314378525 learned_addr learned my addr 172.21.15.32:0/3314378525 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93b7fe700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa93c0b0d10 0x7fa93c13b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 -- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa93c0b0d10 msgr2=0x7fa93c13b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa93c0b0d10 0x7fa93c13b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 -- 172.21.15.32:0/3314378525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa93c0b0560 msgr2=0x7fa93c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa93c0b0560 0x7fa93c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 -- 172.21.15.32:0/3314378525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c00b180 con 0x7fa93c0a9320 2022-08-01T23:03:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93affd700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c145b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fa92c002f70 tx=0x7fa92c014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa92c00d1c0 con 0x7fa93c0a9320 2022-08-01T23:03:45.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c013bf0 con 0x7fa93c0a9320 2022-08-01T23:03:45.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa92c01e400 con 0x7fa93c0a9320 2022-08-01T23:03:45.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c0aaf20 con 0x7fa93c0a9320 2022-08-01T23:03:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.909+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c156880 con 0x7fa93c0a9320 2022-08-01T23:03:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.913+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c0574e0 con 0x7fa93c0a9320 2022-08-01T23:03:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.913+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa92c01c070 con 0x7fa93c0a9320 2022-08-01T23:03:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.913+0000 7fa938ff9700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa920053b30 0x7fa920055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.913+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa92c06ca10 con 0x7fa93c0a9320 2022-08-01T23:03:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.913+0000 7fa9327fc700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa920053b30 0x7fa920055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.913+0000 7fa9327fc700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa920053b30 0x7fa920055ff0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fa92400d000 tx=0x7fa924009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:45.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:45.921+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa92c043720 con 0x7fa93c0a9320 2022-08-01T23:03:46.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.049+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa93c1475c0 con 0x7fa93c0a9320 2022-08-01T23:03:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.769+0000 7fa938ff9700 1 -- 172.21.15.32:0/3314378525 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa92c023020 con 0x7fa93c0a9320 2022-08-01T23:03:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa920053b30 msgr2=0x7fa920055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa920053b30 0x7fa920055ff0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fa92400d000 tx=0x7fa924009040).stop 2022-08-01T23:03:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 msgr2=0x7fa93c145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c145b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fa92c002f70 tx=0x7fa92c014000).stop 2022-08-01T23:03:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 shutdown_connections 2022-08-01T23:03:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa920053b30 0x7fa920055ff0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa93c0b0d10 0x7fa93c13b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa93c0b0560 0x7fa93c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 --2- 172.21.15.32:0/3314378525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa93c0a9320 0x7fa93c145b00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 >> 172.21.15.32:0/3314378525 conn(0x7fa93c0612d0 msgr2=0x7fa93c141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:46.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 shutdown_connections 2022-08-01T23:03:46.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.773+0000 7fa93bfff700 1 -- 172.21.15.32:0/3314378525 wait complete. 2022-08-01T23:03:46.781 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:03:46.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 Processor -- start 2022-08-01T23:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 -- start start 2022-08-01T23:03:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80614b0 0x7fbbc8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbbc81405d0 0x7fbbc8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80aee80 con 0x7fbbc80ad750 2022-08-01T23:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80b70b0 con 0x7fbbc81405d0 2022-08-01T23:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80af0b0 con 0x7fbbc80614b0 2022-08-01T23:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbbdd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbbc80ad750 0x7fbbc80b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34932/0 (socket says 172.21.15.32:34932) 2022-08-01T23:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbbdd9b700 1 -- 172.21.15.32:0/2848807982 learned_addr learned my addr 172.21.15.32:0/2848807982 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc6d9d700 1 --2- 172.21.15.32:0/2848807982 >> v2:172.21.15.55:3300/0 conn(0x7fbbc81405d0 0x7fbbc8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc659c700 1 --2- 172.21.15.32:0/2848807982 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80614b0 0x7fbbc8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc5d9b700 1 -- 172.21.15.32:0/2848807982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439097650 0 0) 0x7fbbc80aee80 con 0x7fbbc80ad750 2022-08-01T23:03:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc5d9b700 1 -- 172.21.15.32:0/2848807982 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba8003260 con 0x7fbbc80ad750 2022-08-01T23:03:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc659c700 1 -- 172.21.15.32:0/2848807982 >> v2:172.21.15.55:3300/0 conn(0x7fbbc81405d0 msgr2=0x7fbbc8140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc659c700 1 --2- 172.21.15.32:0/2848807982 >> v2:172.21.15.55:3300/0 conn(0x7fbbc81405d0 0x7fbbc8140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc659c700 1 -- 172.21.15.32:0/2848807982 >> v1:172.21.15.32:6789/0 conn(0x7fbbc80ad750 legacy=0x7fbbc80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc659c700 1 -- 172.21.15.32:0/2848807982 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80af350 con 0x7fbbc80614b0 2022-08-01T23:03:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.897+0000 7fbbc6d9d700 1 --2- 172.21.15.32:0/2848807982 >> v2:172.21.15.55:3300/0 conn(0x7fbbc81405d0 0x7fbbc8140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc659c700 1 --2- 172.21.15.32:0/2848807982 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80614b0 0x7fbbc8061890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fbbb8002f70 tx=0x7fbbb8006df0).ready entity=mon.2 client_cookie=cf8e4df4d10a4d36 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc5d9b700 1 -- 172.21.15.32:0/2848807982 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbb800d4a0 con 0x7fbbc80614b0 2022-08-01T23:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc5d9b700 1 -- 172.21.15.32:0/2848807982 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8000e90 con 0x7fbbc80614b0 2022-08-01T23:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc5d9b700 1 -- 172.21.15.32:0/2848807982 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbb800d7f0 con 0x7fbbc80614b0 2022-08-01T23:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- 172.21.15.32:0/2848807982 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80614b0 msgr2=0x7fbbc8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 --2- 172.21.15.32:0/2848807982 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80614b0 0x7fbbc8061890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fbbb8002f70 tx=0x7fbbb8006df0).stop 2022-08-01T23:03:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- 172.21.15.32:0/2848807982 shutdown_connections 2022-08-01T23:03:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 --2- 172.21.15.32:0/2848807982 >> v2:172.21.15.55:3300/0 conn(0x7fbbc81405d0 0x7fbbc8140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 --2- 172.21.15.32:0/2848807982 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80614b0 0x7fbbc8061890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- 172.21.15.32:0/2848807982 >> 172.21.15.32:0/2848807982 conn(0x7fbbc8066f70 msgr2=0x7fbbc8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- 172.21.15.32:0/2848807982 shutdown_connections 2022-08-01T23:03:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- 172.21.15.32:0/2848807982 wait complete. 2022-08-01T23:03:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 Processor -- start 2022-08-01T23:03:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- start start 2022-08-01T23:03:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc80614b0 0x7fbbc8155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80ad750 0x7fbbc80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 0x7fbbc80ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80b2500 con 0x7fbbc81405d0 2022-08-01T23:03:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc8145fd0 con 0x7fbbc80614b0 2022-08-01T23:03:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80b2730 con 0x7fbbc80ad750 2022-08-01T23:03:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 0x7fbbc80ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 0x7fbbc80ac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47396/0 (socket says 172.21.15.32:47396) 2022-08-01T23:03:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc6d9d700 1 -- 172.21.15.32:0/1812640387 learned_addr learned my addr 172.21.15.32:0/1812640387 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbc659c700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc80614b0 0x7fbbc8155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.901+0000 7fbbbdd9b700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80ad750 0x7fbbc80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc6d9d700 1 -- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80ad750 msgr2=0x7fbbc80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc6d9d700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80ad750 0x7fbbc80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc6d9d700 1 -- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc80614b0 msgr2=0x7fbbc8155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc6d9d700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc80614b0 0x7fbbc8155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc6d9d700 1 -- 172.21.15.32:0/1812640387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009470 con 0x7fbbc81405d0 2022-08-01T23:03:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc6d9d700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 0x7fbbc80ac620 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7fbbc000c000 tx=0x7fbbc00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbc000ee60 con 0x7fbbc81405d0 2022-08-01T23:03:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc000c850 con 0x7fbbc81405d0 2022-08-01T23:03:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb800b180 con 0x7fbbc81405d0 2022-08-01T23:03:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbc000cbb0 con 0x7fbbc81405d0 2022-08-01T23:03:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc80b3c00 con 0x7fbbc81405d0 2022-08-01T23:03:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc8156340 con 0x7fbbc81405d0 2022-08-01T23:03:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbbc0022070 con 0x7fbbc81405d0 2022-08-01T23:03:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbbffff700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbac053950 0x7fbbac055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fbbc006c5b0 con 0x7fbbc81405d0 2022-08-01T23:03:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.905+0000 7fbbc659c700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbac053950 0x7fbbac055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:46.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.909+0000 7fbbc659c700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbac053950 0x7fbbac055e10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fbbb801a000 tx=0x7fbbb8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:46.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:46.913+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbbc00434d0 con 0x7fbbc81405d0 2022-08-01T23:03:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.041+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbc80af5f0 con 0x7fbbc81405d0 2022-08-01T23:03:47.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.769+0000 7fbbbffff700 1 -- 172.21.15.32:0/1812640387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbbc0046af0 con 0x7fbbc81405d0 2022-08-01T23:03:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbac053950 msgr2=0x7fbbac055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbac053950 0x7fbbac055e10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fbbb801a000 tx=0x7fbbb8014000).stop 2022-08-01T23:03:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 msgr2=0x7fbbc80ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 0x7fbbc80ac620 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7fbbc000c000 tx=0x7fbbc00091d0).stop 2022-08-01T23:03:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 shutdown_connections 2022-08-01T23:03:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbbac053950 0x7fbbac055e10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbc81405d0 0x7fbbc80ac620 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbbc80ad750 0x7fbbc80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 --2- 172.21.15.32:0/1812640387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc80614b0 0x7fbbc8155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 >> 172.21.15.32:0/1812640387 conn(0x7fbbc8066f70 msgr2=0x7fbbc805cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 shutdown_connections 2022-08-01T23:03:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.773+0000 7fbbc759e700 1 -- 172.21.15.32:0/1812640387 wait complete. 2022-08-01T23:03:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:03:47.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:03:47.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e960bc700 1 Processor -- start 2022-08-01T23:03:47.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e960bc700 1 -- start start 2022-08-01T23:03:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e960bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e900b3070 0x7f5e90065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e960bc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e900b2c20 con 0x7f5e900b7840 2022-08-01T23:03:47.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e960bc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e900b7c20 con 0x7f5e9014b040 2022-08-01T23:03:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e960bc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e90061830 con 0x7f5e900b3070 2022-08-01T23:03:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e950ba700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5e900b7840 0x7f5e90061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2022-08-01T23:03:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e950ba700 1 -- 172.21.15.32:0/684407219 learned_addr learned my addr 172.21.15.32:0/684407219 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e948b9700 1 --2- 172.21.15.32:0/684407219 >> v2:172.21.15.32:3300/0 conn(0x7f5e900b3070 0x7f5e90065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e87fff700 1 -- 172.21.15.32:0/684407219 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960061800 0 0) 0x7f5e900b2c20 con 0x7f5e900b7840 2022-08-01T23:03:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e87fff700 1 -- 172.21.15.32:0/684407219 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003260 con 0x7f5e900b7840 2022-08-01T23:03:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e948b9700 1 -- 172.21.15.32:0/684407219 >> v1:172.21.15.55:6789/0 conn(0x7f5e9014b040 legacy=0x7f5e9014b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e948b9700 1 -- 172.21.15.32:0/684407219 >> v1:172.21.15.32:6789/0 conn(0x7f5e900b7840 legacy=0x7f5e90061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e948b9700 1 -- 172.21.15.32:0/684407219 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900b2980 con 0x7f5e900b3070 2022-08-01T23:03:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e948b9700 1 --2- 172.21.15.32:0/684407219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b3070 0x7f5e90065720 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f5e80002f70 tx=0x7f5e80006df0).ready entity=mon.0 client_cookie=d153c658e0ba47ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e87fff700 1 -- 172.21.15.32:0/684407219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8000d5c0 con 0x7f5e900b3070 2022-08-01T23:03:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e87fff700 1 -- 172.21.15.32:0/684407219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80000e90 con 0x7f5e900b3070 2022-08-01T23:03:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.905+0000 7f5e87fff700 1 -- 172.21.15.32:0/684407219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8000d910 con 0x7f5e900b3070 2022-08-01T23:03:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- 172.21.15.32:0/684407219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b3070 msgr2=0x7f5e90065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 --2- 172.21.15.32:0/684407219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b3070 0x7f5e90065720 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f5e80002f70 tx=0x7f5e80006df0).stop 2022-08-01T23:03:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- 172.21.15.32:0/684407219 shutdown_connections 2022-08-01T23:03:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 --2- 172.21.15.32:0/684407219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b3070 0x7f5e90065720 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- 172.21.15.32:0/684407219 >> 172.21.15.32:0/684407219 conn(0x7f5e9005f840 msgr2=0x7f5e9005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- 172.21.15.32:0/684407219 shutdown_connections 2022-08-01T23:03:47.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- 172.21.15.32:0/684407219 wait complete. 2022-08-01T23:03:47.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 Processor -- start 2022-08-01T23:03:47.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- start start 2022-08-01T23:03:47.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e900b3070 0x7f5e9005c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:47.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 0x7f5e900a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9014b040 0x7f5e900a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9009eb50 con 0x7f5e900b7840 2022-08-01T23:03:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900b0940 con 0x7f5e900b3070 2022-08-01T23:03:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.909+0000 7f5e960bc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900b06a0 con 0x7f5e9014b040 2022-08-01T23:03:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 0x7f5e900a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 0x7f5e900a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47410/0 (socket says 172.21.15.32:47410) 2022-08-01T23:03:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 -- 172.21.15.32:0/3059420678 learned_addr learned my addr 172.21.15.32:0/3059420678 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e950ba700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e900b3070 0x7f5e9005c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e958bb700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9014b040 0x7f5e900a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 -- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9014b040 msgr2=0x7f5e900a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9014b040 0x7f5e900a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 -- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e900b3070 msgr2=0x7f5e9005c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e900b3070 0x7f5e9005c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 -- 172.21.15.32:0/3059420678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8000b180 con 0x7f5e900b7840 2022-08-01T23:03:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e948b9700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 0x7f5e900a06e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f5e80002f70 tx=0x7f5e80014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e80016070 con 0x7f5e900b7840 2022-08-01T23:03:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8000de40 con 0x7f5e900b7840 2022-08-01T23:03:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9009edb0 con 0x7f5e900b7840 2022-08-01T23:03:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e80013da0 con 0x7f5e900b7840 2022-08-01T23:03:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9009f250 con 0x7f5e900b7840 2022-08-01T23:03:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.913+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e900609e0 con 0x7f5e900b7840 2022-08-01T23:03:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.917+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e80025030 con 0x7f5e900b7840 2022-08-01T23:03:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.917+0000 7f5e867fc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053b00 0x7f5e78055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.917+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5e8006d160 con 0x7f5e900b7840 2022-08-01T23:03:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.917+0000 7f5e950ba700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053b00 0x7f5e78055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.917+0000 7f5e950ba700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053b00 0x7f5e78055fc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f5e8c00d000 tx=0x7f5e8c0027f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:47.921+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e800440f0 con 0x7f5e900b7840 2022-08-01T23:03:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.057+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5e900574e0 con 0x7f5e900b7840 2022-08-01T23:03:48.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e867fc700 1 -- 172.21.15.32:0/3059420678 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5e800230a0 con 0x7f5e900b7840 2022-08-01T23:03:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053b00 msgr2=0x7f5e78055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053b00 0x7f5e78055fc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f5e8c00d000 tx=0x7f5e8c0027f0).stop 2022-08-01T23:03:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 msgr2=0x7f5e900a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 0x7f5e900a06e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f5e80002f70 tx=0x7f5e80014000).stop 2022-08-01T23:03:48.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 shutdown_connections 2022-08-01T23:03:48.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5e78053b00 0x7f5e78055fc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5e9014b040 0x7f5e900a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e900b7840 0x7f5e900a06e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 --2- 172.21.15.32:0/3059420678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e900b3070 0x7f5e9005c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 >> 172.21.15.32:0/3059420678 conn(0x7f5e9005f840 msgr2=0x7f5e900b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:48.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 shutdown_connections 2022-08-01T23:03:48.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.061+0000 7f5e960bc700 1 -- 172.21.15.32:0/3059420678 wait complete. 2022-08-01T23:03:48.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-08-01T23:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.173+0000 7f9cfcf94700 1 Processor -- start 2022-08-01T23:03:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.173+0000 7f9cfcf94700 1 -- start start 2022-08-01T23:03:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9cf80aebd0 0x7f9cf8143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cf80b0610 0x7f9cf8143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80b0b20 con 0x7f9cf80ad190 2022-08-01T23:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80adde0 con 0x7f9cf80b0610 2022-08-01T23:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80aefb0 con 0x7f9cf80aebd0 2022-08-01T23:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9cf80ad190 0x7f9cf8146cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34958/0 (socket says 172.21.15.32:34958) 2022-08-01T23:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf6ffd700 1 -- 172.21.15.32:0/2723285556 learned_addr learned my addr 172.21.15.32:0/2723285556 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf7fff700 1 --2- 172.21.15.32:0/2723285556 >> v2:172.21.15.32:3300/0 conn(0x7f9cf80b0610 0x7f9cf8143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf77fe700 1 --2- 172.21.15.32:0/2723285556 >> v2:172.21.15.55:3300/0 conn(0x7f9cf80aebd0 0x7f9cf8143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf67fc700 1 -- 172.21.15.32:0/2723285556 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365690080 0 0) 0x7f9cf80b0b20 con 0x7f9cf80ad190 2022-08-01T23:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf67fc700 1 -- 172.21.15.32:0/2723285556 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce4003260 con 0x7f9cf80ad190 2022-08-01T23:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf7fff700 1 -- 172.21.15.32:0/2723285556 >> v2:172.21.15.55:3300/0 conn(0x7f9cf80aebd0 msgr2=0x7f9cf8143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf7fff700 1 --2- 172.21.15.32:0/2723285556 >> v2:172.21.15.55:3300/0 conn(0x7f9cf80aebd0 0x7f9cf8143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf7fff700 1 -- 172.21.15.32:0/2723285556 >> v1:172.21.15.32:6789/0 conn(0x7f9cf80ad190 legacy=0x7f9cf8146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf7fff700 1 -- 172.21.15.32:0/2723285556 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8146890 con 0x7f9cf80b0610 2022-08-01T23:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf7fff700 1 --2- 172.21.15.32:0/2723285556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf8143e30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f9cf00032f0 tx=0x7f9cf00091d0).ready entity=mon.0 client_cookie=a6045d0fb8e48c76 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:48.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf67fc700 1 -- 172.21.15.32:0/2723285556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cf0014e90 con 0x7f9cf80b0610 2022-08-01T23:03:48.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf67fc700 1 -- 172.21.15.32:0/2723285556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf001c420 con 0x7f9cf80b0610 2022-08-01T23:03:48.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cf67fc700 1 -- 172.21.15.32:0/2723285556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cf001c780 con 0x7f9cf80b0610 2022-08-01T23:03:48.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- 172.21.15.32:0/2723285556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 msgr2=0x7f9cf8143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/2723285556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf8143e30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f9cf00032f0 tx=0x7f9cf00091d0).stop 2022-08-01T23:03:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- 172.21.15.32:0/2723285556 shutdown_connections 2022-08-01T23:03:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/2723285556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf8143e30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/2723285556 >> v2:172.21.15.55:3300/0 conn(0x7f9cf80aebd0 0x7f9cf8143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- 172.21.15.32:0/2723285556 >> 172.21.15.32:0/2723285556 conn(0x7f9cf80613c0 msgr2=0x7f9cf8062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:48.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- 172.21.15.32:0/2723285556 shutdown_connections 2022-08-01T23:03:48.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.177+0000 7f9cfcf94700 1 -- 172.21.15.32:0/2723285556 wait complete. 2022-08-01T23:03:48.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.181+0000 7f9cfcf94700 1 Processor -- start 2022-08-01T23:03:48.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.181+0000 7f9cfcf94700 1 -- start start 2022-08-01T23:03:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9cf80ad190 0x7f9cf8133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf80aebd0 0x7f9cf813a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf813f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81409e0 con 0x7f9cf80b0610 2022-08-01T23:03:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8137b40 con 0x7f9cf80aebd0 2022-08-01T23:03:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80b1f00 con 0x7f9cf80ad190 2022-08-01T23:03:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf813f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf813f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47426/0 (socket says 172.21.15.32:47426) 2022-08-01T23:03:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 -- 172.21.15.32:0/3771585905 learned_addr learned my addr 172.21.15.32:0/3771585905 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf6ffd700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf80aebd0 0x7f9cf813a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf77fe700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9cf80ad190 0x7f9cf8133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 -- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9cf80ad190 msgr2=0x7f9cf8133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9cf80ad190 0x7f9cf8133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 -- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf80aebd0 msgr2=0x7f9cf813a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf80aebd0 0x7f9cf813a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 -- 172.21.15.32:0/3771585905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009470 con 0x7f9cf80b0610 2022-08-01T23:03:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf77fe700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9cf80ad190 0x7f9cf8133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf7fff700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf813f900 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f9cf8140b20 tx=0x7f9cf001c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cf0022070 con 0x7f9cf80b0610 2022-08-01T23:03:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf001b950 con 0x7f9cf80b0610 2022-08-01T23:03:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf8137360 con 0x7f9cf80b0610 2022-08-01T23:03:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cf001bcb0 con 0x7f9cf80b0610 2022-08-01T23:03:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf81378a0 con 0x7f9cf80b0610 2022-08-01T23:03:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf80b0d50 con 0x7f9cf80b0610 2022-08-01T23:03:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.185+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9cf002d030 con 0x7f9cf80b0610 2022-08-01T23:03:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.189+0000 7f9cf4ff9700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cd8053b00 0x7f9cd8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.189+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9cf0075470 con 0x7f9cf80b0610 2022-08-01T23:03:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.189+0000 7f9cf77fe700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cd8053b00 0x7f9cd8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.189+0000 7f9cf77fe700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cd8053b00 0x7f9cd8055fc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f9cec00b000 tx=0x7f9cec008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.197+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9cf004c550 con 0x7f9cf80b0610 2022-08-01T23:03:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.357+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f9cf80574e0 con 0x7f9cf80b0610 2022-08-01T23:03:48.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.769+0000 7f9cf4ff9700 1 -- 172.21.15.32:0/3771585905 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v357) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9cf004fb30 con 0x7f9cf80b0610 2022-08-01T23:03:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.773+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cd8053b00 msgr2=0x7f9cd8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.773+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cd8053b00 0x7f9cd8055fc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f9cec00b000 tx=0x7f9cec008040).stop 2022-08-01T23:03:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.773+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 msgr2=0x7f9cf813f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.773+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf813f900 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f9cf8140b20 tx=0x7f9cf001c000).stop 2022-08-01T23:03:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 shutdown_connections 2022-08-01T23:03:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9cd8053b00 0x7f9cd8055fc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf80b0610 0x7f9cf813f900 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf80aebd0 0x7f9cf813a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 --2- 172.21.15.32:0/3771585905 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9cf80ad190 0x7f9cf8133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 >> 172.21.15.32:0/3771585905 conn(0x7f9cf80613c0 msgr2=0x7f9cf8144880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 shutdown_connections 2022-08-01T23:03:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.777+0000 7f9cfcf94700 1 -- 172.21.15.32:0/3771585905 wait complete. 2022-08-01T23:03:48.797 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:03:48.798 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:03:48.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-08-01T23:03:48.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 Processor -- start 2022-08-01T23:03:48.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 -- start start 2022-08-01T23:03:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0340647d0 0x7ff034064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0340b38a0 0x7ff0341470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340b4f60 con 0x7ff0341436f0 2022-08-01T23:03:48.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0341475c0 con 0x7ff0340647d0 2022-08-01T23:03:48.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.901+0000 7ff03a167700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340b4a60 con 0x7ff0340b38a0 2022-08-01T23:03:48.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039165700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0340647d0 0x7ff034064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039966700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0341436f0 0x7ff0341403f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34976/0 (socket says 172.21.15.32:34976) 2022-08-01T23:03:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039966700 1 -- 172.21.15.32:0/2335719206 learned_addr learned my addr 172.21.15.32:0/2335719206 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff038964700 1 --2- 172.21.15.32:0/2335719206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0340b38a0 0x7ff0341470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff02bfff700 1 -- 172.21.15.32:0/2335719206 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943651688 0 0) 0x7ff0340b4f60 con 0x7ff0341436f0 2022-08-01T23:03:48.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039165700 1 -- 172.21.15.32:0/2335719206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0340b38a0 msgr2=0x7ff0341470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039165700 1 --2- 172.21.15.32:0/2335719206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0340b38a0 0x7ff0341470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039165700 1 -- 172.21.15.32:0/2335719206 >> v1:172.21.15.32:6789/0 conn(0x7ff0341436f0 legacy=0x7ff0341403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039165700 1 -- 172.21.15.32:0/2335719206 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340b4cc0 con 0x7ff0340647d0 2022-08-01T23:03:48.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff039165700 1 --2- 172.21.15.32:0/2335719206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff034064bb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024006df0).ready entity=mon.0 client_cookie=20b04ccbaa44c5c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff02bfff700 1 -- 172.21.15.32:0/2335719206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02400d570 con 0x7ff0340647d0 2022-08-01T23:03:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff02bfff700 1 -- 172.21.15.32:0/2335719206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff024000e90 con 0x7ff0340647d0 2022-08-01T23:03:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff02bfff700 1 -- 172.21.15.32:0/2335719206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02400d8c0 con 0x7ff0340647d0 2022-08-01T23:03:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 -- 172.21.15.32:0/2335719206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 msgr2=0x7ff034064bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 --2- 172.21.15.32:0/2335719206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff034064bb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024006df0).stop 2022-08-01T23:03:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 -- 172.21.15.32:0/2335719206 shutdown_connections 2022-08-01T23:03:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 --2- 172.21.15.32:0/2335719206 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0340b38a0 0x7ff0341470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 --2- 172.21.15.32:0/2335719206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff034064bb0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 -- 172.21.15.32:0/2335719206 >> 172.21.15.32:0/2335719206 conn(0x7ff03405f840 msgr2=0x7ff03405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:48.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 -- 172.21.15.32:0/2335719206 shutdown_connections 2022-08-01T23:03:48.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.905+0000 7ff03a167700 1 -- 172.21.15.32:0/2335719206 wait complete. 2022-08-01T23:03:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 Processor -- start 2022-08-01T23:03:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 -- start start 2022-08-01T23:03:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff03413ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0340b38a0 0x7ff0341457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0341436f0 0x7ff034145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340aec60 con 0x7ff0340647d0 2022-08-01T23:03:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034144fb0 con 0x7ff0340b38a0 2022-08-01T23:03:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340aee90 con 0x7ff0341436f0 2022-08-01T23:03:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff03413ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff03413ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47436/0 (socket says 172.21.15.32:47436) 2022-08-01T23:03:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 -- 172.21.15.32:0/1397376134 learned_addr learned my addr 172.21.15.32:0/1397376134 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039966700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0341436f0 0x7ff034145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff038964700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0340b38a0 0x7ff0341457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 -- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0341436f0 msgr2=0x7ff034145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0341436f0 0x7ff034145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 -- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0340b38a0 msgr2=0x7ff0341457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0340b38a0 0x7ff0341457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 -- 172.21.15.32:0/1397376134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02400b180 con 0x7ff0340647d0 2022-08-01T23:03:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff038964700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0340b38a0 0x7ff0341457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff039165700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff03413ee80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02401e400 con 0x7ff0340647d0 2022-08-01T23:03:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02401e560 con 0x7ff0340647d0 2022-08-01T23:03:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034145250 con 0x7ff0340647d0 2022-08-01T23:03:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02401e8c0 con 0x7ff0340647d0 2022-08-01T23:03:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.909+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0341461c0 con 0x7ff0340647d0 2022-08-01T23:03:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.913+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034143250 con 0x7ff0340647d0 2022-08-01T23:03:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.913+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff024025030 con 0x7ff0340647d0 2022-08-01T23:03:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.913+0000 7ff02a7fc700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff01c053a10 0x7ff01c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.913+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff02406cbe0 con 0x7ff0340647d0 2022-08-01T23:03:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.913+0000 7ff038964700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff01c053a10 0x7ff01c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.913+0000 7ff038964700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff01c053a10 0x7ff01c055ed0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7ff03000b000 tx=0x7ff030008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:48.917+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff024043c40 con 0x7ff0340647d0 2022-08-01T23:03:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.089+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7ff034069d90 con 0x7ff0340647d0 2022-08-01T23:03:49.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.773+0000 7ff02a7fc700 1 -- 172.21.15.32:0/1397376134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v358) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff024023070 con 0x7ff0340647d0 2022-08-01T23:03:49.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff01c053a10 msgr2=0x7ff01c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:49.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff01c053a10 0x7ff01c055ed0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7ff03000b000 tx=0x7ff030008040).stop 2022-08-01T23:03:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 msgr2=0x7ff03413ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff03413ee80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024014000).stop 2022-08-01T23:03:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 shutdown_connections 2022-08-01T23:03:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff01c053a10 0x7ff01c055ed0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff0341436f0 0x7ff034145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0340b38a0 0x7ff0341457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 --2- 172.21.15.32:0/1397376134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0340647d0 0x7ff03413ee80 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 >> 172.21.15.32:0/1397376134 conn(0x7ff03405f840 msgr2=0x7ff03405ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 shutdown_connections 2022-08-01T23:03:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.777+0000 7ff03a167700 1 -- 172.21.15.32:0/1397376134 wait complete. 2022-08-01T23:03:49.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:03:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 Processor -- start 2022-08-01T23:03:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 -- start start 2022-08-01T23:03:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9da80ad750 0x7f9da8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80aee80 con 0x7f9da80614b0 2022-08-01T23:03:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da80b70b0 con 0x7f9da80ad750 2022-08-01T23:03:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dade8f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80af0b0 con 0x7f9da81405d0 2022-08-01T23:03:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9da7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9da80ad750 0x7f9da8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dace8d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9da80614b0 0x7f9da80b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34988/0 (socket says 172.21.15.32:34988) 2022-08-01T23:03:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.913+0000 7f9dace8d700 1 -- 172.21.15.32:0/3067528363 learned_addr learned my addr 172.21.15.32:0/3067528363 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dad68e700 1 --2- 172.21.15.32:0/3067528363 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da7fff700 1 -- 172.21.15.32:0/3067528363 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 msgr2=0x7f9da8140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da7fff700 1 --2- 172.21.15.32:0/3067528363 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da8140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da7fff700 1 -- 172.21.15.32:0/3067528363 >> v1:172.21.15.32:6789/0 conn(0x7f9da80614b0 legacy=0x7f9da80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da77fe700 1 -- 172.21.15.32:0/3067528363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356394973 0 0) 0x7f9da80aee80 con 0x7f9da80614b0 2022-08-01T23:03:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da7fff700 1 -- 172.21.15.32:0/3067528363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da80af350 con 0x7f9da80ad750 2022-08-01T23:03:49.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da7fff700 1 --2- 172.21.15.32:0/3067528363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80ad750 0x7f9da8061890 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f9d98002f70 tx=0x7f9d98006df0).ready entity=mon.0 client_cookie=5cbb62a9b1a8e8dc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:49.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da77fe700 1 -- 172.21.15.32:0/3067528363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d9800d570 con 0x7f9da80ad750 2022-08-01T23:03:49.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da77fe700 1 -- 172.21.15.32:0/3067528363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d98000e90 con 0x7f9da80ad750 2022-08-01T23:03:49.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9da77fe700 1 -- 172.21.15.32:0/3067528363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d9800d8c0 con 0x7f9da80ad750 2022-08-01T23:03:49.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 -- 172.21.15.32:0/3067528363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80ad750 msgr2=0x7f9da8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:49.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3067528363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80ad750 0x7f9da8061890 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f9d98002f70 tx=0x7f9d98006df0).stop 2022-08-01T23:03:49.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 -- 172.21.15.32:0/3067528363 shutdown_connections 2022-08-01T23:03:49.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3067528363 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da8140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3067528363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80ad750 0x7f9da8061890 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 -- 172.21.15.32:0/3067528363 >> 172.21.15.32:0/3067528363 conn(0x7f9da8066f70 msgr2=0x7f9da8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:49.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 -- 172.21.15.32:0/3067528363 shutdown_connections 2022-08-01T23:03:49.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.917+0000 7f9dade8f700 1 -- 172.21.15.32:0/3067528363 wait complete. 2022-08-01T23:03:49.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 Processor -- start 2022-08-01T23:03:49.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 -- start start 2022-08-01T23:03:49.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 0x7f9da8152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:49.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9da80ad750 0x7f9da80b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:49.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da80b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:49.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8152f50 con 0x7f9da80614b0 2022-08-01T23:03:49.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80ab7f0 con 0x7f9da80ad750 2022-08-01T23:03:49.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80ab550 con 0x7f9da81405d0 2022-08-01T23:03:49.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 0x7f9da8152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:49.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 0x7f9da8152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47454/0 (socket says 172.21.15.32:47454) 2022-08-01T23:03:49.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 -- 172.21.15.32:0/3865904105 learned_addr learned my addr 172.21.15.32:0/3865904105 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:49.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9da7fff700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9da80ad750 0x7f9da80b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:49.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dad68e700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da80b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 -- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 msgr2=0x7f9da80b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da80b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 -- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9da80ad750 msgr2=0x7f9da80b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9da80ad750 0x7f9da80b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:49.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 -- 172.21.15.32:0/3865904105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9800b180 con 0x7f9da80614b0 2022-08-01T23:03:49.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dace8d700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 0x7f9da8152700 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f9d90000ed0 tx=0x7f9d90007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:49.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d900070d0 con 0x7f9da80614b0 2022-08-01T23:03:49.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90007230 con 0x7f9da80614b0 2022-08-01T23:03:49.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d90005610 con 0x7f9da80614b0 2022-08-01T23:03:49.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da8152bf0 con 0x7f9da80614b0 2022-08-01T23:03:49.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.921+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80aad40 con 0x7f9da80614b0 2022-08-01T23:03:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.925+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d90010070 con 0x7f9da80614b0 2022-08-01T23:03:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.925+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da8152d80 con 0x7f9da80614b0 2022-08-01T23:03:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.925+0000 7f9da5ffb700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d8c053a10 0x7f9d8c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.925+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9d9006b530 con 0x7f9da80614b0 2022-08-01T23:03:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.925+0000 7f9da7fff700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d8c053a10 0x7f9d8c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.925+0000 7f9da7fff700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d8c053a10 0x7f9d8c055ed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f9d9801a000 tx=0x7f9d98014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:49.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:49.933+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d900424c0 con 0x7f9da80614b0 2022-08-01T23:03:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.061+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9da80574e0 con 0x7f9da80614b0 2022-08-01T23:03:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.073+0000 7f9da5ffb700 1 -- 172.21.15.32:0/3865904105 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9d90045ae0 con 0x7f9da80614b0 2022-08-01T23:03:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d8c053a10 msgr2=0x7f9d8c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d8c053a10 0x7f9d8c055ed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f9d9801a000 tx=0x7f9d98014000).stop 2022-08-01T23:03:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 msgr2=0x7f9da8152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 0x7f9da8152700 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f9d90000ed0 tx=0x7f9d90007ac0).stop 2022-08-01T23:03:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 shutdown_connections 2022-08-01T23:03:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9d8c053a10 0x7f9d8c055ed0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9da81405d0 0x7f9da80b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9da80ad750 0x7f9da80b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 --2- 172.21.15.32:0/3865904105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80614b0 0x7f9da8152700 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 >> 172.21.15.32:0/3865904105 conn(0x7f9da8066f70 msgr2=0x7f9da80b8920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 shutdown_connections 2022-08-01T23:03:50.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.077+0000 7f9dade8f700 1 -- 172.21.15.32:0/3865904105 wait complete. 2022-08-01T23:03:50.095 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:03:50.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:50.111 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:50.133 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:50.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:03:50.174 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:03:50.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:50.175 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:50.175 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:03:50.183 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:03:50.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:03:50.185 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:03:50.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:03:50.187 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:03:50.188 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:03:50.223 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:03:50.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:03:50.226 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:03:50.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:03:50.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 Processor -- start 2022-08-01T23:03:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 -- start start 2022-08-01T23:03:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f15780b6510 0x7f157813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15780b0e60 0x7f157813d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157813dc80 con 0x7f1578064500 2022-08-01T23:03:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1578066af0 con 0x7f15780b0e60 2022-08-01T23:03:50.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1578066d90 con 0x7f15780b6510 2022-08-01T23:03:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1576ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15780b0e60 0x7f157813d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1577fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1578064500 0x7f15780609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35006/0 (socket says 172.21.15.32:35006) 2022-08-01T23:03:50.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1577fff700 1 -- 172.21.15.32:0/1875732129 learned_addr learned my addr 172.21.15.32:0/1875732129 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:50.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f15777fe700 1 --2- 172.21.15.32:0/1875732129 >> v2:172.21.15.55:3300/0 conn(0x7f15780b6510 0x7f157813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f15767fc700 1 -- 172.21.15.32:0/1875732129 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297457560 0 0) 0x7f157813dc80 con 0x7f1578064500 2022-08-01T23:03:50.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f15767fc700 1 -- 172.21.15.32:0/1875732129 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1558003260 con 0x7f1578064500 2022-08-01T23:03:50.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1576ffd700 1 -- 172.21.15.32:0/1875732129 >> v2:172.21.15.55:3300/0 conn(0x7f15780b6510 msgr2=0x7f157813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:50.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1576ffd700 1 --2- 172.21.15.32:0/1875732129 >> v2:172.21.15.55:3300/0 conn(0x7f15780b6510 0x7f157813d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1576ffd700 1 -- 172.21.15.32:0/1875732129 >> v1:172.21.15.32:6789/0 conn(0x7f1578064500 legacy=0x7f15780609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1576ffd700 1 -- 172.21.15.32:0/1875732129 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1578067030 con 0x7f15780b0e60 2022-08-01T23:03:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f1576ffd700 1 --2- 172.21.15.32:0/1875732129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15780b0e60 0x7f157813d790 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f15680032f0 tx=0x7f15680091d0).ready entity=mon.0 client_cookie=9481f979703f377 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f15767fc700 1 -- 172.21.15.32:0/1875732129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f156800b600 con 0x7f15780b0e60 2022-08-01T23:03:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f15767fc700 1 -- 172.21.15.32:0/1875732129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156800b760 con 0x7f15780b0e60 2022-08-01T23:03:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f15767fc700 1 -- 172.21.15.32:0/1875732129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f156800b8d0 con 0x7f15780b0e60 2022-08-01T23:03:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 -- 172.21.15.32:0/1875732129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15780b0e60 msgr2=0x7f157813d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.337+0000 7f157c97f700 1 --2- 172.21.15.32:0/1875732129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15780b0e60 0x7f157813d790 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f15680032f0 tx=0x7f15680091d0).stop 2022-08-01T23:03:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- 172.21.15.32:0/1875732129 shutdown_connections 2022-08-01T23:03:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 --2- 172.21.15.32:0/1875732129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15780b0e60 0x7f157813d790 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 --2- 172.21.15.32:0/1875732129 >> v2:172.21.15.55:3300/0 conn(0x7f15780b6510 0x7f157813d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- 172.21.15.32:0/1875732129 >> 172.21.15.32:0/1875732129 conn(0x7f1578063800 msgr2=0x7f1578063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- 172.21.15.32:0/1875732129 shutdown_connections 2022-08-01T23:03:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- 172.21.15.32:0/1875732129 wait complete. 2022-08-01T23:03:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 Processor -- start 2022-08-01T23:03:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- start start 2022-08-01T23:03:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 0x7f15780a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15780b0e60 0x7f15780a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f15780b6510 0x7f15780a2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780a1890 con 0x7f1578064500 2022-08-01T23:03:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780a1260 con 0x7f15780b0e60 2022-08-01T23:03:50.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15780a1500 con 0x7f15780b6510 2022-08-01T23:03:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 0x7f15780a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 0x7f15780a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47466/0 (socket says 172.21.15.32:47466) 2022-08-01T23:03:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 -- 172.21.15.32:0/800929983 learned_addr learned my addr 172.21.15.32:0/800929983 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f1577fff700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f15780b6510 0x7f15780a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f1576ffd700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15780b0e60 0x7f15780a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 -- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f15780b6510 msgr2=0x7f15780a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f15780b6510 0x7f15780a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 -- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15780b0e60 msgr2=0x7f15780a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15780b0e60 0x7f15780a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 -- 172.21.15.32:0/800929983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568009470 con 0x7f1578064500 2022-08-01T23:03:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f1576ffd700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15780b0e60 0x7f15780a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f15777fe700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 0x7f15780a0a80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f156000c000 tx=0x7f1560006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:50.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1560010780 con 0x7f1578064500 2022-08-01T23:03:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15600108e0 con 0x7f1578064500 2022-08-01T23:03:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15780b0090 con 0x7f1578064500 2022-08-01T23:03:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1560010c40 con 0x7f1578064500 2022-08-01T23:03:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.341+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15780a1fa0 con 0x7f1578064500 2022-08-01T23:03:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.345+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15780574e0 con 0x7f1578064500 2022-08-01T23:03:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.345+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f156001c460 con 0x7f1578064500 2022-08-01T23:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.345+0000 7f1574ff9700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f155c053a10 0x7f155c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.345+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f156006c240 con 0x7f1578064500 2022-08-01T23:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.345+0000 7f1576ffd700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f155c053a10 0x7f155c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.345+0000 7f1576ffd700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f155c053a10 0x7f155c055ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f1568017000 tx=0x7f1568016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:50.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.349+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f15600431e0 con 0x7f1578064500 2022-08-01T23:03:50.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.477+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f157806ac80 con 0x7f1578064500 2022-08-01T23:03:50.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.477+0000 7f1574ff9700 1 -- 172.21.15.32:0/800929983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v110) v1 ==== 81+0+29227 (secure 0 0 0) 0x7f1560046800 con 0x7f1578064500 2022-08-01T23:03:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:50.486 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:50.486 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 110, 2022-08-01T23:03:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:03:50.487 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:03:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:03:47.763652+0000", 2022-08-01T23:03:50.488 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:03:50.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:03:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:03:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:03:50.490 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:03:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:03:50.491 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:03:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:03:50.492 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:03:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:03:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:03:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:03:50.494 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:03:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 20, 2022-08-01T23:03:50.495 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:03:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:03:50.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:03:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:03:50.497 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:03:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.498 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:03:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:03:50.499 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:03:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:50.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:50.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:50.502 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:50.503 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:03:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:03:50.504 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:03:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:03:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:03:50.505 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:50.506 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:50.507 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:50.508 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:03:50.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:50.510 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:50.511 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:50.512 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:50.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:50.515 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:50.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:50.517 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:50.518 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:50.519 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:50.520 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:50.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:50.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:50.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:50.524 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:50.525 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:50.526 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:03:50.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:03:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.528 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:03:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.530 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:03:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:03:50.531 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:03:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:03:50.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:03:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:03:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:03:50.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:03:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:03:50.534 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:03:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:03:50.535 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:03:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:03:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:03:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:03:50.537 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:03:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:03:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:03:50.538 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:03:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:03:50.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:03:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:03:50.540 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:03:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:03:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:03:50.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:03:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:03:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:03:50.543 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:03:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:03:50.544 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:03:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:03:50.545 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:03:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:03:50.546 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:03:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:03:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:03:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:03:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:03:50.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:03:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:03:50.549 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:03:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:03:50.550 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:03:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:03:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:03:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:03:50.551 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:03:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:03:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:03:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.552 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:03:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:03:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:03:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:03:50.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:03:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:03:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:03:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:03:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:03:50.554 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:03:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:03:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:03:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:03:50.555 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:03:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:50.556 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:03:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:03:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.557 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.558 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.559 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:03:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:03:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:03:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:03:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:03:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:03:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:03:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:03:50.571 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:03:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:03:50.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:03:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:03:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:03:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:03:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.575 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.577 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:03:50.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:03:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:03:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:03:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:03:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:03:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:03:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:03:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:03:50.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:03:50.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:03:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:03:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:03:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.591 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.592 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:03:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.593 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 104, 2022-08-01T23:03:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:03:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:03:50.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:03:50.599 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:03:50.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.601 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:03:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:03:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:03:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:03:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:03:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:03:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:03:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:03:50.608 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:03:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:03:50.610 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.611 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.612 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:03:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:03:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.615 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:03:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:03:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:03:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:03:50.621 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:03:50.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.624 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:03:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.628 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.629 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 104, 2022-08-01T23:03:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.630 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:03:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:03:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:03:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:03:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.636 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:03:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:03:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:03:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:03:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:03:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:03:50.643 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:03:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:03:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:03:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.644 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.645 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:03:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.646 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.647 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 104, 2022-08-01T23:03:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:03:50.649 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:03:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.651 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:03:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.653 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:03:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:03:50.664 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:03:50.665 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.666 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:03:50.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:03:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:03:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:03:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:03:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:03:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:03:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:03:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.672 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.673 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.674 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.675 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:03:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.676 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:03:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.678 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:03:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:03:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:03:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:03:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:03:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:03:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:03:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:03:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:03:50.687 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:03:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:03:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:50.689 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:03:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:03:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:03:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:03:50.690 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:03:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:03:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:03:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:03:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 105, 2022-08-01T23:03:50.691 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:03:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:03:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:03:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:03:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:03:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:03:50.695 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:03:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:03:50.697 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:03:50.698 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:03:50.699 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.700 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:03:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.701 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:03:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:03:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:03:50.702 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:03:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:03:50.703 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:03:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:03:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:03:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:03:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:03:50.705 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:03:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:03:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:03:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:03:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:50.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:03:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.707 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:03:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.708 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:03:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:03:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.710 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:03:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:03:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.712 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:03:50.713 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:03:50.714 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:03:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:03:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.717 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:03:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:03:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.719 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:03:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:03:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.721 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:03:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:03:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:03:50.723 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:03:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:03:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:03:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:03:50.724 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:03:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:03:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:03:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:03:50.725 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:03:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:03:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:03:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:03:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/550523700": "2022-08-02T23:03:43.748200+0000", 2022-08-01T23:03:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/550523700": "2022-08-02T23:03:43.748200+0000" 2022-08-01T23:03:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:03:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:03:50.727 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:03:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:03:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:03:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:50.728 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:03:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:03:50.729 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:03:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:03:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:03:50.730 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:03:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:03:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:03:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:03:50.731 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:03:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:03:50.732 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:03:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:03:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:03:50.733 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:03:50.733 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:50.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f155c053a10 msgr2=0x7f155c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f155c053a10 0x7f155c055ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f1568017000 tx=0x7f1568016000).stop 2022-08-01T23:03:50.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 msgr2=0x7f15780a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 0x7f15780a0a80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f156000c000 tx=0x7f1560006df0).stop 2022-08-01T23:03:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 shutdown_connections 2022-08-01T23:03:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f155c053a10 0x7f155c055ed0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f15780b6510 0x7f15780a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15780b0e60 0x7f15780a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 --2- 172.21.15.32:0/800929983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1578064500 0x7f15780a0a80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 >> 172.21.15.32:0/800929983 conn(0x7f1578063800 msgr2=0x7f1578060310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 shutdown_connections 2022-08-01T23:03:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.481+0000 7f157c97f700 1 -- 172.21.15.32:0/800929983 wait complete. 2022-08-01T23:03:50.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 Processor -- start 2022-08-01T23:03:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 -- start start 2022-08-01T23:03:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb094148530 0x7fb0940b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940a7250 0x7fb0941448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0940b04e0 con 0x7fb0940b1f40 2022-08-01T23:03:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940a9250 con 0x7fb094148530 2022-08-01T23:03:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940b0780 con 0x7fb0940a7250 2022-08-01T23:03:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb099592700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb0940b1f40 0x7fb0940b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35016/0 (socket says 172.21.15.32:35016) 2022-08-01T23:03:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb099592700 1 -- 172.21.15.32:0/2719769353 learned_addr learned my addr 172.21.15.32:0/2719769353 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb098d91700 1 --2- 172.21.15.32:0/2719769353 >> v2:172.21.15.55:3300/0 conn(0x7fb094148530 0x7fb0940b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb099d93700 1 --2- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940a7250 0x7fb0941448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb093fff700 1 -- 172.21.15.32:0/2719769353 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387407484 0 0) 0x7fb0940b04e0 con 0x7fb0940b1f40 2022-08-01T23:03:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb093fff700 1 -- 172.21.15.32:0/2719769353 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb074003260 con 0x7fb0940b1f40 2022-08-01T23:03:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb093fff700 1 -- 172.21.15.32:0/2719769353 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658807529 0 0) 0x7fb074003260 con 0x7fb0940b1f40 2022-08-01T23:03:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb098d91700 1 -- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940a7250 msgr2=0x7fb0941448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb098d91700 1 --2- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940a7250 0x7fb0941448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb098d91700 1 -- 172.21.15.32:0/2719769353 >> v1:172.21.15.32:6789/0 conn(0x7fb0940b1f40 legacy=0x7fb0940b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb098d91700 1 -- 172.21.15.32:0/2719769353 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940b0a20 con 0x7fb094148530 2022-08-01T23:03:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb098d91700 1 --2- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb0940b1160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fb084001800 tx=0x7fb08c0091d0).ready entity=mon.1 client_cookie=edd74816ea2261c0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb093fff700 1 -- 172.21.15.32:0/2719769353 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb084001660 con 0x7fb094148530 2022-08-01T23:03:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb093fff700 1 -- 172.21.15.32:0/2719769353 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c00f510 con 0x7fb094148530 2022-08-01T23:03:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb093fff700 1 -- 172.21.15.32:0/2719769353 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08c00f680 con 0x7fb094148530 2022-08-01T23:03:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 -- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 msgr2=0x7fb0940b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.849+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb0940b1160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fb084001800 tx=0x7fb08c0091d0).stop 2022-08-01T23:03:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- 172.21.15.32:0/2719769353 shutdown_connections 2022-08-01T23:03:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940a7250 0x7fb0941448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/2719769353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb0940b1160 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- 172.21.15.32:0/2719769353 >> 172.21.15.32:0/2719769353 conn(0x7fb09405f840 msgr2=0x7fb09405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- 172.21.15.32:0/2719769353 shutdown_connections 2022-08-01T23:03:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- 172.21.15.32:0/2719769353 wait complete. 2022-08-01T23:03:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 Processor -- start 2022-08-01T23:03:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- start start 2022-08-01T23:03:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 0x7fb0940ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940b1f40 0x7fb0940af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb094141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940ad9e0 con 0x7fb0940a7250 2022-08-01T23:03:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940ad720 con 0x7fb094148530 2022-08-01T23:03:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb09b7f6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940adbd0 con 0x7fb0940b1f40 2022-08-01T23:03:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb099592700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 0x7fb0940ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb099592700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 0x7fb0940ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47480/0 (socket says 172.21.15.32:47480) 2022-08-01T23:03:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb099592700 1 -- 172.21.15.32:0/3631969417 learned_addr learned my addr 172.21.15.32:0/3631969417 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.853+0000 7fb098d91700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940b1f40 0x7fb0940af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb099d93700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb094141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb098d91700 1 -- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 msgr2=0x7fb094141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb098d91700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb094141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb098d91700 1 -- 172.21.15.32:0/3631969417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 msgr2=0x7fb0940ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb098d91700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 0x7fb0940ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb098d91700 1 -- 172.21.15.32:0/3631969417 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c009470 con 0x7fb0940b1f40 2022-08-01T23:03:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb099592700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 0x7fb0940ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb098d91700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940b1f40 0x7fb0940af330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fb0940aa860 tx=0x7fb08c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08c00f680 con 0x7fb0940b1f40 2022-08-01T23:03:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0032e0 con 0x7fb0940b1f40 2022-08-01T23:03:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0940afb00 con 0x7fb0940b1f40 2022-08-01T23:03:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08c0205e0 con 0x7fb0940b1f40 2022-08-01T23:03:50.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0940af820 con 0x7fb0940b1f40 2022-08-01T23:03:50.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb08c01e070 con 0x7fb0940b1f40 2022-08-01T23:03:50.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0940add10 con 0x7fb0940b1f40 2022-08-01T23:03:50.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb0927fc700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb078053a70 0x7fb078055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:50.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.857+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb08c06cab0 con 0x7fb0940b1f40 2022-08-01T23:03:50.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.861+0000 7fb099592700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb078053a70 0x7fb078055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:50.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.861+0000 7fb099592700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb078053a70 0x7fb078055f30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb08400d000 tx=0x7fb0840011a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:50.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:50.869+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb08c043bb0 con 0x7fb0940b1f40 2022-08-01T23:03:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0940574e0 con 0x7fb0940b1f40 2022-08-01T23:03:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb0927fc700 1 -- 172.21.15.32:0/3631969417 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fb08c01b020 con 0x7fb0940b1f40 2022-08-01T23:03:51.038 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:51.039 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/2470389680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2470389680},{"type":"v1","addr":"172.21.15.32:6835","nonce":2470389680}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53}],"filesystems":[]} 2022-08-01T23:03:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb078053a70 msgr2=0x7fb078055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb078053a70 0x7fb078055f30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb08400d000 tx=0x7fb0840011a0).stop 2022-08-01T23:03:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940b1f40 msgr2=0x7fb0940af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940b1f40 0x7fb0940af330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fb0940aa860 tx=0x7fb08c016000).stop 2022-08-01T23:03:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 shutdown_connections 2022-08-01T23:03:51.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb078053a70 0x7fb078055f30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb094148530 0x7fb094141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0940b1f40 0x7fb0940af330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 --2- 172.21.15.32:0/3631969417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0940a7250 0x7fb0940ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 >> 172.21.15.32:0/3631969417 conn(0x7fb09405f840 msgr2=0x7fb0940b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:51.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 shutdown_connections 2022-08-01T23:03:51.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.033+0000 7fb09b7f6700 1 -- 172.21.15.32:0/3631969417 wait complete. 2022-08-01T23:03:51.044 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 53 2022-08-01T23:03:51.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 Processor -- start 2022-08-01T23:03:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 -- start start 2022-08-01T23:03:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72f40b31d0 0x7f72f4066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72f41402d0 0x7f72f4140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f4067030 con 0x7f72f40a9da0 2022-08-01T23:03:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f4064a70 con 0x7f72f40b31d0 2022-08-01T23:03:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f881e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f4061f60 con 0x7f72f41402d0 2022-08-01T23:03:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72f40a9da0 0x7f72f4060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35030/0 (socket says 172.21.15.32:35030) 2022-08-01T23:03:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f259c700 1 -- 172.21.15.32:0/226997167 learned_addr learned my addr 172.21.15.32:0/226997167 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f2d9d700 1 --2- 172.21.15.32:0/226997167 >> v2:172.21.15.32:3300/0 conn(0x7f72f40b31d0 0x7f72f4066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f359e700 1 --2- 172.21.15.32:0/226997167 >> v2:172.21.15.55:3300/0 conn(0x7f72f41402d0 0x7f72f4140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f1d9b700 1 -- 172.21.15.32:0/226997167 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442869893 0 0) 0x7f72f4067030 con 0x7f72f40a9da0 2022-08-01T23:03:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f1d9b700 1 -- 172.21.15.32:0/226997167 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d4003260 con 0x7f72f40a9da0 2022-08-01T23:03:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f2d9d700 1 -- 172.21.15.32:0/226997167 >> v2:172.21.15.55:3300/0 conn(0x7f72f41402d0 msgr2=0x7f72f4140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f2d9d700 1 --2- 172.21.15.32:0/226997167 >> v2:172.21.15.55:3300/0 conn(0x7f72f41402d0 0x7f72f4140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f2d9d700 1 -- 172.21.15.32:0/226997167 >> v1:172.21.15.32:6789/0 conn(0x7f72f40a9da0 legacy=0x7f72f4060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f2d9d700 1 -- 172.21.15.32:0/226997167 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40b35b0 con 0x7f72f40b31d0 2022-08-01T23:03:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f2d9d700 1 --2- 172.21.15.32:0/226997167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40b31d0 0x7f72f4066020 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f72f40625d0 tx=0x7f72e4006df0).ready entity=mon.0 client_cookie=5127ddb6dbfc15be server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f1d9b700 1 -- 172.21.15.32:0/226997167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e400d5c0 con 0x7f72f40b31d0 2022-08-01T23:03:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f1d9b700 1 -- 172.21.15.32:0/226997167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e4000e90 con 0x7f72f40b31d0 2022-08-01T23:03:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.153+0000 7f72f1d9b700 1 -- 172.21.15.32:0/226997167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e400d910 con 0x7f72f40b31d0 2022-08-01T23:03:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 -- 172.21.15.32:0/226997167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40b31d0 msgr2=0x7f72f4066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 --2- 172.21.15.32:0/226997167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40b31d0 0x7f72f4066020 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f72f40625d0 tx=0x7f72e4006df0).stop 2022-08-01T23:03:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 -- 172.21.15.32:0/226997167 shutdown_connections 2022-08-01T23:03:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 --2- 172.21.15.32:0/226997167 >> v2:172.21.15.55:3300/0 conn(0x7f72f41402d0 0x7f72f4140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 --2- 172.21.15.32:0/226997167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40b31d0 0x7f72f4066020 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 -- 172.21.15.32:0/226997167 >> 172.21.15.32:0/226997167 conn(0x7f72f4063870 msgr2=0x7f72f4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 -- 172.21.15.32:0/226997167 shutdown_connections 2022-08-01T23:03:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 -- 172.21.15.32:0/226997167 wait complete. 2022-08-01T23:03:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 Processor -- start 2022-08-01T23:03:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.157+0000 7f72f881e700 1 -- start start 2022-08-01T23:03:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 0x7f72f40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72f40b31d0 0x7f72f4143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72f41402d0 0x7f72f414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f414ee70 con 0x7f72f40a9da0 2022-08-01T23:03:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f4145430 con 0x7f72f40b31d0 2022-08-01T23:03:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f414f060 con 0x7f72f41402d0 2022-08-01T23:03:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 0x7f72f40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 0x7f72f40acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47494/0 (socket says 172.21.15.32:47494) 2022-08-01T23:03:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 -- 172.21.15.32:0/3619638626 learned_addr learned my addr 172.21.15.32:0/3619638626 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f359e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72f41402d0 0x7f72f414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f259c700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72f40b31d0 0x7f72f4143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 -- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72f41402d0 msgr2=0x7f72f414f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72f41402d0 0x7f72f414f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 -- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72f40b31d0 msgr2=0x7f72f4143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72f40b31d0 0x7f72f4143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 -- 172.21.15.32:0/3619638626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e400b180 con 0x7f72f40a9da0 2022-08-01T23:03:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f259c700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72f40b31d0 0x7f72f4143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f359e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72f41402d0 0x7f72f414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f2d9d700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 0x7f72f40acd10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f72f40625d0 tx=0x7f72e4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e4012860 con 0x7f72f40a9da0 2022-08-01T23:03:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e40129c0 con 0x7f72f40a9da0 2022-08-01T23:03:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f414f300 con 0x7f72f40a9da0 2022-08-01T23:03:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e4012d20 con 0x7f72f40a9da0 2022-08-01T23:03:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.161+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f414fc90 con 0x7f72f40a9da0 2022-08-01T23:03:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.165+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f4066af0 con 0x7f72f40a9da0 2022-08-01T23:03:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.165+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72e400f030 con 0x7f72f40a9da0 2022-08-01T23:03:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.165+0000 7f72ebfff700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72d8053a70 0x7f72d8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.165+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f72e406f170 con 0x7f72f40a9da0 2022-08-01T23:03:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.165+0000 7f72f259c700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72d8053a70 0x7f72d8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.165+0000 7f72f259c700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72d8053a70 0x7f72d8055f30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f72dc00b000 tx=0x7f72dc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.173+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72e4046220 con 0x7f72f40a9da0 2022-08-01T23:03:51.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72f41455f0 con 0x7f72f40a9da0 2022-08-01T23:03:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72ebfff700 1 -- 172.21.15.32:0/3619638626 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f72e4049840 con 0x7f72f40a9da0 2022-08-01T23:03:51.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:51.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/460721844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":460721844},{"type":"v1","addr":"172.21.15.55:6835","nonce":460721844}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6833/2648966769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2648966769},{"type":"v1","addr":"172.21.15.32:6833","nonce":2648966769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/2470389680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2470389680},{"type":"v1","addr":"172.21.15.32:6835","nonce":2470389680}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53}],"filesystems":[]} 2022-08-01T23:03:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72d8053a70 msgr2=0x7f72d8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72d8053a70 0x7f72d8055f30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f72dc00b000 tx=0x7f72dc009040).stop 2022-08-01T23:03:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 msgr2=0x7f72f40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 0x7f72f40acd10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f72f40625d0 tx=0x7f72e4014000).stop 2022-08-01T23:03:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 shutdown_connections 2022-08-01T23:03:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72d8053a70 0x7f72d8055f30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72f41402d0 0x7f72f414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72f40b31d0 0x7f72f4143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 --2- 172.21.15.32:0/3619638626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f40a9da0 0x7f72f40acd10 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 >> 172.21.15.32:0/3619638626 conn(0x7f72f4063870 msgr2=0x7f72f40b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 shutdown_connections 2022-08-01T23:03:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.341+0000 7f72f881e700 1 -- 172.21.15.32:0/3619638626 wait complete. 2022-08-01T23:03:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 53 2022-08-01T23:03:51.359 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:03:51.360 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:03:51.360 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:03:51.360 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:51.361 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:03:51.361 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:03:51.361 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:03:51.362 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:51.362 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:03:51.362 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:03:51.363 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:03:51.363 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:03:51.369 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:03:51.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:03:51.377 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:03:51.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:03:51.379 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:03:51.379 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:03:51.382 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:03:51.382 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:51.421 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:03:51.422 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:51.423 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:03:51.423 DEBUG:teuthology.parallel:result is None 2022-08-01T23:03:51.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:03:51.461 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:03:51.486 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:03:51.499 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.517+0000 7fcd28c49700 1 Processor -- start 2022-08-01T23:03:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.517+0000 7fcd28c49700 1 -- start start 2022-08-01T23:03:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd240b2750 0x7fcd240ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd241442d0 con 0x7fcd240b2b30 2022-08-01T23:03:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd24144030 con 0x7fcd240b2750 2022-08-01T23:03:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240ae360 con 0x7fcd2405c620 2022-08-01T23:03:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd240b2750 0x7fcd240ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd240b2750 0x7fcd240ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47526/0 (socket says 172.21.15.32:47526) 2022-08-01T23:03:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 -- 172.21.15.32:0/1555898171 learned_addr learned my addr 172.21.15.32:0/1555898171 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd21d9b700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 -- 172.21.15.32:0/1555898171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 msgr2=0x7fcd241468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 -- 172.21.15.32:0/1555898171 >> v1:172.21.15.55:6789/0 conn(0x7fcd240b2b30 legacy=0x7fcd24146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 -- 172.21.15.32:0/1555898171 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd240aecf0 con 0x7fcd240b2750 2022-08-01T23:03:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd21d9b700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2259c700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2750 0x7fcd240ae800 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fcd240ad910 tx=0x7fcd14006eb0).ready entity=mon.0 client_cookie=a1457d96ad5d039e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2159a700 1 -- 172.21.15.32:0/1555898171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1400d570 con 0x7fcd240b2750 2022-08-01T23:03:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2159a700 1 -- 172.21.15.32:0/1555898171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd14000e90 con 0x7fcd240b2750 2022-08-01T23:03:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd2159a700 1 -- 172.21.15.32:0/1555898171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1400d900 con 0x7fcd240b2750 2022-08-01T23:03:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- 172.21.15.32:0/1555898171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2750 msgr2=0x7fcd240ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2750 0x7fcd240ae800 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fcd240ad910 tx=0x7fcd14006eb0).stop 2022-08-01T23:03:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- 172.21.15.32:0/1555898171 shutdown_connections 2022-08-01T23:03:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 --2- 172.21.15.32:0/1555898171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2750 0x7fcd240ae800 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- 172.21.15.32:0/1555898171 >> 172.21.15.32:0/1555898171 conn(0x7fcd2405ceb0 msgr2=0x7fcd24063130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- 172.21.15.32:0/1555898171 shutdown_connections 2022-08-01T23:03:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- 172.21.15.32:0/1555898171 wait complete. 2022-08-01T23:03:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 Processor -- start 2022-08-01T23:03:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.521+0000 7fcd28c49700 1 -- start start 2022-08-01T23:03:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd240b2750 0x7fcd2413f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 0x7fcd2413f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240ab470 con 0x7fcd240b2b30 2022-08-01T23:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240ac080 con 0x7fcd240b2750 2022-08-01T23:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240ab6d0 con 0x7fcd2405c620 2022-08-01T23:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 0x7fcd2413f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 0x7fcd2413f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47536/0 (socket says 172.21.15.32:47536) 2022-08-01T23:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 -- 172.21.15.32:0/2561480689 learned_addr learned my addr 172.21.15.32:0/2561480689 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd21d9b700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd240b2750 0x7fcd2413f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 -- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 msgr2=0x7fcd241448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 -- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd240b2750 msgr2=0x7fcd2413f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:51.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd240b2750 0x7fcd2413f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:51.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 -- 172.21.15.32:0/2561480689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1400b1e0 con 0x7fcd240b2b30 2022-08-01T23:03:51.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd2259c700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:51.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd22d9d700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 0x7fcd2413f4f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fcd1c001250 tx=0x7fcd1c007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:51.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1c00cea0 con 0x7fcd240b2b30 2022-08-01T23:03:51.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c006aa0 con 0x7fcd240b2b30 2022-08-01T23:03:51.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd240ab970 con 0x7fcd240b2b30 2022-08-01T23:03:51.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1c006e00 con 0x7fcd240b2b30 2022-08-01T23:03:51.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2413f9e0 con 0x7fcd240b2b30 2022-08-01T23:03:51.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd240ac630 con 0x7fcd240b2b30 2022-08-01T23:03:51.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.525+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd1c019020 con 0x7fcd240b2b30 2022-08-01T23:03:51.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.529+0000 7fcd137fe700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd000539e0 0x7fcd00055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:51.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.529+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcd1c06bbd0 con 0x7fcd240b2b30 2022-08-01T23:03:51.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.529+0000 7fcd2259c700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd000539e0 0x7fcd00055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:51.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.529+0000 7fcd2259c700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd000539e0 0x7fcd00055ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fcd1401c000 tx=0x7fcd14015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:51.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.533+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd240ac630 con 0x7fcd240b2b30 2022-08-01T23:03:51.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:51.665+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fcd240ac630 con 0x7fcd240b2b30 2022-08-01T23:03:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.521+0000 7fcd137fe700 1 -- 172.21.15.32:0/2561480689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd240ac630 con 0x7fcd240b2b30 2022-08-01T23:03:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd000539e0 msgr2=0x7fcd00055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd000539e0 0x7fcd00055ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fcd1401c000 tx=0x7fcd14015000).stop 2022-08-01T23:03:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 msgr2=0x7fcd2413f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 0x7fcd2413f4f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fcd1c001250 tx=0x7fcd1c007f20).stop 2022-08-01T23:03:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 shutdown_connections 2022-08-01T23:03:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcd000539e0 0x7fcd00055ea0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd240b2b30 0x7fcd2413f4f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd240b2750 0x7fcd2413f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 --2- 172.21.15.32:0/2561480689 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcd2405c620 0x7fcd241448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 >> 172.21.15.32:0/2561480689 conn(0x7fcd2405ceb0 msgr2=0x7fcd241411b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 shutdown_connections 2022-08-01T23:03:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.525+0000 7fcd28c49700 1 -- 172.21.15.32:0/2561480689 wait complete. 2022-08-01T23:03:52.534 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:03:52.545 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:03:52.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:03:52.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 Processor -- start 2022-08-01T23:03:52.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 -- start start 2022-08-01T23:03:52.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecd00b38a0 0x7fecd0064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd01403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd00b4f60 con 0x7fecd00647d0 2022-08-01T23:03:52.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd00b4a60 con 0x7fecd00b38a0 2022-08-01T23:03:52.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd5289700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00b4cc0 con 0x7fecd01470d0 2022-08-01T23:03:52.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7feccffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fecd00647d0 0x7fecd01436f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35086/0 (socket says 172.21.15.32:35086) 2022-08-01T23:03:52.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecc77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecd00b38a0 0x7fecd0064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7feccffff700 1 -- 172.21.15.32:0/663878428 learned_addr learned my addr 172.21.15.32:0/663878428 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:52.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.649+0000 7fecd4a88700 1 --2- 172.21.15.32:0/663878428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd01403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7feccf7fe700 1 -- 172.21.15.32:0/663878428 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178219626 0 0) 0x7fecd00b4f60 con 0x7fecd00647d0 2022-08-01T23:03:52.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7feccf7fe700 1 -- 172.21.15.32:0/663878428 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb0003260 con 0x7fecd00647d0 2022-08-01T23:03:52.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecc77fe700 1 -- 172.21.15.32:0/663878428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 msgr2=0x7fecd01403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecc77fe700 1 --2- 172.21.15.32:0/663878428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd01403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecc77fe700 1 -- 172.21.15.32:0/663878428 >> v1:172.21.15.32:6789/0 conn(0x7fecd00647d0 legacy=0x7fecd01436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecc77fe700 1 -- 172.21.15.32:0/663878428 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0143250 con 0x7fecd00b38a0 2022-08-01T23:03:52.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecc77fe700 1 --2- 172.21.15.32:0/663878428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00b38a0 0x7fecd0064bb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7fecb8002f70 tx=0x7fecb8006df0).ready entity=mon.0 client_cookie=276cef78240be39 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7feccf7fe700 1 -- 172.21.15.32:0/663878428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecb800d4a0 con 0x7fecd00b38a0 2022-08-01T23:03:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7feccf7fe700 1 -- 172.21.15.32:0/663878428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8000e90 con 0x7fecd00b38a0 2022-08-01T23:03:52.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7feccf7fe700 1 -- 172.21.15.32:0/663878428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecb800d7f0 con 0x7fecd00b38a0 2022-08-01T23:03:52.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 -- 172.21.15.32:0/663878428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00b38a0 msgr2=0x7fecd0064bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 --2- 172.21.15.32:0/663878428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00b38a0 0x7fecd0064bb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7fecb8002f70 tx=0x7fecb8006df0).stop 2022-08-01T23:03:52.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 -- 172.21.15.32:0/663878428 shutdown_connections 2022-08-01T23:03:52.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 --2- 172.21.15.32:0/663878428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd01403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 --2- 172.21.15.32:0/663878428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00b38a0 0x7fecd0064bb0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 -- 172.21.15.32:0/663878428 >> 172.21.15.32:0/663878428 conn(0x7fecd005f840 msgr2=0x7fecd005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:52.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 -- 172.21.15.32:0/663878428 shutdown_connections 2022-08-01T23:03:52.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.653+0000 7fecd5289700 1 -- 172.21.15.32:0/663878428 wait complete. 2022-08-01T23:03:52.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 Processor -- start 2022-08-01T23:03:52.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 -- start start 2022-08-01T23:03:52.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 0x7fecd01456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd00b38a0 0x7fecd0152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd0152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd01450a0 con 0x7fecd00647d0 2022-08-01T23:03:52.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00b8310 con 0x7fecd00b38a0 2022-08-01T23:03:52.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd5289700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0144e00 con 0x7fecd01470d0 2022-08-01T23:03:52.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7feccffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 0x7fecd01456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7feccffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 0x7fecd01456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47550/0 (socket says 172.21.15.32:47550) 2022-08-01T23:03:52.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7feccffff700 1 -- 172.21.15.32:0/4243559823 learned_addr learned my addr 172.21.15.32:0/4243559823 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:52.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecd4a88700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd0152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.657+0000 7fecc77fe700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd00b38a0 0x7fecd0152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccffff700 1 -- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 msgr2=0x7fecd0152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccffff700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd0152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccffff700 1 -- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd00b38a0 msgr2=0x7fecd0152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccffff700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd00b38a0 0x7fecd0152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccffff700 1 -- 172.21.15.32:0/4243559823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb800b180 con 0x7fecd00647d0 2022-08-01T23:03:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccffff700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 0x7fecd01456b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7fecc0001110 tx=0x7fecc0007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc00070d0 con 0x7fecd00647d0 2022-08-01T23:03:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc0007230 con 0x7fecd00647d0 2022-08-01T23:03:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd0065060 con 0x7fecd00647d0 2022-08-01T23:03:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc0005610 con 0x7fecd00647d0 2022-08-01T23:03:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd0144a60 con 0x7fecd00647d0 2022-08-01T23:03:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd0147560 con 0x7fecd00647d0 2022-08-01T23:03:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fecc0010070 con 0x7fecd00647d0 2022-08-01T23:03:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccdffb700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fecb4053a70 0x7fecb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.661+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fecc006b3b0 con 0x7fecd00647d0 2022-08-01T23:03:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.665+0000 7fecc77fe700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fecb4053a70 0x7fecb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.665+0000 7fecc77fe700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fecb4053a70 0x7fecb4055f30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fecb801a000 tx=0x7fecb8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.665+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fecc0042430 con 0x7fecd00647d0 2022-08-01T23:03:52.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.833+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fecd00574e0 con 0x7fecd00647d0 2022-08-01T23:03:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.833+0000 7feccdffb700 1 -- 172.21.15.32:0/4243559823 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fecc0045a50 con 0x7fecd00647d0 2022-08-01T23:03:52.841 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:52.841 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:03:52.841 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:03:52.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.842 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:03:52.842 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:03:52.842 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.842 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:52.843 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.843 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:03:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:03:52.844 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.845 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:52.845 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.845 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.845 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.846 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:03:52.846 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:03:52.847 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.847 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:52.847 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.847 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.848 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:03:52.848 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:03:52.849 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.849 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.849 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.849 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.850 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:03:52.850 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:03:52.851 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.852 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:03:52.852 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:03:52.852 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:03:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:03:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.856 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:03:52.856 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:03:52.856 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.857 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.857 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.857 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.858 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:03:52.858 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:03:52.858 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.859 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.859 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.859 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.860 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:03:52.860 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:03:52.860 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.861 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.861 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.861 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.862 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:03:52.862 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:03:52.862 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.863 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.864 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:03:52.864 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:03:52.864 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.865 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:52.865 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:03:52.865 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:03:52.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.866 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:03:52.866 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:03:52.866 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.866 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:03:52.867 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:03:52.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:03:52.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.868 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:03:52.868 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:03:52.868 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:52.869 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:03:52.869 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:03:52.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.870 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:03:52.870 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:03:52.870 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.871 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:52.871 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:03:52.871 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:52.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:52.872 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:03:52.872 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:03:52.872 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:03:52.873 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:03:52.873 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:03:52.873 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:03:52.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:52.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:03:52.874 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:03:52.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fecb4053a70 msgr2=0x7fecb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fecb4053a70 0x7fecb4055f30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fecb801a000 tx=0x7fecb8014000).stop 2022-08-01T23:03:52.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 msgr2=0x7fecd01456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 0x7fecd01456b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7fecc0001110 tx=0x7fecc0007ad0).stop 2022-08-01T23:03:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 shutdown_connections 2022-08-01T23:03:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fecb4053a70 0x7fecb4055f30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fecd01470d0 0x7fecd0152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd00b38a0 0x7fecd0152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 --2- 172.21.15.32:0/4243559823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd00647d0 0x7fecd01456b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 >> 172.21.15.32:0/4243559823 conn(0x7fecd005f840 msgr2=0x7fecd0142930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:52.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.837+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 shutdown_connections 2022-08-01T23:03:52.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.841+0000 7fecd5289700 1 -- 172.21.15.32:0/4243559823 wait complete. 2022-08-01T23:03:52.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:52.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e78909700 1 Processor -- start 2022-08-01T23:03:52.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e78909700 1 -- start start 2022-08-01T23:03:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e78909700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e740b01a0 0x7f1e740a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e78909700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740a7250 con 0x7f1e740b2710 2022-08-01T23:03:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e78909700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740a7480 con 0x7f1e740afd60 2022-08-01T23:03:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e78909700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740a7720 con 0x7f1e740b01a0 2022-08-01T23:03:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e6bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e740b01a0 0x7f1e740a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e6bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e740b01a0 0x7f1e740a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47558/0 (socket says 172.21.15.32:47558) 2022-08-01T23:03:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e737fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e740b2710 0x7f1e7414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35102/0 (socket says 172.21.15.32:35102) 2022-08-01T23:03:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.977+0000 7f1e6bfff700 1 -- 172.21.15.32:0/2803777250 learned_addr learned my addr 172.21.15.32:0/2803777250 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e6bfff700 1 -- 172.21.15.32:0/2803777250 >> v1:172.21.15.55:6789/0 conn(0x7f1e740afd60 legacy=0x7f1e74151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e6bfff700 1 -- 172.21.15.32:0/2803777250 >> v1:172.21.15.32:6789/0 conn(0x7f1e740b2710 legacy=0x7f1e7414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e72ffd700 1 -- 172.21.15.32:0/2803777250 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96042869 0 0) 0x7f1e740a7250 con 0x7f1e740b2710 2022-08-01T23:03:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e6bfff700 1 -- 172.21.15.32:0/2803777250 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740a79c0 con 0x7f1e740b01a0 2022-08-01T23:03:52.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e6bfff700 1 --2- 172.21.15.32:0/2803777250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b01a0 0x7f1e740a8540 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f1e5c002f70 tx=0x7f1e5c006eb0).ready entity=mon.0 client_cookie=f63c52080b6d9d06 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:52.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e72ffd700 1 -- 172.21.15.32:0/2803777250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5c00d660 con 0x7f1e740b01a0 2022-08-01T23:03:52.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e72ffd700 1 -- 172.21.15.32:0/2803777250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c000e90 con 0x7f1e740b01a0 2022-08-01T23:03:52.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e72ffd700 1 -- 172.21.15.32:0/2803777250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e5c00d9b0 con 0x7f1e740b01a0 2022-08-01T23:03:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 -- 172.21.15.32:0/2803777250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b01a0 msgr2=0x7f1e740a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 --2- 172.21.15.32:0/2803777250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b01a0 0x7f1e740a8540 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f1e5c002f70 tx=0x7f1e5c006eb0).stop 2022-08-01T23:03:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 -- 172.21.15.32:0/2803777250 shutdown_connections 2022-08-01T23:03:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 --2- 172.21.15.32:0/2803777250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b01a0 0x7f1e740a8540 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 -- 172.21.15.32:0/2803777250 >> 172.21.15.32:0/2803777250 conn(0x7f1e74060310 msgr2=0x7f1e740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 -- 172.21.15.32:0/2803777250 shutdown_connections 2022-08-01T23:03:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.981+0000 7f1e78909700 1 -- 172.21.15.32:0/2803777250 wait complete. 2022-08-01T23:03:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 Processor -- start 2022-08-01T23:03:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 -- start start 2022-08-01T23:03:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 0x7f1e740a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e740b01a0 0x7f1e741565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 0x7f1e74156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740aa460 con 0x7f1e740b2710 2022-08-01T23:03:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740aaef0 con 0x7f1e740afd60 2022-08-01T23:03:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740b5810 con 0x7f1e740b01a0 2022-08-01T23:03:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 0x7f1e74156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 0x7f1e740a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e6bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e740b01a0 0x7f1e741565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 0x7f1e740a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57350/0 (socket says 172.21.15.32:57350) 2022-08-01T23:03:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e737fe700 1 -- 172.21.15.32:0/3802935236 learned_addr learned my addr 172.21.15.32:0/3802935236 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 0x7f1e74156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47568/0 (socket says 172.21.15.32:47568) 2022-08-01T23:03:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 -- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e740b01a0 msgr2=0x7f1e741565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e740b01a0 0x7f1e741565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 -- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 msgr2=0x7f1e740a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 0x7f1e740a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 -- 172.21.15.32:0/3802935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c00b1e0 con 0x7f1e740b2710 2022-08-01T23:03:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e737fe700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 0x7f1e740a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e73fff700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 0x7f1e74156ae0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f1e74061c10 tx=0x7f1e6c007eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e6c00f470 con 0x7f1e740b2710 2022-08-01T23:03:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00f5d0 con 0x7f1e740b2710 2022-08-01T23:03:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e740aa030 con 0x7f1e740b2710 2022-08-01T23:03:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e6c00f930 con 0x7f1e740b2710 2022-08-01T23:03:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.985+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e740ab7d0 con 0x7f1e740b2710 2022-08-01T23:03:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.989+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e740aa1c0 con 0x7f1e740b2710 2022-08-01T23:03:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.989+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1e6c010070 con 0x7f1e740b2710 2022-08-01T23:03:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.989+0000 7f1e717fa700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e58053b30 0x7f1e58055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.989+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1e6c06b160 con 0x7f1e740b2710 2022-08-01T23:03:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.989+0000 7f1e737fe700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e58053b30 0x7f1e58055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.989+0000 7f1e737fe700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e58053b30 0x7f1e58055ff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f1e6400d000 tx=0x7f1e64007c30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:52.993+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e6c042260 con 0x7f1e740b2710 2022-08-01T23:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.121+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e740abe30 con 0x7f1e740b2710 2022-08-01T23:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.121+0000 7f1e717fa700 1 -- 172.21.15.32:0/3802935236 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f1e6c045880 con 0x7f1e740b2710 2022-08-01T23:03:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:53.129 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:52.515195+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e58053b30 msgr2=0x7f1e58055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e58053b30 0x7f1e58055ff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f1e6400d000 tx=0x7f1e64007c30).stop 2022-08-01T23:03:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 msgr2=0x7f1e74156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 0x7f1e74156ae0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f1e74061c10 tx=0x7f1e6c007eb0).stop 2022-08-01T23:03:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 shutdown_connections 2022-08-01T23:03:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1e58053b30 0x7f1e58055ff0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e740b2710 0x7f1e74156ae0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1e740b01a0 0x7f1e741565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 --2- 172.21.15.32:0/3802935236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e740afd60 0x7f1e740a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 >> 172.21.15.32:0/3802935236 conn(0x7f1e74060310 msgr2=0x7f1e741499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 shutdown_connections 2022-08-01T23:03:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.125+0000 7f1e78909700 1 -- 172.21.15.32:0/3802935236 wait complete. 2022-08-01T23:03:53.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:03:53.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 Processor -- start 2022-08-01T23:03:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 -- start start 2022-08-01T23:03:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38e80b3720 0x7f38e8141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 0x7f38e8141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e80ada80 con 0x7f38e80b3340 2022-08-01T23:03:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e80b0090 con 0x7f38e80b3720 2022-08-01T23:03:53.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.249+0000 7f38f07ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81456f0 con 0x7f38e8145220 2022-08-01T23:03:53.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38edd65700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38e80b3720 0x7f38e8141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 0x7f38e8141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 0x7f38e8141a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59314/0 (socket says 172.21.15.32:59314) 2022-08-01T23:03:53.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 -- 172.21.15.32:0/2986546554 learned_addr learned my addr 172.21.15.32:0/2986546554 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:53.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 -- 172.21.15.32:0/2986546554 >> v2:172.21.15.32:3300/0 conn(0x7f38e80b3720 msgr2=0x7f38e8141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:53.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 --2- 172.21.15.32:0/2986546554 >> v2:172.21.15.32:3300/0 conn(0x7f38e80b3720 0x7f38e8141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 -- 172.21.15.32:0/2986546554 >> v1:172.21.15.55:6789/0 conn(0x7f38e80b3340 legacy=0x7f38e813e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 -- 172.21.15.32:0/2986546554 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80b02f0 con 0x7f38e8145220 2022-08-01T23:03:53.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38eed67700 1 --2- 172.21.15.32:0/2986546554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 0x7f38e8141a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f38e8141320 tx=0x7f38e0009290).ready entity=mon.2 client_cookie=5591ea1866f992f4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38ed564700 1 -- 172.21.15.32:0/2986546554 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e000b860 con 0x7f38e8145220 2022-08-01T23:03:53.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38ed564700 1 -- 172.21.15.32:0/2986546554 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e000b9c0 con 0x7f38e8145220 2022-08-01T23:03:53.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38ed564700 1 -- 172.21.15.32:0/2986546554 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e000bb30 con 0x7f38e8145220 2022-08-01T23:03:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 -- 172.21.15.32:0/2986546554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 msgr2=0x7f38e8141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 --2- 172.21.15.32:0/2986546554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 0x7f38e8141a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f38e8141320 tx=0x7f38e0009290).stop 2022-08-01T23:03:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 -- 172.21.15.32:0/2986546554 shutdown_connections 2022-08-01T23:03:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 --2- 172.21.15.32:0/2986546554 >> v2:172.21.15.32:3300/0 conn(0x7f38e80b3720 0x7f38e8141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 --2- 172.21.15.32:0/2986546554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e8145220 0x7f38e8141a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 -- 172.21.15.32:0/2986546554 >> 172.21.15.32:0/2986546554 conn(0x7f38e8060950 msgr2=0x7f38e8060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 -- 172.21.15.32:0/2986546554 shutdown_connections 2022-08-01T23:03:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.253+0000 7f38f07ca700 1 -- 172.21.15.32:0/2986546554 wait complete. 2022-08-01T23:03:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 Processor -- start 2022-08-01T23:03:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 -- start start 2022-08-01T23:03:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e80b3340 0x7f38e809f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e80b3720 0x7f38e8065190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38e8145220 0x7f38e8066940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81507d0 con 0x7f38e80b3720 2022-08-01T23:03:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8064a50 con 0x7f38e8145220 2022-08-01T23:03:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8064cf0 con 0x7f38e80b3340 2022-08-01T23:03:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38edd65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e80b3720 0x7f38e8065190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38edd65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e80b3720 0x7f38e8065190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47580/0 (socket says 172.21.15.32:47580) 2022-08-01T23:03:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38edd65700 1 -- 172.21.15.32:0/1876852728 learned_addr learned my addr 172.21.15.32:0/1876852728 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:53.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38e8145220 0x7f38e8066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38ee566700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e80b3340 0x7f38e809f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 -- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e80b3340 msgr2=0x7f38e809f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e80b3340 0x7f38e809f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 -- 172.21.15.32:0/1876852728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e80b3720 msgr2=0x7f38e8065190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e80b3720 0x7f38e8065190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 -- 172.21.15.32:0/1876852728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0009530 con 0x7f38e8145220 2022-08-01T23:03:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38eed67700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38e8145220 0x7f38e8066940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f38e8141320 tx=0x7f38e0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e000b860 con 0x7f38e8145220 2022-08-01T23:03:53.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e0021630 con 0x7f38e8145220 2022-08-01T23:03:53.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e8150500 con 0x7f38e8145220 2022-08-01T23:03:53.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e0021990 con 0x7f38e8145220 2022-08-01T23:03:53.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.257+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e8065d70 con 0x7f38e8145220 2022-08-01T23:03:53.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.261+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e8064eb0 con 0x7f38e8145220 2022-08-01T23:03:53.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.261+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f38e0027020 con 0x7f38e8145220 2022-08-01T23:03:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.261+0000 7f38e77fe700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f38c8053b00 0x7f38c8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.261+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f38e006cfb0 con 0x7f38e8145220 2022-08-01T23:03:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.261+0000 7f38ee566700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f38c8053b00 0x7f38c8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.261+0000 7f38ee566700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f38c8053b00 0x7f38c8055fc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f38d800d000 tx=0x7f38d8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.269+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f38e0043fe0 con 0x7f38e8145220 2022-08-01T23:03:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.401+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f38e80b0590 con 0x7f38e8145220 2022-08-01T23:03:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.401+0000 7f38e77fe700 1 -- 172.21.15.32:0/1876852728 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f38e0015020 con 0x7f38e8145220 2022-08-01T23:03:53.407 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:53.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.401+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f38c8053b00 msgr2=0x7f38c8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.401+0000 7f38f07ca700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f38c8053b00 0x7f38c8055fc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f38d800d000 tx=0x7f38d8009040).stop 2022-08-01T23:03:53.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.401+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38e8145220 msgr2=0x7f38e8066940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.401+0000 7f38f07ca700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38e8145220 0x7f38e8066940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f38e8141320 tx=0x7f38e0016000).stop 2022-08-01T23:03:53.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 shutdown_connections 2022-08-01T23:03:53.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f38c8053b00 0x7f38c8055fc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38e8145220 0x7f38e8066940 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e80b3720 0x7f38e8065190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 --2- 172.21.15.32:0/1876852728 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38e80b3340 0x7f38e809f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 >> 172.21.15.32:0/1876852728 conn(0x7f38e8060950 msgr2=0x7f38e80b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 shutdown_connections 2022-08-01T23:03:53.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.405+0000 7f38f07ca700 1 -- 172.21.15.32:0/1876852728 wait complete. 2022-08-01T23:03:53.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa824ca4700 1 Processor -- start 2022-08-01T23:03:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa824ca4700 1 -- start start 2022-08-01T23:03:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa824ca4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa8201445d0 0x7fa820144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa824ca4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa820060670 con 0x7fa8200655b0 2022-08-01T23:03:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa824ca4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8200b2540 con 0x7fa820060290 2022-08-01T23:03:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa824ca4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa820065e10 con 0x7fa8201445d0 2022-08-01T23:03:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa8200655b0 0x7fa82014b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35132/0 (socket says 172.21.15.32:35132) 2022-08-01T23:03:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4118749695 learned_addr learned my addr 172.21.15.32:0/4118749695 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81dd9b700 1 --2- 172.21.15.32:0/4118749695 >> v2:172.21.15.55:3300/0 conn(0x7fa8201445d0 0x7fa820144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654845900 0 0) 0x7fa820060670 con 0x7fa8200655b0 2022-08-01T23:03:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa804003260 con 0x7fa8200655b0 2022-08-01T23:03:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540437902 0 0) 0x7fa804003260 con 0x7fa8200655b0 2022-08-01T23:03:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 >> v2:172.21.15.55:3300/0 conn(0x7fa8201445d0 msgr2=0x7fa820144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 --2- 172.21.15.32:0/4118749695 >> v2:172.21.15.55:3300/0 conn(0x7fa8201445d0 0x7fa820144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 >> v1:172.21.15.55:6789/0 conn(0x7fa820060290 legacy=0x7fa820064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200b27e0 con 0x7fa8200655b0 2022-08-01T23:03:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fa818001a00 con 0x7fa8200655b0 2022-08-01T23:03:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 >> v1:172.21.15.32:6789/0 conn(0x7fa8200655b0 legacy=0x7fa82014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa804003750 0x7fa804003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004500 0x7fa804004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa804004e70 0x7fa80400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa820065e10 con 0x7fa804004e70 2022-08-01T23:03:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80400bd20 con 0x7fa804003750 2022-08-01T23:03:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80400bee0 con 0x7fa804004500 2022-08-01T23:03:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa804004e70 0x7fa80400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81e59c700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa804003750 0x7fa804003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81dd9b700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004500 0x7fa804004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004500 msgr2=0x7fa804004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004500 0x7fa804004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa804003750 msgr2=0x7fa804003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa804003750 0x7fa804003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4118749695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804003260 con 0x7fa804004e70 2022-08-01T23:03:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81dd9b700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004500 0x7fa804004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4118749695 reap_dead start 2022-08-01T23:03:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa804004e70 0x7fa80400b750 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fa818001710 tx=0x7fa8180021b0).ready entity=mon.0 client_cookie=d5e439ee1da0f4af server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81800cc40 con 0x7fa804004e70 2022-08-01T23:03:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81800cda0 con 0x7fa804004e70 2022-08-01T23:03:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.525+0000 7fa81d59a700 1 -- 172.21.15.32:0/4118749695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa818004b50 con 0x7fa804004e70 2022-08-01T23:03:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- 172.21.15.32:0/4118749695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa804004e70 msgr2=0x7fa80400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa804004e70 0x7fa80400b750 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fa818001710 tx=0x7fa8180021b0).stop 2022-08-01T23:03:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- 172.21.15.32:0/4118749695 shutdown_connections 2022-08-01T23:03:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4118749695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa804004e70 0x7fa80400b750 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- 172.21.15.32:0/4118749695 >> 172.21.15.32:0/4118749695 conn(0x7fa82005d0f0 msgr2=0x7fa82005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- 172.21.15.32:0/4118749695 shutdown_connections 2022-08-01T23:03:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- 172.21.15.32:0/4118749695 wait complete. 2022-08-01T23:03:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 Processor -- start 2022-08-01T23:03:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- start start 2022-08-01T23:03:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004e70 0x7fa820140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa82013b9c0 0x7fa820153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 0x7fa8200afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200b1740 con 0x7fa820154220 2022-08-01T23:03:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200b19e0 con 0x7fa82013b9c0 2022-08-01T23:03:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.529+0000 7fa824ca4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82013bf70 con 0x7fa804004e70 2022-08-01T23:03:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 0x7fa8200afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 0x7fa8200afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47600/0 (socket says 172.21.15.32:47600) 2022-08-01T23:03:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4292954804 learned_addr learned my addr 172.21.15.32:0/4292954804 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81e59c700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004e70 0x7fa820140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81dd9b700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa82013b9c0 0x7fa820153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004e70 msgr2=0x7fa820140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004e70 0x7fa820140470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa82013b9c0 msgr2=0x7fa820153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa82013b9c0 0x7fa820153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 -- 172.21.15.32:0/4292954804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82013c210 con 0x7fa820154220 2022-08-01T23:03:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81ed9d700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 0x7fa8200afa30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fa818001710 tx=0x7fa818015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa818007b50 con 0x7fa820154220 2022-08-01T23:03:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818007380 con 0x7fa820154220 2022-08-01T23:03:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82013c4b0 con 0x7fa820154220 2022-08-01T23:03:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8180076e0 con 0x7fa820154220 2022-08-01T23:03:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8200b0b80 con 0x7fa820154220 2022-08-01T23:03:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa820064a00 con 0x7fa820154220 2022-08-01T23:03:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa818014410 con 0x7fa820154220 2022-08-01T23:03:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa80f7fe700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa7fc053b00 0x7fa7fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa81806c070 con 0x7fa820154220 2022-08-01T23:03:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.533+0000 7fa81e59c700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa7fc053b00 0x7fa7fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.537+0000 7fa81e59c700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa7fc053b00 0x7fa7fc055fc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fa81400d000 tx=0x7fa814009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.541+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa818043110 con 0x7fa820154220 2022-08-01T23:03:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.669+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8200574e0 con 0x7fa820154220 2022-08-01T23:03:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.669+0000 7fa80f7fe700 1 -- 172.21.15.32:0/4292954804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa818024030 con 0x7fa820154220 2022-08-01T23:03:53.675 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:03:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa7fc053b00 msgr2=0x7fa7fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa7fc053b00 0x7fa7fc055fc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fa81400d000 tx=0x7fa814009040).stop 2022-08-01T23:03:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 msgr2=0x7fa8200afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 0x7fa8200afa30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fa818001710 tx=0x7fa818015000).stop 2022-08-01T23:03:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 shutdown_connections 2022-08-01T23:03:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa7fc053b00 0x7fa7fc055fc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820154220 0x7fa8200afa30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa82013b9c0 0x7fa820153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 --2- 172.21.15.32:0/4292954804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa804004e70 0x7fa820140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 >> 172.21.15.32:0/4292954804 conn(0x7fa82005d0f0 msgr2=0x7fa820143e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 shutdown_connections 2022-08-01T23:03:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.673+0000 7fa824ca4700 1 -- 172.21.15.32:0/4292954804 wait complete. 2022-08-01T23:03:53.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:03:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 Processor -- start 2022-08-01T23:03:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- start start 2022-08-01T23:03:53.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ef00619b0 0x7f5ef00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ef00b6ee0 0x7f5ef0143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00b5ed0 0x7f5ef00b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef0142f60 con 0x7f5ef00619b0 2022-08-01T23:03:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef0061d90 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00ba5a0 con 0x7f5ef00b5ed0 2022-08-01T23:03:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ef00619b0 0x7f5ef00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ef00619b0 0x7f5ef00b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47608/0 (socket says 172.21.15.32:47608) 2022-08-01T23:03:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 -- 172.21.15.32:0/4007505912 learned_addr learned my addr 172.21.15.32:0/4007505912 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef49f9700 1 --2- 172.21.15.32:0/4007505912 >> v2:172.21.15.55:3300/0 conn(0x7f5ef00b6ee0 0x7f5ef0143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef59fb700 1 --2- 172.21.15.32:0/4007505912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00b5ed0 0x7f5ef00b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 -- 172.21.15.32:0/4007505912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00b5ed0 msgr2=0x7f5ef00b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 --2- 172.21.15.32:0/4007505912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00b5ed0 0x7f5ef00b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 -- 172.21.15.32:0/4007505912 >> v2:172.21.15.55:3300/0 conn(0x7f5ef00b6ee0 msgr2=0x7f5ef0143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 --2- 172.21.15.32:0/4007505912 >> v2:172.21.15.55:3300/0 conn(0x7f5ef00b6ee0 0x7f5ef0143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 -- 172.21.15.32:0/4007505912 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0143150 con 0x7f5ef00619b0 2022-08-01T23:03:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef51fa700 1 --2- 172.21.15.32:0/4007505912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00619b0 0x7f5ef00b48a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f5eec002f70 tx=0x7f5eec006eb0).ready entity=mon.0 client_cookie=74cca97ffde73f0d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ee7fff700 1 -- 172.21.15.32:0/4007505912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec00d570 con 0x7f5ef00619b0 2022-08-01T23:03:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ee7fff700 1 -- 172.21.15.32:0/4007505912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec000e90 con 0x7f5ef00619b0 2022-08-01T23:03:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ee7fff700 1 -- 172.21.15.32:0/4007505912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec00d900 con 0x7f5ef00619b0 2022-08-01T23:03:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- 172.21.15.32:0/4007505912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00619b0 msgr2=0x7f5ef00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/4007505912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00619b0 0x7f5ef00b48a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f5eec002f70 tx=0x7f5eec006eb0).stop 2022-08-01T23:03:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- 172.21.15.32:0/4007505912 shutdown_connections 2022-08-01T23:03:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/4007505912 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00b5ed0 0x7f5ef00b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/4007505912 >> v2:172.21.15.55:3300/0 conn(0x7f5ef00b6ee0 0x7f5ef0143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/4007505912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00619b0 0x7f5ef00b48a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- 172.21.15.32:0/4007505912 >> 172.21.15.32:0/4007505912 conn(0x7f5ef006af80 msgr2=0x7f5ef005fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- 172.21.15.32:0/4007505912 shutdown_connections 2022-08-01T23:03:53.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.793+0000 7f5ef61fc700 1 -- 172.21.15.32:0/4007505912 wait complete. 2022-08-01T23:03:53.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 Processor -- start 2022-08-01T23:03:53.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 -- start start 2022-08-01T23:03:53.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00619b0 0x7f5ef0141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ef00b5ed0 0x7f5ef01562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 0x7f5ef01567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0112a30 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0112c60 con 0x7f5ef00b5ed0 2022-08-01T23:03:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0112f00 con 0x7f5ef00619b0 2022-08-01T23:03:53.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 0x7f5ef01567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 0x7f5ef01567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47616/0 (socket says 172.21.15.32:47616) 2022-08-01T23:03:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef51fa700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00619b0 0x7f5ef0141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 -- 172.21.15.32:0/599593012 learned_addr learned my addr 172.21.15.32:0/599593012 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:53.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef49f9700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ef00b5ed0 0x7f5ef01562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 -- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00619b0 msgr2=0x7f5ef0141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00619b0 0x7f5ef0141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 -- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ef00b5ed0 msgr2=0x7f5ef01562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ef00b5ed0 0x7f5ef01562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 -- 172.21.15.32:0/599593012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec00b1e0 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef49f9700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ef00b5ed0 0x7f5ef01562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef59fb700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 0x7f5ef01567c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f5ee800a000 tx=0x7f5ee8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee8013620 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8013780 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef0152670 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee8013ae0 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.797+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef00aed20 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.801+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0152800 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.801+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5ee8013c40 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.801+0000 7f5ee67fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5ed8053a70 0x7f5ed8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.801+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5ee806baf0 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.801+0000 7f5ef51fa700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5ed8053a70 0x7f5ed8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.801+0000 7f5ef51fa700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5ed8053a70 0x7f5ed8055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f5eec01c000 tx=0x7f5eec015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.809+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ee8042ae0 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.933+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5ef00574e0 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ee67fc700 1 -- 172.21.15.32:0/599593012 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5ee800f040 con 0x7f5ef00b6ee0 2022-08-01T23:03:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:53.941 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:03:53.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:03:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:03:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:03:53.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:03:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:03:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:03:53.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:03:53.943 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:03:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5ed8053a70 msgr2=0x7f5ed8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5ed8053a70 0x7f5ed8055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f5eec01c000 tx=0x7f5eec015000).stop 2022-08-01T23:03:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 msgr2=0x7f5ef01567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 0x7f5ef01567c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f5ee800a000 tx=0x7f5ee8008040).stop 2022-08-01T23:03:53.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 shutdown_connections 2022-08-01T23:03:53.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5ed8053a70 0x7f5ed8055f30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef00b6ee0 0x7f5ef01567c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ef00b5ed0 0x7f5ef01562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 --2- 172.21.15.32:0/599593012 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5ef00619b0 0x7f5ef0141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 >> 172.21.15.32:0/599593012 conn(0x7f5ef006af80 msgr2=0x7f5ef00b97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 shutdown_connections 2022-08-01T23:03:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:53.937+0000 7f5ef61fc700 1 -- 172.21.15.32:0/599593012 wait complete. 2022-08-01T23:03:53.954 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:03:53.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:03:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.053+0000 7fbce9dd0700 1 Processor -- start 2022-08-01T23:03:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.053+0000 7fbce9dd0700 1 -- start start 2022-08-01T23:03:54.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbce40b2050 0x7fbce40b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40b0140 0x7fbce40b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce40b05b0 con 0x7fbce40a7250 2022-08-01T23:03:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce40b2920 con 0x7fbce40b2050 2022-08-01T23:03:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce40b07e0 con 0x7fbce40b0140 2022-08-01T23:03:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbce40b2050 0x7fbce40b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbce40a7250 0x7fbce41413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35164/0 (socket says 172.21.15.32:35164) 2022-08-01T23:03:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbce40b2050 0x7fbce40b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47620/0 (socket says 172.21.15.32:47620) 2022-08-01T23:03:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce3fff700 1 -- 172.21.15.32:0/724709584 learned_addr learned my addr 172.21.15.32:0/724709584 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:54.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce2ffd700 1 --2- 172.21.15.32:0/724709584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40b0140 0x7fbce40b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 -- 172.21.15.32:0/724709584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40b0140 msgr2=0x7fbce40b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:54.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 --2- 172.21.15.32:0/724709584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40b0140 0x7fbce40b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 -- 172.21.15.32:0/724709584 >> v1:172.21.15.32:6789/0 conn(0x7fbce40a7250 legacy=0x7fbce41413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 -- 172.21.15.32:0/724709584 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40b0a80 con 0x7fbce40b2050 2022-08-01T23:03:54.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce37fe700 1 --2- 172.21.15.32:0/724709584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce40b1460 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7fbcd8002f70 tx=0x7fbcd8006df0).ready entity=mon.0 client_cookie=ef5c9587114b15af server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:54.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce27fc700 1 -- 172.21.15.32:0/724709584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcd800d570 con 0x7fbce40b2050 2022-08-01T23:03:54.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce27fc700 1 -- 172.21.15.32:0/724709584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8000e90 con 0x7fbce40b2050 2022-08-01T23:03:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce27fc700 1 -- 172.21.15.32:0/724709584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcd800d8c0 con 0x7fbce40b2050 2022-08-01T23:03:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- 172.21.15.32:0/724709584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 msgr2=0x7fbce40b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/724709584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce40b1460 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7fbcd8002f70 tx=0x7fbcd8006df0).stop 2022-08-01T23:03:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- 172.21.15.32:0/724709584 shutdown_connections 2022-08-01T23:03:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/724709584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40b0140 0x7fbce40b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/724709584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce40b1460 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- 172.21.15.32:0/724709584 >> 172.21.15.32:0/724709584 conn(0x7fbce405f840 msgr2=0x7fbce405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- 172.21.15.32:0/724709584 shutdown_connections 2022-08-01T23:03:54.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.057+0000 7fbce9dd0700 1 -- 172.21.15.32:0/724709584 wait complete. 2022-08-01T23:03:54.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 Processor -- start 2022-08-01T23:03:54.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 -- start start 2022-08-01T23:03:54.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40a7250 0x7fbce41465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce40b0140 0x7fbce415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce41579f0 con 0x7fbce40b2050 2022-08-01T23:03:54.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce415a3a0 con 0x7fbce40b0140 2022-08-01T23:03:54.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce9dd0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce4157710 con 0x7fbce40a7250 2022-08-01T23:03:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce415ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47630/0 (socket says 172.21.15.32:47630) 2022-08-01T23:03:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce40b0140 0x7fbce415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 -- 172.21.15.32:0/1966452061 learned_addr learned my addr 172.21.15.32:0/1966452061 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce37fe700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40a7250 0x7fbce41465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 -- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40a7250 msgr2=0x7fbce41465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40a7250 0x7fbce41465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 -- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce40b0140 msgr2=0x7fbce415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce40b0140 0x7fbce415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 -- 172.21.15.32:0/1966452061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd800b180 con 0x7fbce40b2050 2022-08-01T23:03:54.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce3fff700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce415ad30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7fbce406a280 tx=0x7fbcdc007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:54.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.061+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcdc0070d0 con 0x7fbce40b2050 2022-08-01T23:03:54.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc007230 con 0x7fbce40b2050 2022-08-01T23:03:54.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce414e190 con 0x7fbce40b2050 2022-08-01T23:03:54.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcdc005610 con 0x7fbce40b2050 2022-08-01T23:03:54.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce4142da0 con 0x7fbce40b2050 2022-08-01T23:03:54.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce415a560 con 0x7fbce40b2050 2022-08-01T23:03:54.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbcdc010070 con 0x7fbce40b2050 2022-08-01T23:03:54.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce0ff9700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcc40539e0 0x7fbcc4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fbcdc06b2e0 con 0x7fbce40b2050 2022-08-01T23:03:54.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce37fe700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcc40539e0 0x7fbcc4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.065+0000 7fbce37fe700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcc40539e0 0x7fbcc4055ea0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fbcd801a000 tx=0x7fbcd8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:54.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.069+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbcdc0423e0 con 0x7fbce40b2050 2022-08-01T23:03:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.189+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fbce40b0d20 con 0x7fbce40b2050 2022-08-01T23:03:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.305+0000 7fbce0ff9700 1 -- 172.21.15.32:0/1966452061 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v114) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbcdc045a00 con 0x7fbce40b2050 2022-08-01T23:03:54.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcc40539e0 msgr2=0x7fbcc4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcc40539e0 0x7fbcc4055ea0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fbcd801a000 tx=0x7fbcd8014000).stop 2022-08-01T23:03:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 msgr2=0x7fbce415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce415ad30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7fbce406a280 tx=0x7fbcdc007bc0).stop 2022-08-01T23:03:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 shutdown_connections 2022-08-01T23:03:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcc40539e0 0x7fbcc4055ea0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce40b2050 0x7fbce415ad30 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce40b0140 0x7fbce415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 --2- 172.21.15.32:0/1966452061 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbce40a7250 0x7fbce41465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 >> 172.21.15.32:0/1966452061 conn(0x7fbce405f840 msgr2=0x7fbce413e130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:54.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 shutdown_connections 2022-08-01T23:03:54.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.309+0000 7fbce9dd0700 1 -- 172.21.15.32:0/1966452061 wait complete. 2022-08-01T23:03:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:03:54.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:03:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 Processor -- start 2022-08-01T23:03:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 -- start start 2022-08-01T23:03:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6d140b1bc0 0x7f6d14066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d14140e40 0x7f6d1413e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1405f8e0 con 0x7f6d14066780 2022-08-01T23:03:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140b1630 con 0x7f6d14140e40 2022-08-01T23:03:54.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1405fb10 con 0x7f6d140b1bc0 2022-08-01T23:03:54.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d14066780 0x7f6d1405ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35176/0 (socket says 172.21.15.32:35176) 2022-08-01T23:03:54.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1259c700 1 -- 172.21.15.32:0/3547707207 learned_addr learned my addr 172.21.15.32:0/3547707207 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:54.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d12d9d700 1 --2- 172.21.15.32:0/3547707207 >> v2:172.21.15.32:3300/0 conn(0x7f6d14140e40 0x7f6d1413e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d11d9b700 1 --2- 172.21.15.32:0/3547707207 >> v2:172.21.15.55:3300/0 conn(0x7f6d140b1bc0 0x7f6d14066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1159a700 1 -- 172.21.15.32:0/3547707207 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285338628 0 0) 0x7f6d1405f8e0 con 0x7f6d14066780 2022-08-01T23:03:54.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.433+0000 7f6d1159a700 1 -- 172.21.15.32:0/3547707207 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf8003260 con 0x7f6d14066780 2022-08-01T23:03:54.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d11d9b700 1 -- 172.21.15.32:0/3547707207 >> v2:172.21.15.32:3300/0 conn(0x7f6d14140e40 msgr2=0x7f6d1413e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d11d9b700 1 --2- 172.21.15.32:0/3547707207 >> v2:172.21.15.32:3300/0 conn(0x7f6d14140e40 0x7f6d1413e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d11d9b700 1 -- 172.21.15.32:0/3547707207 >> v1:172.21.15.32:6789/0 conn(0x7f6d14066780 legacy=0x7f6d1405ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d11d9b700 1 -- 172.21.15.32:0/3547707207 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140b18d0 con 0x7f6d140b1bc0 2022-08-01T23:03:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d11d9b700 1 --2- 172.21.15.32:0/3547707207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d140b1bc0 0x7f6d14066b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f6d08002f70 tx=0x7f6d08006df0).ready entity=mon.1 client_cookie=e3087715756627bb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1159a700 1 -- 172.21.15.32:0/3547707207 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d0800d4f0 con 0x7f6d140b1bc0 2022-08-01T23:03:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1159a700 1 -- 172.21.15.32:0/3547707207 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08000e90 con 0x7f6d140b1bc0 2022-08-01T23:03:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1159a700 1 -- 172.21.15.32:0/3547707207 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d0800d840 con 0x7f6d140b1bc0 2022-08-01T23:03:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 -- 172.21.15.32:0/3547707207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d140b1bc0 msgr2=0x7f6d14066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 --2- 172.21.15.32:0/3547707207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d140b1bc0 0x7f6d14066b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f6d08002f70 tx=0x7f6d08006df0).stop 2022-08-01T23:03:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 -- 172.21.15.32:0/3547707207 shutdown_connections 2022-08-01T23:03:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 --2- 172.21.15.32:0/3547707207 >> v2:172.21.15.32:3300/0 conn(0x7f6d14140e40 0x7f6d1413e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 --2- 172.21.15.32:0/3547707207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d140b1bc0 0x7f6d14066b60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 -- 172.21.15.32:0/3547707207 >> 172.21.15.32:0/3547707207 conn(0x7f6d14063870 msgr2=0x7f6d14063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 -- 172.21.15.32:0/3547707207 shutdown_connections 2022-08-01T23:03:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.437+0000 7f6d1359e700 1 -- 172.21.15.32:0/3547707207 wait complete. 2022-08-01T23:03:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 Processor -- start 2022-08-01T23:03:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 -- start start 2022-08-01T23:03:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d14066780 0x7f6d140b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 0x7f6d14139130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d14140e40 0x7f6d14136230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14155420 con 0x7f6d140b1bc0 2022-08-01T23:03:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1413a9d0 con 0x7f6d14066780 2022-08-01T23:03:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1413ac70 con 0x7f6d14140e40 2022-08-01T23:03:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 0x7f6d14139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 0x7f6d14139130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47642/0 (socket says 172.21.15.32:47642) 2022-08-01T23:03:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 -- 172.21.15.32:0/1953435275 learned_addr learned my addr 172.21.15.32:0/1953435275 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d1259c700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d14066780 0x7f6d140b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d12d9d700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d14140e40 0x7f6d14136230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 -- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d14140e40 msgr2=0x7f6d14136230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d14140e40 0x7f6d14136230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 -- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d14066780 msgr2=0x7f6d140b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d14066780 0x7f6d140b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.441+0000 7f6d11d9b700 1 -- 172.21.15.32:0/1953435275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0800b180 con 0x7f6d140b1bc0 2022-08-01T23:03:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d11d9b700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 0x7f6d14139130 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f6d08002f70 tx=0x7f6d08014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d08016070 con 0x7f6d140b1bc0 2022-08-01T23:03:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08013c20 con 0x7f6d140b1bc0 2022-08-01T23:03:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d14155610 con 0x7f6d140b1bc0 2022-08-01T23:03:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d0801e450 con 0x7f6d140b1bc0 2022-08-01T23:03:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d14139860 con 0x7f6d140b1bc0 2022-08-01T23:03:54.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1413a770 con 0x7f6d140b1bc0 2022-08-01T23:03:54.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.445+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6d0800f030 con 0x7f6d140b1bc0 2022-08-01T23:03:54.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.449+0000 7f6d037fe700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cf0053b00 0x7f6cf0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.449+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f6d0806cc80 con 0x7f6d140b1bc0 2022-08-01T23:03:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.449+0000 7f6d1259c700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cf0053b00 0x7f6cf0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.449+0000 7f6d1259c700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cf0053b00 0x7f6cf0055fc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f6d0400d000 tx=0x7f6d04009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:54.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.457+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d08043b30 con 0x7f6d140b1bc0 2022-08-01T23:03:54.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:54.559 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:54.562 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:54.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:03:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:54.585+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f6d141557a0 con 0x7f6d140b1bc0 2022-08-01T23:03:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.313+0000 7f6d037fe700 1 -- 172.21.15.32:0/1953435275 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v115) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6d08023030 con 0x7f6d140b1bc0 2022-08-01T23:03:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cf0053b00 msgr2=0x7f6cf0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cf0053b00 0x7f6cf0055fc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f6d0400d000 tx=0x7f6d04009040).stop 2022-08-01T23:03:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 msgr2=0x7f6d14139130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 0x7f6d14139130 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f6d08002f70 tx=0x7f6d08014000).stop 2022-08-01T23:03:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 shutdown_connections 2022-08-01T23:03:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6cf0053b00 0x7f6cf0055fc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6d14140e40 0x7f6d14136230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d140b1bc0 0x7f6d14139130 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 --2- 172.21.15.32:0/1953435275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d14066780 0x7f6d140b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 >> 172.21.15.32:0/1953435275 conn(0x7f6d14063870 msgr2=0x7f6d1413fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:55.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 shutdown_connections 2022-08-01T23:03:55.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.321+0000 7f6d1359e700 1 -- 172.21.15.32:0/1953435275 wait complete. 2022-08-01T23:03:55.330 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:03:55.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:03:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 Processor -- start 2022-08-01T23:03:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 -- start start 2022-08-01T23:03:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efc200af450 0x7efc200af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad9e0 0x7efc200a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200aff30 con 0x7efc200ad4b0 2022-08-01T23:03:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2005c7a0 con 0x7efc200af450 2022-08-01T23:03:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26e68700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc20140990 con 0x7efc200ad9e0 2022-08-01T23:03:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc25665700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efc200ad4b0 0x7efc201432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35190/0 (socket says 172.21.15.32:35190) 2022-08-01T23:03:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc25665700 1 -- 172.21.15.32:0/3029743245 learned_addr learned my addr 172.21.15.32:0/3029743245 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc26667700 1 --2- 172.21.15.32:0/3029743245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad9e0 0x7efc200a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.445+0000 7efc25e66700 1 --2- 172.21.15.32:0/3029743245 >> v2:172.21.15.55:3300/0 conn(0x7efc200af450 0x7efc200af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc24e64700 1 -- 172.21.15.32:0/3029743245 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925306217 0 0) 0x7efc200aff30 con 0x7efc200ad4b0 2022-08-01T23:03:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc24e64700 1 -- 172.21.15.32:0/3029743245 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc08003260 con 0x7efc200ad4b0 2022-08-01T23:03:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26667700 1 -- 172.21.15.32:0/3029743245 >> v2:172.21.15.55:3300/0 conn(0x7efc200af450 msgr2=0x7efc200af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26667700 1 --2- 172.21.15.32:0/3029743245 >> v2:172.21.15.55:3300/0 conn(0x7efc200af450 0x7efc200af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26667700 1 -- 172.21.15.32:0/3029743245 >> v1:172.21.15.32:6789/0 conn(0x7efc200ad4b0 legacy=0x7efc201432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26667700 1 -- 172.21.15.32:0/3029743245 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc201439f0 con 0x7efc200ad9e0 2022-08-01T23:03:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26667700 1 --2- 172.21.15.32:0/3029743245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad9e0 0x7efc200a8e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7efc200ad900 tx=0x7efc180091d0).ready entity=mon.2 client_cookie=8aa335bc17af1618 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc24e64700 1 -- 172.21.15.32:0/3029743245 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc1800b600 con 0x7efc200ad9e0 2022-08-01T23:03:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc24e64700 1 -- 172.21.15.32:0/3029743245 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800b760 con 0x7efc200ad9e0 2022-08-01T23:03:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc24e64700 1 -- 172.21.15.32:0/3029743245 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc1800b8d0 con 0x7efc200ad9e0 2022-08-01T23:03:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 -- 172.21.15.32:0/3029743245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad9e0 msgr2=0x7efc200a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 --2- 172.21.15.32:0/3029743245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad9e0 0x7efc200a8e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7efc200ad900 tx=0x7efc180091d0).stop 2022-08-01T23:03:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 -- 172.21.15.32:0/3029743245 shutdown_connections 2022-08-01T23:03:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 --2- 172.21.15.32:0/3029743245 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad9e0 0x7efc200a8e40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 --2- 172.21.15.32:0/3029743245 >> v2:172.21.15.55:3300/0 conn(0x7efc200af450 0x7efc200af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 -- 172.21.15.32:0/3029743245 >> 172.21.15.32:0/3029743245 conn(0x7efc200612d0 msgr2=0x7efc20062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 -- 172.21.15.32:0/3029743245 shutdown_connections 2022-08-01T23:03:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.449+0000 7efc26e68700 1 -- 172.21.15.32:0/3029743245 wait complete. 2022-08-01T23:03:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 Processor -- start 2022-08-01T23:03:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 -- start start 2022-08-01T23:03:55.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad4b0 0x7efc200ab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 0x7efc20063af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc200af450 0x7efc20063fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200b55c0 con 0x7efc200ad9e0 2022-08-01T23:03:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20066080 con 0x7efc200af450 2022-08-01T23:03:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26e68700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc20066320 con 0x7efc200ad4b0 2022-08-01T23:03:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc25665700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 0x7efc20063af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc25e66700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad4b0 0x7efc200ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc25665700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 0x7efc20063af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47654/0 (socket says 172.21.15.32:47654) 2022-08-01T23:03:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc25665700 1 -- 172.21.15.32:0/1014408274 learned_addr learned my addr 172.21.15.32:0/1014408274 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.453+0000 7efc26667700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc200af450 0x7efc20063fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25665700 1 -- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad4b0 msgr2=0x7efc200ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25665700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad4b0 0x7efc200ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25665700 1 -- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc200af450 msgr2=0x7efc20063fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25665700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc200af450 0x7efc20063fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25665700 1 -- 172.21.15.32:0/1014408274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009470 con 0x7efc200ad9e0 2022-08-01T23:03:55.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25e66700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad4b0 0x7efc200ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25665700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 0x7efc20063af0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7efc10001110 tx=0x7efc10007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc100070d0 con 0x7efc200ad9e0 2022-08-01T23:03:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10007230 con 0x7efc200ad9e0 2022-08-01T23:03:55.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc200665c0 con 0x7efc200ad9e0 2022-08-01T23:03:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc10005610 con 0x7efc200ad9e0 2022-08-01T23:03:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc20140250 con 0x7efc200ad9e0 2022-08-01T23:03:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efc10010070 con 0x7efc200ad9e0 2022-08-01T23:03:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc200574e0 con 0x7efc200ad9e0 2022-08-01T23:03:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc0effd700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc00053b00 0x7efc00055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6304+0+0 (secure 0 0 0) 0x7efc1006b5f0 con 0x7efc200ad9e0 2022-08-01T23:03:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.457+0000 7efc25e66700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc00053b00 0x7efc00055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.461+0000 7efc25e66700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc00053b00 0x7efc00055fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7efc1c00b000 tx=0x7efc1c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:55.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.465+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efc10042430 con 0x7efc200ad9e0 2022-08-01T23:03:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.633+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7efc201437d0 con 0x7efc200ad9e0 2022-08-01T23:03:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.641+0000 7efc0effd700 1 -- 172.21.15.32:0/1014408274 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v55) v1 ==== 189+0+0 (secure 0 0 0) 0x7efc10045a50 con 0x7efc200ad9e0 2022-08-01T23:03:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc00053b00 msgr2=0x7efc00055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc00053b00 0x7efc00055fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7efc1c00b000 tx=0x7efc1c006df0).stop 2022-08-01T23:03:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 msgr2=0x7efc20063af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 0x7efc20063af0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7efc10001110 tx=0x7efc10007ad0).stop 2022-08-01T23:03:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 shutdown_connections 2022-08-01T23:03:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efc00053b00 0x7efc00055fc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc200af450 0x7efc20063fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc200ad9e0 0x7efc20063af0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 --2- 172.21.15.32:0/1014408274 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efc200ad4b0 0x7efc200ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 >> 172.21.15.32:0/1014408274 conn(0x7efc200612d0 msgr2=0x7efc200af830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 shutdown_connections 2022-08-01T23:03:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.649+0000 7efc26e68700 1 -- 172.21.15.32:0/1014408274 wait complete. 2022-08-01T23:03:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 21 and data pool 22 2022-08-01T23:03:55.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 Processor -- start 2022-08-01T23:03:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 -- start start 2022-08-01T23:03:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 0x7f80cc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80cc0ad9e0 0x7f80cc0a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0aff30 con 0x7f80cc0ad4b0 2022-08-01T23:03:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc05c7a0 con 0x7f80cc0ad9e0 2022-08-01T23:03:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc140990 con 0x7f80cc0af450 2022-08-01T23:03:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80ca59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 0x7f80cc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80cad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80cc0ad9e0 0x7f80cc0a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80ca59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 0x7f80cc0af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59410/0 (socket says 172.21.15.32:59410) 2022-08-01T23:03:55.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.753+0000 7f80ca59c700 1 -- 172.21.15.32:0/329276957 learned_addr learned my addr 172.21.15.32:0/329276957 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80ca59c700 1 -- 172.21.15.32:0/329276957 >> v2:172.21.15.55:3300/0 conn(0x7f80cc0ad9e0 msgr2=0x7f80cc0a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80ca59c700 1 --2- 172.21.15.32:0/329276957 >> v2:172.21.15.55:3300/0 conn(0x7f80cc0ad9e0 0x7f80cc0a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80ca59c700 1 -- 172.21.15.32:0/329276957 >> v1:172.21.15.55:6789/0 conn(0x7f80cc0ad4b0 legacy=0x7f80cc1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80ca59c700 1 -- 172.21.15.32:0/329276957 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc1439f0 con 0x7f80cc0af450 2022-08-01T23:03:55.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80ca59c700 1 --2- 172.21.15.32:0/329276957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 0x7f80cc0af830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f80bc002f70 tx=0x7f80bc006eb0).ready entity=mon.2 client_cookie=d73ea4a3465eaf0a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:55.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80c959a700 1 -- 172.21.15.32:0/329276957 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80bc00d570 con 0x7f80cc0af450 2022-08-01T23:03:55.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80c959a700 1 -- 172.21.15.32:0/329276957 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc000e90 con 0x7f80cc0af450 2022-08-01T23:03:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80c959a700 1 -- 172.21.15.32:0/329276957 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80bc00d900 con 0x7f80cc0af450 2022-08-01T23:03:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 -- 172.21.15.32:0/329276957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 msgr2=0x7f80cc0af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 --2- 172.21.15.32:0/329276957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 0x7f80cc0af830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f80bc002f70 tx=0x7f80bc006eb0).stop 2022-08-01T23:03:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 -- 172.21.15.32:0/329276957 shutdown_connections 2022-08-01T23:03:55.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 --2- 172.21.15.32:0/329276957 >> v2:172.21.15.55:3300/0 conn(0x7f80cc0ad9e0 0x7f80cc0a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 --2- 172.21.15.32:0/329276957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0af450 0x7f80cc0af830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 -- 172.21.15.32:0/329276957 >> 172.21.15.32:0/329276957 conn(0x7f80cc0612d0 msgr2=0x7f80cc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 -- 172.21.15.32:0/329276957 shutdown_connections 2022-08-01T23:03:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.757+0000 7f80cb59e700 1 -- 172.21.15.32:0/329276957 wait complete. 2022-08-01T23:03:55.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 Processor -- start 2022-08-01T23:03:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 -- start start 2022-08-01T23:03:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc0ad4b0 0x7f80cc0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0ad9e0 0x7f80cc152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 0x7f80cc064600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc153630 con 0x7f80cc0af450 2022-08-01T23:03:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0a9f90 con 0x7f80cc0ad4b0 2022-08-01T23:03:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc14fc40 con 0x7f80cc0ad9e0 2022-08-01T23:03:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 0x7f80cc064600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 0x7f80cc064600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47682/0 (socket says 172.21.15.32:47682) 2022-08-01T23:03:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 -- 172.21.15.32:0/2999397417 learned_addr learned my addr 172.21.15.32:0/2999397417 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80ca59c700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc0ad4b0 0x7f80cc0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80c9d9b700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0ad9e0 0x7f80cc152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 -- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0ad9e0 msgr2=0x7f80cc152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0ad9e0 0x7f80cc152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 -- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc0ad4b0 msgr2=0x7f80cc0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc0ad4b0 0x7f80cc0b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 -- 172.21.15.32:0/2999397417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc00b1e0 con 0x7f80cc0af450 2022-08-01T23:03:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cad9d700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 0x7f80cc064600 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f80c4006e00 tx=0x7f80c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80c400c640 con 0x7f80cc0af450 2022-08-01T23:03:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c400c7a0 con 0x7f80cc0af450 2022-08-01T23:03:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc14fee0 con 0x7f80cc0af450 2022-08-01T23:03:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80c400cb00 con 0x7f80cc0af450 2022-08-01T23:03:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.761+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc064af0 con 0x7f80cc0af450 2022-08-01T23:03:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.765+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc0574e0 con 0x7f80cc0af450 2022-08-01T23:03:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.765+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f80c4022070 con 0x7f80cc0af450 2022-08-01T23:03:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.765+0000 7f80bb7fe700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f80a8053b00 0x7f80a8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.765+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f80c406c670 con 0x7f80cc0af450 2022-08-01T23:03:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.765+0000 7f80ca59c700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f80a8053b00 0x7f80a8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.765+0000 7f80ca59c700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f80a8053b00 0x7f80a8055fc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f80bc01c000 tx=0x7f80bc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.773+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f80c4043430 con 0x7f80cc0af450 2022-08-01T23:03:55.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.905+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80cc061b20 con 0x7f80cc0af450 2022-08-01T23:03:55.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80bb7fe700 1 -- 172.21.15.32:0/2999397417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f80c4046a50 con 0x7f80cc0af450 2022-08-01T23:03:55.914 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:55.914 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":117,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:55.637703+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:54.249192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:54.638843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:55.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f80a8053b00 msgr2=0x7f80a8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f80a8053b00 0x7f80a8055fc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f80bc01c000 tx=0x7f80bc015000).stop 2022-08-01T23:03:55.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 msgr2=0x7f80cc064600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:55.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 0x7f80cc064600 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f80c4006e00 tx=0x7f80c40091d0).stop 2022-08-01T23:03:55.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 shutdown_connections 2022-08-01T23:03:55.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f80a8053b00 0x7f80a8055fc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80cc0af450 0x7f80cc064600 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f80cc0ad9e0 0x7f80cc152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 --2- 172.21.15.32:0/2999397417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc0ad4b0 0x7f80cc0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:55.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 >> 172.21.15.32:0/2999397417 conn(0x7f80cc0612d0 msgr2=0x7f80cc0a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:55.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 shutdown_connections 2022-08-01T23:03:55.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:55.909+0000 7f80cb59e700 1 -- 172.21.15.32:0/2999397417 wait complete. 2022-08-01T23:03:55.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:56.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146d22b700 1 Processor -- start 2022-08-01T23:03:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146d22b700 1 -- start start 2022-08-01T23:03:56.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146d22b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14680b1f40 0x7f14680b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146d22b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14680b04e0 con 0x7f14680a7250 2022-08-01T23:03:56.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146d22b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14680b0780 con 0x7f1468148530 2022-08-01T23:03:56.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146d22b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14680a9250 con 0x7f14680b1f40 2022-08-01T23:03:56.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146759e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14680a7250 0x7f14681448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35232/0 (socket says 172.21.15.32:35232) 2022-08-01T23:03:56.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f146759e700 1 -- 172.21.15.32:0/2802396138 learned_addr learned my addr 172.21.15.32:0/2802396138 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:56.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1466d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14680b1f40 0x7f14680b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1465d9b700 1 -- 172.21.15.32:0/2802396138 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316936317 0 0) 0x7f14680b04e0 con 0x7f14680a7250 2022-08-01T23:03:56.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1465d9b700 1 -- 172.21.15.32:0/2802396138 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003260 con 0x7f14680a7250 2022-08-01T23:03:56.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1466d9d700 1 -- 172.21.15.32:0/2802396138 >> v1:172.21.15.55:6789/0 conn(0x7f1468148530 legacy=0x7f14680b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1465d9b700 1 -- 172.21.15.32:0/2802396138 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188277304 0 0) 0x7f14680b0780 con 0x7f1468148530 2022-08-01T23:03:56.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1466d9d700 1 -- 172.21.15.32:0/2802396138 >> v1:172.21.15.32:6789/0 conn(0x7f14680a7250 legacy=0x7f14681448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1466d9d700 1 -- 172.21.15.32:0/2802396138 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680b0a20 con 0x7f14680b1f40 2022-08-01T23:03:56.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1466d9d700 1 --2- 172.21.15.32:0/2802396138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680b1f40 0x7f14680b1160 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f14681478c0 tx=0x7f145c006df0).ready entity=mon.0 client_cookie=beae5750d5168132 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:56.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1465d9b700 1 -- 172.21.15.32:0/2802396138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f145c00d610 con 0x7f14680b1f40 2022-08-01T23:03:56.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1465d9b700 1 -- 172.21.15.32:0/2802396138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c000e90 con 0x7f14680b1f40 2022-08-01T23:03:56.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.025+0000 7f1465d9b700 1 -- 172.21.15.32:0/2802396138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f145c00d960 con 0x7f14680b1f40 2022-08-01T23:03:56.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- 172.21.15.32:0/2802396138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680b1f40 msgr2=0x7f14680b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 --2- 172.21.15.32:0/2802396138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680b1f40 0x7f14680b1160 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f14681478c0 tx=0x7f145c006df0).stop 2022-08-01T23:03:56.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- 172.21.15.32:0/2802396138 shutdown_connections 2022-08-01T23:03:56.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 --2- 172.21.15.32:0/2802396138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680b1f40 0x7f14680b1160 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- 172.21.15.32:0/2802396138 >> 172.21.15.32:0/2802396138 conn(0x7f146805f840 msgr2=0x7f146805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:56.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- 172.21.15.32:0/2802396138 shutdown_connections 2022-08-01T23:03:56.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- 172.21.15.32:0/2802396138 wait complete. 2022-08-01T23:03:56.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 Processor -- start 2022-08-01T23:03:56.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- start start 2022-08-01T23:03:56.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 0x7f14680a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14680b1f40 0x7f14680ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1468148530 0x7f1468148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1468156f40 con 0x7f14680a7250 2022-08-01T23:03:56.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14681571e0 con 0x7f14680b1f40 2022-08-01T23:03:56.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f146d22b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1468157480 con 0x7f1468148530 2022-08-01T23:03:56.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f1466d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 0x7f14680a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f1466d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 0x7f14680a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47692/0 (socket says 172.21.15.32:47692) 2022-08-01T23:03:56.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.029+0000 7f1466d9d700 1 -- 172.21.15.32:0/3685838999 learned_addr learned my addr 172.21.15.32:0/3685838999 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f146759e700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1468148530 0x7f1468148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f146659c700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14680b1f40 0x7f14680ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1466d9d700 1 -- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1468148530 msgr2=0x7f1468148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1466d9d700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1468148530 0x7f1468148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1466d9d700 1 -- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14680b1f40 msgr2=0x7f14680ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1466d9d700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14680b1f40 0x7f14680ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1466d9d700 1 -- 172.21.15.32:0/3685838999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c00b180 con 0x7f14680a7250 2022-08-01T23:03:56.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1466d9d700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 0x7f14680a97d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f14681478c0 tx=0x7f145c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:56.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f145c013e90 con 0x7f14680a7250 2022-08-01T23:03:56.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c002f70 con 0x7f14680a7250 2022-08-01T23:03:56.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14680a9cc0 con 0x7f14680a7250 2022-08-01T23:03:56.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f145c01e5f0 con 0x7f14680a7250 2022-08-01T23:03:56.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1468157920 con 0x7f14680a7250 2022-08-01T23:03:56.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146814a990 con 0x7f14680a7250 2022-08-01T23:03:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f145c025030 con 0x7f14680a7250 2022-08-01T23:03:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1457fff700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1444053aa0 0x7f1444055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f145c06cd00 con 0x7f14680a7250 2022-08-01T23:03:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.033+0000 7f146659c700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1444053aa0 0x7f1444055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.037+0000 7f146659c700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1444053aa0 0x7f1444055f60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f145800b000 tx=0x7f1458009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:56.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.041+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f146814a990 con 0x7f14680a7250 2022-08-01T23:03:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.169+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f146814a990 con 0x7f14680a7250 2022-08-01T23:03:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f1457fff700 1 -- 172.21.15.32:0/3685838999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f146814a990 con 0x7f14680a7250 2022-08-01T23:03:56.178 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:56.178 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":117,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:55.637703+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:54.249192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:54.638843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1444053aa0 msgr2=0x7f1444055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1444053aa0 0x7f1444055f60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f145800b000 tx=0x7f1458009040).stop 2022-08-01T23:03:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 msgr2=0x7f14680a97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 0x7f14680a97d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f14681478c0 tx=0x7f145c014000).stop 2022-08-01T23:03:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 shutdown_connections 2022-08-01T23:03:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1444053aa0 0x7f1444055f60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1468148530 0x7f1468148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14680b1f40 0x7f14680ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 --2- 172.21.15.32:0/3685838999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680a7250 0x7f14680a97d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 >> 172.21.15.32:0/3685838999 conn(0x7f146805f840 msgr2=0x7f14681470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 shutdown_connections 2022-08-01T23:03:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.173+0000 7f146d22b700 1 -- 172.21.15.32:0/3685838999 wait complete. 2022-08-01T23:03:56.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:03:56.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.293+0000 7fb97beb5700 1 Processor -- start 2022-08-01T23:03:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.293+0000 7fb97beb5700 1 -- start start 2022-08-01T23:03:56.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb97beb5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb9741403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb97beb5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb974143cf0 con 0x7fb9740b4310 2022-08-01T23:03:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb97beb5700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb974143f90 con 0x7fb9740b4820 2022-08-01T23:03:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb97beb5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741476a0 con 0x7fb974146ae0 2022-08-01T23:03:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979450700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9740b4310 0x7fb9741408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35242/0 (socket says 172.21.15.32:35242) 2022-08-01T23:03:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979450700 1 -- 172.21.15.32:0/42979802 learned_addr learned my addr 172.21.15.32:0/42979802 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb9741403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605327498 0 0) 0x7fb974143cf0 con 0x7fb9740b4310 2022-08-01T23:03:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb95c003260 con 0x7fb9740b4310 2022-08-01T23:03:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829871108 0 0) 0x7fb974143f90 con 0x7fb9740b4820 2022-08-01T23:03:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb974143cf0 con 0x7fb9740b4820 2022-08-01T23:03:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1417589695 0 0) 0x7fb95c003260 con 0x7fb9740b4310 2022-08-01T23:03:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 msgr2=0x7fb9741403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb9741403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 >> v1:172.21.15.55:6789/0 conn(0x7fb9740b4820 legacy=0x7fb9740b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740b0e20 con 0x7fb9740b4310 2022-08-01T23:03:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fb970001660 con 0x7fb9740b4310 2022-08-01T23:03:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb9741403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 >> v1:172.21.15.32:6789/0 conn(0x7fb9740b4310 legacy=0x7fb9741408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c003910 0x7fb95c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb95c004500 0x7fb95c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb95c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741476a0 con 0x7fb95c003910 2022-08-01T23:03:56.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c00b750 con 0x7fb95c004500 2022-08-01T23:03:56.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c00b910 con 0x7fb974146ae0 2022-08-01T23:03:56.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c003910 0x7fb95c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979450700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb95c004500 0x7fb95c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb97a452700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb95c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 -- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 msgr2=0x7fb95c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:56.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974146ae0 0x7fb95c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 -- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb95c004500 msgr2=0x7fb95c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:56.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb95c004500 0x7fb95c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 -- 172.21.15.32:0/42979802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96400b180 con 0x7fb95c003910 2022-08-01T23:03:56.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb979c51700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c003910 0x7fb95c003d10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7fb96400c000 tx=0x7fb964006df0).ready entity=mon.0 client_cookie=490ac59f203ece7b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96400e830 con 0x7fb95c003910 2022-08-01T23:03:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96400e990 con 0x7fb95c003910 2022-08-01T23:03:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.297+0000 7fb978c4f700 1 -- 172.21.15.32:0/42979802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96400ecf0 con 0x7fb95c003910 2022-08-01T23:03:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- 172.21.15.32:0/42979802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c003910 msgr2=0x7fb95c003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 --2- 172.21.15.32:0/42979802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c003910 0x7fb95c003d10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7fb96400c000 tx=0x7fb964006df0).stop 2022-08-01T23:03:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97a452700 1 -- 172.21.15.32:0/42979802 reap_dead start 2022-08-01T23:03:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- 172.21.15.32:0/42979802 shutdown_connections 2022-08-01T23:03:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- 172.21.15.32:0/42979802 >> 172.21.15.32:0/42979802 conn(0x7fb97405c620 msgr2=0x7fb97405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:56.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- 172.21.15.32:0/42979802 shutdown_connections 2022-08-01T23:03:56.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- 172.21.15.32:0/42979802 wait complete. 2022-08-01T23:03:56.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 Processor -- start 2022-08-01T23:03:56.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- start start 2022-08-01T23:03:56.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 0x7fb9740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97413d540 0x7fb9740b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974152a10 0x7fb974152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974145c20 con 0x7fb974145840 2022-08-01T23:03:56.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740b94e0 con 0x7fb97413d540 2022-08-01T23:03:56.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97beb5700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740b8530 con 0x7fb974152a10 2022-08-01T23:03:56.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 0x7fb9740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 0x7fb9740b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47712/0 (socket says 172.21.15.32:47712) 2022-08-01T23:03:56.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 -- 172.21.15.32:0/2968860528 learned_addr learned my addr 172.21.15.32:0/2968860528 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:56.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979450700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97413d540 0x7fb9740b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97a452700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974152a10 0x7fb974152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 -- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974152a10 msgr2=0x7fb974152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974152a10 0x7fb974152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 -- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97413d540 msgr2=0x7fb9740b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:56.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97413d540 0x7fb9740b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 -- 172.21.15.32:0/2968860528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740aeb70 con 0x7fb974145840 2022-08-01T23:03:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb97a452700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974152a10 0x7fb974152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.301+0000 7fb979c51700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 0x7fb9740b8b00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7fb96400c000 tx=0x7fb964019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96401b610 con 0x7fb974145840 2022-08-01T23:03:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964000e90 con 0x7fb974145840 2022-08-01T23:03:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9740aee10 con 0x7fb974145840 2022-08-01T23:03:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96401b960 con 0x7fb974145840 2022-08-01T23:03:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb974152570 con 0x7fb974145840 2022-08-01T23:03:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb96401bac0 con 0x7fb974145840 2022-08-01T23:03:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97413dd40 con 0x7fb974145840 2022-08-01T23:03:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb962ffd700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb954053aa0 0x7fb954055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.305+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb96406d7c0 con 0x7fb974145840 2022-08-01T23:03:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.309+0000 7fb979450700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb954053aa0 0x7fb954055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.309+0000 7fb979450700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb954053aa0 0x7fb954055f60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fb97000d000 tx=0x7fb970007040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.313+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb964044470 con 0x7fb974145840 2022-08-01T23:03:56.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:56.477+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb9740574e0 con 0x7fb974145840 2022-08-01T23:03:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.333+0000 7fb962ffd700 1 -- 172.21.15.32:0/2968860528 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb964047a90 con 0x7fb974145840 2022-08-01T23:03:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb954053aa0 msgr2=0x7fb954055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb954053aa0 0x7fb954055f60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fb97000d000 tx=0x7fb970007040).stop 2022-08-01T23:03:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 msgr2=0x7fb9740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 0x7fb9740b8b00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7fb96400c000 tx=0x7fb964019000).stop 2022-08-01T23:03:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 shutdown_connections 2022-08-01T23:03:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb954053aa0 0x7fb954055f60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb974152a10 0x7fb974152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb97413d540 0x7fb9740b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 --2- 172.21.15.32:0/2968860528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb974145840 0x7fb9740b8b00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 >> 172.21.15.32:0/2968860528 conn(0x7fb97405c620 msgr2=0x7fb9740616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 shutdown_connections 2022-08-01T23:03:57.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.337+0000 7fb97beb5700 1 -- 172.21.15.32:0/2968860528 wait complete. 2022-08-01T23:03:57.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:57.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.461+0000 7fa2f2fbb700 1 Processor -- start 2022-08-01T23:03:57.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.461+0000 7fa2f2fbb700 1 -- start start 2022-08-01T23:03:57.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0b2710 0x7fa2ec14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2ec0b01a0 0x7fa2ec14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2ec0afd60 0x7fa2ec0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0a7250 con 0x7fa2ec0b01a0 2022-08-01T23:03:57.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0a7480 con 0x7fa2ec0afd60 2022-08-01T23:03:57.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0a7720 con 0x7fa2ec0b2710 2022-08-01T23:03:57.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2ec0b01a0 0x7fa2ec14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2ec0b01a0 0x7fa2ec14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47720/0 (socket says 172.21.15.32:47720) 2022-08-01T23:03:57.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 -- 172.21.15.32:0/510578784 learned_addr learned my addr 172.21.15.32:0/510578784 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:57.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 -- 172.21.15.32:0/510578784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0b2710 msgr2=0x7fa2ec14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:57.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f1fb9700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0b2710 0x7fa2ec14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f27ba700 1 --2- 172.21.15.32:0/510578784 >> v2:172.21.15.55:3300/0 conn(0x7fa2ec0afd60 0x7fa2ec0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0b2710 0x7fa2ec14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 -- 172.21.15.32:0/510578784 >> v2:172.21.15.55:3300/0 conn(0x7fa2ec0afd60 msgr2=0x7fa2ec0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:57.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 --2- 172.21.15.32:0/510578784 >> v2:172.21.15.55:3300/0 conn(0x7fa2ec0afd60 0x7fa2ec0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 -- 172.21.15.32:0/510578784 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0a79c0 con 0x7fa2ec0b01a0 2022-08-01T23:03:57.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f27ba700 1 --2- 172.21.15.32:0/510578784 >> v2:172.21.15.55:3300/0 conn(0x7fa2ec0afd60 0x7fa2ec0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f1fb9700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0b2710 0x7fa2ec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f17b8700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec14b4c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fa2e80032f0 tx=0x7fa2e8009290).ready entity=mon.0 client_cookie=ef69562f1b90be4c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f0fb7700 1 -- 172.21.15.32:0/510578784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e800b870 con 0x7fa2ec0b01a0 2022-08-01T23:03:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f0fb7700 1 -- 172.21.15.32:0/510578784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e800b9d0 con 0x7fa2ec0b01a0 2022-08-01T23:03:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f0fb7700 1 -- 172.21.15.32:0/510578784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e800bb40 con 0x7fa2ec0b01a0 2022-08-01T23:03:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/510578784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b01a0 msgr2=0x7fa2ec14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec14b4c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fa2e80032f0 tx=0x7fa2e8009290).stop 2022-08-01T23:03:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/510578784 shutdown_connections 2022-08-01T23:03:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec14b4c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/510578784 >> v2:172.21.15.55:3300/0 conn(0x7fa2ec0afd60 0x7fa2ec0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/510578784 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0b2710 0x7fa2ec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/510578784 >> 172.21.15.32:0/510578784 conn(0x7fa2ec060310 msgr2=0x7fa2ec0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/510578784 shutdown_connections 2022-08-01T23:03:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/510578784 wait complete. 2022-08-01T23:03:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 Processor -- start 2022-08-01T23:03:57.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.465+0000 7fa2f2fbb700 1 -- start start 2022-08-01T23:03:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0afd60 0x7fa2ec0b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec05c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 0x7fa2ec14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec157170 con 0x7fa2ec0b2710 2022-08-01T23:03:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0abdb0 con 0x7fa2ec0b01a0 2022-08-01T23:03:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0ac050 con 0x7fa2ec0afd60 2022-08-01T23:03:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 0x7fa2ec14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 0x7fa2ec14cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47730/0 (socket says 172.21.15.32:47730) 2022-08-01T23:03:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 -- 172.21.15.32:0/2220959927 learned_addr learned my addr 172.21.15.32:0/2220959927 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:57.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f1fb9700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0afd60 0x7fa2ec0b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f17b8700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec05c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 -- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0afd60 msgr2=0x7fa2ec0b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0afd60 0x7fa2ec0b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 -- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ec0b01a0 msgr2=0x7fa2ec05c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec05c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 -- 172.21.15.32:0/2220959927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8009530 con 0x7fa2ec0b2710 2022-08-01T23:03:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f27ba700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 0x7fa2ec14cb40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fa2e4007000 tx=0x7fa2e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e4015070 con 0x7fa2ec0b2710 2022-08-01T23:03:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4010580 con 0x7fa2ec0b2710 2022-08-01T23:03:57.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec0ac2f0 con 0x7fa2ec0b2710 2022-08-01T23:03:57.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2e40108e0 con 0x7fa2ec0b2710 2022-08-01T23:03:57.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec14d030 con 0x7fa2ec0b2710 2022-08-01T23:03:57.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec0574e0 con 0x7fa2ec0b2710 2022-08-01T23:03:57.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa2e4010a40 con 0x7fa2ec0b2710 2022-08-01T23:03:57.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.469+0000 7fa2daffd700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2cc053a70 0x7fa2cc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.473+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa2e406bec0 con 0x7fa2ec0b2710 2022-08-01T23:03:57.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.473+0000 7fa2f1fb9700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2cc053a70 0x7fa2cc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.473+0000 7fa2f1fb9700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2cc053a70 0x7fa2cc055f30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fa2dc00b000 tx=0x7fa2dc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:57.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.477+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2e4042c80 con 0x7fa2ec0b2710 2022-08-01T23:03:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2ec069a70 con 0x7fa2ec0b2710 2022-08-01T23:03:57.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2daffd700 1 -- 172.21.15.32:0/2220959927 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fa2e40462a0 con 0x7fa2ec0b2710 2022-08-01T23:03:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:57.646 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1851140084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1851140084},{"type":"v1","addr":"172.21.15.55:6835","nonce":1851140084}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1909728706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1909728706},{"type":"v1","addr":"172.21.15.32:6833","nonce":1909728706}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:55.638124+0000","modified":"2022-08-01T23:03:57.319377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/519921040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":519921040},{"type":"v1","addr":"172.21.15.32:6835","nonce":519921040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T23:03:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2cc053a70 msgr2=0x7fa2cc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2cc053a70 0x7fa2cc055f30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fa2dc00b000 tx=0x7fa2dc008040).stop 2022-08-01T23:03:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 msgr2=0x7fa2ec14cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 0x7fa2ec14cb40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fa2e4007000 tx=0x7fa2e4008040).stop 2022-08-01T23:03:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 shutdown_connections 2022-08-01T23:03:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2cc053a70 0x7fa2cc055f30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0b2710 0x7fa2ec14cb40 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2ec0b01a0 0x7fa2ec05c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 --2- 172.21.15.32:0/2220959927 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2ec0afd60 0x7fa2ec0b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 >> 172.21.15.32:0/2220959927 conn(0x7fa2ec060310 msgr2=0x7fa2ec148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:57.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 shutdown_connections 2022-08-01T23:03:57.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.641+0000 7fa2f2fbb700 1 -- 172.21.15.32:0/2220959927 wait complete. 2022-08-01T23:03:57.651 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 57 2022-08-01T23:03:57.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:57.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 Processor -- start 2022-08-01T23:03:57.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 -- start start 2022-08-01T23:03:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3640ad750 0x7fe364061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe3641405d0 0x7fe364140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640aee80 con 0x7fe3640614b0 2022-08-01T23:03:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3640b70b0 con 0x7fe3640ad750 2022-08-01T23:03:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3640af0b0 con 0x7fe3641405d0 2022-08-01T23:03:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3640ad750 0x7fe364061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3640ad750 0x7fe364061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47736/0 (socket says 172.21.15.32:47736) 2022-08-01T23:03:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 -- 172.21.15.32:0/276707882 learned_addr learned my addr 172.21.15.32:0/276707882 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:57.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe362d9d700 1 --2- 172.21.15.32:0/276707882 >> v2:172.21.15.55:3300/0 conn(0x7fe3641405d0 0x7fe364140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 -- 172.21.15.32:0/276707882 >> v2:172.21.15.55:3300/0 conn(0x7fe3641405d0 msgr2=0x7fe364140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 --2- 172.21.15.32:0/276707882 >> v2:172.21.15.55:3300/0 conn(0x7fe3641405d0 0x7fe364140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 -- 172.21.15.32:0/276707882 >> v1:172.21.15.55:6789/0 conn(0x7fe3640614b0 legacy=0x7fe3640b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 -- 172.21.15.32:0/276707882 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3640af350 con 0x7fe3640ad750 2022-08-01T23:03:57.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe362d9d700 1 --2- 172.21.15.32:0/276707882 >> v2:172.21.15.55:3300/0 conn(0x7fe3641405d0 0x7fe364140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:57.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe361d9b700 1 --2- 172.21.15.32:0/276707882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364061890 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fe34c002f70 tx=0x7fe34c006eb0).ready entity=mon.0 client_cookie=d5d8eaad50ce0a7f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:57.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36159a700 1 -- 172.21.15.32:0/276707882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c00d610 con 0x7fe3640ad750 2022-08-01T23:03:57.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36159a700 1 -- 172.21.15.32:0/276707882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c000e90 con 0x7fe3640ad750 2022-08-01T23:03:57.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.761+0000 7fe36159a700 1 -- 172.21.15.32:0/276707882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c00d9a0 con 0x7fe3640ad750 2022-08-01T23:03:57.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- 172.21.15.32:0/276707882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 msgr2=0x7fe364061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 --2- 172.21.15.32:0/276707882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364061890 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fe34c002f70 tx=0x7fe34c006eb0).stop 2022-08-01T23:03:57.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- 172.21.15.32:0/276707882 shutdown_connections 2022-08-01T23:03:57.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 --2- 172.21.15.32:0/276707882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364061890 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 --2- 172.21.15.32:0/276707882 >> v2:172.21.15.55:3300/0 conn(0x7fe3641405d0 0x7fe364140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- 172.21.15.32:0/276707882 >> 172.21.15.32:0/276707882 conn(0x7fe364066f70 msgr2=0x7fe364067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:57.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- 172.21.15.32:0/276707882 shutdown_connections 2022-08-01T23:03:57.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- 172.21.15.32:0/276707882 wait complete. 2022-08-01T23:03:57.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 Processor -- start 2022-08-01T23:03:57.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- start start 2022-08-01T23:03:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3640614b0 0x7fe3640affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3641405d0 0x7fe364152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364156980 con 0x7fe3640ad750 2022-08-01T23:03:57.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36409f070 con 0x7fe3640614b0 2022-08-01T23:03:57.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640b35d0 con 0x7fe3641405d0 2022-08-01T23:03:57.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe361d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe361d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47742/0 (socket says 172.21.15.32:47742) 2022-08-01T23:03:57.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe361d9b700 1 -- 172.21.15.32:0/3911221832 learned_addr learned my addr 172.21.15.32:0/3911221832 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:57.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe36259c700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3640614b0 0x7fe3640affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.765+0000 7fe362d9d700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3641405d0 0x7fe364152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe361d9b700 1 -- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3641405d0 msgr2=0x7fe364152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe361d9b700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3641405d0 0x7fe364152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe361d9b700 1 -- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3640614b0 msgr2=0x7fe3640affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe361d9b700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3640614b0 0x7fe3640affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe361d9b700 1 -- 172.21.15.32:0/3911221832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c00b1e0 con 0x7fe3640ad750 2022-08-01T23:03:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe362d9d700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3641405d0 0x7fe364152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe361d9b700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364152550 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fe36409f1b0 tx=0x7fe34c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c011860 con 0x7fe3640ad750 2022-08-01T23:03:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0119c0 con 0x7fe3640ad750 2022-08-01T23:03:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe34c011d20 con 0x7fe3640ad750 2022-08-01T23:03:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe364156550 con 0x7fe3640ad750 2022-08-01T23:03:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36409ece0 con 0x7fe3640ad750 2022-08-01T23:03:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3640af5f0 con 0x7fe3640ad750 2022-08-01T23:03:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe34c025030 con 0x7fe3640ad750 2022-08-01T23:03:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.769+0000 7fe35b7fe700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3480539e0 0x7fe348055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.773+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe34c06d270 con 0x7fe3640ad750 2022-08-01T23:03:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.773+0000 7fe36259c700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3480539e0 0x7fe348055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.773+0000 7fe36259c700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3480539e0 0x7fe348055ea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fe35400b000 tx=0x7fe354009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:57.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.777+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe34c044030 con 0x7fe3640ad750 2022-08-01T23:03:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.905+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3640574e0 con 0x7fe3640ad750 2022-08-01T23:03:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.905+0000 7fe35b7fe700 1 -- 172.21.15.32:0/3911221832 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fe34c023090 con 0x7fe3640ad750 2022-08-01T23:03:57.911 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:57.911 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:56.641397+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:54.249192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:54.638843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.905+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3480539e0 msgr2=0x7fe348055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.905+0000 7fe36359e700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3480539e0 0x7fe348055ea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fe35400b000 tx=0x7fe354009040).stop 2022-08-01T23:03:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 msgr2=0x7fe364152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364152550 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fe36409f1b0 tx=0x7fe34c015000).stop 2022-08-01T23:03:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 shutdown_connections 2022-08-01T23:03:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe3480539e0 0x7fe348055ea0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe3641405d0 0x7fe364152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3640ad750 0x7fe364152550 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 --2- 172.21.15.32:0/3911221832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3640614b0 0x7fe3640affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 >> 172.21.15.32:0/3911221832 conn(0x7fe364066f70 msgr2=0x7fe3640b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:57.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 shutdown_connections 2022-08-01T23:03:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:57.909+0000 7fe36359e700 1 -- 172.21.15.32:0/3911221832 wait complete. 2022-08-01T23:03:57.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:03:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6aa75700 1 Processor -- start 2022-08-01T23:03:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6aa75700 1 -- start start 2022-08-01T23:03:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6aa75700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c640b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6aa75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c640b2510 con 0x7f5c640aadb0 2022-08-01T23:03:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6aa75700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c641443e0 con 0x7f5c640b31f0 2022-08-01T23:03:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6aa75700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640b3700 con 0x7f5c640b7500 2022-08-01T23:03:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6a274700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c640aadb0 0x7f5c64064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35294/0 (socket says 172.21.15.32:35294) 2022-08-01T23:03:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c6a274700 1 -- 172.21.15.32:0/2249624632 learned_addr learned my addr 172.21.15.32:0/2249624632 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c69a73700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c640b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295563112 0 0) 0x7f5c640b2510 con 0x7f5c640aadb0 2022-08-01T23:03:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c4c003260 con 0x7f5c640aadb0 2022-08-01T23:03:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.033+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826130620 0 0) 0x7f5c641443e0 con 0x7f5c640b31f0 2022-08-01T23:03:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c640b2510 con 0x7f5c640b31f0 2022-08-01T23:03:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458230841 0 0) 0x7f5c4c003260 con 0x7f5c640aadb0 2022-08-01T23:03:58.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 msgr2=0x7f5c640b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:58.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c640b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 >> v1:172.21.15.55:6789/0 conn(0x7f5c640b31f0 legacy=0x7f5c6413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640b2740 con 0x7f5c640aadb0 2022-08-01T23:03:58.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f5c5c0019e0 con 0x7f5c640aadb0 2022-08-01T23:03:58.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 >> v1:172.21.15.32:6789/0 conn(0x7f5c640aadb0 legacy=0x7f5c64064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c003910 0x7f5c4c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c004500 0x7f5c4c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c4c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640b3700 con 0x7f5c4c004500 2022-08-01T23:03:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c00b780 con 0x7f5c4c003910 2022-08-01T23:03:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c00b940 con 0x7f5c640b7500 2022-08-01T23:03:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c69272700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c004500 0x7f5c4c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c4c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c69a73700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c003910 0x7f5c4c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 -- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c003910 msgr2=0x7f5c4c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c4c003910 0x7f5c4c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 -- 172.21.15.32:0/2249624632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c004500 msgr2=0x7f5c4c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c004500 0x7f5c4c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 -- 172.21.15.32:0/2249624632 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c003260 con 0x7f5c640b7500 2022-08-01T23:03:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c69272700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c4c004500 0x7f5c4c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c4c004e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5c5c0017b0 tx=0x7f5c640b39a0).ready entity=mon.2 client_cookie=18f93eb1c32190b9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c5c00cbd0 con 0x7f5c640b7500 2022-08-01T23:03:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c00cd30 con 0x7f5c640b7500 2022-08-01T23:03:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c68a71700 1 -- 172.21.15.32:0/2249624632 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c5c00cea0 con 0x7f5c640b7500 2022-08-01T23:03:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6aa75700 1 -- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 msgr2=0x7f5c4c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/2249624632 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b7500 0x7f5c4c004e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5c5c0017b0 tx=0x7f5c640b39a0).stop 2022-08-01T23:03:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6a274700 1 -- 172.21.15.32:0/2249624632 reap_dead start 2022-08-01T23:03:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6aa75700 1 -- 172.21.15.32:0/2249624632 shutdown_connections 2022-08-01T23:03:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6aa75700 1 -- 172.21.15.32:0/2249624632 >> 172.21.15.32:0/2249624632 conn(0x7f5c6405f840 msgr2=0x7f5c6405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6aa75700 1 -- 172.21.15.32:0/2249624632 shutdown_connections 2022-08-01T23:03:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.037+0000 7f5c6aa75700 1 -- 172.21.15.32:0/2249624632 wait complete. 2022-08-01T23:03:58.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 Processor -- start 2022-08-01T23:03:58.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 -- start start 2022-08-01T23:03:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 0x7f5c641434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c640ab4c0 0x7f5c640ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b6eb0 0x7f5c64150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64150430 con 0x7f5c640b7500 2022-08-01T23:03:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c641543d0 con 0x7f5c640ab4c0 2022-08-01T23:03:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64154670 con 0x7f5c640b6eb0 2022-08-01T23:03:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 0x7f5c641434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 0x7f5c641434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47760/0 (socket says 172.21.15.32:47760) 2022-08-01T23:03:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 -- 172.21.15.32:0/792491078 learned_addr learned my addr 172.21.15.32:0/792491078 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6a274700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b6eb0 0x7f5c64150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69272700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c640ab4c0 0x7f5c640ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 -- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b6eb0 msgr2=0x7f5c64150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b6eb0 0x7f5c64150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 -- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c640ab4c0 msgr2=0x7f5c640ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c640ab4c0 0x7f5c640ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 -- 172.21.15.32:0/792491078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64150620 con 0x7f5c640b7500 2022-08-01T23:03:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6a274700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b6eb0 0x7f5c64150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c69a73700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 0x7f5c641434e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f5c54006000 tx=0x7f5c54006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c54013680 con 0x7f5c640b7500 2022-08-01T23:03:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c540137e0 con 0x7f5c640b7500 2022-08-01T23:03:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c641508c0 con 0x7f5c640b7500 2022-08-01T23:03:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c54013b40 con 0x7f5c640b7500 2022-08-01T23:03:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.041+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6413d860 con 0x7f5c640b7500 2022-08-01T23:03:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.045+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c54013ca0 con 0x7f5c640b7500 2022-08-01T23:03:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.045+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c64154870 con 0x7f5c640b7500 2022-08-01T23:03:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.045+0000 7f5c52ffd700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c44053a70 0x7f5c44055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.045+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5c5406c7c0 con 0x7f5c640b7500 2022-08-01T23:03:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.045+0000 7f5c69272700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c44053a70 0x7f5c44055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.045+0000 7f5c69272700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c44053a70 0x7f5c44055f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f5c60003000 tx=0x7f5c60007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.053+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c54043520 con 0x7f5c640b7500 2022-08-01T23:03:58.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.221+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5c64150a50 con 0x7f5c640b7500 2022-08-01T23:03:58.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.229+0000 7f5c52ffd700 1 -- 172.21.15.32:0/792491078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5c54018020 con 0x7f5c640b7500 2022-08-01T23:03:58.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c44053a70 msgr2=0x7f5c44055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c44053a70 0x7f5c44055f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f5c60003000 tx=0x7f5c60007a00).stop 2022-08-01T23:03:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 msgr2=0x7f5c641434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 0x7f5c641434e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f5c54006000 tx=0x7f5c54006d90).stop 2022-08-01T23:03:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 shutdown_connections 2022-08-01T23:03:58.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5c44053a70 0x7f5c44055f30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c640b6eb0 0x7f5c64150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c640ab4c0 0x7f5c640ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 --2- 172.21.15.32:0/792491078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640b7500 0x7f5c641434e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 >> 172.21.15.32:0/792491078 conn(0x7f5c6405f840 msgr2=0x7f5c6413f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 shutdown_connections 2022-08-01T23:03:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.233+0000 7f5c6aa75700 1 -- 172.21.15.32:0/792491078 wait complete. 2022-08-01T23:03:58.242 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:03:58.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:03:58.366 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 Processor -- start 2022-08-01T23:03:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 -- start start 2022-08-01T23:03:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24c40b65c0 0x7f24c4061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c4061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24c4147d80 0x7f24c4144630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c40b1630 con 0x7f24c40b65c0 2022-08-01T23:03:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c4061fe0 con 0x7f24c4147d80 2022-08-01T23:03:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24caeff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c40b2100 con 0x7f24c40b41f0 2022-08-01T23:03:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24c40b65c0 0x7f24c4061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24c40b65c0 0x7f24c4061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47768/0 (socket says 172.21.15.32:47768) 2022-08-01T23:03:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 -- 172.21.15.32:0/2035495354 learned_addr learned my addr 172.21.15.32:0/2035495354 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24ca6fe700 1 --2- 172.21.15.32:0/2035495354 >> v2:172.21.15.55:3300/0 conn(0x7f24c4147d80 0x7f24c4144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c96fc700 1 --2- 172.21.15.32:0/2035495354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c4061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 -- 172.21.15.32:0/2035495354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 msgr2=0x7f24c4061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 --2- 172.21.15.32:0/2035495354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c4061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 -- 172.21.15.32:0/2035495354 >> v2:172.21.15.55:3300/0 conn(0x7f24c4147d80 msgr2=0x7f24c4144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 --2- 172.21.15.32:0/2035495354 >> v2:172.21.15.55:3300/0 conn(0x7f24c4147d80 0x7f24c4144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 -- 172.21.15.32:0/2035495354 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40b69a0 con 0x7f24c40b65c0 2022-08-01T23:03:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c9efd700 1 --2- 172.21.15.32:0/2035495354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c40b65c0 0x7f24c4061600 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f24c4062120 tx=0x7f24b4006eb0).ready entity=mon.0 client_cookie=f0fd26f4341fe7dc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c8efb700 1 -- 172.21.15.32:0/2035495354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b400d570 con 0x7f24c40b65c0 2022-08-01T23:03:58.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c8efb700 1 -- 172.21.15.32:0/2035495354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4000e90 con 0x7f24c40b65c0 2022-08-01T23:03:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.361+0000 7f24c8efb700 1 -- 172.21.15.32:0/2035495354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b400d8c0 con 0x7f24c40b65c0 2022-08-01T23:03:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 -- 172.21.15.32:0/2035495354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c40b65c0 msgr2=0x7f24c4061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 --2- 172.21.15.32:0/2035495354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c40b65c0 0x7f24c4061600 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f24c4062120 tx=0x7f24b4006eb0).stop 2022-08-01T23:03:58.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 -- 172.21.15.32:0/2035495354 shutdown_connections 2022-08-01T23:03:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 --2- 172.21.15.32:0/2035495354 >> v2:172.21.15.55:3300/0 conn(0x7f24c4147d80 0x7f24c4144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 --2- 172.21.15.32:0/2035495354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c4061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 --2- 172.21.15.32:0/2035495354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c40b65c0 0x7f24c4061600 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 -- 172.21.15.32:0/2035495354 >> 172.21.15.32:0/2035495354 conn(0x7f24c405c620 msgr2=0x7f24c405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 -- 172.21.15.32:0/2035495354 shutdown_connections 2022-08-01T23:03:58.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 -- 172.21.15.32:0/2035495354 wait complete. 2022-08-01T23:03:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 Processor -- start 2022-08-01T23:03:58.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 -- start start 2022-08-01T23:03:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.365+0000 7f24caeff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24c40b65c0 0x7f24c40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 0x7f24c40ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c414a220 con 0x7f24c4147d80 2022-08-01T23:03:58.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c4149d20 con 0x7f24c40b65c0 2022-08-01T23:03:58.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c415abe0 con 0x7f24c40b41f0 2022-08-01T23:03:58.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 0x7f24c40ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24c9efd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 0x7f24c40ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47778/0 (socket says 172.21.15.32:47778) 2022-08-01T23:03:58.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 -- 172.21.15.32:0/2736866064 learned_addr learned my addr 172.21.15.32:0/2736866064 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:58.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24c96fc700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24c40b65c0 0x7f24c40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 -- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 msgr2=0x7f24c415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 -- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24c40b65c0 msgr2=0x7f24c40aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24c40b65c0 0x7f24c40aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 -- 172.21.15.32:0/2736866064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b400b1e0 con 0x7f24c4147d80 2022-08-01T23:03:58.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24ca6fe700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 0x7f24c40ad7f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f24bc00a000 tx=0x7f24bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24bc00f710 con 0x7f24c4147d80 2022-08-01T23:03:58.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24bc00f870 con 0x7f24c4147d80 2022-08-01T23:03:58.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c4149f80 con 0x7f24c4147d80 2022-08-01T23:03:58.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24bc00fbd0 con 0x7f24c4147d80 2022-08-01T23:03:58.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c4148610 con 0x7f24c4147d80 2022-08-01T23:03:58.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c4149390 con 0x7f24c4147d80 2022-08-01T23:03:58.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.369+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f24bc01a030 con 0x7f24c4147d80 2022-08-01T23:03:58.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.373+0000 7f24baffd700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f24ac053a70 0x7f24ac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.373+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f24bc06be40 con 0x7f24c4147d80 2022-08-01T23:03:58.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.373+0000 7f24c9efd700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f24ac053a70 0x7f24ac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.373+0000 7f24c9efd700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f24ac053a70 0x7f24ac055f30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f24b401c000 tx=0x7f24b4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.381+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24c4149390 con 0x7f24c4147d80 2022-08-01T23:03:58.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.549+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f24c4149390 con 0x7f24c4147d80 2022-08-01T23:03:58.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.553+0000 7f24baffd700 1 -- 172.21.15.32:0/2736866064 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7f24c4149390 con 0x7f24c4147d80 2022-08-01T23:03:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f24ac053a70 msgr2=0x7f24ac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f24ac053a70 0x7f24ac055f30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f24b401c000 tx=0x7f24b4015000).stop 2022-08-01T23:03:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 msgr2=0x7f24c40ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 0x7f24c40ad7f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f24bc00a000 tx=0x7f24bc008040).stop 2022-08-01T23:03:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 shutdown_connections 2022-08-01T23:03:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f24ac053a70 0x7f24ac055f30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24c4147d80 0x7f24c40ad7f0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24c40b65c0 0x7f24c40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 --2- 172.21.15.32:0/2736866064 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f24c40b41f0 0x7f24c415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 >> 172.21.15.32:0/2736866064 conn(0x7f24c405c620 msgr2=0x7f24c40b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 shutdown_connections 2022-08-01T23:03:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.557+0000 7f24caeff700 1 -- 172.21.15.32:0/2736866064 wait complete. 2022-08-01T23:03:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:03:58.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:03:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 Processor -- start 2022-08-01T23:03:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 -- start start 2022-08-01T23:03:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f484c064b10 0x7f484c144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f484c0b36e0 0x7f484c141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c064ef0 con 0x7f484c141860 2022-08-01T23:03:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c13ebf0 con 0x7f484c064b10 2022-08-01T23:03:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c066690 con 0x7f484c0b36e0 2022-08-01T23:03:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f484c064b10 0x7f484c144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f484c141860 0x7f484c066ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35328/0 (socket says 172.21.15.32:35328) 2022-08-01T23:03:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f484ad9d700 1 -- 172.21.15.32:0/400207069 learned_addr learned my addr 172.21.15.32:0/400207069 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.673+0000 7f4849d9b700 1 --2- 172.21.15.32:0/400207069 >> v2:172.21.15.55:3300/0 conn(0x7f484c0b36e0 0x7f484c141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484959a700 1 -- 172.21.15.32:0/400207069 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764130269 0 0) 0x7f484c064ef0 con 0x7f484c141860 2022-08-01T23:03:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484959a700 1 -- 172.21.15.32:0/400207069 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4830003260 con 0x7f484c141860 2022-08-01T23:03:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f4849d9b700 1 -- 172.21.15.32:0/400207069 >> v2:172.21.15.32:3300/0 conn(0x7f484c064b10 msgr2=0x7f484c144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f4849d9b700 1 --2- 172.21.15.32:0/400207069 >> v2:172.21.15.32:3300/0 conn(0x7f484c064b10 0x7f484c144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f4849d9b700 1 -- 172.21.15.32:0/400207069 >> v1:172.21.15.32:6789/0 conn(0x7f484c141860 legacy=0x7f484c066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f4849d9b700 1 -- 172.21.15.32:0/400207069 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0b4850 con 0x7f484c0b36e0 2022-08-01T23:03:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f4849d9b700 1 --2- 172.21.15.32:0/400207069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c0b36e0 0x7f484c141370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f483c002f70 tx=0x7f483c006df0).ready entity=mon.1 client_cookie=b60a9fd88ee0c600 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484959a700 1 -- 172.21.15.32:0/400207069 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c00d4f0 con 0x7f484c0b36e0 2022-08-01T23:03:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484959a700 1 -- 172.21.15.32:0/400207069 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c000e90 con 0x7f484c0b36e0 2022-08-01T23:03:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484959a700 1 -- 172.21.15.32:0/400207069 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c00d840 con 0x7f484c0b36e0 2022-08-01T23:03:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 -- 172.21.15.32:0/400207069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c0b36e0 msgr2=0x7f484c141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 --2- 172.21.15.32:0/400207069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c0b36e0 0x7f484c141370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f483c002f70 tx=0x7f483c006df0).stop 2022-08-01T23:03:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 -- 172.21.15.32:0/400207069 shutdown_connections 2022-08-01T23:03:58.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 --2- 172.21.15.32:0/400207069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c0b36e0 0x7f484c141370 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 --2- 172.21.15.32:0/400207069 >> v2:172.21.15.32:3300/0 conn(0x7f484c064b10 0x7f484c144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 -- 172.21.15.32:0/400207069 >> 172.21.15.32:0/400207069 conn(0x7f484c05f840 msgr2=0x7f484c05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 -- 172.21.15.32:0/400207069 shutdown_connections 2022-08-01T23:03:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 -- 172.21.15.32:0/400207069 wait complete. 2022-08-01T23:03:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 Processor -- start 2022-08-01T23:03:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.677+0000 7f484b59e700 1 -- start start 2022-08-01T23:03:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c064b10 0x7f484c13f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 0x7f484c138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f484c141860 0x7f484c138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c13f950 con 0x7f484c0b36e0 2022-08-01T23:03:58.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c13b6a0 con 0x7f484c064b10 2022-08-01T23:03:58.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c13b940 con 0x7f484c141860 2022-08-01T23:03:58.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 0x7f484c138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 0x7f484c138900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47788/0 (socket says 172.21.15.32:47788) 2022-08-01T23:03:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 -- 172.21.15.32:0/3351941892 learned_addr learned my addr 172.21.15.32:0/3351941892 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484a59c700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c064b10 0x7f484c13f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 -- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f484c141860 msgr2=0x7f484c138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484ad9d700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f484c141860 0x7f484c138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f484c141860 0x7f484c138df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 -- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c064b10 msgr2=0x7f484c13f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c064b10 0x7f484c13f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 -- 172.21.15.32:0/3351941892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c00b180 con 0x7f484c0b36e0 2022-08-01T23:03:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484ad9d700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f484c141860 0x7f484c138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:03:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484a59c700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c064b10 0x7f484c13f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f4849d9b700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 0x7f484c138900 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f483c002f70 tx=0x7f483c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c016070 con 0x7f484c0b36e0 2022-08-01T23:03:58.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0139c0 con 0x7f484c0b36e0 2022-08-01T23:03:58.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c13bbe0 con 0x7f484c0b36e0 2022-08-01T23:03:58.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c013d20 con 0x7f484c0b36e0 2022-08-01T23:03:58.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c0b5190 con 0x7f484c0b36e0 2022-08-01T23:03:58.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.681+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c0b4af0 con 0x7f484c0b36e0 2022-08-01T23:03:58.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.685+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f483c025030 con 0x7f484c0b36e0 2022-08-01T23:03:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.685+0000 7f483b7fe700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4828053950 0x7f4828055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.685+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f483c06cfb0 con 0x7f484c0b36e0 2022-08-01T23:03:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.685+0000 7f484a59c700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4828053950 0x7f4828055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.685+0000 7f484a59c700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4828053950 0x7f4828055e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f484000b000 tx=0x7f4840008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.689+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f483c043d70 con 0x7f484c0b36e0 2022-08-01T23:03:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.853+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f484c0574e0 con 0x7f484c0b36e0 2022-08-01T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f483b7fe700 1 -- 172.21.15.32:0/3351941892 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f483c023070 con 0x7f484c0b36e0 2022-08-01T23:03:58.861 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:58.862 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1851140084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1851140084},{"type":"v1","addr":"172.21.15.55:6835","nonce":1851140084}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1909728706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1909728706},{"type":"v1","addr":"172.21.15.32:6833","nonce":1909728706}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:55.638124+0000","modified":"2022-08-01T23:03:57.319377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/519921040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":519921040},{"type":"v1","addr":"172.21.15.32:6835","nonce":519921040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T23:03:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4828053950 msgr2=0x7f4828055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4828053950 0x7f4828055e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f484000b000 tx=0x7f4840008040).stop 2022-08-01T23:03:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 msgr2=0x7f484c138900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 0x7f484c138900 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f483c002f70 tx=0x7f483c014000).stop 2022-08-01T23:03:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 shutdown_connections 2022-08-01T23:03:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4828053950 0x7f4828055e10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f484c141860 0x7f484c138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b36e0 0x7f484c138900 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 --2- 172.21.15.32:0/3351941892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f484c064b10 0x7f484c13f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 >> 172.21.15.32:0/3351941892 conn(0x7f484c05f840 msgr2=0x7f484c140670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 shutdown_connections 2022-08-01T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:58.857+0000 7f484b59e700 1 -- 172.21.15.32:0/3351941892 wait complete. 2022-08-01T23:03:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 57 2022-08-01T23:03:59.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:55.638124+0000', 'modified': '2022-08-01T23:03:57.319377+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7674': {'gid': 7674, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/519921040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 519921040}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 519921040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:03:59.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:03:59.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:03:59.292 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":7674,"want_state":"up:active","state":"up:active","rank_uptime":3.6385039610000001,"mdsmap_epoch":57,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":7.7816765050000001} 2022-08-01T23:03:59.292 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 7674, "mdsmap_epoch": 57, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 3.638503961, "state": "up:active", "uptime": 7.781676505, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:03:59.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:03:59.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.389+0000 7f765ecb3700 1 Processor -- start 2022-08-01T23:03:59.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.389+0000 7f765ecb3700 1 -- start start 2022-08-01T23:03:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.389+0000 7f765ecb3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76580b2710 0x7f765814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76580b01a0 0x7f765814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76580a7250 con 0x7f76580b2710 2022-08-01T23:03:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76580a7480 con 0x7f76580b01a0 2022-08-01T23:03:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580a7720 con 0x7f76580afd60 2022-08-01T23:03:59.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765dcb1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76580b2710 0x7f765814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765e4b2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76580b01a0 0x7f765814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765e4b2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59538/0 (socket says 172.21.15.32:59538) 2022-08-01T23:03:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76580b01a0 0x7f765814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57582/0 (socket says 172.21.15.32:57582) 2022-08-01T23:03:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765dcb1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76580b2710 0x7f765814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47798/0 (socket says 172.21.15.32:47798) 2022-08-01T23:03:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765e4b2700 1 -- 172.21.15.32:0/1080514021 learned_addr learned my addr 172.21.15.32:0/1080514021 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 -- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 msgr2=0x7f76580a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 --2- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 -- 172.21.15.32:0/1080514021 >> v2:172.21.15.32:3300/0 conn(0x7f76580b2710 msgr2=0x7f765814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 --2- 172.21.15.32:0/1080514021 >> v2:172.21.15.32:3300/0 conn(0x7f76580b2710 0x7f765814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 -- 172.21.15.32:0/1080514021 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76580a79c0 con 0x7f76580b01a0 2022-08-01T23:03:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765e4b2700 1 --2- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765d4b0700 1 --2- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f765814b4c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7654002f70 tx=0x7f7654006df0).ready entity=mon.1 client_cookie=d4405b56c95b565b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ccaf700 1 -- 172.21.15.32:0/1080514021 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f765400d4a0 con 0x7f76580b01a0 2022-08-01T23:03:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ccaf700 1 -- 172.21.15.32:0/1080514021 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7654000e90 con 0x7f76580b01a0 2022-08-01T23:03:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ccaf700 1 -- 172.21.15.32:0/1080514021 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f765400d7f0 con 0x7f76580b01a0 2022-08-01T23:03:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 msgr2=0x7f765814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 --2- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f765814b4c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7654002f70 tx=0x7f7654006df0).stop 2022-08-01T23:03:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- 172.21.15.32:0/1080514021 shutdown_connections 2022-08-01T23:03:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 --2- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 --2- 172.21.15.32:0/1080514021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f765814b4c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 --2- 172.21.15.32:0/1080514021 >> v2:172.21.15.32:3300/0 conn(0x7f76580b2710 0x7f765814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- 172.21.15.32:0/1080514021 >> 172.21.15.32:0/1080514021 conn(0x7f7658060310 msgr2=0x7f76580b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- 172.21.15.32:0/1080514021 shutdown_connections 2022-08-01T23:03:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.393+0000 7f765ecb3700 1 -- 172.21.15.32:0/1080514021 wait complete. 2022-08-01T23:03:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 Processor -- start 2022-08-01T23:03:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- start start 2022-08-01T23:03:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f76580aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 0x7f7658064d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580acac0 con 0x7f76580b2710 2022-08-01T23:03:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580ab590 con 0x7f76580b01a0 2022-08-01T23:03:59.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580aaf60 con 0x7f76580afd60 2022-08-01T23:03:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 0x7f7658064d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 0x7f7658064d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47806/0 (socket says 172.21.15.32:47806) 2022-08-01T23:03:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 -- 172.21.15.32:0/3044922813 learned_addr learned my addr 172.21.15.32:0/3044922813 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765dcb1700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765d4b0700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f76580aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 -- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 msgr2=0x7f76580ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 -- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 msgr2=0x7f76580aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f76580aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 -- 172.21.15.32:0/3044922813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650009530 con 0x7f76580b2710 2022-08-01T23:03:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765d4b0700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f76580aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:03:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765e4b2700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 0x7f7658064d60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f76580ab7b0 tx=0x7f7650009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f765000c8a0 con 0x7f76580b2710 2022-08-01T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000ca00 con 0x7f76580b2710 2022-08-01T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765400b180 con 0x7f76580b2710 2022-08-01T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f765000cd60 con 0x7f76580b2710 2022-08-01T23:03:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658159a60 con 0x7f76580b2710 2022-08-01T23:03:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.397+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76580574e0 con 0x7f76580b2710 2022-08-01T23:03:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.401+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7650022070 con 0x7f76580b2710 2022-08-01T23:03:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.401+0000 7f7646ffd700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7638053a70 0x7f7638055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.401+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f765006c8f0 con 0x7f76580b2710 2022-08-01T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.401+0000 7f765dcb1700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7638053a70 0x7f7638055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.401+0000 7f765dcb1700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7638053a70 0x7f7638055f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f764800b000 tx=0x7f7648006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.405+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7650043610 con 0x7f76580b2710 2022-08-01T23:03:59.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.529+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7658069a70 con 0x7f76580b2710 2022-08-01T23:03:59.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.529+0000 7f7646ffd700 1 -- 172.21.15.32:0/3044922813 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f7650016090 con 0x7f76580b2710 2022-08-01T23:03:59.537 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:03:59.537 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:56.641397+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:54.249192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:54.638843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:03:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7638053a70 msgr2=0x7f7638055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7638053a70 0x7f7638055f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f764800b000 tx=0x7f7648006df0).stop 2022-08-01T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 msgr2=0x7f7658064d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 0x7f7658064d60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f76580ab7b0 tx=0x7f7650009290).stop 2022-08-01T23:03:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 shutdown_connections 2022-08-01T23:03:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7638053a70 0x7f7638055f30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76580b2710 0x7f7658064d60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76580b01a0 0x7f76580aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 --2- 172.21.15.32:0/3044922813 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76580afd60 0x7f76580ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 >> 172.21.15.32:0/3044922813 conn(0x7f7658060310 msgr2=0x7f7658149c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 shutdown_connections 2022-08-01T23:03:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.533+0000 7f765ecb3700 1 -- 172.21.15.32:0/3044922813 wait complete. 2022-08-01T23:03:59.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:03:54.249192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:03:54.638843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:03:59.551 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:03:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f431a095700 1 Processor -- start 2022-08-01T23:03:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f431a095700 1 -- start start 2022-08-01T23:03:59.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f431a095700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4310 0x7f4314140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f431a095700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4314143cf0 con 0x7f4314146ae0 2022-08-01T23:03:59.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f431a095700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4314143f90 con 0x7f43140b4820 2022-08-01T23:03:59.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f431a095700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43141476a0 con 0x7f43140b4310 2022-08-01T23:03:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f43137fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4314146ae0 0x7f43141403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35352/0 (socket says 172.21.15.32:35352) 2022-08-01T23:03:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f43137fe700 1 -- 172.21.15.32:0/2665145712 learned_addr learned my addr 172.21.15.32:0/2665145712 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:03:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f4312ffd700 1 --2- 172.21.15.32:0/2665145712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4310 0x7f4314140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f43127fc700 1 -- 172.21.15.32:0/2665145712 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162895947 0 0) 0x7f4314143cf0 con 0x7f4314146ae0 2022-08-01T23:03:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f43127fc700 1 -- 172.21.15.32:0/2665145712 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f4003260 con 0x7f4314146ae0 2022-08-01T23:03:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f4312ffd700 1 -- 172.21.15.32:0/2665145712 >> v1:172.21.15.55:6789/0 conn(0x7f43140b4820 legacy=0x7f43140b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f4312ffd700 1 -- 172.21.15.32:0/2665145712 >> v1:172.21.15.32:6789/0 conn(0x7f4314146ae0 legacy=0x7f43141403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.657+0000 7f4312ffd700 1 -- 172.21.15.32:0/2665145712 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140b0e20 con 0x7f43140b4310 2022-08-01T23:03:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f4312ffd700 1 --2- 172.21.15.32:0/2665145712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4310 0x7f4314140ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4304002f70 tx=0x7f4304006df0).ready entity=mon.2 client_cookie=fa46c7a290a1af54 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f43127fc700 1 -- 172.21.15.32:0/2665145712 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f430400d570 con 0x7f43140b4310 2022-08-01T23:03:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f43127fc700 1 -- 172.21.15.32:0/2665145712 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4304000e90 con 0x7f43140b4310 2022-08-01T23:03:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f43127fc700 1 -- 172.21.15.32:0/2665145712 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f430400d8c0 con 0x7f43140b4310 2022-08-01T23:03:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 -- 172.21.15.32:0/2665145712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4310 msgr2=0x7f4314140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 --2- 172.21.15.32:0/2665145712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4310 0x7f4314140ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4304002f70 tx=0x7f4304006df0).stop 2022-08-01T23:03:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 -- 172.21.15.32:0/2665145712 shutdown_connections 2022-08-01T23:03:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 --2- 172.21.15.32:0/2665145712 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4310 0x7f4314140ad0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 -- 172.21.15.32:0/2665145712 >> 172.21.15.32:0/2665145712 conn(0x7f431405c620 msgr2=0x7f431405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 -- 172.21.15.32:0/2665145712 shutdown_connections 2022-08-01T23:03:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 -- 172.21.15.32:0/2665145712 wait complete. 2022-08-01T23:03:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 Processor -- start 2022-08-01T23:03:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.661+0000 7f431a095700 1 -- start start 2022-08-01T23:03:59.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 0x7f43140b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4820 0x7f43141413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4314146ae0 0x7f4314156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43140b78e0 con 0x7f43140b4310 2022-08-01T23:03:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43140b7b10 con 0x7f4314146ae0 2022-08-01T23:03:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43140b7db0 con 0x7f43140b4820 2022-08-01T23:03:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 0x7f43140b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 0x7f43140b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47816/0 (socket says 172.21.15.32:47816) 2022-08-01T23:03:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 -- 172.21.15.32:0/3938477981 learned_addr learned my addr 172.21.15.32:0/3938477981 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f4312ffd700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4820 0x7f43141413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f4313fff700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4314146ae0 0x7f4314156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 -- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4820 msgr2=0x7f43141413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4820 0x7f43141413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 -- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4314146ae0 msgr2=0x7f4314156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4314146ae0 0x7f4314156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 -- 172.21.15.32:0/3938477981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430400b180 con 0x7f43140b4310 2022-08-01T23:03:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f43137fe700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 0x7f43140b82f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f42fc001110 tx=0x7f42fc007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42fc0070d0 con 0x7f43140b4310 2022-08-01T23:03:59.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc007230 con 0x7f43140b4310 2022-08-01T23:03:59.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4314155b70 con 0x7f43140b4310 2022-08-01T23:03:59.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42fc005610 con 0x7f43140b4310 2022-08-01T23:03:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43141418a0 con 0x7f43140b4310 2022-08-01T23:03:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.665+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4314155d00 con 0x7f43140b4310 2022-08-01T23:03:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.669+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f42fc010070 con 0x7f43140b4310 2022-08-01T23:03:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.669+0000 7f4310ff9700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42f80539e0 0x7f42f8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.669+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f42fc06b600 con 0x7f43140b4310 2022-08-01T23:03:59.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.669+0000 7f4312ffd700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42f80539e0 0x7f42f8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.669+0000 7f4312ffd700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42f80539e0 0x7f42f8055ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f430401a000 tx=0x7f4304014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.673+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f42fc042320 con 0x7f43140b4310 2022-08-01T23:03:59.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.801+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f43140b7f70 con 0x7f43140b4310 2022-08-01T23:03:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.801+0000 7f4310ff9700 1 -- 172.21.15.32:0/3938477981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7f42fc045940 con 0x7f43140b4310 2022-08-01T23:03:59.807 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:03:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.801+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42f80539e0 msgr2=0x7f42f8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42f80539e0 0x7f42f8055ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f430401a000 tx=0x7f4304014000).stop 2022-08-01T23:03:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 msgr2=0x7f43140b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 0x7f43140b82f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f42fc001110 tx=0x7f42fc007ad0).stop 2022-08-01T23:03:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 shutdown_connections 2022-08-01T23:03:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42f80539e0 0x7f42f8055ea0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4314146ae0 0x7f4314156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43140b4820 0x7f43141413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 --2- 172.21.15.32:0/3938477981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43140b4310 0x7f43140b82f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 >> 172.21.15.32:0/3938477981 conn(0x7f431405c620 msgr2=0x7f4314145d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 shutdown_connections 2022-08-01T23:03:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.805+0000 7f431a095700 1 -- 172.21.15.32:0/3938477981 wait complete. 2022-08-01T23:03:59.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:03:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.925+0000 7ff28f59e700 1 Processor -- start 2022-08-01T23:03:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.925+0000 7ff28f59e700 1 -- start start 2022-08-01T23:03:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 0x7ff290147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff29005c620 0x7ff2900a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29014ba70 con 0x7ff2900b2930 2022-08-01T23:03:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff290145340 con 0x7ff29005c620 2022-08-01T23:03:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900b2d10 con 0x7ff2900b1ea0 2022-08-01T23:03:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28ed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff29005c620 0x7ff2900a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 0x7ff290147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 0x7ff290147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59562/0 (socket says 172.21.15.32:59562) 2022-08-01T23:03:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 -- 172.21.15.32:0/831830198 learned_addr learned my addr 172.21.15.32:0/831830198 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 -- 172.21.15.32:0/831830198 >> v2:172.21.15.55:3300/0 conn(0x7ff29005c620 msgr2=0x7ff2900a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:03:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 --2- 172.21.15.32:0/831830198 >> v2:172.21.15.55:3300/0 conn(0x7ff29005c620 0x7ff2900a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 -- 172.21.15.32:0/831830198 >> v1:172.21.15.55:6789/0 conn(0x7ff2900b2930 legacy=0x7ff29014b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 -- 172.21.15.32:0/831830198 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29014ac60 con 0x7ff2900b1ea0 2022-08-01T23:03:59.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28dd9b700 1 --2- 172.21.15.32:0/831830198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 0x7ff290147600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7ff2780032f0 tx=0x7ff278009290).ready entity=mon.2 client_cookie=7f93a1d56c7ed36 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28d59a700 1 -- 172.21.15.32:0/831830198 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff27800b8a0 con 0x7ff2900b1ea0 2022-08-01T23:03:59.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28d59a700 1 -- 172.21.15.32:0/831830198 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800ba00 con 0x7ff2900b1ea0 2022-08-01T23:03:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28d59a700 1 -- 172.21.15.32:0/831830198 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff27800bb70 con 0x7ff2900b1ea0 2022-08-01T23:03:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- 172.21.15.32:0/831830198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 msgr2=0x7ff290147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 --2- 172.21.15.32:0/831830198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 0x7ff290147600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7ff2780032f0 tx=0x7ff278009290).stop 2022-08-01T23:03:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- 172.21.15.32:0/831830198 shutdown_connections 2022-08-01T23:03:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 --2- 172.21.15.32:0/831830198 >> v2:172.21.15.55:3300/0 conn(0x7ff29005c620 0x7ff2900a8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 --2- 172.21.15.32:0/831830198 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b1ea0 0x7ff290147600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- 172.21.15.32:0/831830198 >> 172.21.15.32:0/831830198 conn(0x7ff2900612d0 msgr2=0x7ff290062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:03:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- 172.21.15.32:0/831830198 shutdown_connections 2022-08-01T23:03:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- 172.21.15.32:0/831830198 wait complete. 2022-08-01T23:03:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 Processor -- start 2022-08-01T23:03:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.929+0000 7ff28f59e700 1 -- start start 2022-08-01T23:03:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff29005c620 0x7ff2900ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 0x7ff290146a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b2930 0x7ff2900a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290144090 con 0x7ff2900b1ea0 2022-08-01T23:03:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290066eb0 con 0x7ff29005c620 2022-08-01T23:03:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290063c00 con 0x7ff2900b2930 2022-08-01T23:03:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 0x7ff290146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 0x7ff290146a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47832/0 (socket says 172.21.15.32:47832) 2022-08-01T23:03:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 -- 172.21.15.32:0/2609826279 learned_addr learned my addr 172.21.15.32:0/2609826279 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:03:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28ed9d700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b2930 0x7ff2900a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28e59c700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff29005c620 0x7ff2900ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 -- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b2930 msgr2=0x7ff2900a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b2930 0x7ff2900a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 -- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff29005c620 msgr2=0x7ff2900ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:03:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff29005c620 0x7ff2900ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:03:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 -- 172.21.15.32:0/2609826279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009530 con 0x7ff2900b1ea0 2022-08-01T23:03:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28ed9d700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b2930 0x7ff2900a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:03:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28dd9b700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 0x7ff290146a60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7ff2780032f0 tx=0x7ff278016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff27800b8a0 con 0x7ff2900b1ea0 2022-08-01T23:03:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2900a8ee0 con 0x7ff2900b1ea0 2022-08-01T23:03:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2900670c0 con 0x7ff2900b1ea0 2022-08-01T23:03:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800bb70 con 0x7ff2900b1ea0 2022-08-01T23:03:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff278021820 con 0x7ff2900b1ea0 2022-08-01T23:03:59.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.933+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff268003410 con 0x7ff2900b1ea0 2022-08-01T23:03:59.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.937+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff27800f070 con 0x7ff2900b1ea0 2022-08-01T23:03:59.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.937+0000 7ff2877fe700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff274053b00 0x7ff274055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:03:59.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.937+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff27806d3d0 con 0x7ff2900b1ea0 2022-08-01T23:03:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.937+0000 7ff28e59c700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff274053b00 0x7ff274055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:03:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.937+0000 7ff28e59c700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff274053b00 0x7ff274055fc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff28000d000 tx=0x7ff280009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:03:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:03:59.941+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2780440f0 con 0x7ff2900b1ea0 2022-08-01T23:04:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff268003c40 con 0x7ff2900b1ea0 2022-08-01T23:04:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff2877fe700 1 -- 172.21.15.32:0/2609826279 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff278047710 con 0x7ff2900b1ea0 2022-08-01T23:04:00.074 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff274053b00 msgr2=0x7ff274055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff274053b00 0x7ff274055fc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff28000d000 tx=0x7ff280009040).stop 2022-08-01T23:04:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 msgr2=0x7ff290146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 0x7ff290146a60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7ff2780032f0 tx=0x7ff278016000).stop 2022-08-01T23:04:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 shutdown_connections 2022-08-01T23:04:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff274053b00 0x7ff274055fc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2900b2930 0x7ff2900a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2900b1ea0 0x7ff290146a60 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 --2- 172.21.15.32:0/2609826279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff29005c620 0x7ff2900ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 >> 172.21.15.32:0/2609826279 conn(0x7ff2900612d0 msgr2=0x7ff2900b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 shutdown_connections 2022-08-01T23:04:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.069+0000 7ff28f59e700 1 -- 172.21.15.32:0/2609826279 wait complete. 2022-08-01T23:04:00.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:04:00.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.189+0000 7f9b3b14e700 1 Processor -- start 2022-08-01T23:04:00.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.189+0000 7f9b3b14e700 1 -- start start 2022-08-01T23:04:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b2c048f20 0x7f9b2c0d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c041780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c041350 con 0x7f9b2c041c70 2022-08-01T23:04:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c044b00 con 0x7f9b2c048f20 2022-08-01T23:04:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c049300 con 0x7f9b2c042e70 2022-08-01T23:04:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b2c048f20 0x7f9b2c0d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b2c048f20 0x7f9b2c0d5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47840/0 (socket says 172.21.15.32:47840) 2022-08-01T23:04:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 -- 172.21.15.32:0/1089449410 learned_addr learned my addr 172.21.15.32:0/1089449410 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:00.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3994b700 1 --2- 172.21.15.32:0/1089449410 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c041780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 -- 172.21.15.32:0/1089449410 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 msgr2=0x7f9b2c041780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 --2- 172.21.15.32:0/1089449410 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c041780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 -- 172.21.15.32:0/1089449410 >> v1:172.21.15.55:6789/0 conn(0x7f9b2c041c70 legacy=0x7f9b2c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 -- 172.21.15.32:0/1089449410 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c03fb20 con 0x7f9b2c048f20 2022-08-01T23:04:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3a14c700 1 --2- 172.21.15.32:0/1089449410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c048f20 0x7f9b2c0d5c30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f9b24002f70 tx=0x7f9b24006eb0).ready entity=mon.0 client_cookie=ea79a25d05424888 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3914a700 1 -- 172.21.15.32:0/1089449410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b2400d570 con 0x7f9b2c048f20 2022-08-01T23:04:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3914a700 1 -- 172.21.15.32:0/1089449410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b24000e90 con 0x7f9b2c048f20 2022-08-01T23:04:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3914a700 1 -- 172.21.15.32:0/1089449410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b2400d900 con 0x7f9b2c048f20 2022-08-01T23:04:00.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- 172.21.15.32:0/1089449410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c048f20 msgr2=0x7f9b2c0d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/1089449410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c048f20 0x7f9b2c0d5c30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f9b24002f70 tx=0x7f9b24006eb0).stop 2022-08-01T23:04:00.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- 172.21.15.32:0/1089449410 shutdown_connections 2022-08-01T23:04:00.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/1089449410 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c041780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/1089449410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c048f20 0x7f9b2c0d5c30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- 172.21.15.32:0/1089449410 >> 172.21.15.32:0/1089449410 conn(0x7f9b2c00c0c0 msgr2=0x7f9b2c0484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- 172.21.15.32:0/1089449410 shutdown_connections 2022-08-01T23:04:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 -- 172.21.15.32:0/1089449410 wait complete. 2022-08-01T23:04:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.193+0000 7f9b3b14e700 1 Processor -- start 2022-08-01T23:04:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- start start 2022-08-01T23:04:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 0x7f9b2c0e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c03da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b2c048f20 0x7f9b2c03c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0e0f80 con 0x7f9b2c041c70 2022-08-01T23:04:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c03df10 con 0x7f9b2c048f20 2022-08-01T23:04:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0e1170 con 0x7f9b2c042e70 2022-08-01T23:04:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 0x7f9b2c0e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 0x7f9b2c0e1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47844/0 (socket says 172.21.15.32:47844) 2022-08-01T23:04:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3994b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c03da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 -- 172.21.15.32:0/4010889511 learned_addr learned my addr 172.21.15.32:0/4010889511 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a94d700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b2c048f20 0x7f9b2c03c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 -- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 msgr2=0x7f9b2c03da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c03da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 -- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b2c048f20 msgr2=0x7f9b2c03c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b2c048f20 0x7f9b2c03c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 -- 172.21.15.32:0/4010889511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2400b1e0 con 0x7f9b2c041c70 2022-08-01T23:04:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3a14c700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 0x7f9b2c0e1b50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f9b24002f70 tx=0x7f9b24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b24011860 con 0x7f9b2c041c70 2022-08-01T23:04:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b240119c0 con 0x7f9b2c041c70 2022-08-01T23:04:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c0e1410 con 0x7f9b2c041c70 2022-08-01T23:04:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b24011d20 con 0x7f9b2c041c70 2022-08-01T23:04:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c03e330 con 0x7f9b2c041c70 2022-08-01T23:04:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.197+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c03fd80 con 0x7f9b2c041c70 2022-08-01T23:04:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.201+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b2401a040 con 0x7f9b2c041c70 2022-08-01T23:04:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.201+0000 7f9b237fe700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053a70 0x7f9b14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.201+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b2406cdd0 con 0x7f9b2c041c70 2022-08-01T23:04:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.201+0000 7f9b3994b700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053a70 0x7f9b14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.201+0000 7f9b3994b700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053a70 0x7f9b14055f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f9b3000b000 tx=0x7f9b30006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.209+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b24043c40 con 0x7f9b2c041c70 2022-08-01T23:04:00.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.333+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b2c004d20 con 0x7f9b2c041c70 2022-08-01T23:04:00.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b237fe700 1 -- 172.21.15.32:0/4010889511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9b24023020 con 0x7f9b2c041c70 2022-08-01T23:04:00.341 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:00.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053a70 msgr2=0x7f9b14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053a70 0x7f9b14055f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f9b3000b000 tx=0x7f9b30006df0).stop 2022-08-01T23:04:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 msgr2=0x7f9b2c0e1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 0x7f9b2c0e1b50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f9b24002f70 tx=0x7f9b24015000).stop 2022-08-01T23:04:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 shutdown_connections 2022-08-01T23:04:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053a70 0x7f9b14055f30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b2c048f20 0x7f9b2c03c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b2c042e70 0x7f9b2c03da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 --2- 172.21.15.32:0/4010889511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b2c041c70 0x7f9b2c0e1b50 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 >> 172.21.15.32:0/4010889511 conn(0x7f9b2c00c0c0 msgr2=0x7f9b2c0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 shutdown_connections 2022-08-01T23:04:00.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.337+0000 7f9b3b14e700 1 -- 172.21.15.32:0/4010889511 wait complete. 2022-08-01T23:04:00.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:04:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e238a700 1 Processor -- start 2022-08-01T23:04:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e238a700 1 -- start start 2022-08-01T23:04:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e238a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 0x7f46dc0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e238a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46dc144eb0 con 0x7f46dc0b1110 2022-08-01T23:04:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e238a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46dc145110 con 0x7f46dc144ad0 2022-08-01T23:04:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e238a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0ac390 con 0x7f46dc0aa050 2022-08-01T23:04:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46dc0b1110 0x7f46dc13e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35396/0 (socket says 172.21.15.32:35396) 2022-08-01T23:04:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 -- 172.21.15.32:0/4245132205 learned_addr learned my addr 172.21.15.32:0/4245132205 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e0927700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 0x7f46dc0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919094036 0 0) 0x7f46dc144eb0 con 0x7f46dc0b1110 2022-08-01T23:04:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46bc003260 con 0x7f46dc0b1110 2022-08-01T23:04:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015903927 0 0) 0x7f46bc003260 con 0x7f46dc0b1110 2022-08-01T23:04:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 msgr2=0x7f46dc0abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 0x7f46dc0abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 >> v1:172.21.15.55:6789/0 conn(0x7f46dc144ad0 legacy=0x7f46dc0b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc1453b0 con 0x7f46dc0b1110 2022-08-01T23:04:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f46cc001680 con 0x7f46dc0b1110 2022-08-01T23:04:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 >> v1:172.21.15.32:6789/0 conn(0x7f46dc0b1110 legacy=0x7f46dc13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc003780 0x7f46bc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46bc004500 0x7f46bc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 0x7f46bc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0ac390 con 0x7f46bc004500 2022-08-01T23:04:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc00b780 con 0x7f46bc003780 2022-08-01T23:04:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc00b940 con 0x7f46dc0aa050 2022-08-01T23:04:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46bc004500 0x7f46bc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46e0927700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 0x7f46bc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46dbfff700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc003780 0x7f46bc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 -- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 msgr2=0x7f46bc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc0aa050 0x7f46bc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 -- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc003780 msgr2=0x7f46bc003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc003780 0x7f46bc003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 -- 172.21.15.32:0/4245132205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc003260 con 0x7f46bc004500 2022-08-01T23:04:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46db7fe700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46bc004500 0x7f46bc004980 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f46cc001390 tx=0x7f46dc0b27e0).ready entity=mon.0 client_cookie=dc68e2f76279b49e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46cc007170 con 0x7f46bc004500 2022-08-01T23:04:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc0072d0 con 0x7f46bc004500 2022-08-01T23:04:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.457+0000 7f46daffd700 1 -- 172.21.15.32:0/4245132205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46cc00c9c0 con 0x7f46bc004500 2022-08-01T23:04:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- 172.21.15.32:0/4245132205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46bc004500 msgr2=0x7f46bc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 --2- 172.21.15.32:0/4245132205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46bc004500 0x7f46bc004980 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f46cc001390 tx=0x7f46dc0b27e0).stop 2022-08-01T23:04:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e0927700 1 -- 172.21.15.32:0/4245132205 reap_dead start 2022-08-01T23:04:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- 172.21.15.32:0/4245132205 shutdown_connections 2022-08-01T23:04:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- 172.21.15.32:0/4245132205 >> 172.21.15.32:0/4245132205 conn(0x7f46dc0b2b00 msgr2=0x7f46dc0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- 172.21.15.32:0/4245132205 shutdown_connections 2022-08-01T23:04:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- 172.21.15.32:0/4245132205 wait complete. 2022-08-01T23:04:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 Processor -- start 2022-08-01T23:04:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- start start 2022-08-01T23:04:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 0x7f46dc150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46dc0b0a30 0x7f46dc154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc14d950 0x7f46dc14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc153be0 con 0x7f46dc0b41e0 2022-08-01T23:04:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0ae8b0 con 0x7f46dc0b0a30 2022-08-01T23:04:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e238a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0b45c0 con 0x7f46dc14d950 2022-08-01T23:04:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 0x7f46dc150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 0x7f46dc150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47866/0 (socket says 172.21.15.32:47866) 2022-08-01T23:04:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46dbfff700 1 -- 172.21.15.32:0/446415250 learned_addr learned my addr 172.21.15.32:0/446415250 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46db7fe700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46dc0b0a30 0x7f46dc154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.461+0000 7f46e0927700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc14d950 0x7f46dc14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46dbfff700 1 -- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc14d950 msgr2=0x7f46dc14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46dbfff700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc14d950 0x7f46dc14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46dbfff700 1 -- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46dc0b0a30 msgr2=0x7f46dc154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46dbfff700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46dc0b0a30 0x7f46dc154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46dbfff700 1 -- 172.21.15.32:0/446415250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc14e2c0 con 0x7f46dc0b41e0 2022-08-01T23:04:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46dbfff700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 0x7f46dc150610 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f46dc0b4750 tx=0x7f46c4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46c4014620 con 0x7f46dc0b41e0 2022-08-01T23:04:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c4014780 con 0x7f46dc0b41e0 2022-08-01T23:04:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc0aeae0 con 0x7f46dc0b41e0 2022-08-01T23:04:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46c4014ae0 con 0x7f46dc0b41e0 2022-08-01T23:04:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc154070 con 0x7f46dc0b41e0 2022-08-01T23:04:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46dc154230 con 0x7f46dc0b41e0 2022-08-01T23:04:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f46c4014c40 con 0x7f46dc0b41e0 2022-08-01T23:04:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46d97fa700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46c0053b00 0x7f46c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f46c406bf60 con 0x7f46dc0b41e0 2022-08-01T23:04:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.465+0000 7f46db7fe700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46c0053b00 0x7f46c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.469+0000 7f46db7fe700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46c0053b00 0x7f46c0055fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f46cc01a000 tx=0x7f46cc013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.473+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46c4042c80 con 0x7f46dc0b41e0 2022-08-01T23:04:00.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.597+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46dc145650 con 0x7f46dc0b41e0 2022-08-01T23:04:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.597+0000 7f46d97fa700 1 -- 172.21.15.32:0/446415250 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7f46c40220a0 con 0x7f46dc0b41e0 2022-08-01T23:04:00.605 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46c0053b00 msgr2=0x7f46c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46c0053b00 0x7f46c0055fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f46cc01a000 tx=0x7f46cc013000).stop 2022-08-01T23:04:00.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 msgr2=0x7f46dc150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 0x7f46dc150610 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f46dc0b4750 tx=0x7f46c4009040).stop 2022-08-01T23:04:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 shutdown_connections 2022-08-01T23:04:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f46c0053b00 0x7f46c0055fc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46dc14d950 0x7f46dc14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46dc0b0a30 0x7f46dc154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 --2- 172.21.15.32:0/446415250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46dc0b41e0 0x7f46dc150610 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 >> 172.21.15.32:0/446415250 conn(0x7f46dc0b2b00 msgr2=0x7f46dc1414f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 shutdown_connections 2022-08-01T23:04:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.601+0000 7f46e238a700 1 -- 172.21.15.32:0/446415250 wait complete. 2022-08-01T23:04:00.618 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:04:00.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 Processor -- start 2022-08-01T23:04:00.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 -- start start 2022-08-01T23:04:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0ad750 0x7f5aac061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5aac1405d0 0x7f5aac140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aac0aee80 con 0x7f5aac0614b0 2022-08-01T23:04:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac0b70b0 con 0x7f5aac1405d0 2022-08-01T23:04:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2b11700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0af0b0 con 0x7f5aac0ad750 2022-08-01T23:04:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2310700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5aac1405d0 0x7f5aac140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2310700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5aac1405d0 0x7f5aac140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47876/0 (socket says 172.21.15.32:47876) 2022-08-01T23:04:00.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab1b0f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5aac0614b0 0x7f5aac0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35416/0 (socket says 172.21.15.32:35416) 2022-08-01T23:04:00.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.713+0000 7f5ab2310700 1 -- 172.21.15.32:0/1236480458 learned_addr learned my addr 172.21.15.32:0/1236480458 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:00.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab130e700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0ad750 0x7f5aac061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2310700 1 -- 172.21.15.32:0/1236480458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0ad750 msgr2=0x7f5aac061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:00.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2310700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0ad750 0x7f5aac061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2310700 1 -- 172.21.15.32:0/1236480458 >> v1:172.21.15.32:6789/0 conn(0x7f5aac0614b0 legacy=0x7f5aac0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2310700 1 -- 172.21.15.32:0/1236480458 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0af350 con 0x7f5aac1405d0 2022-08-01T23:04:00.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab130e700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0ad750 0x7f5aac061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:00.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2310700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac1405d0 0x7f5aac140a80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f5aa40032f0 tx=0x7f5aa4009290).ready entity=mon.0 client_cookie=1d1accfd20a55949 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab0b0d700 1 -- 172.21.15.32:0/1236480458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aa400b8a0 con 0x7f5aac1405d0 2022-08-01T23:04:00.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab0b0d700 1 -- 172.21.15.32:0/1236480458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400ba00 con 0x7f5aac1405d0 2022-08-01T23:04:00.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab0b0d700 1 -- 172.21.15.32:0/1236480458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aa400bb70 con 0x7f5aac1405d0 2022-08-01T23:04:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 -- 172.21.15.32:0/1236480458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac1405d0 msgr2=0x7f5aac140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac1405d0 0x7f5aac140a80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f5aa40032f0 tx=0x7f5aa4009290).stop 2022-08-01T23:04:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 -- 172.21.15.32:0/1236480458 shutdown_connections 2022-08-01T23:04:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac1405d0 0x7f5aac140a80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/1236480458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0ad750 0x7f5aac061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 -- 172.21.15.32:0/1236480458 >> 172.21.15.32:0/1236480458 conn(0x7f5aac066f70 msgr2=0x7f5aac067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 -- 172.21.15.32:0/1236480458 shutdown_connections 2022-08-01T23:04:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 -- 172.21.15.32:0/1236480458 wait complete. 2022-08-01T23:04:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 Processor -- start 2022-08-01T23:04:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.717+0000 7f5ab2b11700 1 -- start start 2022-08-01T23:04:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0614b0 0x7f5aac141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 0x7f5aac145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5aac1405d0 0x7f5aac145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0b02f0 con 0x7f5aac0ad750 2022-08-01T23:04:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac156550 con 0x7f5aac1405d0 2022-08-01T23:04:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac1567f0 con 0x7f5aac0614b0 2022-08-01T23:04:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 0x7f5aac145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 0x7f5aac145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47880/0 (socket says 172.21.15.32:47880) 2022-08-01T23:04:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 -- 172.21.15.32:0/3266048651 learned_addr learned my addr 172.21.15.32:0/3266048651 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab1b0f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0614b0 0x7f5aac141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2310700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5aac1405d0 0x7f5aac145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 -- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0614b0 msgr2=0x7f5aac141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0614b0 0x7f5aac141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 -- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5aac1405d0 msgr2=0x7f5aac145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5aac1405d0 0x7f5aac145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 -- 172.21.15.32:0/3266048651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa800b180 con 0x7f5aac0ad750 2022-08-01T23:04:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2310700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5aac1405d0 0x7f5aac145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab130e700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 0x7f5aac145590 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f5aa800c000 tx=0x7f5aa8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aa800c890 con 0x7f5aac0ad750 2022-08-01T23:04:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa800c9f0 con 0x7f5aac0ad750 2022-08-01T23:04:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa4009530 con 0x7f5aac0ad750 2022-08-01T23:04:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aa800cd50 con 0x7f5aac0ad750 2022-08-01T23:04:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac0b0a80 con 0x7f5aac0ad750 2022-08-01T23:04:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac156390 con 0x7f5aac0ad750 2022-08-01T23:04:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.721+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5aa801c460 con 0x7f5aac0ad750 2022-08-01T23:04:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.725+0000 7f5a9affd700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a8c053a70 0x7f5a8c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.725+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5aa806c620 con 0x7f5aac0ad750 2022-08-01T23:04:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.725+0000 7f5ab1b0f700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a8c053a70 0x7f5a8c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.725+0000 7f5ab1b0f700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a8c053a70 0x7f5a8c055f30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f5a9c00d000 tx=0x7f5a9c007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:00.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.729+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5aa8043380 con 0x7f5aac0ad750 2022-08-01T23:04:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aac0af5f0 con 0x7f5aac0ad750 2022-08-01T23:04:00.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5a9affd700 1 -- 172.21.15.32:0/3266048651 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f5aa80469a0 con 0x7f5aac0ad750 2022-08-01T23:04:00.894 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:00.894 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1851140084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1851140084},{"type":"v1","addr":"172.21.15.55:6835","nonce":1851140084}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1909728706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1909728706},{"type":"v1","addr":"172.21.15.32:6833","nonce":1909728706}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:55.638124+0000","modified":"2022-08-01T23:03:57.319377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/519921040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":519921040},{"type":"v1","addr":"172.21.15.32:6835","nonce":519921040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T23:04:00.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a8c053a70 msgr2=0x7f5a8c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a8c053a70 0x7f5a8c055f30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f5a9c00d000 tx=0x7f5a9c007b80).stop 2022-08-01T23:04:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 msgr2=0x7f5aac145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 0x7f5aac145590 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f5aa800c000 tx=0x7f5aa8006df0).stop 2022-08-01T23:04:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 shutdown_connections 2022-08-01T23:04:00.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a8c053a70 0x7f5a8c055f30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5aac1405d0 0x7f5aac145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aac0ad750 0x7f5aac145590 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 --2- 172.21.15.32:0/3266048651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5aac0614b0 0x7f5aac141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:00.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 >> 172.21.15.32:0/3266048651 conn(0x7f5aac066f70 msgr2=0x7f5aac0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 shutdown_connections 2022-08-01T23:04:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:00.889+0000 7f5ab2b11700 1 -- 172.21.15.32:0/3266048651 wait complete. 2022-08-01T23:04:00.898 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 57 2022-08-01T23:04:01.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:55.638124+0000', 'modified': '2022-08-01T23:03:57.319377+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7674': {'gid': 7674, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/519921040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 519921040}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 519921040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:04:01.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:04:01.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:04:01.326 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":7674,"want_state":"up:active","state":"up:active","rank_uptime":5.6730417419999997,"mdsmap_epoch":57,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":9.8162146860000004} 2022-08-01T23:04:01.327 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 7674, "mdsmap_epoch": 57, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 5.673041742, "state": "up:active", "uptime": 9.816214686, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:04:01.328 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:04:01.328 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:04:01.328 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:04:01.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:01.329 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:01.338 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:04:01.355 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:01.356 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:01.411 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:01.412 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:01.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:01.413 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:04:01.420 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:04:01.421 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:01.465 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:04:01.483 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:01.484 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:01.493 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:01.494 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:01.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:01.495 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:04:01.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:04:01.535+0000 7fb652699080 -1 init, newargv = 0x563cd38cb840 newargc=9 2022-08-01T23:04:01.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23408]: starting ceph client 2022-08-01T23:04:01.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23408]: starting fuse 2022-08-01T23:04:02.603 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:04:02.621 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:02.622 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:02.679 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:02.680 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:02.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:02.681 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:04:02.687 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:04:02.687 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:04:02.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:02.688 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:04:02.688 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:04:02.689 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:04:02.689 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:04:02.690 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:04:02.690 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:02.691 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:04:02.691 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:04:02.691 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:04:02.692 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:02.692 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:04:02.693 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:04:02.693 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:04:02.694 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:04:02.694 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:02.694 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:04:02.695 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:04:02.695 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:04:02.696 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:04:02.696 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:04:02.696 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:02.697 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:04:02.697 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:04:02.743 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23408.asok 2022-08-01T23:04:02.754 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23408.asok 2022-08-01T23:04:02.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:02.755 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23408.asok status 2022-08-01T23:04:02.848 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:04:02.848 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:04:02.849 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:04:02.849 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:04:02.849 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:04:02.849 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:04:02.850 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:04:02.850 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23408", 2022-08-01T23:04:02.850 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:04:02.850 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:02.851 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:04:02.851 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:04:02.851 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 6782, 2022-08-01T23:04:02.851 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:04:02.852 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:04:02.852 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:04:02.852 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 6782 2022-08-01T23:04:02.852 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:02.852 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:04:02.853 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:04:02.853 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:04:02.853 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3204662052 2022-08-01T23:04:02.853 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:04:02.854 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:02.854 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:04:02.854 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:04:02.854 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:04:02.854 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 3204662052 2022-08-01T23:04:02.855 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:02.855 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.6782 172.21.15.130:0/3204662052", 2022-08-01T23:04:02.855 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/3204662052", 2022-08-01T23:04:02.855 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:04:02.856 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 57, 2022-08-01T23:04:02.856 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 118, 2022-08-01T23:04:02.856 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:04:02.856 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:04:02.857 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:04:02.857 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:04:02.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:02.859 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:02.859 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:02.865 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:04:02.866 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:02.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:02.867 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:02.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:02.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 Processor -- start 2022-08-01T23:04:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 -- start start 2022-08-01T23:04:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76140a7250 0x7f76141448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76140b04e0 con 0x7f7614148530 2022-08-01T23:04:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76140a9250 con 0x7f76140a7250 2022-08-01T23:04:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76140b0780 con 0x7f76140b1f40 2022-08-01T23:04:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:02.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f76195dc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76140a7250 0x7f76141448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:02.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59624/0 (socket says 172.21.15.32:59624) 2022-08-01T23:04:02.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 -- 172.21.15.32:0/707798056 learned_addr learned my addr 172.21.15.32:0/707798056 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:02.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 -- 172.21.15.32:0/707798056 >> v2:172.21.15.55:3300/0 conn(0x7f76140a7250 msgr2=0x7f76141448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:02.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 --2- 172.21.15.32:0/707798056 >> v2:172.21.15.55:3300/0 conn(0x7f76140a7250 0x7f76141448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 -- 172.21.15.32:0/707798056 >> v1:172.21.15.55:6789/0 conn(0x7f7614148530 legacy=0x7f76140b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 -- 172.21.15.32:0/707798056 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140b0a20 con 0x7f76140b1f40 2022-08-01T23:04:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f7618ddb700 1 --2- 172.21.15.32:0/707798056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b1160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f75fc002f70 tx=0x7f75fc006eb0).ready entity=mon.2 client_cookie=3a3aeeecfd4e1d3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f76137fe700 1 -- 172.21.15.32:0/707798056 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75fc00d570 con 0x7f76140b1f40 2022-08-01T23:04:03.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f76137fe700 1 -- 172.21.15.32:0/707798056 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc000e90 con 0x7f76140b1f40 2022-08-01T23:04:03.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f76137fe700 1 -- 172.21.15.32:0/707798056 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75fc00d900 con 0x7f76140b1f40 2022-08-01T23:04:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.989+0000 7f761b03f700 1 -- 172.21.15.32:0/707798056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 msgr2=0x7f76140b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 --2- 172.21.15.32:0/707798056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b1160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f75fc002f70 tx=0x7f75fc006eb0).stop 2022-08-01T23:04:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- 172.21.15.32:0/707798056 shutdown_connections 2022-08-01T23:04:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 --2- 172.21.15.32:0/707798056 >> v2:172.21.15.55:3300/0 conn(0x7f76140a7250 0x7f76141448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 --2- 172.21.15.32:0/707798056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b1160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- 172.21.15.32:0/707798056 >> 172.21.15.32:0/707798056 conn(0x7f761405f840 msgr2=0x7f761405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- 172.21.15.32:0/707798056 shutdown_connections 2022-08-01T23:04:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- 172.21.15.32:0/707798056 wait complete. 2022-08-01T23:04:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 Processor -- start 2022-08-01T23:04:03.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- start start 2022-08-01T23:04:03.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140a7250 0x7f761415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7614148530 0x7f76140a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761415a1e0 con 0x7f76140a7250 2022-08-01T23:04:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761415a480 con 0x7f7614148530 2022-08-01T23:04:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f761b03f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761415a720 con 0x7f76140b1f40 2022-08-01T23:04:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7618ddb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140a7250 0x7f761415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7618ddb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140a7250 0x7f761415abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47892/0 (socket says 172.21.15.32:47892) 2022-08-01T23:04:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7618ddb700 1 -- 172.21.15.32:0/3477347768 learned_addr learned my addr 172.21.15.32:0/3477347768 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f76195dc700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7614148530 0x7f76140a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 -- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7614148530 msgr2=0x7f76140a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7614148530 0x7f76140a96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 -- 172.21.15.32:0/3477347768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140a7250 msgr2=0x7f761415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140a7250 0x7f761415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 -- 172.21.15.32:0/3477347768 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc00b1e0 con 0x7f76140b1f40 2022-08-01T23:04:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.993+0000 7f7613fff700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b3a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7614061530 tx=0x7f760c007f10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f760c00f470 con 0x7f76140b1f40 2022-08-01T23:04:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76140aaa10 con 0x7f76140b1f40 2022-08-01T23:04:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c00f5d0 con 0x7f76140b1f40 2022-08-01T23:04:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f760c00f930 con 0x7f76140b1f40 2022-08-01T23:04:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76140abab0 con 0x7f76140b1f40 2022-08-01T23:04:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7614159ff0 con 0x7f76140b1f40 2022-08-01T23:04:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f760c019030 con 0x7f76140b1f40 2022-08-01T23:04:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f7611ffb700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f75ec053b00 0x7f75ec055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:02.997+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f760c06c0b0 con 0x7f76140b1f40 2022-08-01T23:04:03.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.001+0000 7f7618ddb700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f75ec053b00 0x7f75ec055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.001+0000 7f7618ddb700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f75ec053b00 0x7f75ec055fc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f75fc01c000 tx=0x7f75fc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.005+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f760c042d60 con 0x7f76140b1f40 2022-08-01T23:04:03.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.137+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76140574e0 con 0x7f76140b1f40 2022-08-01T23:04:03.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.137+0000 7f7611ffb700 1 -- 172.21.15.32:0/3477347768 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f760c046380 con 0x7f76140b1f40 2022-08-01T23:04:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:03.144 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:03:56.641397+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:03:54.249192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T23:03:54.638843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:03.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f75ec053b00 msgr2=0x7f75ec055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f75ec053b00 0x7f75ec055fc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f75fc01c000 tx=0x7f75fc015000).stop 2022-08-01T23:04:03.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 msgr2=0x7f76140b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b3a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7614061530 tx=0x7f760c007f10).stop 2022-08-01T23:04:03.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 shutdown_connections 2022-08-01T23:04:03.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f75ec053b00 0x7f75ec055fc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7614148530 0x7f76140a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76140b1f40 0x7f76140b3a40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 --2- 172.21.15.32:0/3477347768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140a7250 0x7f761415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 >> 172.21.15.32:0/3477347768 conn(0x7f761405f840 msgr2=0x7f7614143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 shutdown_connections 2022-08-01T23:04:03.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.141+0000 7f761b03f700 1 -- 172.21.15.32:0/3477347768 wait complete. 2022-08-01T23:04:03.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:03:54.249192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:03:54.638843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:04:03.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:04:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.265+0000 7f0df912b700 1 Processor -- start 2022-08-01T23:04:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.265+0000 7f0df912b700 1 -- start start 2022-08-01T23:04:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0df4146ae0 0x7f0df41403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df40b4310 0x7f0df41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df4143cf0 con 0x7f0df40b4820 2022-08-01T23:04:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df41476a0 con 0x7f0df4146ae0 2022-08-01T23:04:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4143f90 con 0x7f0df40b4310 2022-08-01T23:04:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0df4146ae0 0x7f0df41403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df259c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df40b4310 0x7f0df41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0df4146ae0 0x7f0df41403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57686/0 (socket says 172.21.15.32:57686) 2022-08-01T23:04:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 -- 172.21.15.32:0/1742078193 learned_addr learned my addr 172.21.15.32:0/1742078193 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 -- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df40b4310 msgr2=0x7f0df41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 --2- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df40b4310 0x7f0df41408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 -- 172.21.15.32:0/1742078193 >> v1:172.21.15.55:6789/0 conn(0x7f0df40b4820 legacy=0x7f0df40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 -- 172.21.15.32:0/1742078193 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40b0e20 con 0x7f0df4146ae0 2022-08-01T23:04:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df2d9d700 1 --2- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df4146ae0 0x7f0df41403c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f0de4002f70 tx=0x7f0de4006eb0).ready entity=mon.1 client_cookie=d76d2ff5a51e2c93 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df1d9b700 1 -- 172.21.15.32:0/1742078193 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de400d570 con 0x7f0df4146ae0 2022-08-01T23:04:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df1d9b700 1 -- 172.21.15.32:0/1742078193 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de4000e90 con 0x7f0df4146ae0 2022-08-01T23:04:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df1d9b700 1 -- 172.21.15.32:0/1742078193 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de400d900 con 0x7f0df4146ae0 2022-08-01T23:04:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 -- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df4146ae0 msgr2=0x7f0df41403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.269+0000 7f0df912b700 1 --2- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df4146ae0 0x7f0df41403c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f0de4002f70 tx=0x7f0de4006eb0).stop 2022-08-01T23:04:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- 172.21.15.32:0/1742078193 shutdown_connections 2022-08-01T23:04:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 --2- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df40b4310 0x7f0df41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 --2- 172.21.15.32:0/1742078193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df4146ae0 0x7f0df41403c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- 172.21.15.32:0/1742078193 >> 172.21.15.32:0/1742078193 conn(0x7f0df405c620 msgr2=0x7f0df405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- 172.21.15.32:0/1742078193 shutdown_connections 2022-08-01T23:04:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- 172.21.15.32:0/1742078193 wait complete. 2022-08-01T23:04:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 Processor -- start 2022-08-01T23:04:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- start start 2022-08-01T23:04:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df40b4310 0x7f0df4152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 0x7f0df40b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df4146ae0 0x7f0df40adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40ad890 con 0x7f0df40b4820 2022-08-01T23:04:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4142e70 con 0x7f0df40b4310 2022-08-01T23:04:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df912b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4142bd0 con 0x7f0df4146ae0 2022-08-01T23:04:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 0x7f0df40b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df4146ae0 0x7f0df40adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 0x7f0df40b8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47908/0 (socket says 172.21.15.32:47908) 2022-08-01T23:04:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df40b4310 0x7f0df4152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df259c700 1 -- 172.21.15.32:0/2523420584 learned_addr learned my addr 172.21.15.32:0/2523420584 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.273+0000 7f0df259c700 1 -- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df4146ae0 msgr2=0x7f0df40adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df259c700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df4146ae0 0x7f0df40adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df259c700 1 -- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df40b4310 msgr2=0x7f0df4152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df259c700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df40b4310 0x7f0df4152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df259c700 1 -- 172.21.15.32:0/2523420584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de400b1e0 con 0x7f0df40b4820 2022-08-01T23:04:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df259c700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 0x7f0df40b8680 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f0de800c000 tx=0x7f0de8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de8010780 con 0x7f0df40b4820 2022-08-01T23:04:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de80108e0 con 0x7f0df40b4820 2022-08-01T23:04:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df40adaf0 con 0x7f0df40b4820 2022-08-01T23:04:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de8010c40 con 0x7f0df40b4820 2022-08-01T23:04:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df41430b0 con 0x7f0df40b4820 2022-08-01T23:04:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df40adc80 con 0x7f0df40b4820 2022-08-01T23:04:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0de801c460 con 0x7f0df40b4820 2022-08-01T23:04:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0de3fff700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0dd0053950 0x7f0dd0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0de806c4e0 con 0x7f0df40b4820 2022-08-01T23:04:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.277+0000 7f0df2d9d700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0dd0053950 0x7f0dd0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.281+0000 7f0df2d9d700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0dd0053950 0x7f0dd0055e10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f0de401c000 tx=0x7f0de4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.285+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0de80431a0 con 0x7f0df40b4820 2022-08-01T23:04:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0df40574e0 con 0x7f0df40b4820 2022-08-01T23:04:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0de3fff700 1 -- 172.21.15.32:0/2523420584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0de80467c0 con 0x7f0df40b4820 2022-08-01T23:04:03.419 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:04:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0dd0053950 msgr2=0x7f0dd0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0dd0053950 0x7f0dd0055e10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f0de401c000 tx=0x7f0de4015000).stop 2022-08-01T23:04:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 msgr2=0x7f0df40b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 0x7f0df40b8680 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f0de800c000 tx=0x7f0de8006df0).stop 2022-08-01T23:04:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 shutdown_connections 2022-08-01T23:04:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0dd0053950 0x7f0dd0055e10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0df4146ae0 0x7f0df40adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df40b4820 0x7f0df40b8680 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 --2- 172.21.15.32:0/2523420584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0df40b4310 0x7f0df4152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 >> 172.21.15.32:0/2523420584 conn(0x7f0df405c620 msgr2=0x7f0df40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 shutdown_connections 2022-08-01T23:04:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.413+0000 7f0df912b700 1 -- 172.21.15.32:0/2523420584 wait complete. 2022-08-01T23:04:03.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:04:03.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 Processor -- start 2022-08-01T23:04:03.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 -- start start 2022-08-01T23:04:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9954064b10 0x7f9954144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954141a80 0x7f9954066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9954064ef0 con 0x7f99540b36e0 2022-08-01T23:04:03.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995413ebf0 con 0x7f9954064b10 2022-08-01T23:04:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9958d67700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9954066690 con 0x7f9954141a80 2022-08-01T23:04:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99537fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9954064b10 0x7f9954144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9952ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f99540b36e0 0x7f9954141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35456/0 (socket says 172.21.15.32:35456) 2022-08-01T23:04:03.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9952ffd700 1 -- 172.21.15.32:0/1604995429 learned_addr learned my addr 172.21.15.32:0/1604995429 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f9953fff700 1 --2- 172.21.15.32:0/1604995429 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954141a80 0x7f9954066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99537fe700 1 -- 172.21.15.32:0/1604995429 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954141a80 msgr2=0x7f9954066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99537fe700 1 --2- 172.21.15.32:0/1604995429 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954141a80 0x7f9954066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99537fe700 1 -- 172.21.15.32:0/1604995429 >> v1:172.21.15.32:6789/0 conn(0x7f99540b36e0 legacy=0x7f9954141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99527fc700 1 -- 172.21.15.32:0/1604995429 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788470665 0 0) 0x7f9954064ef0 con 0x7f99540b36e0 2022-08-01T23:04:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99537fe700 1 -- 172.21.15.32:0/1604995429 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540b4850 con 0x7f9954064b10 2022-08-01T23:04:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99537fe700 1 --2- 172.21.15.32:0/1604995429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9954064b10 0x7f9954144e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f9944002f70 tx=0x7f9944006df0).ready entity=mon.0 client_cookie=8c4575fd1776ed8f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99527fc700 1 -- 172.21.15.32:0/1604995429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f994400d570 con 0x7f9954064b10 2022-08-01T23:04:03.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99527fc700 1 -- 172.21.15.32:0/1604995429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944000e90 con 0x7f9954064b10 2022-08-01T23:04:03.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.537+0000 7f99527fc700 1 -- 172.21.15.32:0/1604995429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f994400d8c0 con 0x7f9954064b10 2022-08-01T23:04:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- 172.21.15.32:0/1604995429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9954064b10 msgr2=0x7f9954144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 --2- 172.21.15.32:0/1604995429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9954064b10 0x7f9954144e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f9944002f70 tx=0x7f9944006df0).stop 2022-08-01T23:04:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- 172.21.15.32:0/1604995429 shutdown_connections 2022-08-01T23:04:03.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 --2- 172.21.15.32:0/1604995429 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954141a80 0x7f9954066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 --2- 172.21.15.32:0/1604995429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9954064b10 0x7f9954144e60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- 172.21.15.32:0/1604995429 >> 172.21.15.32:0/1604995429 conn(0x7f995405f840 msgr2=0x7f995405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- 172.21.15.32:0/1604995429 shutdown_connections 2022-08-01T23:04:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- 172.21.15.32:0/1604995429 wait complete. 2022-08-01T23:04:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 Processor -- start 2022-08-01T23:04:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- start start 2022-08-01T23:04:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954064b10 0x7f99541380e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 0x7f99540af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954141a80 0x7f99540afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954143dd0 con 0x7f99540b36e0 2022-08-01T23:04:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b0150 con 0x7f9954141a80 2022-08-01T23:04:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9958d67700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99541447d0 con 0x7f9954064b10 2022-08-01T23:04:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9952ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 0x7f99540af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9952ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 0x7f99540af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47920/0 (socket says 172.21.15.32:47920) 2022-08-01T23:04:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.541+0000 7f9952ffd700 1 -- 172.21.15.32:0/1492602829 learned_addr learned my addr 172.21.15.32:0/1492602829 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f99537fe700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954064b10 0x7f99541380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9953fff700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954141a80 0x7f99540afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9952ffd700 1 -- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954064b10 msgr2=0x7f99541380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9952ffd700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954064b10 0x7f99541380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9952ffd700 1 -- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954141a80 msgr2=0x7f99540afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9952ffd700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954141a80 0x7f99540afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9952ffd700 1 -- 172.21.15.32:0/1492602829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994400b180 con 0x7f99540b36e0 2022-08-01T23:04:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9953fff700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954141a80 0x7f99540afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9952ffd700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 0x7f99540af770 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f9948000ed0 tx=0x7f9948007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f994800ce50 con 0x7f99540b36e0 2022-08-01T23:04:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99480072e0 con 0x7f99540b36e0 2022-08-01T23:04:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99540b5630 con 0x7f99540b36e0 2022-08-01T23:04:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9948005610 con 0x7f99540b36e0 2022-08-01T23:04:03.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99540b5b70 con 0x7f99540b36e0 2022-08-01T23:04:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9954141ec0 con 0x7f99540b36e0 2022-08-01T23:04:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f994801a060 con 0x7f99540b36e0 2022-08-01T23:04:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9950ff9700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99340539e0 0x7f9934055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.545+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f994806b900 con 0x7f99540b36e0 2022-08-01T23:04:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.549+0000 7f99537fe700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99340539e0 0x7f9934055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.549+0000 7f99537fe700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99340539e0 0x7f9934055ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f994401a000 tx=0x7f9944015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.553+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f99480426c0 con 0x7f99540b36e0 2022-08-01T23:04:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9954144910 con 0x7f99540b36e0 2022-08-01T23:04:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9950ff9700 1 -- 172.21.15.32:0/1492602829 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9948018020 con 0x7f99540b36e0 2022-08-01T23:04:03.686 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99340539e0 msgr2=0x7f9934055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99340539e0 0x7f9934055ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f994401a000 tx=0x7f9944015000).stop 2022-08-01T23:04:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 msgr2=0x7f99540af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 0x7f99540af770 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f9948000ed0 tx=0x7f9948007ac0).stop 2022-08-01T23:04:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 shutdown_connections 2022-08-01T23:04:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99340539e0 0x7f9934055ea0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954141a80 0x7f99540afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99540b36e0 0x7f99540af770 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 --2- 172.21.15.32:0/1492602829 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9954064b10 0x7f99541380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 >> 172.21.15.32:0/1492602829 conn(0x7f995405f840 msgr2=0x7f995405cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 shutdown_connections 2022-08-01T23:04:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.681+0000 7f9958d67700 1 -- 172.21.15.32:0/1492602829 wait complete. 2022-08-01T23:04:03.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:04:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.801+0000 7f79c520e700 1 Processor -- start 2022-08-01T23:04:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.801+0000 7f79c520e700 1 -- start start 2022-08-01T23:04:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 0x7f79c0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 0x7f79c0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00aee80 con 0x7f79c00614b0 2022-08-01T23:04:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00b70b0 con 0x7f79c00ad750 2022-08-01T23:04:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00af0b0 con 0x7f79c01405d0 2022-08-01T23:04:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 0x7f79c0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79bf7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 0x7f79c0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79bf7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 0x7f79c0061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57714/0 (socket says 172.21.15.32:57714) 2022-08-01T23:04:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 0x7f79c0140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59666/0 (socket says 172.21.15.32:59666) 2022-08-01T23:04:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79bf7fe700 1 -- 172.21.15.32:0/3597312943 learned_addr learned my addr 172.21.15.32:0/3597312943 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79bffff700 1 --1- 172.21.15.32:0/3597312943 >> v1:172.21.15.55:6789/0 conn(0x7f79c00614b0 0x7f79c00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:56374/0 (socket says 172.21.15.32:56374) 2022-08-01T23:04:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 -- 172.21.15.32:0/3597312943 >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 msgr2=0x7f79c0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 --2- 172.21.15.32:0/3597312943 >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 0x7f79c0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 -- 172.21.15.32:0/3597312943 >> v1:172.21.15.55:6789/0 conn(0x7f79c00614b0 legacy=0x7f79c00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 -- 172.21.15.32:0/3597312943 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00af350 con 0x7f79c01405d0 2022-08-01T23:04:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79beffd700 1 -- 172.21.15.32:0/3597312943 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448951772 0 0) 0x7f79c00aee80 con 0x7f79c00614b0 2022-08-01T23:04:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79bf7fe700 1 --2- 172.21.15.32:0/3597312943 >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 0x7f79c0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c4a0d700 1 --2- 172.21.15.32:0/3597312943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 0x7f79c0140a80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f79b80032f0 tx=0x7f79b80091d0).ready entity=mon.2 client_cookie=d5500cfb1fcb9cd9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79beffd700 1 -- 172.21.15.32:0/3597312943 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79b800b6c0 con 0x7f79c01405d0 2022-08-01T23:04:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79beffd700 1 -- 172.21.15.32:0/3597312943 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b800b820 con 0x7f79c01405d0 2022-08-01T23:04:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79beffd700 1 -- 172.21.15.32:0/3597312943 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79b800b990 con 0x7f79c01405d0 2022-08-01T23:04:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- 172.21.15.32:0/3597312943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 msgr2=0x7f79c0140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 --2- 172.21.15.32:0/3597312943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 0x7f79c0140a80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f79b80032f0 tx=0x7f79b80091d0).stop 2022-08-01T23:04:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- 172.21.15.32:0/3597312943 shutdown_connections 2022-08-01T23:04:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 --2- 172.21.15.32:0/3597312943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c01405d0 0x7f79c0140a80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 --2- 172.21.15.32:0/3597312943 >> v2:172.21.15.55:3300/0 conn(0x7f79c00ad750 0x7f79c0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- 172.21.15.32:0/3597312943 >> 172.21.15.32:0/3597312943 conn(0x7f79c0066f70 msgr2=0x7f79c0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- 172.21.15.32:0/3597312943 shutdown_connections 2022-08-01T23:04:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.805+0000 7f79c520e700 1 -- 172.21.15.32:0/3597312943 wait complete. 2022-08-01T23:04:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 Processor -- start 2022-08-01T23:04:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 -- start start 2022-08-01T23:04:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c00614b0 0x7f79c00b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 0x7f79c00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79c01405d0 0x7f79c00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b3f10 con 0x7f79c00ad750 2022-08-01T23:04:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0152eb0 con 0x7f79c01405d0 2022-08-01T23:04:03.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0153150 con 0x7f79c00614b0 2022-08-01T23:04:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 0x7f79c00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 0x7f79c00b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47934/0 (socket says 172.21.15.32:47934) 2022-08-01T23:04:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 -- 172.21.15.32:0/43421386 learned_addr learned my addr 172.21.15.32:0/43421386 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:03.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bffff700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c00614b0 0x7f79c00b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c4a0d700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79c01405d0 0x7f79c00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 -- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c00614b0 msgr2=0x7f79c00b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c00614b0 0x7f79c00b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 -- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79c01405d0 msgr2=0x7f79c00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79c01405d0 0x7f79c00b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 -- 172.21.15.32:0/43421386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b000b1e0 con 0x7f79c00ad750 2022-08-01T23:04:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bf7fe700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 0x7f79c00b3530 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f79b000c000 tx=0x7f79b0006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79b000ee10 con 0x7f79c00ad750 2022-08-01T23:04:03.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b8009470 con 0x7f79c00ad750 2022-08-01T23:04:03.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b000c850 con 0x7f79c00ad750 2022-08-01T23:04:03.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79b000cbb0 con 0x7f79c00ad750 2022-08-01T23:04:03.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.809+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c00b1180 con 0x7f79c00ad750 2022-08-01T23:04:03.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.813+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c0145ff0 con 0x7f79c00ad750 2022-08-01T23:04:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.813+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f79b0022070 con 0x7f79c00ad750 2022-08-01T23:04:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.813+0000 7f79bd7fa700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79a4053950 0x7f79a4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.813+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f79b006c660 con 0x7f79c00ad750 2022-08-01T23:04:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.813+0000 7f79bffff700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79a4053950 0x7f79a4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.813+0000 7f79bffff700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79a4053950 0x7f79a4055e10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f79a800b000 tx=0x7f79a8007160).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.821+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79c0145ff0 con 0x7f79c00ad750 2022-08-01T23:04:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79c0145ff0 con 0x7f79c00ad750 2022-08-01T23:04:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79bd7fa700 1 -- 172.21.15.32:0/43421386 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f79c0145ff0 con 0x7f79c00ad750 2022-08-01T23:04:03.950 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:03.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79a4053950 msgr2=0x7f79a4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79a4053950 0x7f79a4055e10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f79a800b000 tx=0x7f79a8007160).stop 2022-08-01T23:04:03.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 msgr2=0x7f79c00b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:03.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 0x7f79c00b3530 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f79b000c000 tx=0x7f79b0006eb0).stop 2022-08-01T23:04:03.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 shutdown_connections 2022-08-01T23:04:03.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79a4053950 0x7f79a4055e10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79c01405d0 0x7f79c00b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c00ad750 0x7f79c00b3530 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 --2- 172.21.15.32:0/43421386 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79c00614b0 0x7f79c00b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:03.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 >> 172.21.15.32:0/43421386 conn(0x7f79c0066f70 msgr2=0x7f79c00b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 shutdown_connections 2022-08-01T23:04:03.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:03.945+0000 7f79c520e700 1 -- 172.21.15.32:0/43421386 wait complete. 2022-08-01T23:04:03.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:04:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 Processor -- start 2022-08-01T23:04:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 -- start start 2022-08-01T23:04:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a000b2710 0x7f0a0014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a0014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a000a7250 con 0x7f0a000afd60 2022-08-01T23:04:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a000a7480 con 0x7f0a000b2710 2022-08-01T23:04:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a063af700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000a7720 con 0x7f0a000b01a0 2022-08-01T23:04:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a000b2710 0x7f0a0014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a000b2710 0x7f0a0014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47940/0 (socket says 172.21.15.32:47940) 2022-08-01T23:04:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 -- 172.21.15.32:0/3271570781 learned_addr learned my addr 172.21.15.32:0/3271570781 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a04bac700 1 --2- 172.21.15.32:0/3271570781 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a0014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 -- 172.21.15.32:0/3271570781 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 msgr2=0x7f0a0014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 --2- 172.21.15.32:0/3271570781 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a0014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 -- 172.21.15.32:0/3271570781 >> v1:172.21.15.55:6789/0 conn(0x7f0a000afd60 legacy=0x7f0a000a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 -- 172.21.15.32:0/3271570781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000a79c0 con 0x7f0a000b2710 2022-08-01T23:04:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f0a053ad700 1 --2- 172.21.15.32:0/3271570781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a0014afd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f09fc002f70 tx=0x7f09fc006eb0).ready entity=mon.0 client_cookie=55924e499a5b9c1e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f09f3fff700 1 -- 172.21.15.32:0/3271570781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09fc00d570 con 0x7f0a000b2710 2022-08-01T23:04:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f09f3fff700 1 -- 172.21.15.32:0/3271570781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc000e90 con 0x7f0a000b2710 2022-08-01T23:04:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.069+0000 7f09f3fff700 1 -- 172.21.15.32:0/3271570781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09fc00d900 con 0x7f0a000b2710 2022-08-01T23:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- 172.21.15.32:0/3271570781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 msgr2=0x7f0a0014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 --2- 172.21.15.32:0/3271570781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a0014afd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f09fc002f70 tx=0x7f09fc006eb0).stop 2022-08-01T23:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- 172.21.15.32:0/3271570781 shutdown_connections 2022-08-01T23:04:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 --2- 172.21.15.32:0/3271570781 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a0014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 --2- 172.21.15.32:0/3271570781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a0014afd0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- 172.21.15.32:0/3271570781 >> 172.21.15.32:0/3271570781 conn(0x7f0a00060310 msgr2=0x7f0a000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- 172.21.15.32:0/3271570781 shutdown_connections 2022-08-01T23:04:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- 172.21.15.32:0/3271570781 wait complete. 2022-08-01T23:04:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 Processor -- start 2022-08-01T23:04:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- start start 2022-08-01T23:04:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a000afd60 0x7f0a000aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a000af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a000ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000ac370 con 0x7f0a000b2710 2022-08-01T23:04:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000b3940 con 0x7f0a000afd60 2022-08-01T23:04:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a063af700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000b3b70 con 0x7f0a000b01a0 2022-08-01T23:04:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a000ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a000ab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47950/0 (socket says 172.21.15.32:47950) 2022-08-01T23:04:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 -- 172.21.15.32:0/3573564192 learned_addr learned my addr 172.21.15.32:0/3573564192 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a04bac700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a000af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a053ad700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a000afd60 0x7f0a000aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 -- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 msgr2=0x7f0a000af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a000af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 -- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a000afd60 msgr2=0x7f0a000aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a000afd60 0x7f0a000aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 -- 172.21.15.32:0/3573564192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc00b1e0 con 0x7f0a000b2710 2022-08-01T23:04:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.073+0000 7f0a05bae700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a000ab100 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f0a000ac4b0 tx=0x7f09f8007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09f800e4a0 con 0x7f0a000b2710 2022-08-01T23:04:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f800e600 con 0x7f0a000b2710 2022-08-01T23:04:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09f800e960 con 0x7f0a000b2710 2022-08-01T23:04:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a000aa420 con 0x7f0a000b2710 2022-08-01T23:04:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a000af630 con 0x7f0a000b2710 2022-08-01T23:04:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a000574e0 con 0x7f0a000b2710 2022-08-01T23:04:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f09f8019030 con 0x7f0a000b2710 2022-08-01T23:04:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f09f27fc700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09e8053a70 0x7f09e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f09f806b7d0 con 0x7f0a000b2710 2022-08-01T23:04:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.077+0000 7f0a053ad700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09e8053a70 0x7f09e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.081+0000 7f0a053ad700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09e8053a70 0x7f09e8055f30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f09fc01a000 tx=0x7f09fc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.089+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09f8042590 con 0x7f0a000b2710 2022-08-01T23:04:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.213+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a00069a70 con 0x7f0a000b2710 2022-08-01T23:04:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.213+0000 7f09f27fc700 1 -- 172.21.15.32:0/3573564192 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7f09f8017020 con 0x7f0a000b2710 2022-08-01T23:04:04.220 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09e8053a70 msgr2=0x7f09e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09e8053a70 0x7f09e8055f30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f09fc01a000 tx=0x7f09fc015000).stop 2022-08-01T23:04:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 msgr2=0x7f0a000ab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a000ab100 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f0a000ac4b0 tx=0x7f09f8007f40).stop 2022-08-01T23:04:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 shutdown_connections 2022-08-01T23:04:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09e8053a70 0x7f09e8055f30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a000b2710 0x7f0a000ab100 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0a000b01a0 0x7f0a000af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 --2- 172.21.15.32:0/3573564192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a000afd60 0x7f0a000aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 >> 172.21.15.32:0/3573564192 conn(0x7f0a00060310 msgr2=0x7f0a0014a090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:04.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 shutdown_connections 2022-08-01T23:04:04.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.217+0000 7f0a063af700 1 -- 172.21.15.32:0/3573564192 wait complete. 2022-08-01T23:04:04.233 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:04:04.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.325+0000 7f3a78403700 1 Processor -- start 2022-08-01T23:04:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.325+0000 7f3a78403700 1 -- start start 2022-08-01T23:04:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a700b6710 0x7f3a700b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a700b6af0 0x7f3a7012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a70144240 con 0x7f3a700b4550 2022-08-01T23:04:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a70143d00 con 0x7f3a700b6710 2022-08-01T23:04:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a70143fa0 con 0x7f3a700b6af0 2022-08-01T23:04:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a700b6710 0x7f3a700b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a700b6710 0x7f3a700b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47954/0 (socket says 172.21.15.32:47954) 2022-08-01T23:04:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 -- 172.21.15.32:0/2583614358 learned_addr learned my addr 172.21.15.32:0/2583614358 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a769a0700 1 --2- 172.21.15.32:0/2583614358 >> v2:172.21.15.55:3300/0 conn(0x7f3a700b6af0 0x7f3a7012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 -- 172.21.15.32:0/2583614358 >> v2:172.21.15.55:3300/0 conn(0x7f3a700b6af0 msgr2=0x7f3a7012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 --2- 172.21.15.32:0/2583614358 >> v2:172.21.15.55:3300/0 conn(0x7f3a700b6af0 0x7f3a7012bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 -- 172.21.15.32:0/2583614358 >> v1:172.21.15.55:6789/0 conn(0x7f3a700b4550 legacy=0x7f3a70061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 -- 172.21.15.32:0/2583614358 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a70061780 con 0x7f3a700b6710 2022-08-01T23:04:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a769a0700 1 --2- 172.21.15.32:0/2583614358 >> v2:172.21.15.55:3300/0 conn(0x7f3a700b6af0 0x7f3a7012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7619f700 1 --2- 172.21.15.32:0/2583614358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a700b5d80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f3a701432b0 tx=0x7f3a68009290).ready entity=mon.0 client_cookie=c1b9ffbc2da4778c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7519d700 1 -- 172.21.15.32:0/2583614358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6800b820 con 0x7f3a700b6710 2022-08-01T23:04:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7519d700 1 -- 172.21.15.32:0/2583614358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800b980 con 0x7f3a700b6710 2022-08-01T23:04:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a7519d700 1 -- 172.21.15.32:0/2583614358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6800bb30 con 0x7f3a700b6710 2022-08-01T23:04:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- 172.21.15.32:0/2583614358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 msgr2=0x7f3a700b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 --2- 172.21.15.32:0/2583614358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a700b5d80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f3a701432b0 tx=0x7f3a68009290).stop 2022-08-01T23:04:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- 172.21.15.32:0/2583614358 shutdown_connections 2022-08-01T23:04:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 --2- 172.21.15.32:0/2583614358 >> v2:172.21.15.55:3300/0 conn(0x7f3a700b6af0 0x7f3a7012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 --2- 172.21.15.32:0/2583614358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a700b5d80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- 172.21.15.32:0/2583614358 >> 172.21.15.32:0/2583614358 conn(0x7f3a7005c620 msgr2=0x7f3a7005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- 172.21.15.32:0/2583614358 shutdown_connections 2022-08-01T23:04:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- 172.21.15.32:0/2583614358 wait complete. 2022-08-01T23:04:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 Processor -- start 2022-08-01T23:04:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.329+0000 7f3a78403700 1 -- start start 2022-08-01T23:04:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a700b4550 0x7f3a7012c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a70156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a700b6af0 0x7f3a7012ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70156b00 con 0x7f3a700b6710 2022-08-01T23:04:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70129910 con 0x7f3a700b4550 2022-08-01T23:04:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70129bb0 con 0x7f3a700b6af0 2022-08-01T23:04:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a70156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a70156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47962/0 (socket says 172.21.15.32:47962) 2022-08-01T23:04:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 -- 172.21.15.32:0/3059341804 learned_addr learned my addr 172.21.15.32:0/3059341804 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a769a0700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a700b6af0 0x7f3a7012ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7619f700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a700b4550 0x7f3a7012c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 -- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a700b6af0 msgr2=0x7f3a7012ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a700b6af0 0x7f3a7012ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 -- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a700b4550 msgr2=0x7f3a7012c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a700b4550 0x7f3a7012c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 -- 172.21.15.32:0/3059341804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68009530 con 0x7f3a700b6710 2022-08-01T23:04:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a769a0700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a700b6af0 0x7f3a7012ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a7599e700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a70156610 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f3a64001fa0 tx=0x7f3a64007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6400ce80 con 0x7f3a700b6710 2022-08-01T23:04:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64007240 con 0x7f3a700b6710 2022-08-01T23:04:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a70129e50 con 0x7f3a700b6710 2022-08-01T23:04:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a64006510 con 0x7f3a700b6710 2022-08-01T23:04:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a70147eb0 con 0x7f3a700b6710 2022-08-01T23:04:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a70061a20 con 0x7f3a700b6710 2022-08-01T23:04:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.333+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a64010030 con 0x7f3a700b6710 2022-08-01T23:04:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.337+0000 7f3a637fe700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a50053b30 0x7f3a50055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.337+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3a6406bcc0 con 0x7f3a700b6710 2022-08-01T23:04:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.337+0000 7f3a7619f700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a50053b30 0x7f3a50055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.337+0000 7f3a7619f700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a50053b30 0x7f3a50055ff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f3a68019000 tx=0x7f3a68016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.341+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a64042a80 con 0x7f3a700b6710 2022-08-01T23:04:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.501+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a700574e0 con 0x7f3a700b6710 2022-08-01T23:04:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a637fe700 1 -- 172.21.15.32:0/3059341804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f3a640460a0 con 0x7f3a700b6710 2022-08-01T23:04:04.509 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:04.509 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1851140084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1851140084},{"type":"v1","addr":"172.21.15.55:6835","nonce":1851140084}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1909728706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1909728706},{"type":"v1","addr":"172.21.15.32:6833","nonce":1909728706}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:03:55.638124+0000","modified":"2022-08-01T23:03:57.319377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7674":{"gid":7674,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/519921040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":519921040},{"type":"v1","addr":"172.21.15.32:6835","nonce":519921040}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T23:04:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a50053b30 msgr2=0x7f3a50055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a50053b30 0x7f3a50055ff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f3a68019000 tx=0x7f3a68016000).stop 2022-08-01T23:04:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 msgr2=0x7f3a70156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a70156610 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f3a64001fa0 tx=0x7f3a64007c00).stop 2022-08-01T23:04:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 shutdown_connections 2022-08-01T23:04:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3a50053b30 0x7f3a50055ff0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3a700b6af0 0x7f3a7012ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a700b6710 0x7f3a70156610 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 --2- 172.21.15.32:0/3059341804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a700b4550 0x7f3a7012c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 >> 172.21.15.32:0/3059341804 conn(0x7f3a7005c620 msgr2=0x7f3a701425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:04.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 shutdown_connections 2022-08-01T23:04:04.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:04.505+0000 7f3a78403700 1 -- 172.21.15.32:0/3059341804 wait complete. 2022-08-01T23:04:04.515 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 57 2022-08-01T23:04:04.829 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:03:55.638124+0000', 'modified': '2022-08-01T23:03:57.319377+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7674': {'gid': 7674, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/519921040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 519921040}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 519921040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:04:04.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:04:04.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:04:04.948 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":7674,"want_state":"up:active","state":"up:active","rank_uptime":9.2947981839999994,"mdsmap_epoch":57,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":13.437980241} 2022-08-01T23:04:04.948 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 7674, "mdsmap_epoch": 57, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 9.294798184, "state": "up:active", "uptime": 13.437980241, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:04:04.949 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:04:04.949 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:04:04.949 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:04:04.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:04.950 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:04.959 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:05.017 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:05.018 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:05.028 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:05.029 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:05.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.030 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:05.036 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:04:05.036 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.039 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:05.057 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:05.057 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:05.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:04:05.109+0000 7f70f541c080 -1 init, newargv = 0x5621eaf4d660 newargc=9ceph-fuse[22924]: starting ceph client 2022-08-01T23:04:05.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr: 2022-08-01T23:04:05.121 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:05.122 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:05.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.123 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:05.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22924]: starting fuse 2022-08-01T23:04:05.131 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:04:05.131 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:04:05.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:04:05.133 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:04:05.133 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:04:05.134 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:04:05.134 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:04:05.134 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:05.135 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:04:05.135 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:04:05.135 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:04:05.136 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:05.136 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:04:05.137 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:04:05.137 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:04:05.137 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:04:05.138 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:05.138 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:04:05.139 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:04:05.139 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:04:05.140 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:04:05.140 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:04:05.140 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:05.141 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:04:05.141 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:04:05.185 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.22924.asok 2022-08-01T23:04:05.190 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22924.asok 2022-08-01T23:04:05.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22924.asok status 2022-08-01T23:04:05.283 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:04:05.283 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:04:05.284 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:04:05.284 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:04:05.284 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:04:05.284 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:04:05.284 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:04:05.285 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "22924", 2022-08-01T23:04:05.285 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:04:05.285 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:05.285 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:04:05.286 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:04:05.286 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6821, 2022-08-01T23:04:05.286 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:04:05.286 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:04:05.287 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:04:05.287 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6821 2022-08-01T23:04:05.287 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:05.287 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:04:05.287 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:04:05.288 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:04:05.288 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1954758082 2022-08-01T23:04:05.288 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:04:05.288 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:05.288 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:04:05.289 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:04:05.289 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:04:05.289 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1954758082 2022-08-01T23:04:05.289 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:05.290 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6821 172.21.15.157:0/1954758082", 2022-08-01T23:04:05.290 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/1954758082", 2022-08-01T23:04:05.290 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:04:05.290 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 57, 2022-08-01T23:04:05.291 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 118, 2022-08-01T23:04:05.291 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:04:05.291 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:04:05.291 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:04:05.291 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:04:05.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.293 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.294 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.301 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:05.302 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:05.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.303 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.364 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.364 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:05.372 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:04:05.373 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:05.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.374 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.375 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:05.423 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:04:05.424 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:05.425 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:04:05.425 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:04:05.426 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:05.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23408]: fuse finished with error 0 and tester_r 0 2022-08-01T23:04:05.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.443 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.444 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:05.449 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:04:05.450 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:04:05.451 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T23:04:05.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:05.539 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:05.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.547 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.548 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.556 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:05.557 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:05.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.558 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.559 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.566 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:05.567 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:05.568 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:04:05.568 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:04:05.568 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[22924]: fuse finished with error 0 and tester_r 0 2022-08-01T23:04:05.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:05.626 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:05.627 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.634 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:04:05.635 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:04:05.636 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T23:04:05.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:05.706 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:05.713 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:04:05.713 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:04:05.714 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:04:05.714 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:04:05.714 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:04:05.715 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:04:05.729 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:04:05.730 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:05.730 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:04:05.731 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:05.732 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:04:05.732 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:05.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2022-08-01T23:04:05.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2022-08-01T23:04:05.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2022-08-01T23:04:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.865+0000 7f1623bee700 1 Processor -- start 2022-08-01T23:04:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.865+0000 7f1623bee700 1 -- start start 2022-08-01T23:04:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f161c060290 0x7f161c0617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f161c065fb0 0x7f161c066e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 Processor -- start 2022-08-01T23:04:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c060670 con 0x7f161c065b00 2022-08-01T23:04:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c060e00 con 0x7f161c065fb0 2022-08-01T23:04:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c0610a0 con 0x7f161c060290 2022-08-01T23:04:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 -- start start 2022-08-01T23:04:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f161c065fb0 0x7f161c066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f161c065fb0 0x7f161c066e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47972/0 (socket says 172.21.15.32:47972) 2022-08-01T23:04:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 -- 172.21.15.32:0/3589205214 learned_addr learned my addr 172.21.15.32:0/3589205214 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162198a700 1 --2- 172.21.15.32:0/3589205214 >> v2:172.21.15.55:3300/0 conn(0x7f161c060290 0x7f161c0617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 -- 172.21.15.32:0/3589205214 >> v2:172.21.15.55:3300/0 conn(0x7f161c060290 msgr2=0x7f161c0617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 --2- 172.21.15.32:0/3589205214 >> v2:172.21.15.55:3300/0 conn(0x7f161c060290 0x7f161c0617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 -- 172.21.15.32:0/3589205214 >> v1:172.21.15.55:6789/0 conn(0x7f161c065b00 legacy=0x7f161c066760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 -- 172.21.15.32:0/3589205214 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c061340 con 0x7f161c065fb0 2022-08-01T23:04:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f162218b700 1 --2- 172.21.15.32:0/3589205214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065fb0 0x7f161c066e70 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f161c0aeac0 tx=0x7f1614009290).ready entity=mon.0 client_cookie=d56acb785700e5bc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee740b68f0 0x7fee7413ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 0x7fee7413d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1620988700 1 -- 172.21.15.32:0/3589205214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f161400b830 con 0x7f161c065fb0 2022-08-01T23:04:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1620988700 1 -- 172.21.15.32:0/3589205214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400b990 con 0x7f161c065fb0 2022-08-01T23:04:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1620988700 1 -- 172.21.15.32:0/3589205214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f161400bb40 con 0x7f161c065fb0 2022-08-01T23:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7413d860 con 0x7fee7405ff40 2022-08-01T23:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee740ac6e0 con 0x7fee740b68f0 2022-08-01T23:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee7b92d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee740ac980 con 0x7fee740aa210 2022-08-01T23:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee79eca700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee7405ff40 0x7fee74146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35518/0 (socket says 172.21.15.32:35518) 2022-08-01T23:04:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee79eca700 1 -- 172.21.15.32:0/1016173219 learned_addr learned my addr 172.21.15.32:0/1016173219 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 --2- 172.21.15.32:0/1016173219 >> v2:172.21.15.55:3300/0 conn(0x7fee740b68f0 0x7fee7413ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932879816 0 0) 0x7fee7413d860 con 0x7fee7405ff40 2022-08-01T23:04:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee5c003260 con 0x7fee7405ff40 2022-08-01T23:04:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee78ec8700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 0x7fee7413d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- 172.21.15.32:0/3589205214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065fb0 msgr2=0x7f161c066e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 --2- 172.21.15.32:0/3589205214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065fb0 0x7f161c066e70 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f161c0aeac0 tx=0x7f1614009290).stop 2022-08-01T23:04:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569513227 0 0) 0x7fee5c003260 con 0x7fee7405ff40 2022-08-01T23:04:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 msgr2=0x7fee7413d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- 172.21.15.32:0/3589205214 shutdown_connections 2022-08-01T23:04:05.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 0x7fee7413d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 >> v2:172.21.15.55:3300/0 conn(0x7fee740b68f0 msgr2=0x7fee7413ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 --2- 172.21.15.32:0/3589205214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065fb0 0x7f161c066e70 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 --2- 172.21.15.32:0/3589205214 >> v2:172.21.15.55:3300/0 conn(0x7f161c060290 0x7f161c0617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- 172.21.15.32:0/3589205214 >> 172.21.15.32:0/3589205214 conn(0x7f161c05d0c0 msgr2=0x7f161c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 --2- 172.21.15.32:0/1016173219 >> v2:172.21.15.55:3300/0 conn(0x7fee740b68f0 0x7fee7413ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee740acc20 con 0x7fee7405ff40 2022-08-01T23:04:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fee6c0019e0 con 0x7fee7405ff40 2022-08-01T23:04:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 >> v1:172.21.15.32:6789/0 conn(0x7fee7405ff40 legacy=0x7fee74146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee5c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c004500 0x7fee5c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 0x7fee5c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740ac6e0 con 0x7fee5c0037d0 2022-08-01T23:04:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740ac980 con 0x7fee5c004500 2022-08-01T23:04:05.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c00b750 con 0x7fee740aa210 2022-08-01T23:04:05.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- 172.21.15.32:0/3589205214 shutdown_connections 2022-08-01T23:04:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee5c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7f1623bee700 1 -- 172.21.15.32:0/3589205214 wait complete. 2022-08-01T23:04:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee78ec8700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c004500 0x7fee5c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee79eca700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 0x7fee5c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 -- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 msgr2=0x7fee5c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740aa210 0x7fee5c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 -- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c004500 msgr2=0x7fee5c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c004500 0x7fee5c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 -- 172.21.15.32:0/1016173219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee740acc20 con 0x7fee5c0037d0 2022-08-01T23:04:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee78ec8700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee5c004500 0x7fee5c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee796c9700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee5c003bf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7fee7000c000 tx=0x7fee70006df0).ready entity=mon.0 client_cookie=fc45c4a19fcd91bc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7000e7d0 con 0x7fee5c0037d0 2022-08-01T23:04:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7000e930 con 0x7fee5c0037d0 2022-08-01T23:04:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.869+0000 7fee63fff700 1 -- 172.21.15.32:0/1016173219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7000ec90 con 0x7fee5c0037d0 2022-08-01T23:04:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 Processor -- start 2022-08-01T23:04:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- start start 2022-08-01T23:04:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- 172.21.15.32:0/1016173219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 msgr2=0x7fee5c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 --2- 172.21.15.32:0/1016173219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee5c003bf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7fee7000c000 tx=0x7fee70006df0).stop 2022-08-01T23:04:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee79eca700 1 -- 172.21.15.32:0/1016173219 reap_dead start 2022-08-01T23:04:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- 172.21.15.32:0/1016173219 shutdown_connections 2022-08-01T23:04:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- 172.21.15.32:0/1016173219 >> 172.21.15.32:0/1016173219 conn(0x7fee740b3610 msgr2=0x7fee740b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- 172.21.15.32:0/1016173219 shutdown_connections 2022-08-01T23:04:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- 172.21.15.32:0/1016173219 wait complete. 2022-08-01T23:04:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f161c060290 0x7f161c0a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 0x7f161c145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f161c065fb0 0x7f161c0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c146a90 con 0x7f161c065b00 2022-08-01T23:04:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0a50e0 con 0x7f161c065fb0 2022-08-01T23:04:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0a5380 con 0x7f161c060290 2022-08-01T23:04:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 0x7f161c145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 0x7f161c145670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47990/0 (socket says 172.21.15.32:47990) 2022-08-01T23:04:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f162198a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f161c060290 0x7f161c0a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 -- 172.21.15.32:0/1845388393 learned_addr learned my addr 172.21.15.32:0/1845388393 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f162218b700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f161c065fb0 0x7f161c0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 -- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f161c060290 msgr2=0x7f161c0a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f161c060290 0x7f161c0a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 -- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f161c065fb0 msgr2=0x7f161c0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f161c065fb0 0x7f161c0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 -- 172.21.15.32:0/1845388393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009530 con 0x7f161c065b00 2022-08-01T23:04:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1621189700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 0x7f161c145670 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f1618002f90 tx=0x7f1618006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 Processor -- start 2022-08-01T23:04:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- start start 2022-08-01T23:04:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1618013460 con 0x7f161c065b00 2022-08-01T23:04:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16180135c0 con 0x7f161c065b00 2022-08-01T23:04:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c0a5620 con 0x7f161c065b00 2022-08-01T23:04:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1618013920 con 0x7f161c065b00 2022-08-01T23:04:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c1527b0 con 0x7f161c065b00 2022-08-01T23:04:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fa9f74b0700 1 Processor -- start 2022-08-01T23:04:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee74063670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee74063b60 0x7fee740aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740af1a0 0x7fee74145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740ad280 con 0x7fee5c0037d0 2022-08-01T23:04:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee74145840 con 0x7fee74063b60 2022-08-01T23:04:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee7b92d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee74152d70 con 0x7fee740af1a0 2022-08-01T23:04:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fa9f74b0700 1 -- start start 2022-08-01T23:04:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee796c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee74063670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee796c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee74063670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47992/0 (socket says 172.21.15.32:47992) 2022-08-01T23:04:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee796c9700 1 -- 172.21.15.32:0/3494713239 learned_addr learned my addr 172.21.15.32:0/3494713239 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee78ec8700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee74063b60 0x7fee740aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7fee79eca700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740af1a0 0x7fee74145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.873+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c0574e0 con 0x7f161c065b00 2022-08-01T23:04:05.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee796c9700 1 -- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740af1a0 msgr2=0x7fee74145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee796c9700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740af1a0 0x7fee74145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee796c9700 1 -- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee74063b60 msgr2=0x7fee740aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee796c9700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee74063b60 0x7fee740aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee796c9700 1 -- 172.21.15.32:0/3494713239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74063fe0 con 0x7fee5c0037d0 2022-08-01T23:04:05.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee79eca700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740af1a0 0x7fee74145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:05.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee796c9700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee74063670 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7fee7000c000 tx=0x7fee70019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7001b820 con 0x7fee5c0037d0 2022-08-01T23:04:05.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70000e90 con 0x7fee5c0037d0 2022-08-01T23:04:05.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7001bb70 con 0x7fee5c0037d0 2022-08-01T23:04:05.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee740ad4b0 con 0x7fee5c0037d0 2022-08-01T23:04:05.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee740b1c00 con 0x7fee5c0037d0 2022-08-01T23:04:05.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f161801a060 con 0x7f161c065b00 2022-08-01T23:04:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f74b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9f00aa2b0 0x7fa9f00ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f74b0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa9f00b0a10 0x7fa9f0140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7f160effd700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15fc053a60 0x7f15fc055f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f16180461d0 con 0x7f161c065b00 2022-08-01T23:04:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7f162198a700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15fc053a60 0x7f15fc055f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f74b0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00b2c80 con 0x7fa9f00b0630 2022-08-01T23:04:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f74b0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00b2900 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f74b0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00b2eb0 con 0x7fa9f00b0a10 2022-08-01T23:04:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9f00aa2b0 0x7fa9f00ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f5a4d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa9f00b0a10 0x7fa9f0140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9f00aa2b0 0x7fa9f00ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48002/0 (socket says 172.21.15.32:48002) 2022-08-01T23:04:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 -- 172.21.15.32:0/2411665156 learned_addr learned my addr 172.21.15.32:0/2411665156 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 -- 172.21.15.32:0/2411665156 >> v2:172.21.15.55:3300/0 conn(0x7fa9f00b0a10 msgr2=0x7fa9f0140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 --2- 172.21.15.32:0/2411665156 >> v2:172.21.15.55:3300/0 conn(0x7fa9f00b0a10 0x7fa9f0140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 -- 172.21.15.32:0/2411665156 >> v1:172.21.15.55:6789/0 conn(0x7fa9f00b0630 legacy=0x7fa9f00adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 -- 172.21.15.32:0/2411665156 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00a9d00 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f5a4d700 1 --2- 172.21.15.32:0/2411665156 >> v2:172.21.15.55:3300/0 conn(0x7fa9f00b0a10 0x7fa9f0140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fa9f4a4b700 1 --2- 172.21.15.32:0/2411665156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ae5d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7fa9ec002f70 tx=0x7fa9ec006eb0).ready entity=mon.0 client_cookie=ffb4687ebba30a23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee740af7c0 con 0x7fee5c0037d0 2022-08-01T23:04:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee7001bcd0 con 0x7fee5c0037d0 2022-08-01T23:04:05.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7f162198a700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15fc053a60 0x7f15fc055f20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f161000c000 tx=0x7f1610008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee627fc700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee54053a70 0x7fee54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee7006d3d0 con 0x7fee5c0037d0 2022-08-01T23:04:05.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee78ec8700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee54053a70 0x7fee54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.877+0000 7fee78ec8700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee54053a70 0x7fee54055f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fee6400b000 tx=0x7fee64008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.881+0000 7fa9dffff700 1 -- 172.21.15.32:0/2411665156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec00d610 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.881+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1618042d00 con 0x7f161c065b00 2022-08-01T23:04:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9dffff700 1 -- 172.21.15.32:0/2411665156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec000e90 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9dffff700 1 -- 172.21.15.32:0/2411665156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec00d9a0 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee700441d0 con 0x7fee5c0037d0 2022-08-01T23:04:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2411665156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 msgr2=0x7fa9f00ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2411665156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ae5d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7fa9ec002f70 tx=0x7fa9ec006eb0).stop 2022-08-01T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2411665156 shutdown_connections 2022-08-01T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2411665156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ae5d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2411665156 >> v2:172.21.15.55:3300/0 conn(0x7fa9f00b0a10 0x7fa9f0140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.885+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2411665156 >> 172.21.15.32:0/2411665156 conn(0x7fa9f0060950 msgr2=0x7fa9f0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2411665156 shutdown_connections 2022-08-01T23:04:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2411665156 wait complete. 2022-08-01T23:04:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 Processor -- start 2022-08-01T23:04:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- start start 2022-08-01T23:04:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9f00b0630 0x7fa9f0141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9f00b0a10 0x7fa9f0142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0065810 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0065ab0 con 0x7fa9f00b0a10 2022-08-01T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0065d50 con 0x7fa9f00b0630 2022-08-01T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48008/0 (socket says 172.21.15.32:48008) 2022-08-01T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 -- 172.21.15.32:0/2138759415 learned_addr learned my addr 172.21.15.32:0/2138759415 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f4a4b700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9f00b0630 0x7fa9f0141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f5a4d700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9f00b0a10 0x7fa9f0142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 -- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9f00b0630 msgr2=0x7fa9f0141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9f00b0630 0x7fa9f0141b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 -- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9f00b0a10 msgr2=0x7fa9f0142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9f00b0a10 0x7fa9f0142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 -- 172.21.15.32:0/2138759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec00b1e0 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f524c700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ab200 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fa9f00b0240 tx=0x7fa9e80014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e8010510 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8010670 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f0065ff0 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e80107e0 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.889+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f0142510 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.893+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f0066180 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.893+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9e8010070 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.893+0000 7fa9de7fc700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053a70 0x7fa9d0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.893+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa9e806bce0 con 0x7fa9f00aa2b0 2022-08-01T23:04:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.893+0000 7fa9f4a4b700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053a70 0x7fa9d0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:05.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.893+0000 7fa9f4a4b700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053a70 0x7fa9d0055f30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fa9ec01a000 tx=0x7fa9ec015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:05.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:05.897+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9e8042aa0 con 0x7fa9f00aa2b0 2022-08-01T23:04:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.049+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f161c0b2550 con 0x7f161c065b00 2022-08-01T23:04:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.053+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fee740b2360 con 0x7fee5c0037d0 2022-08-01T23:04:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.065+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fa9f00574e0 con 0x7fa9f00aa2b0 2022-08-01T23:04:06.445 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:04:06.441+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:04:06.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.453+0000 7f160effd700 1 -- 172.21.15.32:0/1845388393 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7674 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7f1618018030 con 0x7f161c065b00 2022-08-01T23:04:06.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.453+0000 7fee627fc700 1 -- 172.21.15.32:0/3494713239 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6653 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7fee7002a070 con 0x7fee5c0037d0 2022-08-01T23:04:06.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.453+0000 7fa9de7fc700 1 -- 172.21.15.32:0/2138759415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7665 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7fa9e80460c0 con 0x7fa9f00aa2b0 2022-08-01T23:04:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053a70 msgr2=0x7fa9d0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053a70 0x7fa9d0055f30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fa9ec01a000 tx=0x7fa9ec015000).stop 2022-08-01T23:04:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 msgr2=0x7fa9f00ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ab200 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fa9f00b0240 tx=0x7fa9e80014a0).stop 2022-08-01T23:04:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15fc053a60 msgr2=0x7f15fc055f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7f1623bee700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15fc053a60 0x7f15fc055f20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f161000c000 tx=0x7f1610008040).stop 2022-08-01T23:04:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 msgr2=0x7f161c145670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.457+0000 7f1623bee700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 0x7f161c145670 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f1618002f90 tx=0x7f1618006df0).stop 2022-08-01T23:04:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee54053a70 msgr2=0x7fee54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee54053a70 0x7fee54055f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fee6400b000 tx=0x7fee64008040).stop 2022-08-01T23:04:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 msgr2=0x7fee74063670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee74063670 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7fee7000c000 tx=0x7fee70019000).stop 2022-08-01T23:04:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 shutdown_connections 2022-08-01T23:04:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa9d0053a70 0x7fa9d0055f30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9f00b0a10 0x7fa9f0142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa9f00b0630 0x7fa9f0141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 shutdown_connections 2022-08-01T23:04:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 --2- 172.21.15.32:0/2138759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f00aa2b0 0x7fa9f00ab200 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 >> 172.21.15.32:0/2138759415 conn(0x7fa9f0060950 msgr2=0x7fa9f00b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:06.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15fc053a60 0x7f15fc055f20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f161c065fb0 0x7f161c0adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161c065b00 0x7f161c145670 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 --2- 172.21.15.32:0/1845388393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f161c060290 0x7f161c0a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 >> 172.21.15.32:0/1845388393 conn(0x7f161c05d0c0 msgr2=0x7f161c061700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:06.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 shutdown_connections 2022-08-01T23:04:06.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 shutdown_connections 2022-08-01T23:04:06.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee54053a70 0x7fee54055f30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee740af1a0 0x7fee74145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee74063b60 0x7fee740aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fa9f74b0700 1 -- 172.21.15.32:0/2138759415 wait complete. 2022-08-01T23:04:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 --2- 172.21.15.32:0/3494713239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee5c0037d0 0x7fee74063670 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 >> 172.21.15.32:0/3494713239 conn(0x7fee740b3610 msgr2=0x7fee741402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 shutdown_connections 2022-08-01T23:04:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 shutdown_connections 2022-08-01T23:04:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7fee7b92d700 1 -- 172.21.15.32:0/3494713239 wait complete. 2022-08-01T23:04:06.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.461+0000 7f1623bee700 1 -- 172.21.15.32:0/1845388393 wait complete. 2022-08-01T23:04:06.477 INFO:teuthology.orchestra.run.smithi032.stderr:failed mds gid 7665 2022-08-01T23:04:06.477 INFO:teuthology.orchestra.run.smithi032.stderr:failed mds gid 7674 2022-08-01T23:04:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:failed mds gid 6653 2022-08-01T23:04:06.480 DEBUG:teuthology.parallel:result is '' 2022-08-01T23:04:06.485 DEBUG:teuthology.parallel:result is '' 2022-08-01T23:04:06.486 DEBUG:teuthology.parallel:result is '' 2022-08-01T23:04:06.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:04:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 Processor -- start 2022-08-01T23:04:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 -- start start 2022-08-01T23:04:06.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f42340b2710 0x7f423414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:06.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42340a8c50 0x7f423414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:06.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42340a7250 con 0x7f42340b01a0 2022-08-01T23:04:06.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42340a7480 con 0x7f42340a8c50 2022-08-01T23:04:06.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.589+0000 7f423b0dd700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42340a7720 con 0x7f42340b2710 2022-08-01T23:04:06.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42340a8c50 0x7f423414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:06.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42340a8c50 0x7f423414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48020/0 (socket says 172.21.15.32:48020) 2022-08-01T23:04:06.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 -- 172.21.15.32:0/3113738214 learned_addr learned my addr 172.21.15.32:0/3113738214 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:06.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a0db700 1 --2- 172.21.15.32:0/3113738214 >> v2:172.21.15.55:3300/0 conn(0x7f42340b2710 0x7f423414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:06.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 -- 172.21.15.32:0/3113738214 >> v2:172.21.15.55:3300/0 conn(0x7f42340b2710 msgr2=0x7f423414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:06.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 --2- 172.21.15.32:0/3113738214 >> v2:172.21.15.55:3300/0 conn(0x7f42340b2710 0x7f423414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 -- 172.21.15.32:0/3113738214 >> v1:172.21.15.55:6789/0 conn(0x7f42340b01a0 legacy=0x7f42340a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 -- 172.21.15.32:0/3113738214 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340a79c0 con 0x7f42340a8c50 2022-08-01T23:04:06.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423a8dc700 1 --2- 172.21.15.32:0/3113738214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340a8c50 0x7f423414b4c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f422c002f70 tx=0x7f422c006eb0).ready entity=mon.0 client_cookie=3dc62918c2506ee4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:06.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f42390d9700 1 -- 172.21.15.32:0/3113738214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422c00d610 con 0x7f42340a8c50 2022-08-01T23:04:06.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f42390d9700 1 -- 172.21.15.32:0/3113738214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c000e90 con 0x7f42340a8c50 2022-08-01T23:04:06.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f42390d9700 1 -- 172.21.15.32:0/3113738214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422c00d9a0 con 0x7f42340a8c50 2022-08-01T23:04:06.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 -- 172.21.15.32:0/3113738214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340a8c50 msgr2=0x7f423414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 --2- 172.21.15.32:0/3113738214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340a8c50 0x7f423414b4c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f422c002f70 tx=0x7f422c006eb0).stop 2022-08-01T23:04:06.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 -- 172.21.15.32:0/3113738214 shutdown_connections 2022-08-01T23:04:06.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 --2- 172.21.15.32:0/3113738214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340a8c50 0x7f423414b4c0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 --2- 172.21.15.32:0/3113738214 >> v2:172.21.15.55:3300/0 conn(0x7f42340b2710 0x7f423414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 -- 172.21.15.32:0/3113738214 >> 172.21.15.32:0/3113738214 conn(0x7f4234060310 msgr2=0x7f42340b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:06.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 -- 172.21.15.32:0/3113738214 shutdown_connections 2022-08-01T23:04:06.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 -- 172.21.15.32:0/3113738214 wait complete. 2022-08-01T23:04:06.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 Processor -- start 2022-08-01T23:04:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.593+0000 7f423b0dd700 1 -- start start 2022-08-01T23:04:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42340a8c50 0x7f4234149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 0x7f4234066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42340b2710 0x7f4234066b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42340aacf0 con 0x7f42340b01a0 2022-08-01T23:04:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4234067020 con 0x7f42340a8c50 2022-08-01T23:04:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42340647a0 con 0x7f42340b2710 2022-08-01T23:04:06.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 0x7f4234066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:06.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 0x7f4234066640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48024/0 (socket says 172.21.15.32:48024) 2022-08-01T23:04:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 -- 172.21.15.32:0/2332977900 learned_addr learned my addr 172.21.15.32:0/2332977900 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423a0db700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42340a8c50 0x7f4234149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423a8dc700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42340b2710 0x7f4234066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 -- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42340b2710 msgr2=0x7f4234066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42340b2710 0x7f4234066b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 -- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42340a8c50 msgr2=0x7f4234149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42340a8c50 0x7f4234149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 -- 172.21.15.32:0/2332977900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c00b1e0 con 0x7f42340b01a0 2022-08-01T23:04:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423a0db700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42340a8c50 0x7f4234149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42398da700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 0x7f4234066640 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f423406b950 tx=0x7f4224007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422400e4a0 con 0x7f42340b01a0 2022-08-01T23:04:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422400e600 con 0x7f42340b01a0 2022-08-01T23:04:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4234064a40 con 0x7f42340b01a0 2022-08-01T23:04:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422400e960 con 0x7f42340b01a0 2022-08-01T23:04:06.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4234063190 con 0x7f42340b01a0 2022-08-01T23:04:06.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.597+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42340574e0 con 0x7f42340b01a0 2022-08-01T23:04:06.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.601+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4224019030 con 0x7f42340b01a0 2022-08-01T23:04:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.601+0000 7f42337fe700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42140539e0 0x7f4214055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.601+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f422406b8b0 con 0x7f42340b01a0 2022-08-01T23:04:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.601+0000 7f423a0db700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42140539e0 0x7f4214055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.601+0000 7f423a0db700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42140539e0 0x7f4214055ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f422800c000 tx=0x7f4228008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.609+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4224042520 con 0x7f42340b01a0 2022-08-01T23:04:06.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:06.773+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4234069a70 con 0x7f42340b01a0 2022-08-01T23:04:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.457+0000 7f42337fe700 1 -- 172.21.15.32:0/2332977900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4224045b40 con 0x7f42340b01a0 2022-08-01T23:04:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.461+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42140539e0 msgr2=0x7f4214055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.461+0000 7f423b0dd700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42140539e0 0x7f4214055ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f422800c000 tx=0x7f4228008040).stop 2022-08-01T23:04:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.461+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 msgr2=0x7f4234066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.461+0000 7f423b0dd700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 0x7f4234066640 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f423406b950 tx=0x7f4224007f70).stop 2022-08-01T23:04:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 shutdown_connections 2022-08-01T23:04:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f42140539e0 0x7f4214055ea0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f42340b2710 0x7f4234066b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42340b01a0 0x7f4234066640 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 --2- 172.21.15.32:0/2332977900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42340a8c50 0x7f4234149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 >> 172.21.15.32:0/2332977900 conn(0x7f4234060310 msgr2=0x7f423414c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 shutdown_connections 2022-08-01T23:04:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.465+0000 7f423b0dd700 1 -- 172.21.15.32:0/2332977900 wait complete. 2022-08-01T23:04:07.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:04:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d5fc8700 1 Processor -- start 2022-08-01T23:04:07.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d5fc8700 1 -- start start 2022-08-01T23:04:07.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d5fc8700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51d01489f0 0x7f51d014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:07.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d5fc8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d0067030 con 0x7f51d00b31d0 2022-08-01T23:04:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d5fc8700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d00b35b0 con 0x7f51d00a9da0 2022-08-01T23:04:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d5fc8700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d0064a70 con 0x7f51d01489f0 2022-08-01T23:04:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d4fc6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51d00b31d0 0x7f51d0066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35572/0 (socket says 172.21.15.32:35572) 2022-08-01T23:04:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d4fc6700 1 -- 172.21.15.32:0/3282216363 learned_addr learned my addr 172.21.15.32:0/3282216363 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d57c7700 1 --2- 172.21.15.32:0/3282216363 >> v2:172.21.15.55:3300/0 conn(0x7f51d01489f0 0x7f51d014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51cf7fe700 1 -- 172.21.15.32:0/3282216363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415166444 0 0) 0x7f51d0067030 con 0x7f51d00b31d0 2022-08-01T23:04:07.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51cf7fe700 1 -- 172.21.15.32:0/3282216363 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0003260 con 0x7f51d00b31d0 2022-08-01T23:04:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d57c7700 1 -- 172.21.15.32:0/3282216363 >> v1:172.21.15.55:6789/0 conn(0x7f51d00a9da0 legacy=0x7f51d0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d57c7700 1 -- 172.21.15.32:0/3282216363 >> v1:172.21.15.32:6789/0 conn(0x7f51d00b31d0 legacy=0x7f51d0066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d57c7700 1 -- 172.21.15.32:0/3282216363 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0066af0 con 0x7f51d01489f0 2022-08-01T23:04:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.593+0000 7f51d57c7700 1 --2- 172.21.15.32:0/3282216363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d01489f0 0x7f51d014ade0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f51c8002f70 tx=0x7f51c8006df0).ready entity=mon.1 client_cookie=4d854b8d9e329584 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51cf7fe700 1 -- 172.21.15.32:0/3282216363 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c800d610 con 0x7f51d01489f0 2022-08-01T23:04:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51cf7fe700 1 -- 172.21.15.32:0/3282216363 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c8000e90 con 0x7f51d01489f0 2022-08-01T23:04:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51cf7fe700 1 -- 172.21.15.32:0/3282216363 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c800d960 con 0x7f51d01489f0 2022-08-01T23:04:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 -- 172.21.15.32:0/3282216363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d01489f0 msgr2=0x7f51d014ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/3282216363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d01489f0 0x7f51d014ade0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f51c8002f70 tx=0x7f51c8006df0).stop 2022-08-01T23:04:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 -- 172.21.15.32:0/3282216363 shutdown_connections 2022-08-01T23:04:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/3282216363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d01489f0 0x7f51d014ade0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 -- 172.21.15.32:0/3282216363 >> 172.21.15.32:0/3282216363 conn(0x7f51d0063870 msgr2=0x7f51d0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 -- 172.21.15.32:0/3282216363 shutdown_connections 2022-08-01T23:04:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 -- 172.21.15.32:0/3282216363 wait complete. 2022-08-01T23:04:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 Processor -- start 2022-08-01T23:04:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.597+0000 7f51d5fc8700 1 -- start start 2022-08-01T23:04:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d00a9da0 0x7f51d00ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51d00b31d0 0x7f51d0147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 0x7f51d00aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0136bd0 con 0x7f51d01489f0 2022-08-01T23:04:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0136e70 con 0x7f51d00a9da0 2022-08-01T23:04:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d013ba80 con 0x7f51d00b31d0 2022-08-01T23:04:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 0x7f51d00aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 0x7f51d00aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48040/0 (socket says 172.21.15.32:48040) 2022-08-01T23:04:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d4fc6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d00a9da0 0x7f51d00ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 -- 172.21.15.32:0/1932629256 learned_addr learned my addr 172.21.15.32:0/1932629256 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51cffff700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51d00b31d0 0x7f51d0147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 -- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51d00b31d0 msgr2=0x7f51d0147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51d00b31d0 0x7f51d0147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 -- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d00a9da0 msgr2=0x7f51d00ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d00a9da0 0x7f51d00ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 -- 172.21.15.32:0/1932629256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c800b180 con 0x7f51d01489f0 2022-08-01T23:04:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d57c7700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 0x7f51d00aaf10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f51c8002f70 tx=0x7f51c8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c8016070 con 0x7f51d01489f0 2022-08-01T23:04:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800d290 con 0x7f51d01489f0 2022-08-01T23:04:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d00af680 con 0x7f51d01489f0 2022-08-01T23:04:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c8013de0 con 0x7f51d01489f0 2022-08-01T23:04:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.601+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d0148320 con 0x7f51d01489f0 2022-08-01T23:04:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.605+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d00ac670 con 0x7f51d01489f0 2022-08-01T23:04:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.605+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f51c8025030 con 0x7f51d01489f0 2022-08-01T23:04:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.605+0000 7f51cdffb700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51b4053b00 0x7f51b4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.605+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f51c806cf50 con 0x7f51d01489f0 2022-08-01T23:04:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.605+0000 7f51d4fc6700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51b4053b00 0x7f51b4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.605+0000 7f51d4fc6700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51b4053b00 0x7f51b4055fc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f51c000d000 tx=0x7f51c0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.609+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51d00ac670 con 0x7f51d01489f0 2022-08-01T23:04:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:07.737+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51d00ac670 con 0x7f51d01489f0 2022-08-01T23:04:08.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.469+0000 7f51cdffb700 1 -- 172.21.15.32:0/1932629256 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) v1 ==== 185+0+0 (secure 0 0 0) 0x7f51d00ac670 con 0x7f51d01489f0 2022-08-01T23:04:08.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51b4053b00 msgr2=0x7f51b4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:08.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51b4053b00 0x7f51b4055fc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f51c000d000 tx=0x7f51c0009040).stop 2022-08-01T23:04:08.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 msgr2=0x7f51d00aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:08.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 0x7f51d00aaf10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f51c8002f70 tx=0x7f51c8014000).stop 2022-08-01T23:04:08.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 shutdown_connections 2022-08-01T23:04:08.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f51b4053b00 0x7f51b4055fc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d01489f0 0x7f51d00aaf10 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51d00b31d0 0x7f51d0147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 --2- 172.21.15.32:0/1932629256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51d00a9da0 0x7f51d00ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 >> 172.21.15.32:0/1932629256 conn(0x7f51d0063870 msgr2=0x7f51d00b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:08.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 shutdown_connections 2022-08-01T23:04:08.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.473+0000 7f51d5fc8700 1 -- 172.21.15.32:0/1932629256 wait complete. 2022-08-01T23:04:08.483 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:04:08.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:04:08.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 Processor -- start 2022-08-01T23:04:08.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- start start 2022-08-01T23:04:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f043c0b38a0 0x7f043c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:08.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:08.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c143ab0 con 0x7f043c0b0740 2022-08-01T23:04:08.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c143d50 con 0x7f043c0b38a0 2022-08-01T23:04:08.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c143ff0 con 0x7f043c0a9da0 2022-08-01T23:04:08.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f043c0b38a0 0x7f043c0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:08.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f043c0b38a0 0x7f043c0b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48046/0 (socket says 172.21.15.32:48046) 2022-08-01T23:04:08.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 -- 172.21.15.32:0/3262919812 learned_addr learned my addr 172.21.15.32:0/3262919812 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:08.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0441b50700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:08.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 -- 172.21.15.32:0/3262919812 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 msgr2=0x7f043c0b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:08.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c0b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 -- 172.21.15.32:0/3262919812 >> v1:172.21.15.55:6789/0 conn(0x7f043c0b0740 legacy=0x7f043c13d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:08.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 -- 172.21.15.32:0/3262919812 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c144290 con 0x7f043c0b38a0 2022-08-01T23:04:08.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0441b50700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c0b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:08.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0440b4e700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b38a0 0x7f043c0b0b20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f042c002f70 tx=0x7f042c006f40).ready entity=mon.0 client_cookie=73dba4bae2cc567 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:08.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0433fff700 1 -- 172.21.15.32:0/3262919812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c00dc70 con 0x7f043c0b38a0 2022-08-01T23:04:08.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0433fff700 1 -- 172.21.15.32:0/3262919812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c000e90 con 0x7f043c0b38a0 2022-08-01T23:04:08.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f0433fff700 1 -- 172.21.15.32:0/3262919812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c00be90 con 0x7f043c0b38a0 2022-08-01T23:04:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- 172.21.15.32:0/3262919812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b38a0 msgr2=0x7f043c0b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b38a0 0x7f043c0b0b20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f042c002f70 tx=0x7f042c006f40).stop 2022-08-01T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- 172.21.15.32:0/3262919812 shutdown_connections 2022-08-01T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b38a0 0x7f043c0b0b20 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 --2- 172.21.15.32:0/3262919812 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c0b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- 172.21.15.32:0/3262919812 >> 172.21.15.32:0/3262919812 conn(0x7f043c060950 msgr2=0x7f043c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.609+0000 7f04435b3700 1 -- 172.21.15.32:0/3262919812 shutdown_connections 2022-08-01T23:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- 172.21.15.32:0/3262919812 wait complete. 2022-08-01T23:04:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 Processor -- start 2022-08-01T23:04:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- start start 2022-08-01T23:04:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c13d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 0x7f043c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f043c0b38a0 0x7f043c09ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c153650 con 0x7f043c0b0740 2022-08-01T23:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0aaf30 con 0x7f043c0b38a0 2022-08-01T23:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0aa690 con 0x7f043c0a9da0 2022-08-01T23:04:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 0x7f043c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 0x7f043c153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48050/0 (socket says 172.21.15.32:48050) 2022-08-01T23:04:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 -- 172.21.15.32:0/1140654629 learned_addr learned my addr 172.21.15.32:0/1140654629 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0441b50700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f043c0b38a0 0x7f043c09ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f044134f700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c13d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 -- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 msgr2=0x7f043c13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c13d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 -- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f043c0b38a0 msgr2=0x7f043c09ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f043c0b38a0 0x7f043c09ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 -- 172.21.15.32:0/1140654629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c00b460 con 0x7f043c0b0740 2022-08-01T23:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0441b50700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f043c0b38a0 0x7f043c09ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f0440b4e700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 0x7f043c153160 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f042c002f70 tx=0x7f043c0b0660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c018410 con 0x7f043c0b0740 2022-08-01T23:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c018570 con 0x7f043c0b0740 2022-08-01T23:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c0aa8f0 con 0x7f043c0b0740 2022-08-01T23:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c0188d0 con 0x7f043c0b0740 2022-08-01T23:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.613+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c0abcb0 con 0x7f043c0b0740 2022-08-01T23:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.617+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c13ddf0 con 0x7f043c0b0740 2022-08-01T23:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.617+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f042c029070 con 0x7f043c0b0740 2022-08-01T23:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.617+0000 7f04327fc700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0424053980 0x7f0424055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.617+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f042c06d960 con 0x7f043c0b0740 2022-08-01T23:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.617+0000 7f044134f700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0424053980 0x7f0424055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.621+0000 7f044134f700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0424053980 0x7f0424055e40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f043800b000 tx=0x7f0438009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:08.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.625+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f042c044770 con 0x7f043c0b0740 2022-08-01T23:04:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:08.753+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f043c0574e0 con 0x7f043c0b0740 2022-08-01T23:04:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.481+0000 7f04327fc700 1 -- 172.21.15.32:0/1140654629 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v123) v1 ==== 133+0+0 (secure 0 0 0) 0x7f042c047d90 con 0x7f043c0b0740 2022-08-01T23:04:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0424053980 msgr2=0x7f0424055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0424053980 0x7f0424055e40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f043800b000 tx=0x7f0438009040).stop 2022-08-01T23:04:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 msgr2=0x7f043c153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:09.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 0x7f043c153160 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f042c002f70 tx=0x7f043c0b0660).stop 2022-08-01T23:04:09.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 shutdown_connections 2022-08-01T23:04:09.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0424053980 0x7f0424055e40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f043c0b38a0 0x7f043c09ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043c0b0740 0x7f043c153160 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 --2- 172.21.15.32:0/1140654629 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f043c0a9da0 0x7f043c13d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 >> 172.21.15.32:0/1140654629 conn(0x7f043c060950 msgr2=0x7f043c0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 shutdown_connections 2022-08-01T23:04:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:09.485+0000 7f04435b3700 1 -- 172.21.15.32:0/1140654629 wait complete. 2022-08-01T23:04:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:04:09.508 DEBUG:teuthology.orchestra.run.smithi130:> sudo rados -p cephfs_metadata put key /etc/fstab 2022-08-01T23:04:09.554 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 Processor -- start 2022-08-01T23:04:09.554 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 -- start start 2022-08-01T23:04:09.554 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c3ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.555 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.555 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c46460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.555 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa6c34b10 con 0x55caa6c31400 2022-08-01T23:04:09.555 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa6c3bb80 con 0x55caa6c372b0 2022-08-01T23:04:09.556 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.547+0000 7f7b9d37bb80 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa6c46950 con 0x55caa6c31da0 2022-08-01T23:04:09.556 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c46460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.556 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9afb7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.556 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c46460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.130:37878/0 (socket says 172.21.15.130:37878) 2022-08-01T23:04:09.557 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 -- 172.21.15.130:0/1520673369 learned_addr learned my addr 172.21.15.130:0/1520673369 (peer_addr_for_me v2:172.21.15.130:0/0) 2022-08-01T23:04:09.557 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9afb7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c3b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.130:34786/0 (socket says 172.21.15.130:34786) 2022-08-01T23:04:09.557 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9b7b8700 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c3ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.557 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 -- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 msgr2=0x55caa6c3b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:09.557 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c3b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.558 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 -- 172.21.15.130:0/1520673369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 msgr2=0x55caa6c3ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:09.558 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c3ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.558 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 -- 172.21.15.130:0/1520673369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84005d90 con 0x55caa6c372b0 2022-08-01T23:04:09.558 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9afb7700 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:09.559 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9b7b8700 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c3ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:09.559 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c46460 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x55caa6c3e280 tx=0x7f7b9400c040).ready entity=mon.1 client_cookie=1c74a697eaf84ec7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:09.559 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b997b4700 1 -- 172.21.15.130:0/1520673369 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9400f040 con 0x55caa6c372b0 2022-08-01T23:04:09.559 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b997b4700 1 -- 172.21.15.130:0/1520673369 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b940088f0 con 0x55caa6c372b0 2022-08-01T23:04:09.560 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b997b4700 1 -- 172.21.15.130:0/1520673369 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b94008c50 con 0x55caa6c372b0 2022-08-01T23:04:09.565 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 msgr2=0x55caa6c46460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:09.565 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c46460 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x55caa6c3e280 tx=0x7f7b9400c040).stop 2022-08-01T23:04:09.565 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/1520673369 shutdown_connections 2022-08-01T23:04:09.565 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c46460 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.566 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.566 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/1520673369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c3ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.566 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/1520673369 >> 172.21.15.130:0/1520673369 conn(0x55caa6ba4640 msgr2=0x55caa6c34d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:09.566 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/1520673369 shutdown_connections 2022-08-01T23:04:09.567 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/1520673369 wait complete. 2022-08-01T23:04:09.567 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 Processor -- start 2022-08-01T23:04:09.567 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- start start 2022-08-01T23:04:09.567 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c46080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.567 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.568 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.568 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa6c35cf0 con 0x55caa6c31400 2022-08-01T23:04:09.568 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa6c41bc0 con 0x55caa6c372b0 2022-08-01T23:04:09.569 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9d37bb80 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa6c41d80 con 0x55caa6c31da0 2022-08-01T23:04:09.569 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9b7b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.570 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9afb7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.570 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.571 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9b7b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c46080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.130:37702/0 (socket says 172.21.15.130:37702) 2022-08-01T23:04:09.571 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9afb7700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.130:34792/0 (socket says 172.21.15.130:34792) 2022-08-01T23:04:09.571 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9b7b8700 1 -- 172.21.15.130:0/2964812556 learned_addr learned my addr 172.21.15.130:0/2964812556 (peer_addr_for_me v2:172.21.15.130:0/0) 2022-08-01T23:04:09.571 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.551+0000 7f7b9bfb9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.130:37884/0 (socket says 172.21.15.130:37884) 2022-08-01T23:04:09.572 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9afb7700 1 -- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 msgr2=0x55caa6c416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:09.572 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9afb7700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55caa6c372b0 0x55caa6c416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.572 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9afb7700 1 -- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 msgr2=0x55caa6c46080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:09.572 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9afb7700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caa6c31400 0x55caa6c46080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:09.573 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9afb7700 1 -- 172.21.15.130:0/2964812556 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009630 con 0x55caa6c31da0 2022-08-01T23:04:09.573 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9afb7700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c411e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x55caa6c427e0 tx=0x7f7b8c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:09.573 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b98fb3700 1 -- 172.21.15.130:0/2964812556 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c005ea0 con 0x55caa6c31da0 2022-08-01T23:04:09.573 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b98fb3700 1 -- 172.21.15.130:0/2964812556 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00de10 con 0x55caa6c31da0 2022-08-01T23:04:09.573 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55caa6b3fef0 con 0x55caa6c31da0 2022-08-01T23:04:09.574 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b98fb3700 1 -- 172.21.15.130:0/2964812556 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c01d640 con 0x55caa6c31da0 2022-08-01T23:04:09.574 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b94009490 con 0x55caa6c31da0 2022-08-01T23:04:09.574 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b98fb3700 1 -- 172.21.15.130:0/2964812556 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b8c016070 con 0x55caa6c31da0 2022-08-01T23:04:09.574 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.555+0000 7f7b98fb3700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b7c0525f0 0x7f7b7c054ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.575 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b98fb3700 1 -- 172.21.15.130:0/2964812556 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f7b8c06c450 con 0x55caa6c31da0 2022-08-01T23:04:09.575 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b9b7b8700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b7c0525f0 0x7f7b7c054ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.575 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x55caa6c4ebc0 0x55caa6c50fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:09.575 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 --> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x55caa6c51580 con 0x55caa6c4ebc0 2022-08-01T23:04:09.576 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b9bfb9700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x55caa6c4ebc0 0x55caa6c50fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:09.576 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b9b7b8700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b7c0525f0 0x7f7b7c054ab0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f7b8400a000 tx=0x7f7b8400c040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:09.576 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:09.559+0000 7f7b9bfb9700 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x55caa6c4ebc0 0x55caa6c50fb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:10.517 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.514+0000 7f7b98fb3700 1 -- 172.21.15.130:0/2964812556 <== osd.0 v2:172.21.15.32:6824/23003 1 ==== osd_map(124..124 src has 1..124) v4 ==== 320+0+0 (crc 0 0 0) 0x7f7b94008b80 con 0x55caa6c4ebc0 2022-08-01T23:04:10.521 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.514+0000 7f7b9bfb9700 1 -- 172.21.15.130:0/2964812556 <== osd.0 v2:172.21.15.32:6824/23003 2 ==== osd_op_reply(1 key [writefull 0~617] v124'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f7b94008d40 con 0x55caa6c4ebc0 2022-08-01T23:04:10.522 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x55caa6c4ebc0 msgr2=0x55caa6c50fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.522 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x55caa6c4ebc0 0x55caa6c50fb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.522 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b7c0525f0 msgr2=0x7f7b7c054ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.522 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b7c0525f0 0x7f7b7c054ab0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f7b8400a000 tx=0x7f7b8400c040).stop 2022-08-01T23:04:10.523 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 msgr2=0x55caa6c411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.523 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 --2- 172.21.15.130:0/2964812556 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x55caa6c31da0 0x55caa6c411e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x55caa6c427e0 tx=0x7f7b8c014000).stop 2022-08-01T23:04:10.523 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9bfb9700 1 -- 172.21.15.130:0/2964812556 reap_dead start 2022-08-01T23:04:10.523 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 shutdown_connections 2022-08-01T23:04:10.524 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 >> 172.21.15.130:0/2964812556 conn(0x55caa6ba4640 msgr2=0x55caa6c3af80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:10.524 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 shutdown_connections 2022-08-01T23:04:10.524 INFO:teuthology.orchestra.run.smithi130.stderr:2022-08-01T23:04:10.518+0000 7f7b9d37bb80 1 -- 172.21.15.130:0/2964812556 wait complete. 2022-08-01T23:04:10.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-08-01T23:04:10.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 Processor -- start 2022-08-01T23:04:10.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 -- start start 2022-08-01T23:04:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08ac0b3f90 0x7f08ac144610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac061bd0 con 0x7f08ac144b00 2022-08-01T23:04:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0a9fd0 con 0x7f08ac0b3f90 2022-08-01T23:04:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0a88e0 con 0x7f08ac0617f0 2022-08-01T23:04:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08ab7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08ac0b3f90 0x7f08ac144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08ab7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac0b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:59796/0 (socket says 172.21.15.32:59796) 2022-08-01T23:04:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08abfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f08ac144b00 0x7f08ac14aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:56510/0 (socket says 172.21.15.32:56510) 2022-08-01T23:04:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08ab7fe700 1 -- 172.21.15.32:0/746600175 learned_addr learned my addr 172.21.15.32:0/746600175 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08ac0b3f90 0x7f08ac144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57846/0 (socket says 172.21.15.32:57846) 2022-08-01T23:04:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aa7fc700 1 -- 172.21.15.32:0/746600175 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248313305 0 0) 0x7f08ac061bd0 con 0x7f08ac144b00 2022-08-01T23:04:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aa7fc700 1 -- 172.21.15.32:0/746600175 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0898003260 con 0x7f08ac144b00 2022-08-01T23:04:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 -- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 msgr2=0x7f08ac0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 --2- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 -- 172.21.15.32:0/746600175 >> v1:172.21.15.55:6789/0 conn(0x7f08ac144b00 legacy=0x7f08ac14aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 -- 172.21.15.32:0/746600175 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0a8b80 con 0x7f08ac0b3f90 2022-08-01T23:04:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08ab7fe700 1 --2- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aaffd700 1 --2- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac144610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f08a0002f70 tx=0x7f08a0006df0).ready entity=mon.1 client_cookie=94ccd4c8541a7fa2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aa7fc700 1 -- 172.21.15.32:0/746600175 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a000d4a0 con 0x7f08ac0b3f90 2022-08-01T23:04:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aa7fc700 1 -- 172.21.15.32:0/746600175 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0000e90 con 0x7f08ac0b3f90 2022-08-01T23:04:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08aa7fc700 1 -- 172.21.15.32:0/746600175 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a000d7f0 con 0x7f08ac0b3f90 2022-08-01T23:04:10.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 -- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 msgr2=0x7f08ac144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.625+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac144610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f08a0002f70 tx=0x7f08a0006df0).stop 2022-08-01T23:04:10.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- 172.21.15.32:0/746600175 shutdown_connections 2022-08-01T23:04:10.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac144610 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/746600175 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- 172.21.15.32:0/746600175 >> 172.21.15.32:0/746600175 conn(0x7f08ac066f70 msgr2=0x7f08ac067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- 172.21.15.32:0/746600175 shutdown_connections 2022-08-01T23:04:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- 172.21.15.32:0/746600175 wait complete. 2022-08-01T23:04:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 Processor -- start 2022-08-01T23:04:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- start start 2022-08-01T23:04:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac065a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac144b00 0x7f08ac063790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0663a0 con 0x7f08ac144b00 2022-08-01T23:04:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac062e40 con 0x7f08ac0b3f90 2022-08-01T23:04:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac066100 con 0x7f08ac0617f0 2022-08-01T23:04:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac144b00 0x7f08ac063790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac065a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac144b00 0x7f08ac063790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48068/0 (socket says 172.21.15.32:48068) 2022-08-01T23:04:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08abfff700 1 -- 172.21.15.32:0/580964223 learned_addr learned my addr 172.21.15.32:0/580964223 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 -- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 msgr2=0x7f08ac1481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac1481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 -- 172.21.15.32:0/580964223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac144b00 msgr2=0x7f08ac063790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac144b00 0x7f08ac063790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 -- 172.21.15.32:0/580964223 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c00b1e0 con 0x7f08ac0617f0 2022-08-01T23:04:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08ab7fe700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac065a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f08ac156af0 tx=0x7f089c006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f089c012600 con 0x7f08ac0617f0 2022-08-01T23:04:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c012760 con 0x7f08ac0617f0 2022-08-01T23:04:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a000b180 con 0x7f08ac0617f0 2022-08-01T23:04:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f089c012ac0 con 0x7f08ac0617f0 2022-08-01T23:04:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.629+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac1486a0 con 0x7f08ac0617f0 2022-08-01T23:04:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.633+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac065f40 con 0x7f08ac0617f0 2022-08-01T23:04:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.633+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f089c012c20 con 0x7f08ac0617f0 2022-08-01T23:04:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.633+0000 7f08a8ff9700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f088c053a70 0x7f088c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.633+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f089c06c5e0 con 0x7f08ac0617f0 2022-08-01T23:04:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.633+0000 7f08aaffd700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f088c053a70 0x7f088c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.637+0000 7f08aaffd700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f088c053a70 0x7f088c055f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f08a001a000 tx=0x7f08a0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.645+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f089c043210 con 0x7f08ac0617f0 2022-08-01T23:04:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.813+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f08ac0b5f00 con 0x7f08ac0617f0 2022-08-01T23:04:10.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08a8ff9700 1 -- 172.21.15.32:0/580964223 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1906 (secure 0 0 0) 0x7f089c046830 con 0x7f08ac0617f0 2022-08-01T23:04:10.821 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:10.822 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:10.822 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:10.822 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2022-08-01T23:04:10.823 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764491874304, 2022-08-01T23:04:10.823 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 12304384, 2022-08-01T23:04:10.823 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8602238976, 2022-08-01T23:04:10.823 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011127026751637459, 2022-08-01T23:04:10.824 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2022-08-01T23:04:10.824 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2022-08-01T23:04:10.824 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2022-08-01T23:04:10.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:10.824 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2022-08-01T23:04:10.825 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2022-08-01T23:04:10.825 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2022-08-01T23:04:10.825 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764491874304, 2022-08-01T23:04:10.825 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 12304384, 2022-08-01T23:04:10.826 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8602238976, 2022-08-01T23:04:10.826 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011127026751637459 2022-08-01T23:04:10.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:10.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:10.826 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:04:10.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:10.827 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2022-08-01T23:04:10.827 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2022-08-01T23:04:10.827 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:10.827 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:10.828 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2022-08-01T23:04:10.828 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:10.828 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:10.828 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:10.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918543360 2022-08-01T23:04:10.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:10.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:10.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:10.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2022-08-01T23:04:10.830 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2022-08-01T23:04:10.830 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:10.830 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:10.830 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:04:10.831 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:10.831 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:10.831 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:10.831 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918543360 2022-08-01T23:04:10.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:10.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:10.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:10.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2022-08-01T23:04:10.832 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 22, 2022-08-01T23:04:10.833 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:10.833 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:10.833 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:04:10.833 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:10.833 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:10.834 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:10.834 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918543360 2022-08-01T23:04:10.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:10.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:10.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:10.835 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2022-08-01T23:04:10.835 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 23, 2022-08-01T23:04:10.835 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:10.836 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:10.836 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:04:10.836 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:10.836 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:10.836 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:10.837 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918543360 2022-08-01T23:04:10.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:10.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:10.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:10.837 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:10.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f088c053a70 msgr2=0x7f088c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f088c053a70 0x7f088c055f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f08a001a000 tx=0x7f08a0014000).stop 2022-08-01T23:04:10.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 msgr2=0x7f08ac065a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:10.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac065a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f08ac156af0 tx=0x7f089c006eb0).stop 2022-08-01T23:04:10.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 shutdown_connections 2022-08-01T23:04:10.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f088c053a70 0x7f088c055f30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac144b00 0x7f08ac063790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08ac0b3f90 0x7f08ac1481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 --2- 172.21.15.32:0/580964223 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f08ac0617f0 0x7f08ac065a50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 >> 172.21.15.32:0/580964223 conn(0x7f08ac066f70 msgr2=0x7f08ac0b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 shutdown_connections 2022-08-01T23:04:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:10.817+0000 7f08b0bb3700 1 -- 172.21.15.32:0/580964223 wait complete. 2022-08-01T23:04:10.842 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2022-08-01T23:04:15.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-08-01T23:04:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 Processor -- start 2022-08-01T23:04:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 -- start start 2022-08-01T23:04:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b7840 0x7f8d74065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d7414b040 0x7f8d7414b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d740b2c20 con 0x7f8d740b3070 2022-08-01T23:04:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d74061830 con 0x7f8d7414b040 2022-08-01T23:04:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d73fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74065c10 con 0x7f8d740b7840 2022-08-01T23:04:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d727fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8d740b3070 0x7f8d74061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35614/0 (socket says 172.21.15.32:35614) 2022-08-01T23:04:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d727fc700 1 -- 172.21.15.32:0/2051894785 learned_addr learned my addr 172.21.15.32:0/2051894785 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.949+0000 7f8d737fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d7414b040 0x7f8d7414b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d72ffd700 1 --2- 172.21.15.32:0/2051894785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b7840 0x7f8d74065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d71ffb700 1 -- 172.21.15.32:0/2051894785 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012387104 0 0) 0x7f8d740b2c20 con 0x7f8d740b3070 2022-08-01T23:04:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d71ffb700 1 -- 172.21.15.32:0/2051894785 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d58003260 con 0x7f8d740b3070 2022-08-01T23:04:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d737fe700 1 -- 172.21.15.32:0/2051894785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b7840 msgr2=0x7f8d74065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d737fe700 1 --2- 172.21.15.32:0/2051894785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b7840 0x7f8d74065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d737fe700 1 -- 172.21.15.32:0/2051894785 >> v1:172.21.15.32:6789/0 conn(0x7f8d740b3070 legacy=0x7f8d74061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d737fe700 1 -- 172.21.15.32:0/2051894785 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740b7c20 con 0x7f8d7414b040 2022-08-01T23:04:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d737fe700 1 --2- 172.21.15.32:0/2051894785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d7414b040 0x7f8d7414b4f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f8d6c002f70 tx=0x7f8d6c006df0).ready entity=mon.0 client_cookie=dc260a461a447e30 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d71ffb700 1 -- 172.21.15.32:0/2051894785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d6c00d570 con 0x7f8d7414b040 2022-08-01T23:04:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d71ffb700 1 -- 172.21.15.32:0/2051894785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c000e90 con 0x7f8d7414b040 2022-08-01T23:04:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d71ffb700 1 -- 172.21.15.32:0/2051894785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d6c00d900 con 0x7f8d7414b040 2022-08-01T23:04:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 -- 172.21.15.32:0/2051894785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d7414b040 msgr2=0x7f8d7414b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 --2- 172.21.15.32:0/2051894785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d7414b040 0x7f8d7414b4f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f8d6c002f70 tx=0x7f8d6c006df0).stop 2022-08-01T23:04:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 -- 172.21.15.32:0/2051894785 shutdown_connections 2022-08-01T23:04:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 --2- 172.21.15.32:0/2051894785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d7414b040 0x7f8d7414b4f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 --2- 172.21.15.32:0/2051894785 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b7840 0x7f8d74065720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 -- 172.21.15.32:0/2051894785 >> 172.21.15.32:0/2051894785 conn(0x7f8d7405f840 msgr2=0x7f8d7405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 -- 172.21.15.32:0/2051894785 shutdown_connections 2022-08-01T23:04:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 -- 172.21.15.32:0/2051894785 wait complete. 2022-08-01T23:04:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 Processor -- start 2022-08-01T23:04:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.953+0000 7f8d73fff700 1 -- start start 2022-08-01T23:04:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b3070 0x7f8d740b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 0x7f8d7409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7414b040 0x7f8d740aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b1d10 con 0x7f8d740b7840 2022-08-01T23:04:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b1fb0 con 0x7f8d7414b040 2022-08-01T23:04:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b0d00 con 0x7f8d740b3070 2022-08-01T23:04:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 0x7f8d7409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 0x7f8d7409ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48078/0 (socket says 172.21.15.32:48078) 2022-08-01T23:04:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 -- 172.21.15.32:0/1281191968 learned_addr learned my addr 172.21.15.32:0/1281191968 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d72ffd700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b3070 0x7f8d740b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d737fe700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7414b040 0x7f8d740aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 -- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b3070 msgr2=0x7f8d740b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b3070 0x7f8d740b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 -- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7414b040 msgr2=0x7f8d740aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7414b040 0x7f8d740aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 -- 172.21.15.32:0/1281191968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c00b180 con 0x7f8d740b7840 2022-08-01T23:04:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d72ffd700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b3070 0x7f8d740b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d727fc700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 0x7f8d7409ed80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f8d64001490 tx=0x7f8d64007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d64005ba0 con 0x7f8d740b7840 2022-08-01T23:04:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d740b0fa0 con 0x7f8d740b7840 2022-08-01T23:04:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d64005d00 con 0x7f8d740b7840 2022-08-01T23:04:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d640072c0 con 0x7f8d740b7840 2022-08-01T23:04:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d740b1880 con 0x7f8d740b7840 2022-08-01T23:04:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.957+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d740b1130 con 0x7f8d740b7840 2022-08-01T23:04:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.961+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d6401a070 con 0x7f8d740b7840 2022-08-01T23:04:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.961+0000 7f8d63fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8d500539e0 0x7f8d50055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.961+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f8d6406b7a0 con 0x7f8d740b7840 2022-08-01T23:04:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.961+0000 7f8d72ffd700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8d500539e0 0x7f8d50055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.961+0000 7f8d72ffd700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8d500539e0 0x7f8d50055ea0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f8d6800c000 tx=0x7f8d68008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:15.969+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d640424c0 con 0x7f8d740b7840 2022-08-01T23:04:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.137+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f8d740b2980 con 0x7f8d740b7840 2022-08-01T23:04:16.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.137+0000 7f8d63fff700 1 -- 172.21.15.32:0/1281191968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7f8d64045ae0 con 0x7f8d740b7840 2022-08-01T23:04:16.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:16.143 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:16.143 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:16.143 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2022-08-01T23:04:16.144 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764491522048, 2022-08-01T23:04:16.144 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 12656640, 2022-08-01T23:04:16.144 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8602591232, 2022-08-01T23:04:16.145 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011127482168376446, 2022-08-01T23:04:16.145 INFO:teuthology.orchestra.run.smithi032.stdout: "num_osds": 8, 2022-08-01T23:04:16.145 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_osds": 8, 2022-08-01T23:04:16.145 INFO:teuthology.orchestra.run.smithi032.stdout: "num_per_pool_omap_osds": 8 2022-08-01T23:04:16.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:16.146 INFO:teuthology.orchestra.run.smithi032.stdout: "stats_by_class": { 2022-08-01T23:04:16.146 INFO:teuthology.orchestra.run.smithi032.stdout: "ssd": { 2022-08-01T23:04:16.146 INFO:teuthology.orchestra.run.smithi032.stdout: "total_bytes": 773094113280, 2022-08-01T23:04:16.147 INFO:teuthology.orchestra.run.smithi032.stdout: "total_avail_bytes": 764491522048, 2022-08-01T23:04:16.147 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_bytes": 12656640, 2022-08-01T23:04:16.147 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_bytes": 8602591232, 2022-08-01T23:04:16.147 INFO:teuthology.orchestra.run.smithi032.stdout: "total_used_raw_ratio": 0.011127482168376446 2022-08-01T23:04:16.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:16.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:16.149 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:04:16.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:16.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_health_metrics", 2022-08-01T23:04:16.149 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 1, 2022-08-01T23:04:16.150 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:16.150 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:16.150 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 2, 2022-08-01T23:04:16.150 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:16.151 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:16.151 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:16.151 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918379520 2022-08-01T23:04:16.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:16.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:16.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:16.152 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd", 2022-08-01T23:04:16.152 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2, 2022-08-01T23:04:16.153 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:16.153 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:16.153 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:04:16.153 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:16.153 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:16.154 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:16.154 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918379520 2022-08-01T23:04:16.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:16.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:16.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:16.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_data", 2022-08-01T23:04:16.155 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 22, 2022-08-01T23:04:16.155 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:16.156 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 0, 2022-08-01T23:04:16.156 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 0, 2022-08-01T23:04:16.156 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 0, 2022-08-01T23:04:16.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 0, 2022-08-01T23:04:16.156 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 0, 2022-08-01T23:04:16.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918379520 2022-08-01T23:04:16.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:16.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:16.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:16.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs_metadata", 2022-08-01T23:04:16.158 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 23, 2022-08-01T23:04:16.158 INFO:teuthology.orchestra.run.smithi032.stdout: "stats": { 2022-08-01T23:04:16.158 INFO:teuthology.orchestra.run.smithi032.stdout: "stored": 617, 2022-08-01T23:04:16.158 INFO:teuthology.orchestra.run.smithi032.stdout: "objects": 1, 2022-08-01T23:04:16.159 INFO:teuthology.orchestra.run.smithi032.stdout: "kb_used": 8, 2022-08-01T23:04:16.159 INFO:teuthology.orchestra.run.smithi032.stdout: "bytes_used": 8192, 2022-08-01T23:04:16.159 INFO:teuthology.orchestra.run.smithi032.stdout: "percent_used": 1.1286283907452344e-08, 2022-08-01T23:04:16.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max_avail": 362918379520 2022-08-01T23:04:16.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:16.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:16.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:16.160 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:16.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8d500539e0 msgr2=0x7f8d50055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8d500539e0 0x7f8d50055ea0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f8d6800c000 tx=0x7f8d68008040).stop 2022-08-01T23:04:16.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 msgr2=0x7f8d7409ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 0x7f8d7409ed80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f8d64001490 tx=0x7f8d64007e20).stop 2022-08-01T23:04:16.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 shutdown_connections 2022-08-01T23:04:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8d500539e0 0x7f8d50055ea0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d7414b040 0x7f8d740aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d740b7840 0x7f8d7409ed80 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 --2- 172.21.15.32:0/1281191968 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8d740b3070 0x7f8d740b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 >> 172.21.15.32:0/1281191968 conn(0x7f8d7405f840 msgr2=0x7f8d740b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 shutdown_connections 2022-08-01T23:04:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.141+0000 7f8d73fff700 1 -- 172.21.15.32:0/1281191968 wait complete. 2022-08-01T23:04:16.166 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2022-08-01T23:04:16.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-08-01T23:04:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 Processor -- start 2022-08-01T23:04:16.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 -- start start 2022-08-01T23:04:16.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200614b0 0x7ff120061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1201405d0 0x7ff120140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1200aee80 con 0x7ff1200ad750 2022-08-01T23:04:16.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1200b70b0 con 0x7ff1201405d0 2022-08-01T23:04:16.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff126773700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200af0b0 con 0x7ff1200614b0 2022-08-01T23:04:16.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff125f72700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1201405d0 0x7ff120140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff125771700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200614b0 0x7ff120061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff125f72700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1201405d0 0x7ff120140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:57876/0 (socket says 172.21.15.32:57876) 2022-08-01T23:04:16.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff125f72700 1 -- 172.21.15.32:0/435841218 learned_addr learned my addr 172.21.15.32:0/435841218 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:16.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.269+0000 7ff124f70700 1 --1- 172.21.15.32:0/435841218 >> v1:172.21.15.55:6789/0 conn(0x7ff1200ad750 0x7ff1200b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:56534/0 (socket says 172.21.15.32:56534) 2022-08-01T23:04:16.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff125f72700 1 -- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200614b0 msgr2=0x7ff120061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff125f72700 1 --2- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200614b0 0x7ff120061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff125f72700 1 -- 172.21.15.32:0/435841218 >> v1:172.21.15.55:6789/0 conn(0x7ff1200ad750 legacy=0x7ff1200b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff125f72700 1 -- 172.21.15.32:0/435841218 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200af350 con 0x7ff1201405d0 2022-08-01T23:04:16.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff125f72700 1 --2- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff120140a60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7ff1180032f0 tx=0x7ff118009290).ready entity=mon.1 client_cookie=81d51ca715460dfe server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff117fff700 1 -- 172.21.15.32:0/435841218 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff11800b8a0 con 0x7ff1201405d0 2022-08-01T23:04:16.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff117fff700 1 -- 172.21.15.32:0/435841218 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800ba00 con 0x7ff1201405d0 2022-08-01T23:04:16.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff117fff700 1 -- 172.21.15.32:0/435841218 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff11800bb70 con 0x7ff1201405d0 2022-08-01T23:04:16.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 -- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 msgr2=0x7ff120140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 --2- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff120140a60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7ff1180032f0 tx=0x7ff118009290).stop 2022-08-01T23:04:16.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 -- 172.21.15.32:0/435841218 shutdown_connections 2022-08-01T23:04:16.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 --2- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff120140a60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 --2- 172.21.15.32:0/435841218 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200614b0 0x7ff120061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 -- 172.21.15.32:0/435841218 >> 172.21.15.32:0/435841218 conn(0x7ff120066f70 msgr2=0x7ff120067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 -- 172.21.15.32:0/435841218 shutdown_connections 2022-08-01T23:04:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 -- 172.21.15.32:0/435841218 wait complete. 2022-08-01T23:04:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 Processor -- start 2022-08-01T23:04:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.273+0000 7ff126773700 1 -- start start 2022-08-01T23:04:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1200614b0 0x7ff1200b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200ad750 0x7ff1200b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff1200b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120153360 con 0x7ff1200614b0 2022-08-01T23:04:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200aaa40 con 0x7ff1201405d0 2022-08-01T23:04:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200aace0 con 0x7ff1200ad750 2022-08-01T23:04:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1200614b0 0x7ff1200b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1200614b0 0x7ff1200b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48092/0 (socket says 172.21.15.32:48092) 2022-08-01T23:04:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 -- 172.21.15.32:0/2951267890 learned_addr learned my addr 172.21.15.32:0/2951267890 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff124f70700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200ad750 0x7ff1200b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125f72700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff1200b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 -- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200ad750 msgr2=0x7ff1200b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff1200ad750 0x7ff1200b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 -- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 msgr2=0x7ff1200b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff1200b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 -- 172.21.15.32:0/2951267890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118009530 con 0x7ff1200614b0 2022-08-01T23:04:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125f72700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1201405d0 0x7ff1200b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff125771700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1200614b0 0x7ff1200b3730 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7ff11000c000 tx=0x7ff110006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff110010780 con 0x7ff1200614b0 2022-08-01T23:04:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1100108e0 con 0x7ff1200614b0 2022-08-01T23:04:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 -- 172.21.15.32:0/2951267890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff120152f60 con 0x7ff1200614b0 2022-08-01T23:04:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff110010c40 con 0x7ff1200614b0 2022-08-01T23:04:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.277+0000 7ff126773700 1 -- 172.21.15.32:0/2951267890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1200ab9d0 con 0x7ff1200614b0 2022-08-01T23:04:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.281+0000 7ff126773700 1 -- 172.21.15.32:0/2951267890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1200af5f0 con 0x7ff1200614b0 2022-08-01T23:04:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.281+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff11001c460 con 0x7ff1200614b0 2022-08-01T23:04:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.281+0000 7ff1167fc700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff108053950 0x7ff108055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.281+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6422+0+0 (secure 0 0 0) 0x7ff11006c560 con 0x7ff1200614b0 2022-08-01T23:04:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.281+0000 7ff124f70700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff108053950 0x7ff108055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.281+0000 7ff124f70700 1 --2- 172.21.15.32:0/2951267890 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff108053950 0x7ff108055e10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7ff11c00d000 tx=0x7ff11c007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.285+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1100431b0 con 0x7ff1200614b0 2022-08-01T23:04:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.449+0000 7ff126773700 1 -- 172.21.15.32:0/2951267890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff1200574e0 con 0x7ff1200614b0 2022-08-01T23:04:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.449+0000 7ff1167fc700 1 -- 172.21.15.32:0/2951267890 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v59) v1 ==== 209+0+0 (secure 0 0 0) 0x7ff1100467d0 con 0x7ff1200614b0 2022-08-01T23:04:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2022-08-01T23:04:16.462 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-08-01T23:04:16.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:04:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 Processor -- start 2022-08-01T23:04:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 -- start start 2022-08-01T23:04:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87d0067490 0x7f87d0143a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87d0143f70 0x7f87d014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d00aad40 con 0x7f87d0060290 2022-08-01T23:04:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d0060670 con 0x7f87d0067490 2022-08-01T23:04:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87d5f2d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d0067030 con 0x7f87d0143f70 2022-08-01T23:04:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87d0067490 0x7f87d0143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87d0067490 0x7f87d0143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48102/0 (socket says 172.21.15.32:48102) 2022-08-01T23:04:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 -- 172.21.15.32:0/3113236100 learned_addr learned my addr 172.21.15.32:0/3113236100 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87cffff700 1 --2- 172.21.15.32:0/3113236100 >> v2:172.21.15.55:3300/0 conn(0x7f87d0143f70 0x7f87d014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 -- 172.21.15.32:0/3113236100 >> v2:172.21.15.55:3300/0 conn(0x7f87d0143f70 msgr2=0x7f87d014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 --2- 172.21.15.32:0/3113236100 >> v2:172.21.15.55:3300/0 conn(0x7f87d0143f70 0x7f87d014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 -- 172.21.15.32:0/3113236100 >> v1:172.21.15.55:6789/0 conn(0x7f87d0060290 legacy=0x7f87d00b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 -- 172.21.15.32:0/3113236100 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00b3540 con 0x7f87d0067490 2022-08-01T23:04:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ceffd700 1 --2- 172.21.15.32:0/3113236100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0067490 0x7f87d0143a80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f87c4002f70 tx=0x7f87c4006eb0).ready entity=mon.0 client_cookie=b56b821ef4ac413e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ce7fc700 1 -- 172.21.15.32:0/3113236100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c400d5c0 con 0x7f87d0067490 2022-08-01T23:04:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ce7fc700 1 -- 172.21.15.32:0/3113236100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4000e90 con 0x7f87d0067490 2022-08-01T23:04:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.573+0000 7f87ce7fc700 1 -- 172.21.15.32:0/3113236100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c400d950 con 0x7f87d0067490 2022-08-01T23:04:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3113236100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0067490 msgr2=0x7f87d0143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3113236100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0067490 0x7f87d0143a80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f87c4002f70 tx=0x7f87c4006eb0).stop 2022-08-01T23:04:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3113236100 shutdown_connections 2022-08-01T23:04:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3113236100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0067490 0x7f87d0143a80 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3113236100 >> v2:172.21.15.55:3300/0 conn(0x7f87d0143f70 0x7f87d014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3113236100 >> 172.21.15.32:0/3113236100 conn(0x7f87d005d0c0 msgr2=0x7f87d005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3113236100 shutdown_connections 2022-08-01T23:04:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3113236100 wait complete. 2022-08-01T23:04:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 Processor -- start 2022-08-01T23:04:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.577+0000 7f87d5f2d700 1 -- start start 2022-08-01T23:04:16.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d0060290 0x7f87d0148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d0067490 0x7f87d009f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 0x7f87d009f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00a18c0 con 0x7f87d0143f70 2022-08-01T23:04:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00ae0d0 con 0x7f87d0067490 2022-08-01T23:04:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00a10e0 con 0x7f87d0060290 2022-08-01T23:04:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 0x7f87d009f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 0x7f87d009f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48110/0 (socket says 172.21.15.32:48110) 2022-08-01T23:04:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 -- 172.21.15.32:0/3932545451 learned_addr learned my addr 172.21.15.32:0/3932545451 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87ceffd700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d0067490 0x7f87d009f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d0060290 0x7f87d0148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 -- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d0060290 msgr2=0x7f87d0148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d0060290 0x7f87d0148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 -- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d0067490 msgr2=0x7f87d009f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d0067490 0x7f87d009f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 -- 172.21.15.32:0/3932545451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c400b1e0 con 0x7f87d0143f70 2022-08-01T23:04:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87cffff700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 0x7f87d009f520 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f87c8006e00 tx=0x7f87c80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c8012600 con 0x7f87d0143f70 2022-08-01T23:04:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8012760 con 0x7f87d0143f70 2022-08-01T23:04:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d00a1380 con 0x7f87d0143f70 2022-08-01T23:04:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87c8012ac0 con 0x7f87d0143f70 2022-08-01T23:04:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.581+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d00a0dd0 con 0x7f87d0143f70 2022-08-01T23:04:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.585+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f87c8022070 con 0x7f87d0143f70 2022-08-01T23:04:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.585+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d009ed70 con 0x7f87d0143f70 2022-08-01T23:04:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.585+0000 7f87ccff9700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053b00 0x7f87b0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.585+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f87c806c700 con 0x7f87d0143f70 2022-08-01T23:04:16.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.585+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053b00 0x7f87b0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.585+0000 7f87cf7fe700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053b00 0x7f87b0055fc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f87c000d000 tx=0x7f87c0007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.593+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87c8043330 con 0x7f87d0143f70 2022-08-01T23:04:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.757+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f87d00574e0 con 0x7f87d0143f70 2022-08-01T23:04:16.762 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:04:16.757+0000 7f9825d5c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:04:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.769+0000 7f87ccff9700 1 -- 172.21.15.32:0/3932545451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v60) v1 ==== 189+0+0 (secure 0 0 0) 0x7f87c8016020 con 0x7f87d0143f70 2022-08-01T23:04:16.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053b00 msgr2=0x7f87b0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053b00 0x7f87b0055fc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f87c000d000 tx=0x7f87c0007b90).stop 2022-08-01T23:04:16.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 msgr2=0x7f87d009f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 0x7f87d009f520 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f87c8006e00 tx=0x7f87c80091d0).stop 2022-08-01T23:04:16.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 shutdown_connections 2022-08-01T23:04:16.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f87b0053b00 0x7f87b0055fc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87d0143f70 0x7f87d009f520 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87d0067490 0x7f87d009f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 --2- 172.21.15.32:0/3932545451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f87d0060290 0x7f87d0148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 >> 172.21.15.32:0/3932545451 conn(0x7f87d005d0c0 msgr2=0x7f87d0143850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 shutdown_connections 2022-08-01T23:04:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.773+0000 7f87d5f2d700 1 -- 172.21.15.32:0/3932545451 wait complete. 2022-08-01T23:04:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 23 and data pool 22 2022-08-01T23:04:16.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:04:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.897+0000 7f8efdfbf700 1 Processor -- start 2022-08-01T23:04:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.897+0000 7f8efdfbf700 1 -- start start 2022-08-01T23:04:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef8152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80a7250 con 0x7f8ef80b2710 2022-08-01T23:04:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80a7480 con 0x7f8ef80b01a0 2022-08-01T23:04:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80a7720 con 0x7f8ef80a8c50 2022-08-01T23:04:16.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efcfbd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ef80b2710 0x7f8ef814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35656/0 (socket says 172.21.15.32:35656) 2022-08-01T23:04:16.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efcfbd700 1 -- 172.21.15.32:0/2166394233 learned_addr learned my addr 172.21.15.32:0/2166394233 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:16.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efd7be700 1 --2- 172.21.15.32:0/2166394233 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef8152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339527911 0 0) 0x7f8ef80a7250 con 0x7f8ef80b2710 2022-08-01T23:04:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee4003260 con 0x7f8ef80b2710 2022-08-01T23:04:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089775536 0 0) 0x7f8ef80a7480 con 0x7f8ef80b01a0 2022-08-01T23:04:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef80a7250 con 0x7f8ef80b01a0 2022-08-01T23:04:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efd7be700 1 -- 172.21.15.32:0/2166394233 >> v1:172.21.15.55:6789/0 conn(0x7f8ef80b01a0 legacy=0x7f8ef80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efd7be700 1 -- 172.21.15.32:0/2166394233 >> v1:172.21.15.32:6789/0 conn(0x7f8ef80b2710 legacy=0x7f8ef814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691486967 0 0) 0x7f8ee4003260 con 0x7f8ef80b2710 2022-08-01T23:04:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efd7be700 1 -- 172.21.15.32:0/2166394233 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80a79c0 con 0x7f8ef80a8c50 2022-08-01T23:04:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efd7be700 1 --2- 172.21.15.32:0/2166394233 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef8152d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8eec001110 tx=0x7f8ef0006df0).ready entity=mon.2 client_cookie=84777cb7ef5e889c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eec001660 con 0x7f8ef80a8c50 2022-08-01T23:04:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0000e90 con 0x7f8ef80a8c50 2022-08-01T23:04:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8ef77fe700 1 -- 172.21.15.32:0/2166394233 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ef0012460 con 0x7f8ef80a8c50 2022-08-01T23:04:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2166394233 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 msgr2=0x7f8ef8152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2166394233 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef8152d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8eec001110 tx=0x7f8ef0006df0).stop 2022-08-01T23:04:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2166394233 shutdown_connections 2022-08-01T23:04:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2166394233 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef8152d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2166394233 >> 172.21.15.32:0/2166394233 conn(0x7f8ef8060310 msgr2=0x7f8ef80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2166394233 shutdown_connections 2022-08-01T23:04:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.901+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2166394233 wait complete. 2022-08-01T23:04:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 Processor -- start 2022-08-01T23:04:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 -- start start 2022-08-01T23:04:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef811d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef80b01a0 0x7f8ef8153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 0x7f8ef8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8157850 con 0x7f8ef80b2710 2022-08-01T23:04:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8154600 con 0x7f8ef80b01a0 2022-08-01T23:04:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef811d6a0 con 0x7f8ef80a8c50 2022-08-01T23:04:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 0x7f8ef8154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 0x7f8ef8154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48122/0 (socket says 172.21.15.32:48122) 2022-08-01T23:04:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 -- 172.21.15.32:0/2347944917 learned_addr learned my addr 172.21.15.32:0/2347944917 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efcfbd700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef811d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8ef7fff700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef80b01a0 0x7f8ef8153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 -- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 msgr2=0x7f8ef811d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef811d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 -- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef80b01a0 msgr2=0x7f8ef8153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef80b01a0 0x7f8ef8153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 -- 172.21.15.32:0/2347944917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef000b180 con 0x7f8ef80b2710 2022-08-01T23:04:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efd7be700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 0x7f8ef8154110 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f8eec001110 tx=0x7f8ef0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ef000da20 con 0x7f8ef80b2710 2022-08-01T23:04:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef000db80 con 0x7f8ef80b2710 2022-08-01T23:04:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef8063ea0 con 0x7f8ef80b2710 2022-08-01T23:04:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ef001e400 con 0x7f8ef80b2710 2022-08-01T23:04:16.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.905+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef80643e0 con 0x7f8ef80b2710 2022-08-01T23:04:16.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.909+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ef0025030 con 0x7f8ef80b2710 2022-08-01T23:04:16.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.909+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef80574e0 con 0x7f8ef80b2710 2022-08-01T23:04:16.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.909+0000 7f8ef5ffb700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ed8053a70 0x7f8ed8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:16.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.909+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f8ef006cd60 con 0x7f8ef80b2710 2022-08-01T23:04:16.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.909+0000 7f8efcfbd700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ed8053a70 0x7f8ed8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:16.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.909+0000 7f8efcfbd700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ed8053a70 0x7f8ed8055f30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f8eec00b000 tx=0x7f8eec000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:16.913+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ef00439c0 con 0x7f8ef80b2710 2022-08-01T23:04:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.077+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ef8064a60 con 0x7f8ef80b2710 2022-08-01T23:04:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.777+0000 7f8ef5ffb700 1 -- 172.21.15.32:0/2347944917 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ef00230a0 con 0x7f8ef80b2710 2022-08-01T23:04:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ed8053a70 msgr2=0x7f8ed8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ed8053a70 0x7f8ed8055f30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f8eec00b000 tx=0x7f8eec000bc0).stop 2022-08-01T23:04:17.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 msgr2=0x7f8ef8154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:17.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 0x7f8ef8154110 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f8eec001110 tx=0x7f8ef0014000).stop 2022-08-01T23:04:17.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 shutdown_connections 2022-08-01T23:04:17.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ed8053a70 0x7f8ed8055f30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef80b2710 0x7f8ef8154110 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef80b01a0 0x7f8ef8153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 --2- 172.21.15.32:0/2347944917 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ef80a8c50 0x7f8ef811d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 >> 172.21.15.32:0/2347944917 conn(0x7f8ef8060310 msgr2=0x7f8ef80af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 shutdown_connections 2022-08-01T23:04:17.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.781+0000 7f8efdfbf700 1 -- 172.21.15.32:0/2347944917 wait complete. 2022-08-01T23:04:17.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:04:17.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 Processor -- start 2022-08-01T23:04:17.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 -- start start 2022-08-01T23:04:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510060290 0x7f35100b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3510067490 0x7f3510143a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35100aad40 con 0x7f35100b4dd0 2022-08-01T23:04:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3510060670 con 0x7f3510067490 2022-08-01T23:04:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f35151ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3510067030 con 0x7f3510060290 2022-08-01T23:04:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f350e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3510067490 0x7f3510143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f350e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3510067490 0x7f3510143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48130/0 (socket says 172.21.15.32:48130) 2022-08-01T23:04:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f350e59c700 1 -- 172.21.15.32:0/994275331 learned_addr learned my addr 172.21.15.32:0/994275331 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.909+0000 7f350ed9d700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510060290 0x7f35100b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350e59c700 1 -- 172.21.15.32:0/994275331 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510060290 msgr2=0x7f35100b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350e59c700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510060290 0x7f35100b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350e59c700 1 -- 172.21.15.32:0/994275331 >> v1:172.21.15.55:6789/0 conn(0x7f35100b4dd0 legacy=0x7f35101446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350e59c700 1 -- 172.21.15.32:0/994275331 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35100b3540 con 0x7f3510067490 2022-08-01T23:04:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350ed9d700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510060290 0x7f35100b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350e59c700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3510067490 0x7f3510143a80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f3504002f70 tx=0x7f3504006eb0).ready entity=mon.0 client_cookie=3c99a05ca178b358 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350dd9b700 1 -- 172.21.15.32:0/994275331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350400d5c0 con 0x7f3510067490 2022-08-01T23:04:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350dd9b700 1 -- 172.21.15.32:0/994275331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3504000e90 con 0x7f3510067490 2022-08-01T23:04:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f350dd9b700 1 -- 172.21.15.32:0/994275331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350400d950 con 0x7f3510067490 2022-08-01T23:04:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 -- 172.21.15.32:0/994275331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3510067490 msgr2=0x7f3510143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3510067490 0x7f3510143a80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f3504002f70 tx=0x7f3504006eb0).stop 2022-08-01T23:04:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 -- 172.21.15.32:0/994275331 shutdown_connections 2022-08-01T23:04:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3510067490 0x7f3510143a80 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 --2- 172.21.15.32:0/994275331 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510060290 0x7f35100b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 -- 172.21.15.32:0/994275331 >> 172.21.15.32:0/994275331 conn(0x7f351005d0c0 msgr2=0x7f351005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 -- 172.21.15.32:0/994275331 shutdown_connections 2022-08-01T23:04:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 -- 172.21.15.32:0/994275331 wait complete. 2022-08-01T23:04:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 Processor -- start 2022-08-01T23:04:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.913+0000 7f35151ca700 1 -- start start 2022-08-01T23:04:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3510060290 0x7f3510144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510067490 0x7f35100a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 0x7f35100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100ae610 con 0x7f35100b4dd0 2022-08-01T23:04:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100aca20 con 0x7f3510060290 2022-08-01T23:04:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35100a5630 con 0x7f3510067490 2022-08-01T23:04:17.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 0x7f35100a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:17.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510067490 0x7f35100a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:17.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 0x7f35100a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48136/0 (socket says 172.21.15.32:48136) 2022-08-01T23:04:17.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 -- 172.21.15.32:0/1211207284 learned_addr learned my addr 172.21.15.32:0/1211207284 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350ed9d700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3510060290 0x7f3510144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 -- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510067490 msgr2=0x7f35100a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510067490 0x7f35100a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 -- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3510060290 msgr2=0x7f3510144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3510060290 0x7f3510144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 -- 172.21.15.32:0/1211207284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350400b1e0 con 0x7f35100b4dd0 2022-08-01T23:04:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f350f59e700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 0x7f35100a5140 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f3510067370 tx=0x7f3508007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350800ce70 con 0x7f35100b4dd0 2022-08-01T23:04:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508006aa0 con 0x7f35100b4dd0 2022-08-01T23:04:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3508006e00 con 0x7f35100b4dd0 2022-08-01T23:04:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35100ad8c0 con 0x7f35100b4dd0 2022-08-01T23:04:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35100ade00 con 0x7f35100b4dd0 2022-08-01T23:04:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.917+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3508010030 con 0x7f35100b4dd0 2022-08-01T23:04:17.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.921+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35100ae9e0 con 0x7f35100b4dd0 2022-08-01T23:04:17.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.921+0000 7f34fffff700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34ec053a70 0x7f34ec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:17.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.921+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f350806bf00 con 0x7f35100b4dd0 2022-08-01T23:04:17.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.921+0000 7f350ed9d700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34ec053a70 0x7f34ec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.921+0000 7f350ed9d700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34ec053a70 0x7f34ec055f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f350000c000 tx=0x7f3500008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:17.929+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3508042b60 con 0x7f35100b4dd0 2022-08-01T23:04:18.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.053+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f35100574e0 con 0x7f35100b4dd0 2022-08-01T23:04:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.793+0000 7f34fffff700 1 -- 172.21.15.32:0/1211207284 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3508046180 con 0x7f35100b4dd0 2022-08-01T23:04:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34ec053a70 msgr2=0x7f34ec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34ec053a70 0x7f34ec055f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f350000c000 tx=0x7f3500008040).stop 2022-08-01T23:04:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 msgr2=0x7f35100a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 0x7f35100a5140 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f3510067370 tx=0x7f3508007f40).stop 2022-08-01T23:04:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 shutdown_connections 2022-08-01T23:04:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34ec053a70 0x7f34ec055f30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35100b4dd0 0x7f35100a5140 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3510067490 0x7f35100a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 --2- 172.21.15.32:0/1211207284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3510060290 0x7f3510144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 >> 172.21.15.32:0/1211207284 conn(0x7f351005d0c0 msgr2=0x7f35100b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 shutdown_connections 2022-08-01T23:04:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.801+0000 7f35151ca700 1 -- 172.21.15.32:0/1211207284 wait complete. 2022-08-01T23:04:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:04:18.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:04:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 Processor -- start 2022-08-01T23:04:18.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 -- start start 2022-08-01T23:04:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b2050 0x7f33c80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33c80b0140 0x7f33c80b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80b2920 con 0x7f33c80a7250 2022-08-01T23:04:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c80b0580 con 0x7f33c80b0140 2022-08-01T23:04:18.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33ccae9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b07e0 con 0x7f33c80b2050 2022-08-01T23:04:18.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33c80b0140 0x7f33c80b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:18.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33c80a7250 0x7f33c81413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35688/0 (socket says 172.21.15.32:35688) 2022-08-01T23:04:18.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c6d9d700 1 -- 172.21.15.32:0/884655027 learned_addr learned my addr 172.21.15.32:0/884655027 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:18.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c659c700 1 --2- 172.21.15.32:0/884655027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b2050 0x7f33c80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c5d9b700 1 -- 172.21.15.32:0/884655027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b2050 msgr2=0x7f33c80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c5d9b700 1 --2- 172.21.15.32:0/884655027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b2050 0x7f33c80b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c5d9b700 1 -- 172.21.15.32:0/884655027 >> v1:172.21.15.32:6789/0 conn(0x7f33c80a7250 legacy=0x7f33c81413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c5d9b700 1 -- 172.21.15.32:0/884655027 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80b0a80 con 0x7f33c80b0140 2022-08-01T23:04:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c559a700 1 -- 172.21.15.32:0/884655027 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368526199 0 0) 0x7f33c80b2920 con 0x7f33c80a7250 2022-08-01T23:04:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c5d9b700 1 --2- 172.21.15.32:0/884655027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80b0140 0x7f33c80b2430 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f33bc002f70 tx=0x7f33bc006df0).ready entity=mon.0 client_cookie=5022058592aa41e2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.921+0000 7f33c559a700 1 -- 172.21.15.32:0/884655027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33bc00d570 con 0x7f33c80b0140 2022-08-01T23:04:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33c559a700 1 -- 172.21.15.32:0/884655027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc000e90 con 0x7f33c80b0140 2022-08-01T23:04:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33c559a700 1 -- 172.21.15.32:0/884655027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33bc00d8c0 con 0x7f33c80b0140 2022-08-01T23:04:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- 172.21.15.32:0/884655027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80b0140 msgr2=0x7f33c80b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 --2- 172.21.15.32:0/884655027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80b0140 0x7f33c80b2430 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f33bc002f70 tx=0x7f33bc006df0).stop 2022-08-01T23:04:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- 172.21.15.32:0/884655027 shutdown_connections 2022-08-01T23:04:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 --2- 172.21.15.32:0/884655027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80b0140 0x7f33c80b2430 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 --2- 172.21.15.32:0/884655027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b2050 0x7f33c80b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- 172.21.15.32:0/884655027 >> 172.21.15.32:0/884655027 conn(0x7f33c805f840 msgr2=0x7f33c805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- 172.21.15.32:0/884655027 shutdown_connections 2022-08-01T23:04:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- 172.21.15.32:0/884655027 wait complete. 2022-08-01T23:04:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 Processor -- start 2022-08-01T23:04:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- start start 2022-08-01T23:04:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 0x7f33c80ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b0140 0x7f33c813c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33c80b2050 0x7f33c813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c815a570 con 0x7f33c80a7250 2022-08-01T23:04:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c815a810 con 0x7f33c80b2050 2022-08-01T23:04:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33ccae9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c815aab0 con 0x7f33c80b0140 2022-08-01T23:04:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.925+0000 7f33c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 0x7f33c80ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 0x7f33c80ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48148/0 (socket says 172.21.15.32:48148) 2022-08-01T23:04:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 -- 172.21.15.32:0/2569636359 learned_addr learned my addr 172.21.15.32:0/2569636359 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c5d9b700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b0140 0x7f33c813c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c6d9d700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33c80b2050 0x7f33c813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 -- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b0140 msgr2=0x7f33c813c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b0140 0x7f33c813c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 -- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33c80b2050 msgr2=0x7f33c813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33c80b2050 0x7f33c813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 -- 172.21.15.32:0/2569636359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc00b180 con 0x7f33c80a7250 2022-08-01T23:04:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c6d9d700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33c80b2050 0x7f33c813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33c659c700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 0x7f33c80ae960 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f33b8006e00 tx=0x7f33b80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33b800c850 con 0x7f33c80a7250 2022-08-01T23:04:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b800c9b0 con 0x7f33c80a7250 2022-08-01T23:04:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c815ad50 con 0x7f33c80a7250 2022-08-01T23:04:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33b800cd10 con 0x7f33c80a7250 2022-08-01T23:04:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c813d390 con 0x7f33c80a7250 2022-08-01T23:04:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33b8022070 con 0x7f33c80a7250 2022-08-01T23:04:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.929+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c80b0d20 con 0x7f33c80a7250 2022-08-01T23:04:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.933+0000 7f33b77fe700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33a4053a10 0x7f33a4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.933+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f33b806c7a0 con 0x7f33c80a7250 2022-08-01T23:04:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.933+0000 7f33c5d9b700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33a4053a10 0x7f33a4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:18.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.933+0000 7f33c5d9b700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33a4053a10 0x7f33a4055ed0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f33bc018000 tx=0x7f33bc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:18.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:18.937+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33b80435b0 con 0x7f33c80a7250 2022-08-01T23:04:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.065+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f33c80574e0 con 0x7f33c80a7250 2022-08-01T23:04:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.797+0000 7f33b77fe700 1 -- 172.21.15.32:0/2569636359 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v128) v1 ==== 133+0+0 (secure 0 0 0) 0x7f33b8016020 con 0x7f33c80a7250 2022-08-01T23:04:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33a4053a10 msgr2=0x7f33a4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33a4053a10 0x7f33a4055ed0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f33bc018000 tx=0x7f33bc014000).stop 2022-08-01T23:04:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 msgr2=0x7f33c80ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 0x7f33c80ae960 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f33b8006e00 tx=0x7f33b80091d0).stop 2022-08-01T23:04:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 shutdown_connections 2022-08-01T23:04:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f33a4053a10 0x7f33a4055ed0 secure :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0x7f33bc018000 tx=0x7f33bc014000).stop 2022-08-01T23:04:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33c80b2050 0x7f33c813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33c80b0140 0x7f33c813c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 --2- 172.21.15.32:0/2569636359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c80a7250 0x7f33c80ae960 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.801+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 >> 172.21.15.32:0/2569636359 conn(0x7f33c805f840 msgr2=0x7f33c813f940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.805+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 shutdown_connections 2022-08-01T23:04:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.805+0000 7f33ccae9700 1 -- 172.21.15.32:0/2569636359 wait complete. 2022-08-01T23:04:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:04:19.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-08-01T23:04:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.929+0000 7f707cbbd700 1 Processor -- start 2022-08-01T23:04:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.929+0000 7f707cbbd700 1 -- start start 2022-08-01T23:04:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70780b2930 0x7f70780612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780671e0 con 0x7f70780b7840 2022-08-01T23:04:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780617e0 con 0x7f7078143400 2022-08-01T23:04:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7078065540 con 0x7f70780b2930 2022-08-01T23:04:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f7076ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70780b2930 0x7f70780612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f70777fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f70780b7840 0x7f7078060be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35698/0 (socket says 172.21.15.32:35698) 2022-08-01T23:04:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f70777fe700 1 -- 172.21.15.32:0/1601791338 learned_addr learned my addr 172.21.15.32:0/1601791338 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:19.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f7076ffd700 1 -- 172.21.15.32:0/1601791338 >> v1:172.21.15.55:6789/0 conn(0x7f7078143400 legacy=0x7f7078143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f7076ffd700 1 -- 172.21.15.32:0/1601791338 >> v1:172.21.15.32:6789/0 conn(0x7f70780b7840 legacy=0x7f7078060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f70767fc700 1 -- 172.21.15.32:0/1601791338 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614977906 0 0) 0x7f70780617e0 con 0x7f7078143400 2022-08-01T23:04:19.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f7076ffd700 1 -- 172.21.15.32:0/1601791338 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780b7c20 con 0x7f70780b2930 2022-08-01T23:04:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f7076ffd700 1 --2- 172.21.15.32:0/1601791338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780b2930 0x7f70780612f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f7068002f70 tx=0x7f7068006df0).ready entity=mon.0 client_cookie=328668fce2948742 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f70767fc700 1 -- 172.21.15.32:0/1601791338 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706800d610 con 0x7f70780b2930 2022-08-01T23:04:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f70767fc700 1 -- 172.21.15.32:0/1601791338 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068000e90 con 0x7f70780b2930 2022-08-01T23:04:19.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f70767fc700 1 -- 172.21.15.32:0/1601791338 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706800d960 con 0x7f70780b2930 2022-08-01T23:04:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- 172.21.15.32:0/1601791338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780b2930 msgr2=0x7f70780612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 --2- 172.21.15.32:0/1601791338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780b2930 0x7f70780612f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f7068002f70 tx=0x7f7068006df0).stop 2022-08-01T23:04:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- 172.21.15.32:0/1601791338 shutdown_connections 2022-08-01T23:04:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 --2- 172.21.15.32:0/1601791338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780b2930 0x7f70780612f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- 172.21.15.32:0/1601791338 >> 172.21.15.32:0/1601791338 conn(0x7f707805f840 msgr2=0x7f707805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- 172.21.15.32:0/1601791338 shutdown_connections 2022-08-01T23:04:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- 172.21.15.32:0/1601791338 wait complete. 2022-08-01T23:04:19.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 Processor -- start 2022-08-01T23:04:19.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.933+0000 7f707cbbd700 1 -- start start 2022-08-01T23:04:19.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70780b2930 0x7f70780b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:19.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70780b7840 0x7f7078142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:19.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 0x7f7078142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:19.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780b1570 con 0x7f7078143400 2022-08-01T23:04:19.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780ac570 con 0x7f70780b7840 2022-08-01T23:04:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780ac810 con 0x7f70780b2930 2022-08-01T23:04:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 0x7f7078142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 0x7f7078142c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48164/0 (socket says 172.21.15.32:48164) 2022-08-01T23:04:19.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 -- 172.21.15.32:0/2246759647 learned_addr learned my addr 172.21.15.32:0/2246759647 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7076ffd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70780b7840 0x7f7078142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f70777fe700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70780b2930 0x7f70780b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 -- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70780b2930 msgr2=0x7f70780b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70780b2930 0x7f70780b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:19.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 -- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70780b7840 msgr2=0x7f7078142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:19.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70780b7840 0x7f7078142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:19.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 -- 172.21.15.32:0/2246759647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706800b180 con 0x7f7078143400 2022-08-01T23:04:19.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7077fff700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 0x7f7078142c40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f7070000f40 tx=0x7f7070007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:19.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7070006aa0 con 0x7f7078143400 2022-08-01T23:04:19.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070006c00 con 0x7f7078143400 2022-08-01T23:04:19.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780b12d0 con 0x7f7078143400 2022-08-01T23:04:19.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7070007030 con 0x7f7078143400 2022-08-01T23:04:19.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.937+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780b0150 con 0x7f7078143400 2022-08-01T23:04:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.941+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70780574e0 con 0x7f7078143400 2022-08-01T23:04:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.941+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7070010030 con 0x7f7078143400 2022-08-01T23:04:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.941+0000 7f7074ff9700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f705c053b00 0x7f705c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.941+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f707006bf50 con 0x7f7078143400 2022-08-01T23:04:19.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.941+0000 7f70777fe700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f705c053b00 0x7f705c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:19.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.941+0000 7f70777fe700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f705c053b00 0x7f705c055fc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f706000d000 tx=0x7f7060009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:19.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:19.945+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7070042c00 con 0x7f7078143400 2022-08-01T23:04:20.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.113+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7078060330 con 0x7f7078143400 2022-08-01T23:04:20.118 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:04:20.113+0000 7f9825d5c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:04:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.121+0000 7f7074ff9700 1 -- 172.21.15.32:0/2246759647 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7070046220 con 0x7f7078143400 2022-08-01T23:04:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f705c053b00 msgr2=0x7f705c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f705c053b00 0x7f705c055fc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f706000d000 tx=0x7f7060009040).stop 2022-08-01T23:04:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 msgr2=0x7f7078142c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 0x7f7078142c40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f7070000f40 tx=0x7f7070007f90).stop 2022-08-01T23:04:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 shutdown_connections 2022-08-01T23:04:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f705c053b00 0x7f705c055fc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7078143400 0x7f7078142c40 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70780b7840 0x7f7078142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 --2- 172.21.15.32:0/2246759647 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f70780b2930 0x7f70780b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.125+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 >> 172.21.15.32:0/2246759647 conn(0x7f707805f840 msgr2=0x7f707805cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.129+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 shutdown_connections 2022-08-01T23:04:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.129+0000 7f707cbbd700 1 -- 172.21.15.32:0/2246759647 wait complete. 2022-08-01T23:04:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 24 and data pool 22 2022-08-01T23:04:20.147 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:04:20.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:20.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:20.181 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:20.203 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:20.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:04:20.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:20.225 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:20.225 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:20.250 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:04:20.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:20.253 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:20.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:20.255 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:20.262 INFO:teuthology.orchestra.run.smithi130.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:04:20.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:20.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:20.266 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:20.267 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:20.274 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:04:20.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:20.277 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:20.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:20.279 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:20.327 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:04:20.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:20.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:04:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 Processor -- start 2022-08-01T23:04:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 -- start start 2022-08-01T23:04:20.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91e00b6a90 0x7f91e00acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:20.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91e0060aa0 0x7f91e00aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:20.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00b6e70 con 0x7f91e00aa540 2022-08-01T23:04:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e0061370 con 0x7f91e0060aa0 2022-08-01T23:04:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91e56c3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e0061610 con 0x7f91e00b6a90 2022-08-01T23:04:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91df7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91e00aa540 0x7f91e00b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35714/0 (socket says 172.21.15.32:35714) 2022-08-01T23:04:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91de7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91e0060aa0 0x7f91e00aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91df7fe700 1 -- 172.21.15.32:0/987329600 learned_addr learned my addr 172.21.15.32:0/987329600 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91deffd700 1 --2- 172.21.15.32:0/987329600 >> v2:172.21.15.55:3300/0 conn(0x7f91e00b6a90 0x7f91e00acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91ddffb700 1 -- 172.21.15.32:0/987329600 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132290765 0 0) 0x7f91e00b6e70 con 0x7f91e00aa540 2022-08-01T23:04:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91ddffb700 1 -- 172.21.15.32:0/987329600 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c4003260 con 0x7f91e00aa540 2022-08-01T23:04:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91de7fc700 1 -- 172.21.15.32:0/987329600 >> v2:172.21.15.55:3300/0 conn(0x7f91e00b6a90 msgr2=0x7f91e00acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91de7fc700 1 --2- 172.21.15.32:0/987329600 >> v2:172.21.15.55:3300/0 conn(0x7f91e00b6a90 0x7f91e00acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91de7fc700 1 -- 172.21.15.32:0/987329600 >> v1:172.21.15.32:6789/0 conn(0x7f91e00aa540 legacy=0x7f91e00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91de7fc700 1 -- 172.21.15.32:0/987329600 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00618b0 con 0x7f91e0060aa0 2022-08-01T23:04:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91deffd700 1 --2- 172.21.15.32:0/987329600 >> v2:172.21.15.55:3300/0 conn(0x7f91e00b6a90 0x7f91e00acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.441+0000 7f91de7fc700 1 --2- 172.21.15.32:0/987329600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e0060aa0 0x7f91e00aa050 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f91d4002f70 tx=0x7f91d4006df0).ready entity=mon.0 client_cookie=1dc1deadcefdc879 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91ddffb700 1 -- 172.21.15.32:0/987329600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d400d4f0 con 0x7f91e0060aa0 2022-08-01T23:04:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91ddffb700 1 -- 172.21.15.32:0/987329600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d4000e90 con 0x7f91e0060aa0 2022-08-01T23:04:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91ddffb700 1 -- 172.21.15.32:0/987329600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d400d840 con 0x7f91e0060aa0 2022-08-01T23:04:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 -- 172.21.15.32:0/987329600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e0060aa0 msgr2=0x7f91e00aa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 --2- 172.21.15.32:0/987329600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e0060aa0 0x7f91e00aa050 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f91d4002f70 tx=0x7f91d4006df0).stop 2022-08-01T23:04:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 -- 172.21.15.32:0/987329600 shutdown_connections 2022-08-01T23:04:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 --2- 172.21.15.32:0/987329600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e0060aa0 0x7f91e00aa050 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 --2- 172.21.15.32:0/987329600 >> v2:172.21.15.55:3300/0 conn(0x7f91e00b6a90 0x7f91e00acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 -- 172.21.15.32:0/987329600 >> 172.21.15.32:0/987329600 conn(0x7f91e00b3610 msgr2=0x7f91e00b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 -- 172.21.15.32:0/987329600 shutdown_connections 2022-08-01T23:04:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 -- 172.21.15.32:0/987329600 wait complete. 2022-08-01T23:04:20.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 Processor -- start 2022-08-01T23:04:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.445+0000 7f91e56c3700 1 -- start start 2022-08-01T23:04:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e0060aa0 0x7f91e00b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:20.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 0x7f91e00b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:20.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e00b6a90 0x7f91e00b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:20.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e013e890 con 0x7f91e00aa540 2022-08-01T23:04:20.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e013eaf0 con 0x7f91e00b6a90 2022-08-01T23:04:20.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0152000 con 0x7f91e0060aa0 2022-08-01T23:04:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 0x7f91e00b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 0x7f91e00b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48178/0 (socket says 172.21.15.32:48178) 2022-08-01T23:04:20.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 -- 172.21.15.32:0/3646049471 learned_addr learned my addr 172.21.15.32:0/3646049471 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91df7fe700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e00b6a90 0x7f91e00b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91deffd700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e0060aa0 0x7f91e00b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 -- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e0060aa0 msgr2=0x7f91e00b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e0060aa0 0x7f91e00b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 -- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e00b6a90 msgr2=0x7f91e00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e00b6a90 0x7f91e00b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 -- 172.21.15.32:0/3646049471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d000b180 con 0x7f91e00aa540 2022-08-01T23:04:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91de7fc700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 0x7f91e00b4770 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f91d4002f70 tx=0x7f91d4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d4016070 con 0x7f91e00aa540 2022-08-01T23:04:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d4013850 con 0x7f91e00aa540 2022-08-01T23:04:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91d4013bb0 con 0x7f91e00aa540 2022-08-01T23:04:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d400b180 con 0x7f91e00aa540 2022-08-01T23:04:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e00b2540 con 0x7f91e00aa540 2022-08-01T23:04:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.449+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e0151e40 con 0x7f91e00aa540 2022-08-01T23:04:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.453+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f91d400f030 con 0x7f91e00aa540 2022-08-01T23:04:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.453+0000 7f91cffff700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91bc053a70 0x7f91bc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.453+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f91d406cc80 con 0x7f91e00aa540 2022-08-01T23:04:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.453+0000 7f91deffd700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91bc053a70 0x7f91bc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.453+0000 7f91deffd700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91bc053a70 0x7f91bc055f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f91d000b000 tx=0x7f91d0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.461+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f91d40438e0 con 0x7f91e00aa540 2022-08-01T23:04:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.589+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f91e0061b50 con 0x7f91e00aa540 2022-08-01T23:04:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.589+0000 7f91cffff700 1 -- 172.21.15.32:0/3646049471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v129) v1 ==== 81+0+34216 (secure 0 0 0) 0x7f91d4023020 con 0x7f91e00aa540 2022-08-01T23:04:20.598 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:20.599 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 129, 2022-08-01T23:04:20.599 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:04:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:04:20.600 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:04:20.115467+0000", 2022-08-01T23:04:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:20.601 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:04:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:04:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:04:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:04:20.603 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:04:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:04:20.604 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:04:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:04:20.605 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:04:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:04:20.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:04:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:04:20.607 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 24, 2022-08-01T23:04:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:04:20.608 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:04:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:04:20.609 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:04:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:04:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.610 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:04:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:04:20.611 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:04:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:20.612 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:20.613 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:20.614 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:20.615 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:04:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:04:20.616 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:04:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:04:20.617 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:04:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:20.618 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:20.619 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:20.620 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.621 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:04:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:20.622 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:20.623 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:20.625 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:20.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:20.627 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:20.628 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:20.629 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:20.630 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:20.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:20.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.633 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:20.634 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:20.635 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:20.636 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:20.637 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:04:20.638 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:04:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.639 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:04:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.641 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:04:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:04:20.642 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:04:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:20.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:20.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:20.645 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:20.646 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:20.647 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:20.648 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:20.650 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:20.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:04:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:20.652 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:20.654 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:20.655 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:20.656 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:20.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:20.660 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:20.661 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:20.662 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:20.663 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:04:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:04:20.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 22, 2022-08-01T23:04:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:04:20.665 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:03:54.638843+0000", 2022-08-01T23:04:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:20.666 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:20.667 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:20.668 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:20.669 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:20.670 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "129", 2022-08-01T23:04:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:20.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:20.672 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:20.673 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:20.674 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:20.675 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:20.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:20.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:20.678 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:20.679 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:20.680 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:04:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:04:20.681 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:04:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 24, 2022-08-01T23:04:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:04:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:04:19.125078+0000", 2022-08-01T23:04:20.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 32769, 2022-08-01T23:04:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,creating", 2022-08-01T23:04:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:20.684 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:20.685 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:20.686 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:20.687 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.688 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "129", 2022-08-01T23:04:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:20.689 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:20.690 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:20.691 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:20.692 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:20.693 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:20.694 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:20.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:20.696 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:20.697 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:20.698 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:04:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:04:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:04:20.699 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:04:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:04:20.700 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:04:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:20.701 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:04:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:04:20.702 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.703 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:20.704 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:04:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:04:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.708 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:04:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.710 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:04:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:04:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:04:20.714 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:04:20.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.717 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:04:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:04:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:04:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:04:20.719 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:04:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:04:20.721 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.722 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.723 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:04:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:04:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.726 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:04:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:04:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.730 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:04:20.731 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.732 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:04:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:04:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:04:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:20.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:04:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:04:20.737 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:04:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:04:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.738 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:04:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.740 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.741 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2022-08-01T23:04:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.742 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:04:20.743 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:04:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:04:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:04:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:04:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:04:20.751 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.752 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.753 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:04:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:04:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.755 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:04:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:04:20.756 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:04:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:04:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.757 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:04:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.759 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.760 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:20.761 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:04:20.763 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.764 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:04:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.765 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:04:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:04:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.769 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.770 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:04:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:04:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:04:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:04:20.775 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.776 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:04:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:04:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:04:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:04:20.777 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:20.779 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:04:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.780 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.781 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2022-08-01T23:04:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.782 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:04:20.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:04:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:04:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:04:20.788 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.789 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:04:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:04:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:04:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:04:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:04:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:04:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:04:20.797 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:04:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.798 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:04:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.800 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.801 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.802 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:04:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:04:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:04:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:04:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:04:20.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:04:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:04:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:04:20.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.817 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:04:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:04:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:04:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:04:20.818 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:04:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.820 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.821 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.822 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:04:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:04:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:04:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:04:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:04:20.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:04:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:04:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:04:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:20.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:04:20.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:04:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:04:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:04:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.837 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:04:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:20.839 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:20.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:20.841 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:04:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:04:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:04:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:04:20.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:20.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:04:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:04:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:04:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:04:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:20.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:04:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:04:20.855 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:04:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:04:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:20.856 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:04:20.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.858 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:04:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.860 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.861 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:04:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.862 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:04:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.865 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:04:20.866 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.867 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.868 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:04:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:20.869 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:04:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.872 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:04:20.873 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:20.874 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:20.875 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:04:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.876 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:04:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:04:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:04:20.877 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:04:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:04:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/519921040": "2022-08-02T23:04:06.054811+0000", 2022-08-01T23:04:20.878 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/519921040": "2022-08-02T23:04:06.054811+0000" 2022-08-01T23:04:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:04:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:04:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:04:20.879 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:04:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:04:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:04:20.880 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:04:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:04:20.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:04:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:20.882 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:04:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:04:20.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:04:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:04:20.884 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:04:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:04:20.885 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:04:20.885 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91bc053a70 msgr2=0x7f91bc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91bc053a70 0x7f91bc055f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f91d000b000 tx=0x7f91d0006df0).stop 2022-08-01T23:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 msgr2=0x7f91e00b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 0x7f91e00b4770 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f91d4002f70 tx=0x7f91d4014000).stop 2022-08-01T23:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 shutdown_connections 2022-08-01T23:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91bc053a70 0x7f91bc055f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91e00b6a90 0x7f91e00b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e00aa540 0x7f91e00b4770 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 --2- 172.21.15.32:0/3646049471 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91e0060aa0 0x7f91e00b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 >> 172.21.15.32:0/3646049471 conn(0x7f91e00b3610 msgr2=0x7f91e00b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 shutdown_connections 2022-08-01T23:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:20.593+0000 7f91e56c3700 1 -- 172.21.15.32:0/3646049471 wait complete. 2022-08-01T23:04:20.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:21.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 Processor -- start 2022-08-01T23:04:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 -- start start 2022-08-01T23:04:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a440b0270 0x7f9a44143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9a440ac880 0x7f9a441434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a44067a10 con 0x7f9a440b2710 2022-08-01T23:04:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a440ad740 con 0x7f9a440b0270 2022-08-01T23:04:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a4a3cf700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a44067cb0 con 0x7f9a440ac880 2022-08-01T23:04:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a48bcc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a440b0270 0x7f9a44143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a48bcc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a440b0270 0x7f9a44143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48184/0 (socket says 172.21.15.32:48184) 2022-08-01T23:04:21.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a493cd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a440b2710 0x7f9a441401e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35726/0 (socket says 172.21.15.32:35726) 2022-08-01T23:04:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.005+0000 7f9a48bcc700 1 -- 172.21.15.32:0/3359379363 learned_addr learned my addr 172.21.15.32:0/3359379363 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a49bce700 1 --2- 172.21.15.32:0/3359379363 >> v2:172.21.15.55:3300/0 conn(0x7f9a440ac880 0x7f9a441434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a48bcc700 1 -- 172.21.15.32:0/3359379363 >> v2:172.21.15.55:3300/0 conn(0x7f9a440ac880 msgr2=0x7f9a441434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a48bcc700 1 --2- 172.21.15.32:0/3359379363 >> v2:172.21.15.55:3300/0 conn(0x7f9a440ac880 0x7f9a441434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a48bcc700 1 -- 172.21.15.32:0/3359379363 >> v1:172.21.15.32:6789/0 conn(0x7f9a440b2710 legacy=0x7f9a441401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a48bcc700 1 -- 172.21.15.32:0/3359379363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44147160 con 0x7f9a440b0270 2022-08-01T23:04:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a3bfff700 1 -- 172.21.15.32:0/3359379363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173382189 0 0) 0x7f9a44067a10 con 0x7f9a440b2710 2022-08-01T23:04:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a48bcc700 1 --2- 172.21.15.32:0/3359379363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440b0270 0x7f9a44143000 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f9a40002f70 tx=0x7f9a40006eb0).ready entity=mon.0 client_cookie=f29b1752fdc801fb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a3bfff700 1 -- 172.21.15.32:0/3359379363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4000d610 con 0x7f9a440b0270 2022-08-01T23:04:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a3bfff700 1 -- 172.21.15.32:0/3359379363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a40000e90 con 0x7f9a440b0270 2022-08-01T23:04:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a3bfff700 1 -- 172.21.15.32:0/3359379363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4000d960 con 0x7f9a440b0270 2022-08-01T23:04:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3359379363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440b0270 msgr2=0x7f9a44143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3359379363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440b0270 0x7f9a44143000 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f9a40002f70 tx=0x7f9a40006eb0).stop 2022-08-01T23:04:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3359379363 shutdown_connections 2022-08-01T23:04:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3359379363 >> v2:172.21.15.55:3300/0 conn(0x7f9a440ac880 0x7f9a441434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3359379363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440b0270 0x7f9a44143000 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3359379363 >> 172.21.15.32:0/3359379363 conn(0x7f9a44060310 msgr2=0x7f9a440b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3359379363 shutdown_connections 2022-08-01T23:04:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3359379363 wait complete. 2022-08-01T23:04:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 Processor -- start 2022-08-01T23:04:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.009+0000 7f9a4a3cf700 1 -- start start 2022-08-01T23:04:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 0x7f9a440ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a440b2710 0x7f9a44142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a44155ff0 0x7f9a441564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4409ea40 con 0x7f9a440ac880 2022-08-01T23:04:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44143810 con 0x7f9a440b2710 2022-08-01T23:04:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44143ab0 con 0x7f9a44155ff0 2022-08-01T23:04:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 0x7f9a440ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 0x7f9a440ab580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48190/0 (socket says 172.21.15.32:48190) 2022-08-01T23:04:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 -- 172.21.15.32:0/3625262750 learned_addr learned my addr 172.21.15.32:0/3625262750 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a49bce700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a44155ff0 0x7f9a441564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a48bcc700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a440b2710 0x7f9a44142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 -- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a44155ff0 msgr2=0x7f9a441564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a44155ff0 0x7f9a441564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 -- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a440b2710 msgr2=0x7f9a44142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a440b2710 0x7f9a44142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 -- 172.21.15.32:0/3625262750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4000b1e0 con 0x7f9a440ac880 2022-08-01T23:04:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a493cd700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 0x7f9a440ab580 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f9a34000ed0 tx=0x7f9a34007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a340070d0 con 0x7f9a440ac880 2022-08-01T23:04:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34007230 con 0x7f9a440ac880 2022-08-01T23:04:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a44143d50 con 0x7f9a440ac880 2022-08-01T23:04:21.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a34005550 con 0x7f9a440ac880 2022-08-01T23:04:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a440aba70 con 0x7f9a440ac880 2022-08-01T23:04:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a440574e0 con 0x7f9a440ac880 2022-08-01T23:04:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.013+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a3401a060 con 0x7f9a440ac880 2022-08-01T23:04:21.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.017+0000 7f9a3a7fc700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053a70 0x7f9a2c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.017+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9a3406b8f0 con 0x7f9a440ac880 2022-08-01T23:04:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.017+0000 7f9a48bcc700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053a70 0x7f9a2c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.017+0000 7f9a48bcc700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053a70 0x7f9a2c055f30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9a4001a000 tx=0x7f9a40015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:21.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.021+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a34042660 con 0x7f9a440ac880 2022-08-01T23:04:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.189+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a440b1bb0 con 0x7f9a440ac880 2022-08-01T23:04:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.189+0000 7f9a3a7fc700 1 -- 172.21.15.32:0/3625262750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f9a34018020 con 0x7f9a440ac880 2022-08-01T23:04:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:21.195 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:20.115815+0000","modified":"2022-08-01T23:04:20.115938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-08-01T23:04:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053a70 msgr2=0x7f9a2c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053a70 0x7f9a2c055f30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9a4001a000 tx=0x7f9a40015000).stop 2022-08-01T23:04:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 msgr2=0x7f9a440ab580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 0x7f9a440ab580 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f9a34000ed0 tx=0x7f9a34007c80).stop 2022-08-01T23:04:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 shutdown_connections 2022-08-01T23:04:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9a2c053a70 0x7f9a2c055f30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9a44155ff0 0x7f9a441564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9a440b2710 0x7f9a44142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 --2- 172.21.15.32:0/3625262750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a440ac880 0x7f9a440ab580 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 >> 172.21.15.32:0/3625262750 conn(0x7f9a44060310 msgr2=0x7f9a44146410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 shutdown_connections 2022-08-01T23:04:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.193+0000 7f9a4a3cf700 1 -- 172.21.15.32:0/3625262750 wait complete. 2022-08-01T23:04:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2022-08-01T23:04:21.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:04:21.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 Processor -- start 2022-08-01T23:04:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- start start 2022-08-01T23:04:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c6813c870 0x7f9c6813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c680ad0b0 0x7f9c680ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c680b2dc0 con 0x7f9c680b29e0 2022-08-01T23:04:21.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c680b1340 con 0x7f9c6813c870 2022-08-01T23:04:21.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680b17f0 con 0x7f9c680ad0b0 2022-08-01T23:04:21.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c6813c870 0x7f9c6813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6e134700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9c680b29e0 0x7f9c680b0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35740/0 (socket says 172.21.15.32:35740) 2022-08-01T23:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c6813c870 0x7f9c6813ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48200/0 (socket says 172.21.15.32:48200) 2022-08-01T23:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6e134700 1 -- 172.21.15.32:0/2596146956 learned_addr learned my addr 172.21.15.32:0/2596146956 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6e935700 1 --2- 172.21.15.32:0/2596146956 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c680ad0b0 0x7f9c680ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 -- 172.21.15.32:0/2596146956 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c680ad0b0 msgr2=0x7f9c680ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 --2- 172.21.15.32:0/2596146956 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c680ad0b0 0x7f9c680ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 -- 172.21.15.32:0/2596146956 >> v1:172.21.15.32:6789/0 conn(0x7f9c680b29e0 legacy=0x7f9c680b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 -- 172.21.15.32:0/2596146956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c680b1a90 con 0x7f9c6813c870 2022-08-01T23:04:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d933700 1 --2- 172.21.15.32:0/2596146956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6813c870 0x7f9c6813ccb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f9c580032f0 tx=0x7f9c580091d0).ready entity=mon.0 client_cookie=7d1f7567442daf02 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d132700 1 -- 172.21.15.32:0/2596146956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c58009ea0 con 0x7f9c6813c870 2022-08-01T23:04:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d132700 1 -- 172.21.15.32:0/2596146956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5800cba0 con 0x7f9c6813c870 2022-08-01T23:04:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6d132700 1 -- 172.21.15.32:0/2596146956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c58010410 con 0x7f9c6813c870 2022-08-01T23:04:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- 172.21.15.32:0/2596146956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6813c870 msgr2=0x7f9c6813ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 --2- 172.21.15.32:0/2596146956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6813c870 0x7f9c6813ccb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f9c580032f0 tx=0x7f9c580091d0).stop 2022-08-01T23:04:21.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- 172.21.15.32:0/2596146956 shutdown_connections 2022-08-01T23:04:21.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 --2- 172.21.15.32:0/2596146956 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c680ad0b0 0x7f9c680ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 --2- 172.21.15.32:0/2596146956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c6813c870 0x7f9c6813ccb0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- 172.21.15.32:0/2596146956 >> 172.21.15.32:0/2596146956 conn(0x7f9c680613c0 msgr2=0x7f9c68062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- 172.21.15.32:0/2596146956 shutdown_connections 2022-08-01T23:04:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.309+0000 7f9c6f136700 1 -- 172.21.15.32:0/2596146956 wait complete. 2022-08-01T23:04:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.313+0000 7f9c6f136700 1 Processor -- start 2022-08-01T23:04:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.313+0000 7f9c6f136700 1 -- start start 2022-08-01T23:04:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c680ad0b0 0x7f9c68142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 0x7f9c6813e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c6813c870 0x7f9c6813ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680678a0 con 0x7f9c680b29e0 2022-08-01T23:04:21.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68065160 con 0x7f9c680ad0b0 2022-08-01T23:04:21.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68067600 con 0x7f9c6813c870 2022-08-01T23:04:21.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 0x7f9c6813e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 0x7f9c6813e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48206/0 (socket says 172.21.15.32:48206) 2022-08-01T23:04:21.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 -- 172.21.15.32:0/1444448780 learned_addr learned my addr 172.21.15.32:0/1444448780 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:21.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6e134700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c680ad0b0 0x7f9c68142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6e935700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c6813c870 0x7f9c6813ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 -- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c6813c870 msgr2=0x7f9c6813ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c6813c870 0x7f9c6813ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 -- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c680ad0b0 msgr2=0x7f9c68142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:21.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c680ad0b0 0x7f9c68142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:21.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 -- 172.21.15.32:0/1444448780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c58009470 con 0x7f9c680b29e0 2022-08-01T23:04:21.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6e935700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c6813c870 0x7f9c6813ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:21.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6d933700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 0x7f9c6813e590 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f9c580032f0 tx=0x7f9c58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:21.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c58009ea0 con 0x7f9c680b29e0 2022-08-01T23:04:21.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c58010410 con 0x7f9c680b29e0 2022-08-01T23:04:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c68067b60 con 0x7f9c680b29e0 2022-08-01T23:04:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c5801a7f0 con 0x7f9c680b29e0 2022-08-01T23:04:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.317+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c68065400 con 0x7f9c680b29e0 2022-08-01T23:04:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.321+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c68067d20 con 0x7f9c680b29e0 2022-08-01T23:04:21.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.321+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c5800e040 con 0x7f9c680b29e0 2022-08-01T23:04:21.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.321+0000 7f9c5f7fe700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9c480539e0 0x7f9c48055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:21.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.321+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9c5806de10 con 0x7f9c680b29e0 2022-08-01T23:04:21.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.321+0000 7f9c6e134700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9c480539e0 0x7f9c48055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:21.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.321+0000 7f9c6e134700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9c480539e0 0x7f9c48055ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f9c6400d000 tx=0x7f9c64009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:21.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.325+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c58044b80 con 0x7f9c680b29e0 2022-08-01T23:04:21.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:21.485+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9c680574e0 con 0x7f9c680b29e0 2022-08-01T23:04:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.249+0000 7f9c5f7fe700 1 -- 172.21.15.32:0/1444448780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9c580481a0 con 0x7f9c680b29e0 2022-08-01T23:04:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9c480539e0 msgr2=0x7f9c48055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9c480539e0 0x7f9c48055ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f9c6400d000 tx=0x7f9c64009040).stop 2022-08-01T23:04:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 msgr2=0x7f9c6813e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 0x7f9c6813e590 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f9c580032f0 tx=0x7f9c58017000).stop 2022-08-01T23:04:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 shutdown_connections 2022-08-01T23:04:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9c480539e0 0x7f9c48055ea0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9c6813c870 0x7f9c6813ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c680b29e0 0x7f9c6813e590 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 --2- 172.21.15.32:0/1444448780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c680ad0b0 0x7f9c68142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.253+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 >> 172.21.15.32:0/1444448780 conn(0x7f9c680613c0 msgr2=0x7f9c680b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.257+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 shutdown_connections 2022-08-01T23:04:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.257+0000 7f9c6f136700 1 -- 172.21.15.32:0/1444448780 wait complete. 2022-08-01T23:04:22.264 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:04:22.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 Processor -- start 2022-08-01T23:04:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 -- start start 2022-08-01T23:04:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b400b1f40 0x7f0b400b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b400a7250 0x7f0b401448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400b04e0 con 0x7f0b40148530 2022-08-01T23:04:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400a9250 con 0x7f0b400b1f40 2022-08-01T23:04:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b454c8700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400b0780 con 0x7f0b400a7250 2022-08-01T23:04:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b3effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b400b1f40 0x7f0b400b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b3effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b400b1f40 0x7f0b400b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48214/0 (socket says 172.21.15.32:48214) 2022-08-01T23:04:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b3effd700 1 -- 172.21.15.32:0/1575897201 learned_addr learned my addr 172.21.15.32:0/1575897201 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.381+0000 7f0b3f7fe700 1 --2- 172.21.15.32:0/1575897201 >> v2:172.21.15.55:3300/0 conn(0x7f0b400a7250 0x7f0b401448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3effd700 1 -- 172.21.15.32:0/1575897201 >> v2:172.21.15.55:3300/0 conn(0x7f0b400a7250 msgr2=0x7f0b401448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3effd700 1 --2- 172.21.15.32:0/1575897201 >> v2:172.21.15.55:3300/0 conn(0x7f0b400a7250 0x7f0b401448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3effd700 1 -- 172.21.15.32:0/1575897201 >> v1:172.21.15.55:6789/0 conn(0x7f0b40148530 legacy=0x7f0b400b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3dffb700 1 -- 172.21.15.32:0/1575897201 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28740019 0 0) 0x7f0b400b04e0 con 0x7f0b40148530 2022-08-01T23:04:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3effd700 1 -- 172.21.15.32:0/1575897201 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400b0a20 con 0x7f0b400b1f40 2022-08-01T23:04:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3effd700 1 --2- 172.21.15.32:0/1575897201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b400b1f40 0x7f0b400b1160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f0b401478c0 tx=0x7f0b34006eb0).ready entity=mon.0 client_cookie=fadc23fd2a10f33c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3dffb700 1 -- 172.21.15.32:0/1575897201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b3400d610 con 0x7f0b400b1f40 2022-08-01T23:04:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3dffb700 1 -- 172.21.15.32:0/1575897201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34000e90 con 0x7f0b400b1f40 2022-08-01T23:04:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b3dffb700 1 -- 172.21.15.32:0/1575897201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b3400d960 con 0x7f0b400b1f40 2022-08-01T23:04:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 -- 172.21.15.32:0/1575897201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b400b1f40 msgr2=0x7f0b400b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 --2- 172.21.15.32:0/1575897201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b400b1f40 0x7f0b400b1160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f0b401478c0 tx=0x7f0b34006eb0).stop 2022-08-01T23:04:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 -- 172.21.15.32:0/1575897201 shutdown_connections 2022-08-01T23:04:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 --2- 172.21.15.32:0/1575897201 >> v2:172.21.15.55:3300/0 conn(0x7f0b400a7250 0x7f0b401448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 --2- 172.21.15.32:0/1575897201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b400b1f40 0x7f0b400b1160 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 -- 172.21.15.32:0/1575897201 >> 172.21.15.32:0/1575897201 conn(0x7f0b4005f840 msgr2=0x7f0b4005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 -- 172.21.15.32:0/1575897201 shutdown_connections 2022-08-01T23:04:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 -- 172.21.15.32:0/1575897201 wait complete. 2022-08-01T23:04:22.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 Processor -- start 2022-08-01T23:04:22.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.385+0000 7f0b454c8700 1 -- start start 2022-08-01T23:04:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b400a7250 0x7f0b400af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b400b1f40 0x7f0b400abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 0x7f0b400ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400abdd0 con 0x7f0b40148530 2022-08-01T23:04:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400ab3f0 con 0x7f0b400b1f40 2022-08-01T23:04:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400ab690 con 0x7f0b400a7250 2022-08-01T23:04:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 0x7f0b400ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 0x7f0b400ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48220/0 (socket says 172.21.15.32:48220) 2022-08-01T23:04:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 -- 172.21.15.32:0/88132897 learned_addr learned my addr 172.21.15.32:0/88132897 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3e7fc700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b400b1f40 0x7f0b400abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3effd700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b400a7250 0x7f0b400af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 -- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b400a7250 msgr2=0x7f0b400af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b400a7250 0x7f0b400af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 -- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b400b1f40 msgr2=0x7f0b400abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b400b1f40 0x7f0b400abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 -- 172.21.15.32:0/88132897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3400b1e0 con 0x7f0b40148530 2022-08-01T23:04:22.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3e7fc700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b400b1f40 0x7f0b400abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b3f7fe700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 0x7f0b400ac460 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f0b3800c000 tx=0x7f0b38006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b38010780 con 0x7f0b40148530 2022-08-01T23:04:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b380108e0 con 0x7f0b40148530 2022-08-01T23:04:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b38010c40 con 0x7f0b40148530 2022-08-01T23:04:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b400ab930 con 0x7f0b40148530 2022-08-01T23:04:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b400ac950 con 0x7f0b40148530 2022-08-01T23:04:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.389+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b400aed30 con 0x7f0b40148530 2022-08-01T23:04:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.393+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b3801c460 con 0x7f0b40148530 2022-08-01T23:04:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.393+0000 7f0b2ffff700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0b1c053a10 0x7f0b1c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.393+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0b3806c3e0 con 0x7f0b40148530 2022-08-01T23:04:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.393+0000 7f0b3effd700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0b1c053a10 0x7f0b1c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.393+0000 7f0b3effd700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0b1c053a10 0x7f0b1c055ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f0b3401a000 tx=0x7f0b34015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.397+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b400aed30 con 0x7f0b40148530 2022-08-01T23:04:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.557+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b400aed30 con 0x7f0b40148530 2022-08-01T23:04:22.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.557+0000 7f0b2ffff700 1 -- 172.21.15.32:0/88132897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f0b400aed30 con 0x7f0b40148530 2022-08-01T23:04:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:22.563 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":63,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:20.115815+0000","modified":"2022-08-01T23:04:22.247882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-08-01T23:04:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0b1c053a10 msgr2=0x7f0b1c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0b1c053a10 0x7f0b1c055ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f0b3401a000 tx=0x7f0b34015000).stop 2022-08-01T23:04:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 msgr2=0x7f0b400ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 0x7f0b400ac460 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f0b3800c000 tx=0x7f0b38006df0).stop 2022-08-01T23:04:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 shutdown_connections 2022-08-01T23:04:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0b1c053a10 0x7f0b1c055ed0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b40148530 0x7f0b400ac460 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b400b1f40 0x7f0b400abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 --2- 172.21.15.32:0/88132897 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0b400a7250 0x7f0b400af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 >> 172.21.15.32:0/88132897 conn(0x7f0b4005f840 msgr2=0x7f0b400b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 shutdown_connections 2022-08-01T23:04:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.561+0000 7f0b454c8700 1 -- 172.21.15.32:0/88132897 wait complete. 2022-08-01T23:04:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 63 2022-08-01T23:04:22.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:04:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 Processor -- start 2022-08-01T23:04:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 -- start start 2022-08-01T23:04:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f240b13f0 0x7f8f240afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8f240b1920 0x7f8f240b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240b0740 con 0x7f8f240b2710 2022-08-01T23:04:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f240adf10 con 0x7f8f240b13f0 2022-08-01T23:04:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.681+0000 7f8f2359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f240ae1b0 con 0x7f8f240b1920 2022-08-01T23:04:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f240b13f0 0x7f8f240afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f240b13f0 0x7f8f240afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48228/0 (socket says 172.21.15.32:48228) 2022-08-01T23:04:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f240b2710 0x7f8f240abca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35768/0 (socket says 172.21.15.32:35768) 2022-08-01T23:04:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 -- 172.21.15.32:0/3967103611 learned_addr learned my addr 172.21.15.32:0/3967103611 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f22d9d700 1 --2- 172.21.15.32:0/3967103611 >> v2:172.21.15.55:3300/0 conn(0x7f8f240b1920 0x7f8f240b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 -- 172.21.15.32:0/3967103611 >> v2:172.21.15.55:3300/0 conn(0x7f8f240b1920 msgr2=0x7f8f240b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 --2- 172.21.15.32:0/3967103611 >> v2:172.21.15.55:3300/0 conn(0x7f8f240b1920 0x7f8f240b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 -- 172.21.15.32:0/3967103611 >> v1:172.21.15.32:6789/0 conn(0x7f8f240b2710 legacy=0x7f8f240abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 -- 172.21.15.32:0/3967103611 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240a9520 con 0x7f8f240b13f0 2022-08-01T23:04:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f21d9b700 1 --2- 172.21.15.32:0/3967103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b13f0 0x7f8f240afd60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f8f0c002f70 tx=0x7f8f0c006eb0).ready entity=mon.0 client_cookie=2d558d29e1f7fdf1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2159a700 1 -- 172.21.15.32:0/3967103611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0c00d5c0 con 0x7f8f240b13f0 2022-08-01T23:04:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2159a700 1 -- 172.21.15.32:0/3967103611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c000e90 con 0x7f8f240b13f0 2022-08-01T23:04:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2159a700 1 -- 172.21.15.32:0/3967103611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0c00d950 con 0x7f8f240b13f0 2022-08-01T23:04:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 -- 172.21.15.32:0/3967103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b13f0 msgr2=0x7f8f240afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 --2- 172.21.15.32:0/3967103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b13f0 0x7f8f240afd60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f8f0c002f70 tx=0x7f8f0c006eb0).stop 2022-08-01T23:04:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 -- 172.21.15.32:0/3967103611 shutdown_connections 2022-08-01T23:04:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 --2- 172.21.15.32:0/3967103611 >> v2:172.21.15.55:3300/0 conn(0x7f8f240b1920 0x7f8f240b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 --2- 172.21.15.32:0/3967103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b13f0 0x7f8f240afd60 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 -- 172.21.15.32:0/3967103611 >> 172.21.15.32:0/3967103611 conn(0x7f8f24060310 msgr2=0x7f8f240b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 -- 172.21.15.32:0/3967103611 shutdown_connections 2022-08-01T23:04:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.685+0000 7f8f2359e700 1 -- 172.21.15.32:0/3967103611 wait complete. 2022-08-01T23:04:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 Processor -- start 2022-08-01T23:04:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- start start 2022-08-01T23:04:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b13f0 0x7f8f240abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 0x7f8f240aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b2710 0x7f8f240ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2414edc0 con 0x7f8f240b1920 2022-08-01T23:04:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240665c0 con 0x7f8f240b13f0 2022-08-01T23:04:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24066860 con 0x7f8f240b2710 2022-08-01T23:04:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 0x7f8f240aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 0x7f8f240aabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48234/0 (socket says 172.21.15.32:48234) 2022-08-01T23:04:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 -- 172.21.15.32:0/1893001487 learned_addr learned my addr 172.21.15.32:0/1893001487 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2259c700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b13f0 0x7f8f240abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f22d9d700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b2710 0x7f8f240ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 -- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b2710 msgr2=0x7f8f240ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b2710 0x7f8f240ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 -- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b13f0 msgr2=0x7f8f240abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b13f0 0x7f8f240abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 -- 172.21.15.32:0/1893001487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c00b1e0 con 0x7f8f240b1920 2022-08-01T23:04:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f21d9b700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 0x7f8f240aabe0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f8f0c002f70 tx=0x7f8f0c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0c011860 con 0x7f8f240b1920 2022-08-01T23:04:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0119c0 con 0x7f8f240b1920 2022-08-01T23:04:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f24066b00 con 0x7f8f240b1920 2022-08-01T23:04:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0c011d20 con 0x7f8f240b1920 2022-08-01T23:04:22.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f24066fa0 con 0x7f8f240b1920 2022-08-01T23:04:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.689+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f240574e0 con 0x7f8f240b1920 2022-08-01T23:04:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.693+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8f0c01a040 con 0x7f8f240b1920 2022-08-01T23:04:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.693+0000 7f8f1b7fe700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f08053a70 0x7f8f08055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.693+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f8f0c06d140 con 0x7f8f240b1920 2022-08-01T23:04:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.693+0000 7f8f2259c700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f08053a70 0x7f8f08055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.693+0000 7f8f2259c700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f08053a70 0x7f8f08055f30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f8f1400b000 tx=0x7f8f14007160).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.697+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f0c043d20 con 0x7f8f240b1920 2022-08-01T23:04:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:22.857+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8f24067860 con 0x7f8f240b1920 2022-08-01T23:04:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.253+0000 7f8f1b7fe700 1 -- 172.21.15.32:0/1893001487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f0c047340 con 0x7f8f240b1920 2022-08-01T23:04:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f08053a70 msgr2=0x7f8f08055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f08053a70 0x7f8f08055f30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f8f1400b000 tx=0x7f8f14007160).stop 2022-08-01T23:04:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 msgr2=0x7f8f240aabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 0x7f8f240aabe0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f8f0c002f70 tx=0x7f8f0c015000).stop 2022-08-01T23:04:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 shutdown_connections 2022-08-01T23:04:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f08053a70 0x7f8f08055f30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b2710 0x7f8f240ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b1920 0x7f8f240aabe0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 --2- 172.21.15.32:0/1893001487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b13f0 0x7f8f240abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 >> 172.21.15.32:0/1893001487 conn(0x7f8f24060310 msgr2=0x7f8f2413fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:23.266 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 shutdown_connections 2022-08-01T23:04:23.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.257+0000 7f8f2359e700 1 -- 172.21.15.32:0/1893001487 wait complete. 2022-08-01T23:04:23.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:04:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.389+0000 7f174d04f700 1 Processor -- start 2022-08-01T23:04:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.389+0000 7f174d04f700 1 -- start start 2022-08-01T23:04:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f17480b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f174805c6f0 0x7f17480a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:23.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17480aa630 con 0x7f17480af550 2022-08-01T23:04:23.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17480b15e0 con 0x7f174805c6f0 2022-08-01T23:04:23.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17480af930 con 0x7f1748060330 2022-08-01T23:04:23.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f1747fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17480af550 0x7f17481445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35782/0 (socket says 172.21.15.32:35782) 2022-08-01T23:04:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174c84e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f174805c6f0 0x7f17480a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f1747fff700 1 -- 172.21.15.32:0/1669893180 learned_addr learned my addr 172.21.15.32:0/1669893180 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:23.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f173f7fe700 1 --2- 172.21.15.32:0/1669893180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f17480b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174c84e700 1 -- 172.21.15.32:0/1669893180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 msgr2=0x7f17480b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:23.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174c84e700 1 --2- 172.21.15.32:0/1669893180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f17480b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174c84e700 1 -- 172.21.15.32:0/1669893180 >> v1:172.21.15.32:6789/0 conn(0x7f17480af550 legacy=0x7f17481445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f17477fe700 1 -- 172.21.15.32:0/1669893180 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330035599 0 0) 0x7f17480aa630 con 0x7f17480af550 2022-08-01T23:04:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174c84e700 1 -- 172.21.15.32:0/1669893180 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174805fd10 con 0x7f174805c6f0 2022-08-01T23:04:23.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174c84e700 1 --2- 172.21.15.32:0/1669893180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f174805c6f0 0x7f17480a8d80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f17400032f0 tx=0x7f17400091d0).ready entity=mon.0 client_cookie=86aa9c773b647513 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:23.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f17477fe700 1 -- 172.21.15.32:0/1669893180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174000b6c0 con 0x7f174805c6f0 2022-08-01T23:04:23.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f17477fe700 1 -- 172.21.15.32:0/1669893180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174000b820 con 0x7f174805c6f0 2022-08-01T23:04:23.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f17477fe700 1 -- 172.21.15.32:0/1669893180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174000b990 con 0x7f174805c6f0 2022-08-01T23:04:23.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- 172.21.15.32:0/1669893180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f174805c6f0 msgr2=0x7f17480a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:23.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 --2- 172.21.15.32:0/1669893180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f174805c6f0 0x7f17480a8d80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f17400032f0 tx=0x7f17400091d0).stop 2022-08-01T23:04:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- 172.21.15.32:0/1669893180 shutdown_connections 2022-08-01T23:04:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 --2- 172.21.15.32:0/1669893180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f174805c6f0 0x7f17480a8d80 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 --2- 172.21.15.32:0/1669893180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f17480b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- 172.21.15.32:0/1669893180 >> 172.21.15.32:0/1669893180 conn(0x7f17480612d0 msgr2=0x7f1748062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- 172.21.15.32:0/1669893180 shutdown_connections 2022-08-01T23:04:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.393+0000 7f174d04f700 1 -- 172.21.15.32:0/1669893180 wait complete. 2022-08-01T23:04:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 Processor -- start 2022-08-01T23:04:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 -- start start 2022-08-01T23:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174805c6f0 0x7f1748142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f174813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 0x7f174813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748143f80 con 0x7f17480af550 2022-08-01T23:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17481410e0 con 0x7f174805c6f0 2022-08-01T23:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1748142410 con 0x7f1748060330 2022-08-01T23:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 0x7f174813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f173f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f174813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 0x7f174813df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48250/0 (socket says 172.21.15.32:48250) 2022-08-01T23:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 -- 172.21.15.32:0/2181738943 learned_addr learned my addr 172.21.15.32:0/2181738943 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f1747fff700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174805c6f0 0x7f1748142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 -- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 msgr2=0x7f174813da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f174813da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 -- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174805c6f0 msgr2=0x7f1748142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174805c6f0 0x7f1748142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 -- 172.21.15.32:0/2181738943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740009470 con 0x7f17480af550 2022-08-01T23:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174c84e700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 0x7f174813df00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f17480672c0 tx=0x7f1740016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174000b6c0 con 0x7f17480af550 2022-08-01T23:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740021630 con 0x7f17480af550 2022-08-01T23:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174805f840 con 0x7f17480af550 2022-08-01T23:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1740021990 con 0x7f17480af550 2022-08-01T23:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.397+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748141310 con 0x7f17480af550 2022-08-01T23:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.401+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1748142810 con 0x7f17480af550 2022-08-01T23:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.401+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f174000f070 con 0x7f17480af550 2022-08-01T23:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.401+0000 7f1745ffb700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f172c053b00 0x7f172c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.401+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f174006d1f0 con 0x7f17480af550 2022-08-01T23:04:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.401+0000 7f1747fff700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f172c053b00 0x7f172c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.401+0000 7f1747fff700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f172c053b00 0x7f172c055fc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f173800d000 tx=0x7f1738007c20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.409+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1740043ee0 con 0x7f17480af550 2022-08-01T23:04:23.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:23.537+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f174805fa70 con 0x7f17480af550 2022-08-01T23:04:24.271 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.264+0000 7f1745ffb700 1 -- 172.21.15.32:0/2181738943 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1740015020 con 0x7f17480af550 2022-08-01T23:04:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f172c053b00 msgr2=0x7f172c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f172c053b00 0x7f172c055fc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f173800d000 tx=0x7f1738007c20).stop 2022-08-01T23:04:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 msgr2=0x7f174813df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 0x7f174813df00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f17480672c0 tx=0x7f1740016000).stop 2022-08-01T23:04:24.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 shutdown_connections 2022-08-01T23:04:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f172c053b00 0x7f172c055fc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17480af550 0x7f174813df00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1748060330 0x7f174813da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 --2- 172.21.15.32:0/2181738943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174805c6f0 0x7f1748142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 >> 172.21.15.32:0/2181738943 conn(0x7f17480612d0 msgr2=0x7f17480aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.268+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 shutdown_connections 2022-08-01T23:04:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.272+0000 7f174d04f700 1 -- 172.21.15.32:0/2181738943 wait complete. 2022-08-01T23:04:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:04:24.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:04:24.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 Processor -- start 2022-08-01T23:04:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 -- start start 2022-08-01T23:04:24.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c300b6710 0x7f4c300b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:24.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c300b4550 0x7f4c300b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:24.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c30144240 con 0x7f4c30061ec0 2022-08-01T23:04:24.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c300b6fe0 con 0x7f4c300b6710 2022-08-01T23:04:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c351a0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c30143d00 con 0x7f4c300b4550 2022-08-01T23:04:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c2ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c300b6710 0x7f4c300b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c2ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c300b6710 0x7f4c300b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48254/0 (socket says 172.21.15.32:48254) 2022-08-01T23:04:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c2ed9d700 1 -- 172.21.15.32:0/3104588899 learned_addr learned my addr 172.21.15.32:0/3104588899 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.396+0000 7f4c2659c700 1 --2- 172.21.15.32:0/3104588899 >> v2:172.21.15.55:3300/0 conn(0x7f4c300b4550 0x7f4c300b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2ed9d700 1 -- 172.21.15.32:0/3104588899 >> v2:172.21.15.55:3300/0 conn(0x7f4c300b4550 msgr2=0x7f4c300b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2ed9d700 1 --2- 172.21.15.32:0/3104588899 >> v2:172.21.15.55:3300/0 conn(0x7f4c300b4550 0x7f4c300b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2ed9d700 1 -- 172.21.15.32:0/3104588899 >> v1:172.21.15.55:6789/0 conn(0x7f4c30061ec0 legacy=0x7f4c30146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2ed9d700 1 -- 172.21.15.32:0/3104588899 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30143fa0 con 0x7f4c300b6710 2022-08-01T23:04:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2ed9d700 1 --2- 172.21.15.32:0/3104588899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c300b5d80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f4c301432b0 tx=0x7f4c20006eb0).ready entity=mon.0 client_cookie=3a42df1c0f809127 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2e59c700 1 -- 172.21.15.32:0/3104588899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c2000d610 con 0x7f4c300b6710 2022-08-01T23:04:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2e59c700 1 -- 172.21.15.32:0/3104588899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20000e90 con 0x7f4c300b6710 2022-08-01T23:04:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c2e59c700 1 -- 172.21.15.32:0/3104588899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c2000d9a0 con 0x7f4c300b6710 2022-08-01T23:04:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 -- 172.21.15.32:0/3104588899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 msgr2=0x7f4c300b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 --2- 172.21.15.32:0/3104588899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c300b5d80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f4c301432b0 tx=0x7f4c20006eb0).stop 2022-08-01T23:04:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 -- 172.21.15.32:0/3104588899 shutdown_connections 2022-08-01T23:04:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 --2- 172.21.15.32:0/3104588899 >> v2:172.21.15.55:3300/0 conn(0x7f4c300b4550 0x7f4c300b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 --2- 172.21.15.32:0/3104588899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c300b5d80 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 -- 172.21.15.32:0/3104588899 >> 172.21.15.32:0/3104588899 conn(0x7f4c3005c620 msgr2=0x7f4c3005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 -- 172.21.15.32:0/3104588899 shutdown_connections 2022-08-01T23:04:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.400+0000 7f4c351a0700 1 -- 172.21.15.32:0/3104588899 wait complete. 2022-08-01T23:04:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 Processor -- start 2022-08-01T23:04:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 -- start start 2022-08-01T23:04:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c30061ec0 0x7f4c30125270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c300b4550 0x7f4c30128e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c30126330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c30126d90 con 0x7f4c300b6710 2022-08-01T23:04:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c30125930 con 0x7f4c30061ec0 2022-08-01T23:04:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c351a0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c30125bd0 con 0x7f4c300b4550 2022-08-01T23:04:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c30126330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c30126330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48264/0 (socket says 172.21.15.32:48264) 2022-08-01T23:04:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 -- 172.21.15.32:0/2721239514 learned_addr learned my addr 172.21.15.32:0/2721239514 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2659c700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c300b4550 0x7f4c30128e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2ed9d700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c30061ec0 0x7f4c30125270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 -- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c300b4550 msgr2=0x7f4c30128e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c300b4550 0x7f4c30128e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 -- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c30061ec0 msgr2=0x7f4c30125270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c30061ec0 0x7f4c30125270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.404+0000 7f4c2f59e700 1 -- 172.21.15.32:0/2721239514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2000b1e0 con 0x7f4c300b6710 2022-08-01T23:04:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2f59e700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c30126330 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f4c300b1e10 tx=0x7f4c280013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c28006aa0 con 0x7f4c300b6710 2022-08-01T23:04:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28006c00 con 0x7f4c300b6710 2022-08-01T23:04:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c30125e70 con 0x7f4c300b6710 2022-08-01T23:04:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c28007030 con 0x7f4c300b6710 2022-08-01T23:04:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3012c760 con 0x7f4c300b6710 2022-08-01T23:04:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c30061780 con 0x7f4c300b6710 2022-08-01T23:04:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c28010030 con 0x7f4c300b6710 2022-08-01T23:04:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2cd99700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c14053a70 0x7f4c14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f4c2806bcb0 con 0x7f4c300b6710 2022-08-01T23:04:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.408+0000 7f4c2ed9d700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c14053a70 0x7f4c14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.412+0000 7f4c2ed9d700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c14053a70 0x7f4c14055f30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f4c2001c000 tx=0x7f4c20015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:24.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.416+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c28042bd0 con 0x7f4c300b6710 2022-08-01T23:04:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:24.544+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c300574e0 con 0x7f4c300b6710 2022-08-01T23:04:24.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:24.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:24.626 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:24.628 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.288+0000 7f4c2cd99700 1 -- 172.21.15.32:0/2721239514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4c280461f0 con 0x7f4c300b6710 2022-08-01T23:04:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c14053a70 msgr2=0x7f4c14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c14053a70 0x7f4c14055f30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f4c2001c000 tx=0x7f4c20015000).stop 2022-08-01T23:04:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 msgr2=0x7f4c30126330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c30126330 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f4c300b1e10 tx=0x7f4c280013a0).stop 2022-08-01T23:04:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 shutdown_connections 2022-08-01T23:04:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4c14053a70 0x7f4c14055f30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c300b6710 0x7f4c30126330 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4c300b4550 0x7f4c30128e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 --2- 172.21.15.32:0/2721239514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c30061ec0 0x7f4c30125270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 >> 172.21.15.32:0/2721239514 conn(0x7f4c3005c620 msgr2=0x7f4c301272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 shutdown_connections 2022-08-01T23:04:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.292+0000 7f4c351a0700 1 -- 172.21.15.32:0/2721239514 wait complete. 2022-08-01T23:04:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:04:25.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:04:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 Processor -- start 2022-08-01T23:04:25.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 -- start start 2022-08-01T23:04:25.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbde40b4310 0x7fbde4140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbde40b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde4143cf0 con 0x7fbde4146ae0 2022-08-01T23:04:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde41476a0 con 0x7fbde40b4310 2022-08-01T23:04:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde890f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4143f90 con 0x7fbde40b4820 2022-08-01T23:04:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbde4146ae0 0x7fbde41403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35810/0 (socket says 172.21.15.32:35810) 2022-08-01T23:04:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde259c700 1 -- 172.21.15.32:0/2960533900 learned_addr learned my addr 172.21.15.32:0/2960533900 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde1d9b700 1 --2- 172.21.15.32:0/2960533900 >> v2:172.21.15.55:3300/0 conn(0x7fbde40b4310 0x7fbde4140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde2d9d700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbde40b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667412361 0 0) 0x7fbde4143cf0 con 0x7fbde4146ae0 2022-08-01T23:04:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc8003260 con 0x7fbde4146ae0 2022-08-01T23:04:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2312018970 0 0) 0x7fbdc8003260 con 0x7fbde4146ae0 2022-08-01T23:04:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 msgr2=0x7fbde40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.420+0000 7fbde159a700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbde40b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 >> v2:172.21.15.55:3300/0 conn(0x7fbde40b4310 msgr2=0x7fbde4140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 --2- 172.21.15.32:0/2960533900 >> v2:172.21.15.55:3300/0 conn(0x7fbde40b4310 0x7fbde4140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40b0e20 con 0x7fbde4146ae0 2022-08-01T23:04:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fbdd8001660 con 0x7fbde4146ae0 2022-08-01T23:04:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde1d9b700 1 --2- 172.21.15.32:0/2960533900 >> v2:172.21.15.55:3300/0 conn(0x7fbde40b4310 0x7fbde4140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 >> v1:172.21.15.32:6789/0 conn(0x7fbde4146ae0 legacy=0x7fbde41403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbdc8003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc8003c80 0x7fbdc8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbdc8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde41476a0 con 0x7fbdc80049f0 2022-08-01T23:04:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4143f90 con 0x7fbdc8003c80 2022-08-01T23:04:25.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800b750 con 0x7fbde40b4820 2022-08-01T23:04:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbdc8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde1d9b700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc8003c80 0x7fbdc8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde259c700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbdc8003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 -- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 msgr2=0x7fbdc8003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbdc8003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 -- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc8003c80 msgr2=0x7fbdc8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc8003c80 0x7fbdc8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 -- 172.21.15.32:0/2960533900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8003260 con 0x7fbdc80049f0 2022-08-01T23:04:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde259c700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40b4820 0x7fbdc8003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbdc8004e70 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7fbdc80054d0 tx=0x7fbddc0091d0).ready entity=mon.0 client_cookie=7ca87442bc31ac7b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbddc00e7e0 con 0x7fbdc80049f0 2022-08-01T23:04:25.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc00e940 con 0x7fbdc80049f0 2022-08-01T23:04:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde159a700 1 -- 172.21.15.32:0/2960533900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbddc00eca0 con 0x7fbdc80049f0 2022-08-01T23:04:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 -- 172.21.15.32:0/2960533900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 msgr2=0x7fbdc8004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 --2- 172.21.15.32:0/2960533900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbdc8004e70 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7fbdc80054d0 tx=0x7fbddc0091d0).stop 2022-08-01T23:04:25.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde2d9d700 1 -- 172.21.15.32:0/2960533900 reap_dead start 2022-08-01T23:04:25.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 -- 172.21.15.32:0/2960533900 shutdown_connections 2022-08-01T23:04:25.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 -- 172.21.15.32:0/2960533900 >> 172.21.15.32:0/2960533900 conn(0x7fbde405c620 msgr2=0x7fbde405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:25.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 -- 172.21.15.32:0/2960533900 shutdown_connections 2022-08-01T23:04:25.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 -- 172.21.15.32:0/2960533900 wait complete. 2022-08-01T23:04:25.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 Processor -- start 2022-08-01T23:04:25.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.424+0000 7fbde890f700 1 -- start start 2022-08-01T23:04:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbde40ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbde40ae170 0x7fbde40b9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40ae5f0 0x7fbde40b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b8ed0 con 0x7fbdc80049f0 2022-08-01T23:04:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b84f0 con 0x7fbde40ae170 2022-08-01T23:04:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b8790 con 0x7fbde40ae5f0 2022-08-01T23:04:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbde40ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbde40ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48280/0 (socket says 172.21.15.32:48280) 2022-08-01T23:04:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 -- 172.21.15.32:0/4010656862 learned_addr learned my addr 172.21.15.32:0/4010656862 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde2d9d700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40ae5f0 0x7fbde40b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde1d9b700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbde40ae170 0x7fbde40b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 -- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40ae5f0 msgr2=0x7fbde40b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40ae5f0 0x7fbde40b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 -- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbde40ae170 msgr2=0x7fbde40b9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbde40ae170 0x7fbde40b9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 -- 172.21.15.32:0/4010656862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40b8a30 con 0x7fbdc80049f0 2022-08-01T23:04:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde259c700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbde40ad980 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7fbdd800b000 tx=0x7fbde40b2240).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd8015630 con 0x7fbdc80049f0 2022-08-01T23:04:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde40ade70 con 0x7fbdc80049f0 2022-08-01T23:04:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd8010040 con 0x7fbdc80049f0 2022-08-01T23:04:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde4152410 con 0x7fbdc80049f0 2022-08-01T23:04:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.428+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd8015790 con 0x7fbdc80049f0 2022-08-01T23:04:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.432+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde40574e0 con 0x7fbdc80049f0 2022-08-01T23:04:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.432+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbdd80103c0 con 0x7fbdc80049f0 2022-08-01T23:04:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.432+0000 7fbdd37fe700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbdc0053980 0x7fbdc0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.432+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbdd806b890 con 0x7fbdc80049f0 2022-08-01T23:04:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.432+0000 7fbde1d9b700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbdc0053980 0x7fbdc0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.432+0000 7fbde1d9b700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbdc0053980 0x7fbdc0055e40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fbdd400b000 tx=0x7fbdd4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:25.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.440+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbdd8042830 con 0x7fbdc80049f0 2022-08-01T23:04:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.572+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbde40b8bc0 con 0x7fbdc80049f0 2022-08-01T23:04:25.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.576+0000 7fbdd37fe700 1 -- 172.21.15.32:0/4010656862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdd8045e50 con 0x7fbdc80049f0 2022-08-01T23:04:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbdc0053980 msgr2=0x7fbdc0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbdc0053980 0x7fbdc0055e40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fbdd400b000 tx=0x7fbdd4008040).stop 2022-08-01T23:04:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 msgr2=0x7fbde40ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbde40ad980 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7fbdd800b000 tx=0x7fbde40b2240).stop 2022-08-01T23:04:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 shutdown_connections 2022-08-01T23:04:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbdc0053980 0x7fbdc0055e40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbde40ae5f0 0x7fbde40b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbde40ae170 0x7fbde40b9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 --2- 172.21.15.32:0/4010656862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80049f0 0x7fbde40ad980 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 >> 172.21.15.32:0/4010656862 conn(0x7fbde405c620 msgr2=0x7fbde413b250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 shutdown_connections 2022-08-01T23:04:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.580+0000 7fbde890f700 1 -- 172.21.15.32:0/4010656862 wait complete. 2022-08-01T23:04:25.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-08-01T23:04:25.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 Processor -- start 2022-08-01T23:04:25.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- start start 2022-08-01T23:04:25.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b4550 0x7ffa040b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa04144240 con 0x7ffa040b6710 2022-08-01T23:04:25.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa04143d00 con 0x7ffa040b6af0 2022-08-01T23:04:25.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04143fa0 con 0x7ffa040b4550 2022-08-01T23:04:25.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa09587700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa040b6710 0x7ffa04061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35830/0 (socket says 172.21.15.32:35830) 2022-08-01T23:04:25.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa09587700 1 -- 172.21.15.32:0/1709266402 learned_addr learned my addr 172.21.15.32:0/1709266402 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:25.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa08d86700 1 --2- 172.21.15.32:0/1709266402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b4550 0x7ffa040b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa08d86700 1 -- 172.21.15.32:0/1709266402 >> v1:172.21.15.55:6789/0 conn(0x7ffa040b6af0 legacy=0x7ffa0412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa08d86700 1 -- 172.21.15.32:0/1709266402 >> v1:172.21.15.32:6789/0 conn(0x7ffa040b6710 legacy=0x7ffa04061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa08d86700 1 -- 172.21.15.32:0/1709266402 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04061780 con 0x7ffa040b4550 2022-08-01T23:04:25.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa08d86700 1 --2- 172.21.15.32:0/1709266402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b4550 0x7ffa040b5d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ff9f40032f0 tx=0x7ff9f40091d0).ready entity=mon.2 client_cookie=2a208636d2155dc3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:25.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa03fff700 1 -- 172.21.15.32:0/1709266402 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9f400b8a0 con 0x7ffa040b4550 2022-08-01T23:04:25.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa03fff700 1 -- 172.21.15.32:0/1709266402 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f400ba00 con 0x7ffa040b4550 2022-08-01T23:04:25.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa03fff700 1 -- 172.21.15.32:0/1709266402 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9f400bb70 con 0x7ffa040b4550 2022-08-01T23:04:25.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/1709266402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b4550 msgr2=0x7ffa040b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/1709266402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b4550 0x7ffa040b5d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ff9f40032f0 tx=0x7ff9f40091d0).stop 2022-08-01T23:04:25.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/1709266402 shutdown_connections 2022-08-01T23:04:25.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/1709266402 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b4550 0x7ffa040b5d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.696+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/1709266402 >> 172.21.15.32:0/1709266402 conn(0x7ffa0405c620 msgr2=0x7ffa0405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:25.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.700+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/1709266402 shutdown_connections 2022-08-01T23:04:25.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.700+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/1709266402 wait complete. 2022-08-01T23:04:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.700+0000 7ffa0b7eb700 1 Processor -- start 2022-08-01T23:04:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.700+0000 7ffa0b7eb700 1 -- start start 2022-08-01T23:04:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa040b4550 0x7ffa04127ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b6710 0x7ffa04142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 0x7ffa04155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04148700 con 0x7ffa040b6af0 2022-08-01T23:04:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04147420 con 0x7ffa040b4550 2022-08-01T23:04:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04142f50 con 0x7ffa040b6710 2022-08-01T23:04:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 0x7ffa04155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09587700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa040b4550 0x7ffa04127ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 0x7ffa04155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48298/0 (socket says 172.21.15.32:48298) 2022-08-01T23:04:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 -- 172.21.15.32:0/4008757663 learned_addr learned my addr 172.21.15.32:0/4008757663 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa08d86700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b6710 0x7ffa04142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 -- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b6710 msgr2=0x7ffa04142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b6710 0x7ffa04142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 -- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa040b4550 msgr2=0x7ffa04127ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa040b4550 0x7ffa04127ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 -- 172.21.15.32:0/4008757663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f4009470 con 0x7ffa040b6af0 2022-08-01T23:04:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09587700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa040b4550 0x7ffa04127ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa09d88700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 0x7ffa04155b10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7ffa04143090 tx=0x7ff9fc007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9fc007030 con 0x7ffa040b6af0 2022-08-01T23:04:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc007190 con 0x7ffa040b6af0 2022-08-01T23:04:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa041488f0 con 0x7ffa040b6af0 2022-08-01T23:04:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9fc006510 con 0x7ffa040b6af0 2022-08-01T23:04:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa04148e30 con 0x7ffa040b6af0 2022-08-01T23:04:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.704+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa04061a20 con 0x7ffa040b6af0 2022-08-01T23:04:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.708+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff9fc010030 con 0x7ffa040b6af0 2022-08-01T23:04:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.708+0000 7ffa027fc700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9e80539e0 0x7ff9e8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.708+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff9fc06b930 con 0x7ffa040b6af0 2022-08-01T23:04:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.708+0000 7ffa09587700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9e80539e0 0x7ff9e8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.708+0000 7ffa09587700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9e80539e0 0x7ff9e8055ea0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7ff9ec00d000 tx=0x7ff9ec007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.712+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff9fc042930 con 0x7ffa040b6af0 2022-08-01T23:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:25.880+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7ffa040574e0 con 0x7ffa040b6af0 2022-08-01T23:04:26.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.292+0000 7ffa027fc700 1 -- 172.21.15.32:0/4008757663 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v391) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9fc045f50 con 0x7ffa040b6af0 2022-08-01T23:04:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9e80539e0 msgr2=0x7ff9e8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:26.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9e80539e0 0x7ff9e8055ea0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7ff9ec00d000 tx=0x7ff9ec007b80).stop 2022-08-01T23:04:26.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 msgr2=0x7ffa04155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:26.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 0x7ffa04155b10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7ffa04143090 tx=0x7ff9fc007c20).stop 2022-08-01T23:04:26.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 shutdown_connections 2022-08-01T23:04:26.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9e80539e0 0x7ff9e8055ea0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa040b6af0 0x7ffa04155b10 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa040b6710 0x7ffa04142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 --2- 172.21.15.32:0/4008757663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa040b4550 0x7ffa04127ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 >> 172.21.15.32:0/4008757663 conn(0x7ffa0405c620 msgr2=0x7ffa04061f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:26.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 shutdown_connections 2022-08-01T23:04:26.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.296+0000 7ffa0b7eb700 1 -- 172.21.15.32:0/4008757663 wait complete. 2022-08-01T23:04:26.318 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:04:26.319 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:04:26.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-08-01T23:04:26.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 Processor -- start 2022-08-01T23:04:26.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 -- start start 2022-08-01T23:04:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff6f40b1f40 0x7ff6f40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 0x7ff6f40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:26.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40b04e0 con 0x7ff6f40a7250 2022-08-01T23:04:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f40a9250 con 0x7ff6f40b1f40 2022-08-01T23:04:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fbebb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b0780 con 0x7ff6f4148530 2022-08-01T23:04:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 0x7ff6f40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9c57700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff6f40b1f40 0x7ff6f40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:26.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 0x7ff6f40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60040/0 (socket says 172.21.15.32:60040) 2022-08-01T23:04:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6fa458700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff6f40a7250 0x7ff6f41448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:56754/0 (socket says 172.21.15.32:56754) 2022-08-01T23:04:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 -- 172.21.15.32:0/1374917615 learned_addr learned my addr 172.21.15.32:0/1374917615 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 -- 172.21.15.32:0/1374917615 >> v2:172.21.15.55:3300/0 conn(0x7ff6f40b1f40 msgr2=0x7ff6f40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:26.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 --2- 172.21.15.32:0/1374917615 >> v2:172.21.15.55:3300/0 conn(0x7ff6f40b1f40 0x7ff6f40b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 -- 172.21.15.32:0/1374917615 >> v1:172.21.15.55:6789/0 conn(0x7ff6f40a7250 legacy=0x7ff6f41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:26.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 -- 172.21.15.32:0/1374917615 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40b0a20 con 0x7ff6f4148530 2022-08-01T23:04:26.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f9456700 1 --2- 172.21.15.32:0/1374917615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 0x7ff6f40b2320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7ff6e40032f0 tx=0x7ff6e4009290).ready entity=mon.2 client_cookie=fa5a3ed18c3f2ffe server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:26.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f8c55700 1 -- 172.21.15.32:0/1374917615 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e400b8a0 con 0x7ff6f4148530 2022-08-01T23:04:26.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f8c55700 1 -- 172.21.15.32:0/1374917615 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400ba00 con 0x7ff6f4148530 2022-08-01T23:04:26.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.424+0000 7ff6f8c55700 1 -- 172.21.15.32:0/1374917615 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e400bb70 con 0x7ff6f4148530 2022-08-01T23:04:26.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- 172.21.15.32:0/1374917615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 msgr2=0x7ff6f40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:26.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/1374917615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 0x7ff6f40b2320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7ff6e40032f0 tx=0x7ff6e4009290).stop 2022-08-01T23:04:26.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- 172.21.15.32:0/1374917615 shutdown_connections 2022-08-01T23:04:26.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/1374917615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f4148530 0x7ff6f40b2320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/1374917615 >> v2:172.21.15.55:3300/0 conn(0x7ff6f40b1f40 0x7ff6f40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- 172.21.15.32:0/1374917615 >> 172.21.15.32:0/1374917615 conn(0x7ff6f405f840 msgr2=0x7ff6f405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:26.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- 172.21.15.32:0/1374917615 shutdown_connections 2022-08-01T23:04:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- 172.21.15.32:0/1374917615 wait complete. 2022-08-01T23:04:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 Processor -- start 2022-08-01T23:04:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- start start 2022-08-01T23:04:26.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f40a7250 0x7ff6f414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f40b1f40 0x7ff6f4143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 0x7ff6f4143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40aaef0 con 0x7ff6f4148530 2022-08-01T23:04:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40ab190 con 0x7ff6f40b1f40 2022-08-01T23:04:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fbebb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40ab430 con 0x7ff6f40a7250 2022-08-01T23:04:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fa458700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 0x7ff6f4143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fa458700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 0x7ff6f4143f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48312/0 (socket says 172.21.15.32:48312) 2022-08-01T23:04:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6f9c57700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f40a7250 0x7ff6f414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6fa458700 1 -- 172.21.15.32:0/2066933708 learned_addr learned my addr 172.21.15.32:0/2066933708 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.428+0000 7ff6f9456700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f40b1f40 0x7ff6f4143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fa458700 1 -- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f40a7250 msgr2=0x7ff6f414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fa458700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f40a7250 0x7ff6f414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fa458700 1 -- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f40b1f40 msgr2=0x7ff6f4143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:26.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fa458700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f40b1f40 0x7ff6f4143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:26.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fa458700 1 -- 172.21.15.32:0/2066933708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4009530 con 0x7ff6f4148530 2022-08-01T23:04:26.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fa458700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 0x7ff6f4143f30 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7ff6f4061530 tx=0x7ff6ec007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:26.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6ec0070d0 con 0x7ff6f4148530 2022-08-01T23:04:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec007230 con 0x7ff6f4148530 2022-08-01T23:04:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f40a99c0 con 0x7ff6f4148530 2022-08-01T23:04:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6ec005740 con 0x7ff6f4148530 2022-08-01T23:04:26.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f4148d90 con 0x7ff6f4148530 2022-08-01T23:04:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f40ab5f0 con 0x7ff6f4148530 2022-08-01T23:04:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff6ec010070 con 0x7ff6f4148530 2022-08-01T23:04:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f2ffd700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff6d4053aa0 0x7ff6d4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff6ec06b1c0 con 0x7ff6f4148530 2022-08-01T23:04:26.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.432+0000 7ff6f9c57700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff6d4053aa0 0x7ff6d4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:26.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.436+0000 7ff6f9c57700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff6d4053aa0 0x7ff6d4055f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7ff6e800b000 tx=0x7ff6e8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:26.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.440+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff6ec042260 con 0x7ff6f4148530 2022-08-01T23:04:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:26.608+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7ff6f40b0cc0 con 0x7ff6f4148530 2022-08-01T23:04:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.300+0000 7ff6f2ffd700 1 -- 172.21.15.32:0/2066933708 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v392) v1 ==== 131+0+0 (secure 0 0 0) 0x7ff6ec045880 con 0x7ff6f4148530 2022-08-01T23:04:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff6d4053aa0 msgr2=0x7ff6d4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff6d4053aa0 0x7ff6d4055f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7ff6e800b000 tx=0x7ff6e8006df0).stop 2022-08-01T23:04:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 msgr2=0x7ff6f4143f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 0x7ff6f4143f30 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7ff6f4061530 tx=0x7ff6ec007b80).stop 2022-08-01T23:04:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 shutdown_connections 2022-08-01T23:04:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff6d4053aa0 0x7ff6d4055f60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f4148530 0x7ff6f4143f30 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f40b1f40 0x7ff6f4143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 --2- 172.21.15.32:0/2066933708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6f40a7250 0x7ff6f414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 >> 172.21.15.32:0/2066933708 conn(0x7ff6f405f840 msgr2=0x7ff6f40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 shutdown_connections 2022-08-01T23:04:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.304+0000 7ff6fbebb700 1 -- 172.21.15.32:0/2066933708 wait complete. 2022-08-01T23:04:27.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:04:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.440+0000 7f0fe3436700 1 Processor -- start 2022-08-01T23:04:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.440+0000 7f0fe3436700 1 -- start start 2022-08-01T23:04:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fdc0612f0 0x7f0fdc143b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc0671e0 con 0x7f0fdc0b2930 2022-08-01T23:04:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc065540 con 0x7f0fdc0b7840 2022-08-01T23:04:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fdc0b7c20 con 0x7f0fdc0612f0 2022-08-01T23:04:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe2c35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fdc0612f0 0x7f0fdc143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe1c33700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fdc0b2930 0x7f0fdc143400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35860/0 (socket says 172.21.15.32:35860) 2022-08-01T23:04:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe1c33700 1 -- 172.21.15.32:0/3037547157 learned_addr learned my addr 172.21.15.32:0/3037547157 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe2c35700 1 -- 172.21.15.32:0/3037547157 >> v1:172.21.15.55:6789/0 conn(0x7f0fdc0b7840 legacy=0x7f0fdc060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe2c35700 1 -- 172.21.15.32:0/3037547157 >> v1:172.21.15.32:6789/0 conn(0x7f0fdc0b2930 legacy=0x7f0fdc143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe2c35700 1 -- 172.21.15.32:0/3037547157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc066ce0 con 0x7f0fdc0612f0 2022-08-01T23:04:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe2c35700 1 --2- 172.21.15.32:0/3037547157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc143b10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f0fd40032f0 tx=0x7f0fd40091d0).ready entity=mon.0 client_cookie=2e9ccc3b35d5ed2a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe1432700 1 -- 172.21.15.32:0/3037547157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd4009ea0 con 0x7f0fdc0612f0 2022-08-01T23:04:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe1432700 1 -- 172.21.15.32:0/3037547157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd400b8a0 con 0x7f0fdc0612f0 2022-08-01T23:04:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe1432700 1 -- 172.21.15.32:0/3037547157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd400ba10 con 0x7f0fdc0612f0 2022-08-01T23:04:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- 172.21.15.32:0/3037547157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0612f0 msgr2=0x7f0fdc143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 --2- 172.21.15.32:0/3037547157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc143b10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f0fd40032f0 tx=0x7f0fd40091d0).stop 2022-08-01T23:04:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- 172.21.15.32:0/3037547157 shutdown_connections 2022-08-01T23:04:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 --2- 172.21.15.32:0/3037547157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc143b10 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- 172.21.15.32:0/3037547157 >> 172.21.15.32:0/3037547157 conn(0x7f0fdc05f840 msgr2=0x7f0fdc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- 172.21.15.32:0/3037547157 shutdown_connections 2022-08-01T23:04:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.444+0000 7f0fe3436700 1 -- 172.21.15.32:0/3037547157 wait complete. 2022-08-01T23:04:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 Processor -- start 2022-08-01T23:04:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 -- start start 2022-08-01T23:04:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 0x7f0fdc155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0fdc0b7840 0x7f0fdc156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc144df0 con 0x7f0fdc0b2930 2022-08-01T23:04:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0adb70 con 0x7f0fdc0612f0 2022-08-01T23:04:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc1484e0 con 0x7f0fdc0b7840 2022-08-01T23:04:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 0x7f0fdc155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe2c35700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0fdc0b7840 0x7f0fdc156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 0x7f0fdc155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48326/0 (socket says 172.21.15.32:48326) 2022-08-01T23:04:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe2434700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe2c35700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0fdc0b7840 0x7f0fdc156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60062/0 (socket says 172.21.15.32:60062) 2022-08-01T23:04:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 -- 172.21.15.32:0/4157385825 learned_addr learned my addr 172.21.15.32:0/4157385825 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 -- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0fdc0b7840 msgr2=0x7f0fdc156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0fdc0b7840 0x7f0fdc156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 -- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fdc0612f0 msgr2=0x7f0fdc148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 -- 172.21.15.32:0/4157385825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009470 con 0x7f0fdc0b2930 2022-08-01T23:04:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe1c33700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 0x7f0fdc155c70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f0fdc0643c0 tx=0x7f0fd8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd80070d0 con 0x7f0fdc0b2930 2022-08-01T23:04:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8007230 con 0x7f0fdc0b2930 2022-08-01T23:04:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd8005610 con 0x7f0fdc0b2930 2022-08-01T23:04:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc148780 con 0x7f0fdc0b2930 2022-08-01T23:04:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.448+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc156650 con 0x7f0fdc0b2930 2022-08-01T23:04:27.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.452+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc0574e0 con 0x7f0fdc0b2930 2022-08-01T23:04:27.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.452+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fd8010070 con 0x7f0fdc0b2930 2022-08-01T23:04:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.452+0000 7f0fcb7fe700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fbc053aa0 0x7f0fbc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.452+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0fd806b370 con 0x7f0fdc0b2930 2022-08-01T23:04:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.452+0000 7f0fe2434700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fbc053aa0 0x7f0fbc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.456+0000 7f0fe2434700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fbc053aa0 0x7f0fbc055f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0fcc00d000 tx=0x7f0fcc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:27.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.460+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fd8042300 con 0x7f0fdc0b2930 2022-08-01T23:04:27.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.584+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0fdc060330 con 0x7f0fdc0b2930 2022-08-01T23:04:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.592+0000 7f0fcb7fe700 1 -- 172.21.15.32:0/4157385825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0fd8018070 con 0x7f0fdc0b2930 2022-08-01T23:04:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fbc053aa0 msgr2=0x7f0fbc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fbc053aa0 0x7f0fbc055f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0fcc00d000 tx=0x7f0fcc009040).stop 2022-08-01T23:04:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 msgr2=0x7f0fdc155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 0x7f0fdc155c70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f0fdc0643c0 tx=0x7f0fd8007c40).stop 2022-08-01T23:04:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 shutdown_connections 2022-08-01T23:04:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fbc053aa0 0x7f0fbc055f60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0fdc0b7840 0x7f0fdc156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fdc0b2930 0x7f0fdc155c70 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 --2- 172.21.15.32:0/4157385825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fdc0612f0 0x7f0fdc148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 >> 172.21.15.32:0/4157385825 conn(0x7f0fdc05f840 msgr2=0x7f0fdc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 shutdown_connections 2022-08-01T23:04:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.596+0000 7f0fe3436700 1 -- 172.21.15.32:0/4157385825 wait complete. 2022-08-01T23:04:27.615 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:04:27.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:27.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:27.657 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:27.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:27.699 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:04:27.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:27.701 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:27.701 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:27.708 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:04:27.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:27.710 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:27.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:27.711 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:27.711 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:27.747 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:04:27.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:27.749 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:27.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:04:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66eb47d700 1 Processor -- start 2022-08-01T23:04:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66eb47d700 1 -- start start 2022-08-01T23:04:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66eb47d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66e40b2710 0x7f66e414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66eb47d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40a7250 con 0x7f66e40b01a0 2022-08-01T23:04:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66eb47d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40a7480 con 0x7f66e40a8c50 2022-08-01T23:04:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66eb47d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e40a7720 con 0x7f66e40b2710 2022-08-01T23:04:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66e9c7a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66e40b01a0 0x7f66e40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35872/0 (socket says 172.21.15.32:35872) 2022-08-01T23:04:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66e9c7a700 1 -- 172.21.15.32:0/2736860752 learned_addr learned my addr 172.21.15.32:0/2736860752 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66ea47b700 1 --2- 172.21.15.32:0/2736860752 >> v2:172.21.15.32:3300/0 conn(0x7f66e40b2710 0x7f66e414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66e9479700 1 -- 172.21.15.32:0/2736860752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039259622 0 0) 0x7f66e40a7250 con 0x7f66e40b01a0 2022-08-01T23:04:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66e9479700 1 -- 172.21.15.32:0/2736860752 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66cc003260 con 0x7f66e40b01a0 2022-08-01T23:04:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66ea47b700 1 -- 172.21.15.32:0/2736860752 >> v1:172.21.15.55:6789/0 conn(0x7f66e40a8c50 legacy=0x7f66e4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66e9479700 1 -- 172.21.15.32:0/2736860752 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778773950 0 0) 0x7f66e40a7480 con 0x7f66e40a8c50 2022-08-01T23:04:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66ea47b700 1 -- 172.21.15.32:0/2736860752 >> v1:172.21.15.32:6789/0 conn(0x7f66e40b01a0 legacy=0x7f66e40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66ea47b700 1 -- 172.21.15.32:0/2736860752 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40a79c0 con 0x7f66e40b2710 2022-08-01T23:04:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.856+0000 7f66ea47b700 1 --2- 172.21.15.32:0/2736860752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40b2710 0x7f66e414afd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f66e0001800 tx=0x7f66d4006df0).ready entity=mon.0 client_cookie=caa6f85c055e3ef4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66e9479700 1 -- 172.21.15.32:0/2736860752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e0001660 con 0x7f66e40b2710 2022-08-01T23:04:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66e9479700 1 -- 172.21.15.32:0/2736860752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d4000e90 con 0x7f66e40b2710 2022-08-01T23:04:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66e9479700 1 -- 172.21.15.32:0/2736860752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d4011400 con 0x7f66e40b2710 2022-08-01T23:04:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- 172.21.15.32:0/2736860752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40b2710 msgr2=0x7f66e414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 --2- 172.21.15.32:0/2736860752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40b2710 0x7f66e414afd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f66e0001800 tx=0x7f66d4006df0).stop 2022-08-01T23:04:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- 172.21.15.32:0/2736860752 shutdown_connections 2022-08-01T23:04:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 --2- 172.21.15.32:0/2736860752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40b2710 0x7f66e414afd0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- 172.21.15.32:0/2736860752 >> 172.21.15.32:0/2736860752 conn(0x7f66e4060310 msgr2=0x7f66e40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- 172.21.15.32:0/2736860752 shutdown_connections 2022-08-01T23:04:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- 172.21.15.32:0/2736860752 wait complete. 2022-08-01T23:04:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 Processor -- start 2022-08-01T23:04:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- start start 2022-08-01T23:04:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 0x7f66e4064ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66e40b01a0 0x7f66e40666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66e40b2710 0x7f66e4066b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4066270 con 0x7f66e40a8c50 2022-08-01T23:04:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40af1c0 con 0x7f66e40b01a0 2022-08-01T23:04:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66eb47d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40af460 con 0x7f66e40b2710 2022-08-01T23:04:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66ea47b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 0x7f66e4064ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66ea47b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 0x7f66e4064ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48334/0 (socket says 172.21.15.32:48334) 2022-08-01T23:04:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.860+0000 7f66ea47b700 1 -- 172.21.15.32:0/3831688497 learned_addr learned my addr 172.21.15.32:0/3831688497 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66eac7c700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66e40b2710 0x7f66e4066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66e9c7a700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66e40b01a0 0x7f66e40666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66ea47b700 1 -- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66e40b2710 msgr2=0x7f66e4066b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66ea47b700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66e40b2710 0x7f66e4066b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66ea47b700 1 -- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66e40b01a0 msgr2=0x7f66e40666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66ea47b700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66e40b01a0 0x7f66e40666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66ea47b700 1 -- 172.21.15.32:0/3831688497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d400b180 con 0x7f66e40a8c50 2022-08-01T23:04:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66ea47b700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 0x7f66e4064ee0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f66e0001800 tx=0x7f66e40a7610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d4015070 con 0x7f66e40a8c50 2022-08-01T23:04:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d400d960 con 0x7f66e40a8c50 2022-08-01T23:04:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e40af700 con 0x7f66e40a8c50 2022-08-01T23:04:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d400dcc0 con 0x7f66e40a8c50 2022-08-01T23:04:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e4159a70 con 0x7f66e40a8c50 2022-08-01T23:04:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e40663b0 con 0x7f66e40a8c50 2022-08-01T23:04:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f66d400f070 con 0x7f66e40a8c50 2022-08-01T23:04:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66d37fe700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66c4053b00 0x7f66c4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:27.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.864+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f66d406ca20 con 0x7f66e40a8c50 2022-08-01T23:04:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.868+0000 7f66e9c7a700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66c4053b00 0x7f66c4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.868+0000 7f66e9c7a700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66c4053b00 0x7f66c4055fc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f66e000b000 tx=0x7f66e00016d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.872+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f66d40439b0 con 0x7f66e40a8c50 2022-08-01T23:04:28.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.996+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f66e40574e0 con 0x7f66e40a8c50 2022-08-01T23:04:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:27.996+0000 7f66d37fe700 1 -- 172.21.15.32:0/3831688497 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+29227 (secure 0 0 0) 0x7f66d4046fd0 con 0x7f66e40a8c50 2022-08-01T23:04:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:28.005 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:28.005 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 132, 2022-08-01T23:04:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:04:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:04:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:04:25.279234+0000", 2022-08-01T23:04:28.006 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:04:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:04:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:04:28.007 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:04:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:04:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:04:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:04:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:28.008 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:04:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:04:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:04:28.009 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:04:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:04:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 24, 2022-08-01T23:04:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:04:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:04:28.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:04:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:04:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:04:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.011 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:04:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:04:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:04:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:28.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:28.013 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:04:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:04:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:04:28.014 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:04:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:04:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.016 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:04:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:28.017 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:28.018 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:28.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:28.020 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:28.021 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:28.022 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:28.023 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:28.024 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:28.025 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:04:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:04:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:04:28.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:04:28.027 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:04:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:04:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:28.028 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:28.029 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:28.030 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:28.031 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:28.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:04:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:28.033 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:28.034 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:28.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:28.036 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:28.037 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:28.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:28.039 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:28.040 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:04:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:28.041 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:04:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:28.042 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:04:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:04:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.043 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.044 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.044 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.044 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.044 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:04:28.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:04:28.048 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.049 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.049 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:04:28.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:04:28.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.052 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:04:28.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.054 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:04:28.054 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.055 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.055 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.056 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:04:28.056 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:04:28.057 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:28.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.058 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:04:28.058 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:04:28.058 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:04:28.059 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:04:28.059 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.059 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.059 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.060 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:28.060 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:04:28.061 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.061 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.061 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.061 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.062 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.062 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.062 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.062 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:28.062 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.063 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.063 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.064 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:04:28.064 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.065 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:04:28.065 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.066 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:04:28.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.068 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:04:28.068 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.069 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.070 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:04:28.070 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.071 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:04:28.071 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.072 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.072 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:04:28.073 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.074 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:04:28.074 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:28.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.075 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:04:28.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:04:28.076 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:04:28.076 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:04:28.076 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.076 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.077 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.077 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.077 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:28.078 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:04:28.078 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.078 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.078 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.079 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.079 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.079 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.079 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.079 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2022-08-01T23:04:28.080 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.080 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.080 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.081 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:04:28.081 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.082 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:04:28.082 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.083 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:04:28.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:04:28.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.086 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.086 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.087 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:04:28.087 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.088 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:04:28.088 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.089 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.089 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.090 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.091 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:04:28.091 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.092 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:04:28.093 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:28.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.093 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.094 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:04:28.094 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:04:28.094 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:04:28.094 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:04:28.095 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.095 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.095 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.095 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.096 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:28.096 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:04:28.096 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.097 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.097 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.097 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.097 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.098 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.098 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.098 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:28.098 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.098 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.099 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.099 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:04:28.100 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.101 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:04:28.101 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.101 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.101 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.102 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.102 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.103 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:04:28.103 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.104 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:04:28.104 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.105 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.105 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.106 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:04:28.106 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.107 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:04:28.107 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.108 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.108 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.109 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:04:28.109 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.110 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:04:28.110 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:28.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.110 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:04:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:04:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:04:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:04:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:04:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2022-08-01T23:04:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:04:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:04:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:04:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:04:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:04:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:04:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:04:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:04:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:04:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:04:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:04:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:04:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:04:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:04:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:04:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:04:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:04:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:04:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.141 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:04:28.141 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.142 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.142 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.142 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.143 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:04:28.143 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.144 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:04:28.144 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:28.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.145 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:04:28.145 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:04:28.145 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:04:28.146 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:04:28.146 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.146 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.146 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.147 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:28.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:04:28.148 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.148 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.148 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.148 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.148 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.149 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.149 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:28.149 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.150 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.150 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.150 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.151 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:04:28.151 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.152 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:04:28.152 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.153 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.153 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.154 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:04:28.154 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.155 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:04:28.155 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.156 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.156 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.157 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:04:28.157 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.158 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:04:28.158 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.158 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.159 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.159 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.159 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.160 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:04:28.160 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.161 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:04:28.161 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:28.162 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.162 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.162 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:04:28.162 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:04:28.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:04:28.163 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:04:28.163 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.163 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.164 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:28.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:04:28.165 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:28.165 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:28.165 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:28.166 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:28.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:28.166 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:28.166 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:28.167 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:28.167 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:28.167 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:28.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:28.167 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.168 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:04:28.168 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.169 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:04:28.169 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.170 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:28.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.171 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:04:28.171 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.172 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:04:28.173 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.174 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:28.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.174 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:04:28.175 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.176 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:04:28.176 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.177 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:28.177 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:28.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:28.178 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:04:28.178 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:28.179 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:04:28.179 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:28.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.180 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:04:28.180 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:04:28.180 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:04:28.180 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:04:28.181 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:28.181 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:28.181 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:28.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:28.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.182 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:28.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:04:28.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:28.183 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.183 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.183 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.183 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.183 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.184 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:04:28.184 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:28.185 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.185 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.185 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.185 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.186 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:04:28.186 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:28.187 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.187 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.187 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.188 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.188 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.188 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:04:28.188 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:28.189 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.189 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.189 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.190 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.190 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.190 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:04:28.190 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:28.191 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.192 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.192 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.192 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.192 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.192 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:04:28.193 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:28.194 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.194 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.194 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.194 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.195 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.195 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:04:28.195 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.196 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:28.196 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.196 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.196 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.196 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.197 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.197 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:04:28.197 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:28.198 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:28.198 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:28.198 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:28.198 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:28.199 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:28.199 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:28.199 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:04:28.199 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:28.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.200 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:28.200 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:04:28.200 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:04:28.200 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:04:28.201 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:04:28.201 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:04:28.201 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/519921040": "2022-08-02T23:04:06.054811+0000", 2022-08-01T23:04:28.201 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/519921040": "2022-08-02T23:04:06.054811+0000" 2022-08-01T23:04:28.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.202 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:04:28.202 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:04:28.202 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:04:28.202 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:28.203 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:04:28.203 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:04:28.203 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:28.203 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:04:28.203 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:28.204 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:28.204 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:04:28.204 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:04:28.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:04:28.205 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:28.205 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:28.205 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:04:28.205 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:28.206 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:28.206 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:04:28.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:28.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:28.207 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:04:28.207 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:04:28.207 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:04:28.207 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:04:28.207 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:04:28.208 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66c4053b00 msgr2=0x7f66c4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66c4053b00 0x7f66c4055fc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f66e000b000 tx=0x7f66e00016d0).stop 2022-08-01T23:04:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 msgr2=0x7f66e4064ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 0x7f66e4064ee0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7f66e0001800 tx=0x7f66e40a7610).stop 2022-08-01T23:04:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 shutdown_connections 2022-08-01T23:04:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f66c4053b00 0x7f66c4055fc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66e40b2710 0x7f66e4066b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66e40b01a0 0x7f66e40666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 --2- 172.21.15.32:0/3831688497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e40a8c50 0x7f66e4064ee0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 >> 172.21.15.32:0/3831688497 conn(0x7f66e4060310 msgr2=0x7f66e40b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 shutdown_connections 2022-08-01T23:04:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.000+0000 7f66eb47d700 1 -- 172.21.15.32:0/3831688497 wait complete. 2022-08-01T23:04:28.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:28.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c71a1700 1 Processor -- start 2022-08-01T23:04:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c71a1700 1 -- start start 2022-08-01T23:04:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c71a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c00b5d80 0x7f72c00b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c71a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c0144240 con 0x7f72c00b6710 2022-08-01T23:04:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c71a1700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c00b6fe0 con 0x7f72c00b4550 2022-08-01T23:04:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c71a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c0143d00 con 0x7f72c00b5d80 2022-08-01T23:04:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c4f3d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72c00b6710 0x7f72c0061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35886/0 (socket says 172.21.15.32:35886) 2022-08-01T23:04:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c4f3d700 1 -- 172.21.15.32:0/407075767 learned_addr learned my addr 172.21.15.32:0/407075767 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c573e700 1 --2- 172.21.15.32:0/407075767 >> v2:172.21.15.32:3300/0 conn(0x7f72c00b5d80 0x7f72c00b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838281200 0 0) 0x7f72c0144240 con 0x7f72c00b6710 2022-08-01T23:04:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a4003260 con 0x7f72c00b6710 2022-08-01T23:04:28.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950247755 0 0) 0x7f72c00b6fe0 con 0x7f72c00b4550 2022-08-01T23:04:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c0144240 con 0x7f72c00b4550 2022-08-01T23:04:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c573e700 1 -- 172.21.15.32:0/407075767 >> v1:172.21.15.55:6789/0 conn(0x7f72c00b4550 legacy=0x7f72c012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991492112 0 0) 0x7f72a4003260 con 0x7f72c00b6710 2022-08-01T23:04:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c573e700 1 -- 172.21.15.32:0/407075767 >> v1:172.21.15.32:6789/0 conn(0x7f72c00b6710 legacy=0x7f72c0061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c573e700 1 -- 172.21.15.32:0/407075767 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0143fa0 con 0x7f72c00b5d80 2022-08-01T23:04:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72c573e700 1 --2- 172.21.15.32:0/407075767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c00b6af0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f72b8001110 tx=0x7f72b0006df0).ready entity=mon.0 client_cookie=59fc9e8ac8285815 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b8001660 con 0x7f72c00b5d80 2022-08-01T23:04:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0000e90 con 0x7f72c00b5d80 2022-08-01T23:04:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.316+0000 7f72bf7fe700 1 -- 172.21.15.32:0/407075767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b0012460 con 0x7f72c00b5d80 2022-08-01T23:04:28.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/407075767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 msgr2=0x7f72c00b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 --2- 172.21.15.32:0/407075767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c00b6af0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f72b8001110 tx=0x7f72b0006df0).stop 2022-08-01T23:04:28.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/407075767 shutdown_connections 2022-08-01T23:04:28.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 --2- 172.21.15.32:0/407075767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c00b6af0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/407075767 >> 172.21.15.32:0/407075767 conn(0x7f72c005c620 msgr2=0x7f72c005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/407075767 shutdown_connections 2022-08-01T23:04:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/407075767 wait complete. 2022-08-01T23:04:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 Processor -- start 2022-08-01T23:04:28.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- start start 2022-08-01T23:04:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72c00b4550 0x7f72c0144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c0152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72c00b6710 0x7f72c0152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0144e10 con 0x7f72c00b5d80 2022-08-01T23:04:28.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0155940 con 0x7f72c00b6710 2022-08-01T23:04:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0155b70 con 0x7f72c00b4550 2022-08-01T23:04:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c0152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c0152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48350/0 (socket says 172.21.15.32:48350) 2022-08-01T23:04:28.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 -- 172.21.15.32:0/2799886315 learned_addr learned my addr 172.21.15.32:0/2799886315 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c4f3d700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72c00b4550 0x7f72c0144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c573e700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72c00b6710 0x7f72c0152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 -- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72c00b4550 msgr2=0x7f72c0144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72c00b4550 0x7f72c0144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 -- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72c00b6710 msgr2=0x7f72c0152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72c00b6710 0x7f72c0152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 -- 172.21.15.32:0/2799886315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b000b180 con 0x7f72c00b5d80 2022-08-01T23:04:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c573e700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72c00b6710 0x7f72c0152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bffff700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c0152370 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f72a8001490 tx=0x7f72a8007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a8002310 con 0x7f72c00b5d80 2022-08-01T23:04:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a8002470 con 0x7f72c00b5d80 2022-08-01T23:04:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c0155e10 con 0x7f72c00b5d80 2022-08-01T23:04:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a8001840 con 0x7f72c00b5d80 2022-08-01T23:04:28.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.320+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c01279a0 con 0x7f72c00b5d80 2022-08-01T23:04:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.324+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c0061780 con 0x7f72c00b5d80 2022-08-01T23:04:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.324+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72a8019030 con 0x7f72c00b5d80 2022-08-01T23:04:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.324+0000 7f72bdffb700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7298053b00 0x7f7298055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.324+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f72a806b600 con 0x7f72c00b5d80 2022-08-01T23:04:28.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.324+0000 7f72c4f3d700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7298053b00 0x7f7298055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.324+0000 7f72c4f3d700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7298053b00 0x7f7298055fc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f72b800b000 tx=0x7f72b8000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.328+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72a80426a0 con 0x7f72c00b5d80 2022-08-01T23:04:28.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72c00574e0 con 0x7f72c00b5d80 2022-08-01T23:04:28.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72bdffb700 1 -- 172.21.15.32:0/2799886315 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+504 (secure 0 0 0) 0x7f72a8045cc0 con 0x7f72c00b5d80 2022-08-01T23:04:28.503 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:28.504 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-08-01T23:04:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7298053b00 msgr2=0x7f7298055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7298053b00 0x7f7298055fc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f72b800b000 tx=0x7f72b8000bc0).stop 2022-08-01T23:04:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 msgr2=0x7f72c0152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c0152370 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f72a8001490 tx=0x7f72a8007e30).stop 2022-08-01T23:04:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 shutdown_connections 2022-08-01T23:04:28.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7298053b00 0x7f7298055fc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72c00b6710 0x7f72c0152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c00b5d80 0x7f72c0152370 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 --2- 172.21.15.32:0/2799886315 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72c00b4550 0x7f72c0144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 >> 172.21.15.32:0/2799886315 conn(0x7f72c005c620 msgr2=0x7f72c01488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 shutdown_connections 2022-08-01T23:04:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.496+0000 7f72c71a1700 1 -- 172.21.15.32:0/2799886315 wait complete. 2022-08-01T23:04:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 64 2022-08-01T23:04:28.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.608+0000 7fa35a130700 1 Processor -- start 2022-08-01T23:04:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.608+0000 7fa35a130700 1 -- start start 2022-08-01T23:04:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa3540b2710 0x7fa35414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3540a8c50 0x7fa35414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540a7250 con 0x7fa3540b01a0 2022-08-01T23:04:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3540a7480 con 0x7fa3540a8c50 2022-08-01T23:04:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3540a7720 con 0x7fa3540b2710 2022-08-01T23:04:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3540a8c50 0x7fa35414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3540a8c50 0x7fa35414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48360/0 (socket says 172.21.15.32:48360) 2022-08-01T23:04:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 -- 172.21.15.32:0/2843614297 learned_addr learned my addr 172.21.15.32:0/2843614297 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35912e700 1 --2- 172.21.15.32:0/2843614297 >> v2:172.21.15.55:3300/0 conn(0x7fa3540b2710 0x7fa35414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 -- 172.21.15.32:0/2843614297 >> v2:172.21.15.55:3300/0 conn(0x7fa3540b2710 msgr2=0x7fa35414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 --2- 172.21.15.32:0/2843614297 >> v2:172.21.15.55:3300/0 conn(0x7fa3540b2710 0x7fa35414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 -- 172.21.15.32:0/2843614297 >> v1:172.21.15.55:6789/0 conn(0x7fa3540b01a0 legacy=0x7fa3540a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa34bfff700 1 -- 172.21.15.32:0/2843614297 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226316215 0 0) 0x7fa3540a7250 con 0x7fa3540b01a0 2022-08-01T23:04:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 -- 172.21.15.32:0/2843614297 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540a79c0 con 0x7fa3540a8c50 2022-08-01T23:04:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35992f700 1 --2- 172.21.15.32:0/2843614297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa35414b4c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fa34c002f70 tx=0x7fa34c006eb0).ready entity=mon.0 client_cookie=a7f8298c0b015ff4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa34bfff700 1 -- 172.21.15.32:0/2843614297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa34c00d5c0 con 0x7fa3540a8c50 2022-08-01T23:04:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa34bfff700 1 -- 172.21.15.32:0/2843614297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c000e90 con 0x7fa3540a8c50 2022-08-01T23:04:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa34bfff700 1 -- 172.21.15.32:0/2843614297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa34c00d910 con 0x7fa3540a8c50 2022-08-01T23:04:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- 172.21.15.32:0/2843614297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 msgr2=0x7fa35414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 --2- 172.21.15.32:0/2843614297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa35414b4c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fa34c002f70 tx=0x7fa34c006eb0).stop 2022-08-01T23:04:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- 172.21.15.32:0/2843614297 shutdown_connections 2022-08-01T23:04:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 --2- 172.21.15.32:0/2843614297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa35414b4c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 --2- 172.21.15.32:0/2843614297 >> v2:172.21.15.55:3300/0 conn(0x7fa3540b2710 0x7fa35414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- 172.21.15.32:0/2843614297 >> 172.21.15.32:0/2843614297 conn(0x7fa354060310 msgr2=0x7fa3540b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- 172.21.15.32:0/2843614297 shutdown_connections 2022-08-01T23:04:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.612+0000 7fa35a130700 1 -- 172.21.15.32:0/2843614297 wait complete. 2022-08-01T23:04:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.616+0000 7fa35a130700 1 Processor -- start 2022-08-01T23:04:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.616+0000 7fa35a130700 1 -- start start 2022-08-01T23:04:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.616+0000 7fa35a130700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa3540a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3540b01a0 0x7fa354156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3540b2710 0x7fa3540aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b5980 con 0x7fa3540a8c50 2022-08-01T23:04:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3541573a0 con 0x7fa3540b2710 2022-08-01T23:04:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b5be0 con 0x7fa3540b01a0 2022-08-01T23:04:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa3540a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa3540a9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48362/0 (socket says 172.21.15.32:48362) 2022-08-01T23:04:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 -- 172.21.15.32:0/1752930655 learned_addr learned my addr 172.21.15.32:0/1752930655 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35892d700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3540b01a0 0x7fa354156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35992f700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3540b2710 0x7fa3540aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 -- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3540b01a0 msgr2=0x7fa354156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3540b01a0 0x7fa354156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 -- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3540b2710 msgr2=0x7fa3540aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3540b2710 0x7fa3540aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 -- 172.21.15.32:0/1752930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c00b1e0 con 0x7fa3540a8c50 2022-08-01T23:04:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35992f700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3540b2710 0x7fa3540aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35912e700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa3540a9610 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7fa34400c000 tx=0x7fa344006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa344010780 con 0x7fa3540a8c50 2022-08-01T23:04:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3440108e0 con 0x7fa3540a8c50 2022-08-01T23:04:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa354063c50 con 0x7fa3540a8c50 2022-08-01T23:04:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa344010c40 con 0x7fa3540a8c50 2022-08-01T23:04:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa354064190 con 0x7fa3540a8c50 2022-08-01T23:04:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3540574e0 con 0x7fa3540a8c50 2022-08-01T23:04:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa34401c460 con 0x7fa3540a8c50 2022-08-01T23:04:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa34a7fc700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa33c053b30 0x7fa33c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.620+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa34406c070 con 0x7fa3540a8c50 2022-08-01T23:04:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.624+0000 7fa35892d700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa33c053b30 0x7fa33c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.624+0000 7fa35892d700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa33c053b30 0x7fa33c055ff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa35000d000 tx=0x7fa350007d80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.628+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3440430b0 con 0x7fa3540a8c50 2022-08-01T23:04:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.788+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3540aa590 con 0x7fa3540a8c50 2022-08-01T23:04:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.788+0000 7fa34a7fc700 1 -- 172.21.15.32:0/1752930655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+504 (secure 0 0 0) 0x7fa3440466d0 con 0x7fa3540a8c50 2022-08-01T23:04:28.796 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:28.796 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-08-01T23:04:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa33c053b30 msgr2=0x7fa33c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa33c053b30 0x7fa33c055ff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa35000d000 tx=0x7fa350007d80).stop 2022-08-01T23:04:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 msgr2=0x7fa3540a9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa3540a9610 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7fa34400c000 tx=0x7fa344006df0).stop 2022-08-01T23:04:28.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 shutdown_connections 2022-08-01T23:04:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa33c053b30 0x7fa33c055ff0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3540b2710 0x7fa3540aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa3540b01a0 0x7fa354156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 --2- 172.21.15.32:0/1752930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3540a8c50 0x7fa3540a9610 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 >> 172.21.15.32:0/1752930655 conn(0x7fa354060310 msgr2=0x7fa35414b130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 shutdown_connections 2022-08-01T23:04:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.792+0000 7fa35a130700 1 -- 172.21.15.32:0/1752930655 wait complete. 2022-08-01T23:04:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 64 2022-08-01T23:04:28.809 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:04:28.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:04:28.810 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:04:28.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:04:28.811 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:04:28.811 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:04:28.813 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:04:28.814 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:28.815 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:04:28.815 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:04:28.816 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:28.816 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:28.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:04:28.875 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:04:28.876 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:04:28.889 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:04:28.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.912+0000 7f97ad6e6700 1 Processor -- start 2022-08-01T23:04:28.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.912+0000 7f97ad6e6700 1 -- start start 2022-08-01T23:04:28.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97a8061ab0 0x7f97a8143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.922 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a8140b10 0x7f97a8128e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a8063e40 con 0x7f97a8063a60 2022-08-01T23:04:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a813ff40 con 0x7f97a8140b10 2022-08-01T23:04:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a8062180 con 0x7f97a8061ab0 2022-08-01T23:04:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a8140b10 0x7f97a8128e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97a8140b10 0x7f97a8128e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48396/0 (socket says 172.21.15.32:48396) 2022-08-01T23:04:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 -- 172.21.15.32:0/3508135669 learned_addr learned my addr 172.21.15.32:0/3508135669 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97a77fe700 1 --2- 172.21.15.32:0/3508135669 >> v2:172.21.15.55:3300/0 conn(0x7f97a8061ab0 0x7f97a8143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 -- 172.21.15.32:0/3508135669 >> v2:172.21.15.55:3300/0 conn(0x7f97a8061ab0 msgr2=0x7f97a8143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 --2- 172.21.15.32:0/3508135669 >> v2:172.21.15.55:3300/0 conn(0x7f97a8061ab0 0x7f97a8143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 -- 172.21.15.32:0/3508135669 >> v1:172.21.15.55:6789/0 conn(0x7f97a8063a60 legacy=0x7f97a80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 -- 172.21.15.32:0/3508135669 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8140130 con 0x7f97a8140b10 2022-08-01T23:04:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97acee5700 1 --2- 172.21.15.32:0/3508135669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8140b10 0x7f97a8128e10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f97a0002f70 tx=0x7f97a0006eb0).ready entity=mon.0 client_cookie=143630bb929a5f15 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97a6ffd700 1 -- 172.21.15.32:0/3508135669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a000f9c0 con 0x7f97a8140b10 2022-08-01T23:04:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97a6ffd700 1 -- 172.21.15.32:0/3508135669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a0000e90 con 0x7f97a8140b10 2022-08-01T23:04:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97a6ffd700 1 -- 172.21.15.32:0/3508135669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a000fd50 con 0x7f97a8140b10 2022-08-01T23:04:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3508135669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8140b10 msgr2=0x7f97a8128e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3508135669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8140b10 0x7f97a8128e10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f97a0002f70 tx=0x7f97a0006eb0).stop 2022-08-01T23:04:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3508135669 shutdown_connections 2022-08-01T23:04:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3508135669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8140b10 0x7f97a8128e10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3508135669 >> v2:172.21.15.55:3300/0 conn(0x7f97a8061ab0 0x7f97a8143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3508135669 >> 172.21.15.32:0/3508135669 conn(0x7f97a805c620 msgr2=0x7f97a805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3508135669 shutdown_connections 2022-08-01T23:04:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.916+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3508135669 wait complete. 2022-08-01T23:04:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 Processor -- start 2022-08-01T23:04:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 -- start start 2022-08-01T23:04:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a8061ab0 0x7f97a8125fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 0x7f97a80b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a8140b10 0x7f97a814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b5690 con 0x7f97a8063a60 2022-08-01T23:04:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b3390 con 0x7f97a8140b10 2022-08-01T23:04:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b3630 con 0x7f97a8061ab0 2022-08-01T23:04:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 0x7f97a80b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 0x7f97a80b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48400/0 (socket says 172.21.15.32:48400) 2022-08-01T23:04:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 -- 172.21.15.32:0/3079042436 learned_addr learned my addr 172.21.15.32:0/3079042436 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a7fff700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a8061ab0 0x7f97a8125fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97acee5700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a8140b10 0x7f97a814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 -- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a8061ab0 msgr2=0x7f97a8125fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a8061ab0 0x7f97a8125fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 -- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a8140b10 msgr2=0x7f97a814e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a8140b10 0x7f97a814e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 -- 172.21.15.32:0/3079042436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a000b1e0 con 0x7f97a8063a60 2022-08-01T23:04:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97acee5700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a8140b10 0x7f97a814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a77fe700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 0x7f97a80b5d20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f979000c000 tx=0x7f9790006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9790010590 con 0x7f97a8063a60 2022-08-01T23:04:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97900106f0 con 0x7f97a8063a60 2022-08-01T23:04:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9790010a90 con 0x7f97a8063a60 2022-08-01T23:04:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.920+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a80b5880 con 0x7f97a8063a60 2022-08-01T23:04:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a814e820 con 0x7f97a8063a60 2022-08-01T23:04:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a80b37f0 con 0x7f97a8063a60 2022-08-01T23:04:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9790010bf0 con 0x7f97a8063a60 2022-08-01T23:04:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97a57fa700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f979006bec0 con 0x7f97a8063a60 2022-08-01T23:04:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97a7fff700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.924+0000 7f97a7fff700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f979800d000 tx=0x7f9798007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:28.932+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9790042ec0 con 0x7f97a8063a60 2022-08-01T23:04:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.072+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f97a80574e0 con 0x7f97a8063a60 2022-08-01T23:04:29.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.920+0000 7f97a57fa700 1 -- 172.21.15.32:0/3079042436 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f97900464e0 con 0x7f97a8063a60 2022-08-01T23:04:29.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 msgr2=0x7f978c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:29.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f979800d000 tx=0x7f9798007f10).stop 2022-08-01T23:04:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 msgr2=0x7f97a80b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 0x7f97a80b5d20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f979000c000 tx=0x7f9790006df0).stop 2022-08-01T23:04:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 shutdown_connections 2022-08-01T23:04:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f978c053b00 0x7f978c055fc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a8140b10 0x7f97a814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97a8063a60 0x7f97a80b5d20 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 --2- 172.21.15.32:0/3079042436 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f97a8061ab0 0x7f97a8125fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 >> 172.21.15.32:0/3079042436 conn(0x7f97a805c620 msgr2=0x7f97a813e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 shutdown_connections 2022-08-01T23:04:29.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:29.924+0000 7f97ad6e6700 1 -- 172.21.15.32:0/3079042436 wait complete. 2022-08-01T23:04:29.934 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:04:29.946 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:04:29.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:04:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27cf40700 1 Processor -- start 2022-08-01T23:04:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27cf40700 1 -- start start 2022-08-01T23:04:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27cf40700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2780b6a90 0x7fe2780acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27cf40700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780b6e70 con 0x7fe2780aa760 2022-08-01T23:04:30.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27cf40700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe278061370 con 0x7fe278060aa0 2022-08-01T23:04:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27cf40700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe278061610 con 0x7fe2780b6a90 2022-08-01T23:04:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe276d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2780aa760 0x7fe27813c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35952/0 (socket says 172.21.15.32:35952) 2022-08-01T23:04:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe276d9d700 1 -- 172.21.15.32:0/3324007803 learned_addr learned my addr 172.21.15.32:0/3324007803 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:30.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 --2- 172.21.15.32:0/3324007803 >> v2:172.21.15.55:3300/0 conn(0x7fe2780b6a90 0x7fe2780acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139532390 0 0) 0x7fe2780b6e70 con 0x7fe2780aa760 2022-08-01T23:04:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe25c003260 con 0x7fe2780aa760 2022-08-01T23:04:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769207011 0 0) 0x7fe278061370 con 0x7fe278060aa0 2022-08-01T23:04:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2780b6e70 con 0x7fe278060aa0 2022-08-01T23:04:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186634560 0 0) 0x7fe25c003260 con 0x7fe2780aa760 2022-08-01T23:04:30.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 >> v2:172.21.15.55:3300/0 conn(0x7fe2780b6a90 msgr2=0x7fe2780acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:30.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 --2- 172.21.15.32:0/3324007803 >> v2:172.21.15.55:3300/0 conn(0x7fe2780b6a90 0x7fe2780acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 >> v1:172.21.15.55:6789/0 conn(0x7fe278060aa0 legacy=0x7fe2780aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780618b0 con 0x7fe2780aa760 2022-08-01T23:04:30.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fe270001a00 con 0x7fe2780aa760 2022-08-01T23:04:30.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 >> v1:172.21.15.32:6789/0 conn(0x7fe2780aa760 legacy=0x7fe27813c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe25c0038e0 0x7fe25c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe25c004500 0x7fe25c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c004e70 0x7fe25c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe278061610 con 0x7fe25c0038e0 2022-08-01T23:04:30.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c00bcf0 con 0x7fe25c004500 2022-08-01T23:04:30.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c00cfc0 con 0x7fe25c004e70 2022-08-01T23:04:30.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 --2- 172.21.15.32:0/3324007803 >> v2:172.21.15.55:3300/0 conn(0x7fe2780b6a90 0x7fe2780acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:30.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe25c0038e0 0x7fe25c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe275d9b700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe25c004500 0x7fe25c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe276d9d700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c004e70 0x7fe25c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 -- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c004e70 msgr2=0x7fe25c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:30.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c004e70 0x7fe25c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 -- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe25c004500 msgr2=0x7fe25c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:30.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe25c004500 0x7fe25c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 -- 172.21.15.32:0/3324007803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c003260 con 0x7fe25c0038e0 2022-08-01T23:04:30.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe276d9d700 1 -- 172.21.15.32:0/3324007803 reap_dead start 2022-08-01T23:04:30.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.052+0000 7fe27659c700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe25c0038e0 0x7fe25c003ce0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fe26c00c000 tx=0x7fe26c006df0).ready entity=mon.0 client_cookie=1e48293eadf789ed server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26c00e800 con 0x7fe25c0038e0 2022-08-01T23:04:30.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c00e960 con 0x7fe25c0038e0 2022-08-01T23:04:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27559a700 1 -- 172.21.15.32:0/3324007803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26c00ecc0 con 0x7fe25c0038e0 2022-08-01T23:04:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- 172.21.15.32:0/3324007803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe25c0038e0 msgr2=0x7fe25c003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe25c0038e0 0x7fe25c003ce0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fe26c00c000 tx=0x7fe26c006df0).stop 2022-08-01T23:04:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- 172.21.15.32:0/3324007803 shutdown_connections 2022-08-01T23:04:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 --2- 172.21.15.32:0/3324007803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe25c0038e0 0x7fe25c003ce0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- 172.21.15.32:0/3324007803 >> 172.21.15.32:0/3324007803 conn(0x7fe2780b3610 msgr2=0x7fe2780b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- 172.21.15.32:0/3324007803 shutdown_connections 2022-08-01T23:04:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- 172.21.15.32:0/3324007803 wait complete. 2022-08-01T23:04:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 Processor -- start 2022-08-01T23:04:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- start start 2022-08-01T23:04:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c0038e0 0x7fe27813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 0x7fe27813ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe278156c70 0x7fe278157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b2800 con 0x7fe2780a96f0 2022-08-01T23:04:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815ad20 con 0x7fe278156c70 2022-08-01T23:04:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27cf40700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815afc0 con 0x7fe25c0038e0 2022-08-01T23:04:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe275d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 0x7fe27813ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe278156c70 0x7fe278157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe275d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 0x7fe27813ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48424/0 (socket says 172.21.15.32:48424) 2022-08-01T23:04:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe275d9b700 1 -- 172.21.15.32:0/973688189 learned_addr learned my addr 172.21.15.32:0/973688189 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.056+0000 7fe27659c700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c0038e0 0x7fe27813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe275d9b700 1 -- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c0038e0 msgr2=0x7fe27813cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe275d9b700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c0038e0 0x7fe27813cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe275d9b700 1 -- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe278156c70 msgr2=0x7fe278157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe275d9b700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe278156c70 0x7fe278157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe275d9b700 1 -- 172.21.15.32:0/973688189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27813c480 con 0x7fe2780a96f0 2022-08-01T23:04:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe275d9b700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 0x7fe27813ab50 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7fe26800d000 tx=0x7fe2680079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26800f8b0 con 0x7fe2780a96f0 2022-08-01T23:04:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26800fa10 con 0x7fe2780a96f0 2022-08-01T23:04:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27813c720 con 0x7fe2780a96f0 2022-08-01T23:04:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26800fd70 con 0x7fe2780a96f0 2022-08-01T23:04:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27813d160 con 0x7fe2780a96f0 2022-08-01T23:04:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27813c8e0 con 0x7fe2780a96f0 2022-08-01T23:04:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe26801b030 con 0x7fe2780a96f0 2022-08-01T23:04:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe2677fe700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe254053aa0 0x7fe254055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe26806be60 con 0x7fe2780a96f0 2022-08-01T23:04:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.060+0000 7fe27659c700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe254053aa0 0x7fe254055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.064+0000 7fe27659c700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe254053aa0 0x7fe254055f60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fe26c01b000 tx=0x7fe26c019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.068+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe268042ec0 con 0x7fe2780a96f0 2022-08-01T23:04:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.236+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2780574e0 con 0x7fe2780a96f0 2022-08-01T23:04:30.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.236+0000 7fe2677fe700 1 -- 172.21.15.32:0/973688189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fe2680464e0 con 0x7fe2780a96f0 2022-08-01T23:04:30.243 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:30.244 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:30.244 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:04:30.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.244 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:04:30.245 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:04:30.245 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.245 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:30.245 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.246 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.246 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.247 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:04:30.247 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:04:30.247 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.247 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:30.248 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.248 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.248 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.249 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:04:30.249 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:04:30.249 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.249 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:30.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.250 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.251 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:04:30.251 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:04:30.251 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.252 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.252 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.252 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.253 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:04:30.253 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:04:30.253 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.254 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.254 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.254 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.255 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:04:30.255 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:04:30.255 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.256 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.256 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.256 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.257 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:04:30.257 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:04:30.257 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.258 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.258 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.258 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.258 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.259 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:04:30.259 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:04:30.259 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.259 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.260 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.260 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.261 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:04:30.261 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:04:30.261 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.261 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.262 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.262 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.263 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:04:30.263 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:04:30.263 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.263 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.264 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.264 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.265 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:04:30.265 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:04:30.265 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.265 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.266 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.266 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.267 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:04:30.267 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:04:30.267 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.267 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:30.268 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:04:30.268 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:04:30.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.269 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:04:30.269 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:04:30.269 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.269 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:30.270 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:04:30.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:04:30.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.271 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:04:30.271 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:04:30.271 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.271 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:04:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.273 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:04:30.273 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:04:30.273 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.274 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:04:30.274 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:04:30.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:30.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:30.275 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:04:30.275 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:04:30.275 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:30.276 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:04:30.276 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:04:30.276 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:30.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:30.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:30.277 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe254053aa0 msgr2=0x7fe254055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe254053aa0 0x7fe254055f60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fe26c01b000 tx=0x7fe26c019000).stop 2022-08-01T23:04:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 msgr2=0x7fe27813ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 0x7fe27813ab50 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7fe26800d000 tx=0x7fe2680079e0).stop 2022-08-01T23:04:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 shutdown_connections 2022-08-01T23:04:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe254053aa0 0x7fe254055f60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe278156c70 0x7fe278157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2780a96f0 0x7fe27813ab50 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 --2- 172.21.15.32:0/973688189 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe25c0038e0 0x7fe27813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 >> 172.21.15.32:0/973688189 conn(0x7fe2780b3610 msgr2=0x7fe2780b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 shutdown_connections 2022-08-01T23:04:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.240+0000 7fe27cf40700 1 -- 172.21.15.32:0/973688189 wait complete. 2022-08-01T23:04:30.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 Processor -- start 2022-08-01T23:04:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 -- start start 2022-08-01T23:04:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f036c0b1f40 0x7f036c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c148530 0x7f036c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0b04e0 con 0x7f036c0a7250 2022-08-01T23:04:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0a9250 con 0x7f036c0b1f40 2022-08-01T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f03740ae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0b0780 con 0x7f036c148530 2022-08-01T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f037264b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f036c0a7250 0x7f036c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35974/0 (socket says 172.21.15.32:35974) 2022-08-01T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f037264b700 1 -- 172.21.15.32:0/3771198562 learned_addr learned my addr 172.21.15.32:0/3771198562 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371649700 1 --2- 172.21.15.32:0/3771198562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c148530 0x7f036c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371e4a700 1 --2- 172.21.15.32:0/3771198562 >> v2:172.21.15.55:3300/0 conn(0x7f036c0b1f40 0x7f036c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0370e48700 1 -- 172.21.15.32:0/3771198562 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160489204 0 0) 0x7f036c0b04e0 con 0x7f036c0a7250 2022-08-01T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0370e48700 1 -- 172.21.15.32:0/3771198562 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0358003260 con 0x7f036c0a7250 2022-08-01T23:04:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371649700 1 -- 172.21.15.32:0/3771198562 >> v2:172.21.15.55:3300/0 conn(0x7f036c0b1f40 msgr2=0x7f036c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371649700 1 --2- 172.21.15.32:0/3771198562 >> v2:172.21.15.55:3300/0 conn(0x7f036c0b1f40 0x7f036c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371649700 1 -- 172.21.15.32:0/3771198562 >> v1:172.21.15.32:6789/0 conn(0x7f036c0a7250 legacy=0x7f036c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371649700 1 -- 172.21.15.32:0/3771198562 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0b0a20 con 0x7f036c148530 2022-08-01T23:04:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371e4a700 1 --2- 172.21.15.32:0/3771198562 >> v2:172.21.15.55:3300/0 conn(0x7f036c0b1f40 0x7f036c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0371649700 1 --2- 172.21.15.32:0/3771198562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c148530 0x7f036c0b2320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f03640032f0 tx=0x7f03640091d0).ready entity=mon.2 client_cookie=b8fc845b30bbdfe4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0370e48700 1 -- 172.21.15.32:0/3771198562 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f036400b600 con 0x7f036c148530 2022-08-01T23:04:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0370e48700 1 -- 172.21.15.32:0/3771198562 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036400b760 con 0x7f036c148530 2022-08-01T23:04:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.384+0000 7f0370e48700 1 -- 172.21.15.32:0/3771198562 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f036400b8d0 con 0x7f036c148530 2022-08-01T23:04:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- 172.21.15.32:0/3771198562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c148530 msgr2=0x7f036c0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 --2- 172.21.15.32:0/3771198562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c148530 0x7f036c0b2320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f03640032f0 tx=0x7f03640091d0).stop 2022-08-01T23:04:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- 172.21.15.32:0/3771198562 shutdown_connections 2022-08-01T23:04:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 --2- 172.21.15.32:0/3771198562 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c148530 0x7f036c0b2320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 --2- 172.21.15.32:0/3771198562 >> v2:172.21.15.55:3300/0 conn(0x7f036c0b1f40 0x7f036c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- 172.21.15.32:0/3771198562 >> 172.21.15.32:0/3771198562 conn(0x7f036c05f840 msgr2=0x7f036c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- 172.21.15.32:0/3771198562 shutdown_connections 2022-08-01T23:04:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- 172.21.15.32:0/3771198562 wait complete. 2022-08-01T23:04:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 Processor -- start 2022-08-01T23:04:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- start start 2022-08-01T23:04:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f036c0a7250 0x7f036c1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c0b1f40 0x7f036c143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 0x7f036c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0a95d0 con 0x7f036c148530 2022-08-01T23:04:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0adba0 con 0x7f036c0a7250 2022-08-01T23:04:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f03740ae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0ad0d0 con 0x7f036c0b1f40 2022-08-01T23:04:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 0x7f036c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f0371649700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c0b1f40 0x7f036c143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 0x7f036c15a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48438/0 (socket says 172.21.15.32:48438) 2022-08-01T23:04:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f0371e4a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f036c0a7250 0x7f036c1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 -- 172.21.15.32:0/464454151 learned_addr learned my addr 172.21.15.32:0/464454151 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 -- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c0b1f40 msgr2=0x7f036c143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c0b1f40 0x7f036c143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 -- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f036c0a7250 msgr2=0x7f036c1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f036c0a7250 0x7f036c1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.388+0000 7f037264b700 1 -- 172.21.15.32:0/464454151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036800b180 con 0x7f036c148530 2022-08-01T23:04:30.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f037264b700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 0x7f036c15a090 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f035c001110 tx=0x7f035c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f035c00ce80 con 0x7f036c148530 2022-08-01T23:04:30.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c0070d0 con 0x7f036c148530 2022-08-01T23:04:30.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f035c007430 con 0x7f036c148530 2022-08-01T23:04:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0364009470 con 0x7f036c148530 2022-08-01T23:04:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c0ae080 con 0x7f036c148530 2022-08-01T23:04:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c0b0cc0 con 0x7f036c148530 2022-08-01T23:04:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f035c010070 con 0x7f036c148530 2022-08-01T23:04:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.392+0000 7f0362ffd700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f034c0539e0 0x7f034c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.396+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f035c06b3a0 con 0x7f036c148530 2022-08-01T23:04:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.396+0000 7f0371e4a700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f034c0539e0 0x7f034c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.396+0000 7f0371e4a700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f034c0539e0 0x7f034c055ea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f036800b000 tx=0x7f0368006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.404+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f035c042400 con 0x7f036c148530 2022-08-01T23:04:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.532+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f036c0574e0 con 0x7f036c148530 2022-08-01T23:04:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.532+0000 7f0362ffd700 1 -- 172.21.15.32:0/464454151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f035c007bc0 con 0x7f036c148530 2022-08-01T23:04:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:30.539 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":133,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:29.916362+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f034c0539e0 msgr2=0x7f034c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f034c0539e0 0x7f034c055ea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f036800b000 tx=0x7f0368006df0).stop 2022-08-01T23:04:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 msgr2=0x7f036c15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 0x7f036c15a090 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f035c001110 tx=0x7f035c007a80).stop 2022-08-01T23:04:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 shutdown_connections 2022-08-01T23:04:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f034c0539e0 0x7f034c055ea0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c148530 0x7f036c15a090 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f036c0b1f40 0x7f036c143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 --2- 172.21.15.32:0/464454151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f036c0a7250 0x7f036c1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 >> 172.21.15.32:0/464454151 conn(0x7f036c05f840 msgr2=0x7f036c1470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 shutdown_connections 2022-08-01T23:04:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.536+0000 7f03740ae700 1 -- 172.21.15.32:0/464454151 wait complete. 2022-08-01T23:04:30.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:04:30.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:04:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 Processor -- start 2022-08-01T23:04:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 -- start start 2022-08-01T23:04:30.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd4140614b0 0x7fd414061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd414140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4140aee80 con 0x7fd4140ad750 2022-08-01T23:04:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4140b70b0 con 0x7fd4140614b0 2022-08-01T23:04:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419c06700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140af0b0 con 0x7fd4141405d0 2022-08-01T23:04:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd413fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4140ad750 0x7fd4140b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35984/0 (socket says 172.21.15.32:35984) 2022-08-01T23:04:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd413fff700 1 -- 172.21.15.32:0/1773068385 learned_addr learned my addr 172.21.15.32:0/1773068385 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419405700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd414140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd418c04700 1 --2- 172.21.15.32:0/1773068385 >> v2:172.21.15.55:3300/0 conn(0x7fd4140614b0 0x7fd414061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd4137fe700 1 -- 172.21.15.32:0/1773068385 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707264315 0 0) 0x7fd4140aee80 con 0x7fd4140ad750 2022-08-01T23:04:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd4137fe700 1 -- 172.21.15.32:0/1773068385 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd400003260 con 0x7fd4140ad750 2022-08-01T23:04:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd418c04700 1 -- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 msgr2=0x7fd414140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd418c04700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd414140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd418c04700 1 -- 172.21.15.32:0/1773068385 >> v1:172.21.15.32:6789/0 conn(0x7fd4140ad750 legacy=0x7fd4140b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd418c04700 1 -- 172.21.15.32:0/1773068385 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140af350 con 0x7fd4140614b0 2022-08-01T23:04:30.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd419405700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd414140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:30.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.656+0000 7fd418c04700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd414061890 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fd4140aed30 tx=0x7fd408006df0).ready entity=mon.1 client_cookie=9aaf0ba84993026b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd4137fe700 1 -- 172.21.15.32:0/1773068385 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd40800d570 con 0x7fd4140614b0 2022-08-01T23:04:30.668 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd4137fe700 1 -- 172.21.15.32:0/1773068385 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408000e90 con 0x7fd4140614b0 2022-08-01T23:04:30.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd4137fe700 1 -- 172.21.15.32:0/1773068385 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd40800d8c0 con 0x7fd4140614b0 2022-08-01T23:04:30.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 -- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 msgr2=0x7fd414061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd414061890 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fd4140aed30 tx=0x7fd408006df0).stop 2022-08-01T23:04:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 -- 172.21.15.32:0/1773068385 shutdown_connections 2022-08-01T23:04:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd414140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 --2- 172.21.15.32:0/1773068385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd414061890 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 -- 172.21.15.32:0/1773068385 >> 172.21.15.32:0/1773068385 conn(0x7fd414066f70 msgr2=0x7fd414067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 -- 172.21.15.32:0/1773068385 shutdown_connections 2022-08-01T23:04:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 -- 172.21.15.32:0/1773068385 wait complete. 2022-08-01T23:04:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 Processor -- start 2022-08-01T23:04:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.660+0000 7fd419c06700 1 -- start start 2022-08-01T23:04:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd4140b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 0x7fd4140b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd4140b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140aca70 con 0x7fd4140ad750 2022-08-01T23:04:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140abf10 con 0x7fd4140614b0 2022-08-01T23:04:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd414156530 con 0x7fd4141405d0 2022-08-01T23:04:30.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 0x7fd4140b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 0x7fd4140b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48450/0 (socket says 172.21.15.32:48450) 2022-08-01T23:04:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 -- 172.21.15.32:0/3888108435 learned_addr learned my addr 172.21.15.32:0/3888108435 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419405700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd4140b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd418c04700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd4140b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 -- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 msgr2=0x7fd4140b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd4140b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 -- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 msgr2=0x7fd4140b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd4140b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 -- 172.21.15.32:0/3888108435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c009470 con 0x7fd4140ad750 2022-08-01T23:04:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd413fff700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 0x7fd4140b35e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7fd404001110 tx=0x7fd404007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4040070d0 con 0x7fd4140ad750 2022-08-01T23:04:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404007230 con 0x7fd4140ad750 2022-08-01T23:04:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40800b180 con 0x7fd4140ad750 2022-08-01T23:04:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd404005610 con 0x7fd4140ad750 2022-08-01T23:04:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4140b3fc0 con 0x7fd4140ad750 2022-08-01T23:04:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.664+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd414156370 con 0x7fd4140ad750 2022-08-01T23:04:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.668+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd404010070 con 0x7fd4140ad750 2022-08-01T23:04:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.668+0000 7fd411ffb700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3f4053950 0x7fd3f4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.668+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd40406b410 con 0x7fd4140ad750 2022-08-01T23:04:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.668+0000 7fd418c04700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3f4053950 0x7fd3f4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.668+0000 7fd418c04700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3f4053950 0x7fd3f4055e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fd40801a000 tx=0x7fd408014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.672+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd404042430 con 0x7fd4140ad750 2022-08-01T23:04:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.796+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd4140ac0a0 con 0x7fd4140ad750 2022-08-01T23:04:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.796+0000 7fd411ffb700 1 -- 172.21.15.32:0/3888108435 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v133) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd404045a50 con 0x7fd4140ad750 2022-08-01T23:04:30.804 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:04:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3f4053950 msgr2=0x7fd3f4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3f4053950 0x7fd3f4055e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fd40801a000 tx=0x7fd408014000).stop 2022-08-01T23:04:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 msgr2=0x7fd4140b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 0x7fd4140b35e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7fd404001110 tx=0x7fd404007a40).stop 2022-08-01T23:04:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 shutdown_connections 2022-08-01T23:04:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3f4053950 0x7fd3f4055e10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd4141405d0 0x7fd4140b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4140ad750 0x7fd4140b35e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 --2- 172.21.15.32:0/3888108435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4140614b0 0x7fd4140b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 >> 172.21.15.32:0/3888108435 conn(0x7fd414066f70 msgr2=0x7fd41405cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 shutdown_connections 2022-08-01T23:04:30.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.800+0000 7fd419c06700 1 -- 172.21.15.32:0/3888108435 wait complete. 2022-08-01T23:04:30.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:04:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3606dfb700 1 Processor -- start 2022-08-01T23:04:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3606dfb700 1 -- start start 2022-08-01T23:04:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3606dfb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f36001401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3606dfb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3600067a10 con 0x7f36000ac880 2022-08-01T23:04:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3606dfb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3600067cb0 con 0x7f36000b0270 2022-08-01T23:04:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3606dfb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36000ad740 con 0x7f36000b2710 2022-08-01T23:04:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f36065fa700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36000ac880 0x7f3600143710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36000/0 (socket says 172.21.15.32:36000) 2022-08-01T23:04:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f36065fa700 1 -- 172.21.15.32:0/3733482940 learned_addr learned my addr 172.21.15.32:0/3733482940 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3605df9700 1 --2- 172.21.15.32:0/3733482940 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f36001401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3604df7700 1 -- 172.21.15.32:0/3733482940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099436106 0 0) 0x7f3600067a10 con 0x7f36000ac880 2022-08-01T23:04:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3604df7700 1 -- 172.21.15.32:0/3733482940 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e8003260 con 0x7f36000ac880 2022-08-01T23:04:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3605df9700 1 -- 172.21.15.32:0/3733482940 >> v1:172.21.15.55:6789/0 conn(0x7f36000b0270 legacy=0x7f3600143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3605df9700 1 -- 172.21.15.32:0/3733482940 >> v1:172.21.15.32:6789/0 conn(0x7f36000ac880 legacy=0x7f3600143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3605df9700 1 -- 172.21.15.32:0/3733482940 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600147160 con 0x7f36000b2710 2022-08-01T23:04:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.920+0000 7f3605df9700 1 --2- 172.21.15.32:0/3733482940 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f36001401e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f35f0002f70 tx=0x7f35f0006df0).ready entity=mon.2 client_cookie=6843adbd4579a2d3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3604df7700 1 -- 172.21.15.32:0/3733482940 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35f000d570 con 0x7f36000b2710 2022-08-01T23:04:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3604df7700 1 -- 172.21.15.32:0/3733482940 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0000e90 con 0x7f36000b2710 2022-08-01T23:04:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3604df7700 1 -- 172.21.15.32:0/3733482940 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35f000d8c0 con 0x7f36000b2710 2022-08-01T23:04:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3733482940 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 msgr2=0x7f36001401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3733482940 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f36001401e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f35f0002f70 tx=0x7f35f0006df0).stop 2022-08-01T23:04:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3733482940 shutdown_connections 2022-08-01T23:04:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3733482940 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f36001401e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3733482940 >> 172.21.15.32:0/3733482940 conn(0x7f3600060310 msgr2=0x7f36000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3733482940 shutdown_connections 2022-08-01T23:04:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3733482940 wait complete. 2022-08-01T23:04:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 Processor -- start 2022-08-01T23:04:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- start start 2022-08-01T23:04:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 0x7f360009ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36000b0270 0x7f3600146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f3600152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36001501a0 con 0x7f36000ac880 2022-08-01T23:04:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36000b43b0 con 0x7f36000b0270 2022-08-01T23:04:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36000b3ce0 con 0x7f36000b2710 2022-08-01T23:04:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 0x7f360009ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 0x7f360009ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48462/0 (socket says 172.21.15.32:48462) 2022-08-01T23:04:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 -- 172.21.15.32:0/3219341390 learned_addr learned my addr 172.21.15.32:0/3219341390 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f36065fa700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f3600152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f36055f8700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36000b0270 0x7f3600146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 -- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 msgr2=0x7f3600152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f3600152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 -- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36000b0270 msgr2=0x7f3600146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36000b0270 0x7f3600146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 -- 172.21.15.32:0/3219341390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f000b180 con 0x7f36000ac880 2022-08-01T23:04:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3605df9700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 0x7f360009ebe0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f36000abfa0 tx=0x7f35f0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35f0016070 con 0x7f36000ac880 2022-08-01T23:04:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36001565d0 con 0x7f36000ac880 2022-08-01T23:04:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f00139c0 con 0x7f36000ac880 2022-08-01T23:04:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3600156870 con 0x7f36000ac880 2022-08-01T23:04:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.924+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35f0013d20 con 0x7f36000ac880 2022-08-01T23:04:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.928+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36000574e0 con 0x7f36000ac880 2022-08-01T23:04:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.928+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f35f000f030 con 0x7f36000ac880 2022-08-01T23:04:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.928+0000 7f35eeffd700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35e0053b00 0x7f35e0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.928+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f35f006cc00 con 0x7f36000ac880 2022-08-01T23:04:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.928+0000 7f36055f8700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35e0053b00 0x7f35e0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.932+0000 7f36055f8700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35e0053b00 0x7f35e0055fc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f35fc00b000 tx=0x7f35fc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:30.936+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35f0043b70 con 0x7f36000ac880 2022-08-01T23:04:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.060+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36000b1bb0 con 0x7f36000ac880 2022-08-01T23:04:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.060+0000 7f35eeffd700 1 -- 172.21.15.32:0/3219341390 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35f0023070 con 0x7f36000ac880 2022-08-01T23:04:31.069 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35e0053b00 msgr2=0x7f35e0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35e0053b00 0x7f35e0055fc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f35fc00b000 tx=0x7f35fc009040).stop 2022-08-01T23:04:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 msgr2=0x7f360009ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 0x7f360009ebe0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f36000abfa0 tx=0x7f35f0014000).stop 2022-08-01T23:04:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 shutdown_connections 2022-08-01T23:04:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f35e0053b00 0x7f35e0055fc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f36000b2710 0x7f3600152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36000b0270 0x7f3600146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 --2- 172.21.15.32:0/3219341390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36000ac880 0x7f360009ebe0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 >> 172.21.15.32:0/3219341390 conn(0x7f3600060310 msgr2=0x7f36000b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 shutdown_connections 2022-08-01T23:04:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.064+0000 7f3606dfb700 1 -- 172.21.15.32:0/3219341390 wait complete. 2022-08-01T23:04:31.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:04:31.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 Processor -- start 2022-08-01T23:04:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 -- start start 2022-08-01T23:04:31.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f013c0b5630 0x7f013c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f013c0b07a0 0x7f013c147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c067730 0x7f013c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013c145470 con 0x7f013c0b5630 2022-08-01T23:04:31.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013c0ae180 con 0x7f013c0b07a0 2022-08-01T23:04:31.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0b5a10 con 0x7f013c067730 2022-08-01T23:04:31.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.176+0000 7f013a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f013c0b5630 0x7f013c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f0139d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f013c0b07a0 0x7f013c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013ad9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c067730 0x7f013c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013ad9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c067730 0x7f013c0acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60210/0 (socket says 172.21.15.32:60210) 2022-08-01T23:04:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f0139d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f013c0b07a0 0x7f013c147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58256/0 (socket says 172.21.15.32:58256) 2022-08-01T23:04:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013ad9d700 1 -- 172.21.15.32:0/288220133 learned_addr learned my addr 172.21.15.32:0/288220133 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013a59c700 1 -- 172.21.15.32:0/288220133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c067730 msgr2=0x7f013c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013a59c700 1 --2- 172.21.15.32:0/288220133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c067730 0x7f013c0acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013a59c700 1 -- 172.21.15.32:0/288220133 >> v2:172.21.15.55:3300/0 conn(0x7f013c0b07a0 msgr2=0x7f013c147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:31.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013a59c700 1 --2- 172.21.15.32:0/288220133 >> v2:172.21.15.55:3300/0 conn(0x7f013c0b07a0 0x7f013c147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013a59c700 1 -- 172.21.15.32:0/288220133 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c0ac990 con 0x7f013c0b5630 2022-08-01T23:04:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013a59c700 1 --2- 172.21.15.32:0/288220133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b5630 0x7f013c0af6e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f0130002f70 tx=0x7f0130006df0).ready entity=mon.0 client_cookie=9c84faddcad28a44 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013959a700 1 -- 172.21.15.32:0/288220133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013000d4a0 con 0x7f013c0b5630 2022-08-01T23:04:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013959a700 1 -- 172.21.15.32:0/288220133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130000e90 con 0x7f013c0b5630 2022-08-01T23:04:31.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013959a700 1 -- 172.21.15.32:0/288220133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013000d7f0 con 0x7f013c0b5630 2022-08-01T23:04:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 -- 172.21.15.32:0/288220133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b5630 msgr2=0x7f013c0af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 --2- 172.21.15.32:0/288220133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b5630 0x7f013c0af6e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f0130002f70 tx=0x7f0130006df0).stop 2022-08-01T23:04:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 -- 172.21.15.32:0/288220133 shutdown_connections 2022-08-01T23:04:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 --2- 172.21.15.32:0/288220133 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c067730 0x7f013c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 --2- 172.21.15.32:0/288220133 >> v2:172.21.15.55:3300/0 conn(0x7f013c0b07a0 0x7f013c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 --2- 172.21.15.32:0/288220133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b5630 0x7f013c0af6e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 -- 172.21.15.32:0/288220133 >> 172.21.15.32:0/288220133 conn(0x7f013c0626f0 msgr2=0x7f013c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:31.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 -- 172.21.15.32:0/288220133 shutdown_connections 2022-08-01T23:04:31.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 -- 172.21.15.32:0/288220133 wait complete. 2022-08-01T23:04:31.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 Processor -- start 2022-08-01T23:04:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.180+0000 7f013b59e700 1 -- start start 2022-08-01T23:04:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f013c067730 0x7f013c064ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 0x7f013c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c0b5630 0x7f013c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c065340 con 0x7f013c0b07a0 2022-08-01T23:04:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c156d00 con 0x7f013c067730 2022-08-01T23:04:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c156fa0 con 0x7f013c0b5630 2022-08-01T23:04:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 0x7f013c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 0x7f013c1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48478/0 (socket says 172.21.15.32:48478) 2022-08-01T23:04:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 -- 172.21.15.32:0/1497452414 learned_addr learned my addr 172.21.15.32:0/1497452414 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013a59c700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f013c067730 0x7f013c064ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013ad9d700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c0b5630 0x7f013c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 -- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c0b5630 msgr2=0x7f013c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c0b5630 0x7f013c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 -- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f013c067730 msgr2=0x7f013c064ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f013c067730 0x7f013c064ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 -- 172.21.15.32:0/1497452414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013000b180 con 0x7f013c0b07a0 2022-08-01T23:04:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013ad9d700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c0b5630 0x7f013c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f0139d9b700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 0x7f013c1576e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f012c00c000 tx=0x7f012c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012c00c850 con 0x7f013c0b07a0 2022-08-01T23:04:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c00c9b0 con 0x7f013c0b07a0 2022-08-01T23:04:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012c00cd10 con 0x7f013c0b07a0 2022-08-01T23:04:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013c157240 con 0x7f013c0b07a0 2022-08-01T23:04:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c0657b0 con 0x7f013c0b07a0 2022-08-01T23:04:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f012c022070 con 0x7f013c0b07a0 2022-08-01T23:04:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.184+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c065a90 con 0x7f013c0b07a0 2022-08-01T23:04:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.188+0000 7f012b7fe700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01180539e0 0x7f0118055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.188+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f012c06c3e0 con 0x7f013c0b07a0 2022-08-01T23:04:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.188+0000 7f013a59c700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01180539e0 0x7f0118055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.188+0000 7f013a59c700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01180539e0 0x7f0118055ea0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f013001a000 tx=0x7f0130014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.192+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f012c0434e0 con 0x7f013c0b07a0 2022-08-01T23:04:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.324+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f013c1573d0 con 0x7f013c0b07a0 2022-08-01T23:04:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f012b7fe700 1 -- 172.21.15.32:0/1497452414 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+143 (secure 0 0 0) 0x7f012c046b00 con 0x7f013c0b07a0 2022-08-01T23:04:31.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:31.333 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:04:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:04:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:04:31.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:04:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:04:31.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:04:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01180539e0 msgr2=0x7f0118055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01180539e0 0x7f0118055ea0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f013001a000 tx=0x7f0130014000).stop 2022-08-01T23:04:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 msgr2=0x7f013c1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 0x7f013c1576e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f012c00c000 tx=0x7f012c0091d0).stop 2022-08-01T23:04:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 shutdown_connections 2022-08-01T23:04:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01180539e0 0x7f0118055ea0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f013c0b5630 0x7f013c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c0b07a0 0x7f013c1576e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 --2- 172.21.15.32:0/1497452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f013c067730 0x7f013c064ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 >> 172.21.15.32:0/1497452414 conn(0x7f013c0626f0 msgr2=0x7f013c143b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 shutdown_connections 2022-08-01T23:04:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.328+0000 7f013b59e700 1 -- 172.21.15.32:0/1497452414 wait complete. 2022-08-01T23:04:31.346 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:04:31.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:04:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 Processor -- start 2022-08-01T23:04:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- start start 2022-08-01T23:04:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb00614b0 0x7f3cb0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3cb00ad750 0x7f3cb00b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb00aee80 con 0x7f3cb01405d0 2022-08-01T23:04:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00b70b0 con 0x7f3cb00614b0 2022-08-01T23:04:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00af0b0 con 0x7f3cb00ad750 2022-08-01T23:04:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb00614b0 0x7f3cb0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3cb01405d0 0x7f3cb01466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36028/0 (socket says 172.21.15.32:36028) 2022-08-01T23:04:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caf7fe700 1 -- 172.21.15.32:0/3988218589 learned_addr learned my addr 172.21.15.32:0/3988218589 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3cae7fc700 1 --2- 172.21.15.32:0/3988218589 >> v2:172.21.15.55:3300/0 conn(0x7f3cb00ad750 0x7f3cb00b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caeffd700 1 -- 172.21.15.32:0/3988218589 >> v2:172.21.15.55:3300/0 conn(0x7f3cb00ad750 msgr2=0x7f3cb00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caeffd700 1 --2- 172.21.15.32:0/3988218589 >> v2:172.21.15.55:3300/0 conn(0x7f3cb00ad750 0x7f3cb00b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caeffd700 1 -- 172.21.15.32:0/3988218589 >> v1:172.21.15.32:6789/0 conn(0x7f3cb01405d0 legacy=0x7f3cb01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3cadffb700 1 -- 172.21.15.32:0/3988218589 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598797450 0 0) 0x7f3cb00aee80 con 0x7f3cb01405d0 2022-08-01T23:04:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caeffd700 1 -- 172.21.15.32:0/3988218589 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00af350 con 0x7f3cb00614b0 2022-08-01T23:04:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caeffd700 1 --2- 172.21.15.32:0/3988218589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00614b0 0x7f3cb0061890 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f3cb00aed30 tx=0x7f3ca4006df0).ready entity=mon.0 client_cookie=80598a6c6bb9966a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3cadffb700 1 -- 172.21.15.32:0/3988218589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca400d5c0 con 0x7f3cb00614b0 2022-08-01T23:04:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3cadffb700 1 -- 172.21.15.32:0/3988218589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca4000e90 con 0x7f3cb00614b0 2022-08-01T23:04:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3cadffb700 1 -- 172.21.15.32:0/3988218589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca400d910 con 0x7f3cb00614b0 2022-08-01T23:04:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- 172.21.15.32:0/3988218589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00614b0 msgr2=0x7f3cb0061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 --2- 172.21.15.32:0/3988218589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00614b0 0x7f3cb0061890 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f3cb00aed30 tx=0x7f3ca4006df0).stop 2022-08-01T23:04:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- 172.21.15.32:0/3988218589 shutdown_connections 2022-08-01T23:04:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 --2- 172.21.15.32:0/3988218589 >> v2:172.21.15.55:3300/0 conn(0x7f3cb00ad750 0x7f3cb00b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 --2- 172.21.15.32:0/3988218589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb00614b0 0x7f3cb0061890 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- 172.21.15.32:0/3988218589 >> 172.21.15.32:0/3988218589 conn(0x7f3cb0066f70 msgr2=0x7f3cb0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:31.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- 172.21.15.32:0/3988218589 shutdown_connections 2022-08-01T23:04:31.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.448+0000 7f3caffff700 1 -- 172.21.15.32:0/3988218589 wait complete. 2022-08-01T23:04:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 Processor -- start 2022-08-01T23:04:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 -- start start 2022-08-01T23:04:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cb00614b0 0x7f3cb0143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3cb00ad750 0x7f3cb01434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 0x7f3cb0152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00b31d0 con 0x7f3cb01405d0 2022-08-01T23:04:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00ad280 con 0x7f3cb00614b0 2022-08-01T23:04:31.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00acd40 con 0x7f3cb00ad750 2022-08-01T23:04:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 0x7f3cb0152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 0x7f3cb0152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48494/0 (socket says 172.21.15.32:48494) 2022-08-01T23:04:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 -- 172.21.15.32:0/1270206451 learned_addr learned my addr 172.21.15.32:0/1270206451 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3cae7fc700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3cb00ad750 0x7f3cb01434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caeffd700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cb00614b0 0x7f3cb0143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 -- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3cb00ad750 msgr2=0x7f3cb01434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3cb00ad750 0x7f3cb01434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 -- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cb00614b0 msgr2=0x7f3cb0143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cb00614b0 0x7f3cb0143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 -- 172.21.15.32:0/1270206451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca400b180 con 0x7f3cb01405d0 2022-08-01T23:04:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.452+0000 7f3caf7fe700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 0x7f3cb0152b80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f3cb0142a00 tx=0x7f3ca8001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca8005ba0 con 0x7f3cb01405d0 2022-08-01T23:04:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8005d00 con 0x7f3cb01405d0 2022-08-01T23:04:31.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb00acfe0 con 0x7f3cb01405d0 2022-08-01T23:04:31.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca8004a90 con 0x7f3cb01405d0 2022-08-01T23:04:31.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb00afac0 con 0x7f3cb01405d0 2022-08-01T23:04:31.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb00af5f0 con 0x7f3cb01405d0 2022-08-01T23:04:31.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ca8004bf0 con 0x7f3cb01405d0 2022-08-01T23:04:31.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3c9ffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c8c053950 0x7f3c8c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:31.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3ca806b840 con 0x7f3cb01405d0 2022-08-01T23:04:31.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.456+0000 7f3caeffd700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c8c053950 0x7f3c8c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:31.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.460+0000 7f3caeffd700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c8c053950 0x7f3c8c055e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f3ca401a000 tx=0x7f3ca4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.464+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ca8042940 con 0x7f3cb01405d0 2022-08-01T23:04:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:31.584+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f3cb00574e0 con 0x7f3cb01405d0 2022-08-01T23:04:32.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.340+0000 7f3c9ffff700 1 -- 172.21.15.32:0/1270206451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v134) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3ca8045f60 con 0x7f3cb01405d0 2022-08-01T23:04:32.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c8c053950 msgr2=0x7f3c8c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:32.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c8c053950 0x7f3c8c055e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f3ca401a000 tx=0x7f3ca4014000).stop 2022-08-01T23:04:32.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 msgr2=0x7f3cb0152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:32.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 0x7f3cb0152b80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f3cb0142a00 tx=0x7f3ca8001360).stop 2022-08-01T23:04:32.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 shutdown_connections 2022-08-01T23:04:32.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3c8c053950 0x7f3c8c055e10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb01405d0 0x7f3cb0152b80 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3cb00ad750 0x7f3cb01434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 --2- 172.21.15.32:0/1270206451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cb00614b0 0x7f3cb0143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 >> 172.21.15.32:0/1270206451 conn(0x7f3cb0066f70 msgr2=0x7f3cb00b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 shutdown_connections 2022-08-01T23:04:32.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.344+0000 7f3caffff700 1 -- 172.21.15.32:0/1270206451 wait complete. 2022-08-01T23:04:32.354 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:04:32.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:04:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.476+0000 7fb89c726700 1 Processor -- start 2022-08-01T23:04:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.476+0000 7fb89c726700 1 -- start start 2022-08-01T23:04:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8940b4310 0x7fb894140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8940b4820 0x7fb8940b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb894143cf0 con 0x7fb894146ae0 2022-08-01T23:04:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8941476a0 con 0x7fb8940b4820 2022-08-01T23:04:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb894143f90 con 0x7fb8940b4310 2022-08-01T23:04:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8940b4820 0x7fb8940b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb899cc1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8940b4310 0x7fb894140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8940b4820 0x7fb8940b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58288/0 (socket says 172.21.15.32:58288) 2022-08-01T23:04:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 -- 172.21.15.32:0/3749507408 learned_addr learned my addr 172.21.15.32:0/3749507408 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 -- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8940b4310 msgr2=0x7fb894140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 --2- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8940b4310 0x7fb894140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 -- 172.21.15.32:0/3749507408 >> v1:172.21.15.55:6789/0 conn(0x7fb894146ae0 legacy=0x7fb8941403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 -- 172.21.15.32:0/3749507408 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940b0e20 con 0x7fb8940b4820 2022-08-01T23:04:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89acc3700 1 --2- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb8940b5060 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fb88c0032f0 tx=0x7fb88c009290).ready entity=mon.1 client_cookie=5453481efdfc130e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb8994c0700 1 -- 172.21.15.32:0/3749507408 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb88c00b8a0 con 0x7fb8940b4820 2022-08-01T23:04:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb8994c0700 1 -- 172.21.15.32:0/3749507408 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00ba00 con 0x7fb8940b4820 2022-08-01T23:04:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb8994c0700 1 -- 172.21.15.32:0/3749507408 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb88c00bb70 con 0x7fb8940b4820 2022-08-01T23:04:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 -- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 msgr2=0x7fb8940b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 --2- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb8940b5060 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fb88c0032f0 tx=0x7fb88c009290).stop 2022-08-01T23:04:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 -- 172.21.15.32:0/3749507408 shutdown_connections 2022-08-01T23:04:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 --2- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb8940b5060 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.480+0000 7fb89c726700 1 --2- 172.21.15.32:0/3749507408 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb8940b4310 0x7fb894140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- 172.21.15.32:0/3749507408 >> 172.21.15.32:0/3749507408 conn(0x7fb89405c620 msgr2=0x7fb89405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- 172.21.15.32:0/3749507408 shutdown_connections 2022-08-01T23:04:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- 172.21.15.32:0/3749507408 wait complete. 2022-08-01T23:04:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 Processor -- start 2022-08-01T23:04:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- start start 2022-08-01T23:04:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940b4310 0x7fb8940ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb894152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb894146ae0 0x7fb894145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940b00c0 con 0x7fb8940b4310 2022-08-01T23:04:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940addb0 con 0x7fb8940b4820 2022-08-01T23:04:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89c726700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941525a0 con 0x7fb894146ae0 2022-08-01T23:04:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb899cc1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb894152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89a4c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940b4310 0x7fb8940ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89acc3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb894146ae0 0x7fb894145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb899cc1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb894152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58292/0 (socket says 172.21.15.32:58292) 2022-08-01T23:04:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89acc3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb894146ae0 0x7fb894145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60246/0 (socket says 172.21.15.32:60246) 2022-08-01T23:04:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb899cc1700 1 -- 172.21.15.32:0/2254130544 learned_addr learned my addr 172.21.15.32:0/2254130544 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89a4c2700 1 -- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb894146ae0 msgr2=0x7fb894145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89a4c2700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb894146ae0 0x7fb894145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89a4c2700 1 -- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 msgr2=0x7fb894152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.484+0000 7fb89a4c2700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb894152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb89a4c2700 1 -- 172.21.15.32:0/2254130544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009530 con 0x7fb8940b4310 2022-08-01T23:04:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb89a4c2700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940b4310 0x7fb8940ad780 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fb894060de0 tx=0x7fb888007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb88800cca0 con 0x7fb8940b4310 2022-08-01T23:04:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb894152840 con 0x7fb8940b4310 2022-08-01T23:04:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88800ce00 con 0x7fb8940b4310 2022-08-01T23:04:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb888007220 con 0x7fb8940b4310 2022-08-01T23:04:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb894146190 con 0x7fb8940b4310 2022-08-01T23:04:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb894152ce0 con 0x7fb8940b4310 2022-08-01T23:04:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb888010030 con 0x7fb8940b4310 2022-08-01T23:04:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb8877fe700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb874053a70 0x7fb874055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.488+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fb88806bb60 con 0x7fb8940b4310 2022-08-01T23:04:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.492+0000 7fb899cc1700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb874053a70 0x7fb874055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.492+0000 7fb899cc1700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb874053a70 0x7fb874055f30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fb89000b000 tx=0x7fb890006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.500+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb894152ce0 con 0x7fb8940b4310 2022-08-01T23:04:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:32.628+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb894152ce0 con 0x7fb8940b4310 2022-08-01T23:04:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.356+0000 7fb8877fe700 1 -- 172.21.15.32:0/2254130544 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v135) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb894152ce0 con 0x7fb8940b4310 2022-08-01T23:04:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb874053a70 msgr2=0x7fb874055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb874053a70 0x7fb874055f30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fb89000b000 tx=0x7fb890006df0).stop 2022-08-01T23:04:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940b4310 msgr2=0x7fb8940ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940b4310 0x7fb8940ad780 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fb894060de0 tx=0x7fb888007b70).stop 2022-08-01T23:04:33.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 shutdown_connections 2022-08-01T23:04:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb874053a70 0x7fb874055f30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb894146ae0 0x7fb894145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8940b4820 0x7fb894152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 --2- 172.21.15.32:0/2254130544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940b4310 0x7fb8940ad780 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 >> 172.21.15.32:0/2254130544 conn(0x7fb89405c620 msgr2=0x7fb89413b350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 shutdown_connections 2022-08-01T23:04:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.364+0000 7fb89c726700 1 -- 172.21.15.32:0/2254130544 wait complete. 2022-08-01T23:04:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:04:33.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:04:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380d672700 1 Processor -- start 2022-08-01T23:04:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380d672700 1 -- start start 2022-08-01T23:04:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380d672700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0d10 0x7f3808146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380d672700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080b12d0 con 0x7f38080b0560 2022-08-01T23:04:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380d672700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080a9700 con 0x7f38080a9320 2022-08-01T23:04:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380d672700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b1500 con 0x7f38080b0d10 2022-08-01T23:04:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380ce71700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38080b0560 0x7f3808140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36060/0 (socket says 172.21.15.32:36060) 2022-08-01T23:04:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f380ce71700 1 -- 172.21.15.32:0/1799449718 learned_addr learned my addr 172.21.15.32:0/1799449718 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f38077fe700 1 --2- 172.21.15.32:0/1799449718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0d10 0x7f3808146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.484+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601045049 0 0) 0x7f38080b12d0 con 0x7f38080b0560 2022-08-01T23:04:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e8003260 con 0x7f38080b0560 2022-08-01T23:04:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166679519 0 0) 0x7f38080a9700 con 0x7f38080a9320 2022-08-01T23:04:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38080b12d0 con 0x7f38080a9320 2022-08-01T23:04:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f38077fe700 1 -- 172.21.15.32:0/1799449718 >> v1:172.21.15.55:6789/0 conn(0x7f38080a9320 legacy=0x7f38081466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f38077fe700 1 -- 172.21.15.32:0/1799449718 >> v1:172.21.15.32:6789/0 conn(0x7f38080b0560 legacy=0x7f3808140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f38077fe700 1 -- 172.21.15.32:0/1799449718 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080b17a0 con 0x7f38080b0d10 2022-08-01T23:04:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f38077fe700 1 --2- 172.21.15.32:0/1799449718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0d10 0x7f3808146df0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f37f8002f70 tx=0x7f37f8006df0).ready entity=mon.2 client_cookie=c54130a960ebf6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f800d5c0 con 0x7f38080b0d10 2022-08-01T23:04:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8000e90 con 0x7f38080b0d10 2022-08-01T23:04:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f3806ffd700 1 -- 172.21.15.32:0/1799449718 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f800d910 con 0x7f38080b0d10 2022-08-01T23:04:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 -- 172.21.15.32:0/1799449718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0d10 msgr2=0x7f3808146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 --2- 172.21.15.32:0/1799449718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0d10 0x7f3808146df0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f37f8002f70 tx=0x7f37f8006df0).stop 2022-08-01T23:04:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 -- 172.21.15.32:0/1799449718 shutdown_connections 2022-08-01T23:04:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 --2- 172.21.15.32:0/1799449718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0d10 0x7f3808146df0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 -- 172.21.15.32:0/1799449718 >> 172.21.15.32:0/1799449718 conn(0x7f38080612d0 msgr2=0x7f3808062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 -- 172.21.15.32:0/1799449718 shutdown_connections 2022-08-01T23:04:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 -- 172.21.15.32:0/1799449718 wait complete. 2022-08-01T23:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.488+0000 7f380d672700 1 Processor -- start 2022-08-01T23:04:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 -- start start 2022-08-01T23:04:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 0x7f3808141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0560 0x7f38080af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38080b0d10 0x7f38081437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b4030 con 0x7f38080a9320 2022-08-01T23:04:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808142550 con 0x7f38080b0d10 2022-08-01T23:04:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3808142780 con 0x7f38080b0560 2022-08-01T23:04:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 0x7f3808141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 0x7f3808141030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48520/0 (socket says 172.21.15.32:48520) 2022-08-01T23:04:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 -- 172.21.15.32:0/3026920617 learned_addr learned my addr 172.21.15.32:0/3026920617 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380ce71700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38080b0d10 0x7f38081437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f38077fe700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0560 0x7f38080af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 -- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0560 msgr2=0x7f38080af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0560 0x7f38080af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 -- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38080b0d10 msgr2=0x7f38081437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38080b0d10 0x7f38081437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 -- 172.21.15.32:0/3026920617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f800b180 con 0x7f38080a9320 2022-08-01T23:04:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f3807fff700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 0x7f3808141030 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f37f0001110 tx=0x7f37f0007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f0012400 con 0x7f38080a9320 2022-08-01T23:04:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0012560 con 0x7f38080a9320 2022-08-01T23:04:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3808142a20 con 0x7f38080a9320 2022-08-01T23:04:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37f00128c0 con 0x7f38080a9320 2022-08-01T23:04:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.492+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38080ae6f0 con 0x7f38080a9320 2022-08-01T23:04:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.496+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38080ac7e0 con 0x7f38080a9320 2022-08-01T23:04:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.496+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f37f0019030 con 0x7f38080a9320 2022-08-01T23:04:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.496+0000 7f38057fa700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37ec053a70 0x7f37ec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.496+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f37f006b410 con 0x7f38080a9320 2022-08-01T23:04:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.496+0000 7f38077fe700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37ec053a70 0x7f37ec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.496+0000 7f38077fe700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37ec053a70 0x7f37ec055f30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f37f801a000 tx=0x7f37f8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:33.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.504+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f37f0042250 con 0x7f38080a9320 2022-08-01T23:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.668+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f38080574e0 con 0x7f38080a9320 2022-08-01T23:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.680+0000 7f38057fa700 1 -- 172.21.15.32:0/3026920617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v67) v1 ==== 189+0+0 (secure 0 0 0) 0x7f37f0045870 con 0x7f38080a9320 2022-08-01T23:04:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37ec053a70 msgr2=0x7f37ec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37ec053a70 0x7f37ec055f30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f37f801a000 tx=0x7f37f8014000).stop 2022-08-01T23:04:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 msgr2=0x7f3808141030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 0x7f3808141030 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f37f0001110 tx=0x7f37f0007ae0).stop 2022-08-01T23:04:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 shutdown_connections 2022-08-01T23:04:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37ec053a70 0x7f37ec055f30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38080b0d10 0x7f38081437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f38080b0560 0x7f38080af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 --2- 172.21.15.32:0/3026920617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080a9320 0x7f3808141030 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 >> 172.21.15.32:0/3026920617 conn(0x7f38080612d0 msgr2=0x7f380813eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:33.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 shutdown_connections 2022-08-01T23:04:33.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.684+0000 7f380d672700 1 -- 172.21.15.32:0/3026920617 wait complete. 2022-08-01T23:04:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 25 and data pool 26 2022-08-01T23:04:33.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 Processor -- start 2022-08-01T23:04:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 -- start start 2022-08-01T23:04:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f49c40af550 0x7f49c40b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49c405c6f0 0x7f49c40a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40aa630 con 0x7f49c4060330 2022-08-01T23:04:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40b15e0 con 0x7f49c405c6f0 2022-08-01T23:04:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c9301700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40af930 con 0x7f49c40af550 2022-08-01T23:04:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c8b00700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49c405c6f0 0x7f49c40a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.792+0000 7f49c37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49c4060330 0x7f49c41445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36082/0 (socket says 172.21.15.32:36082) 2022-08-01T23:04:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c37fe700 1 -- 172.21.15.32:0/769870169 learned_addr learned my addr 172.21.15.32:0/769870169 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c3fff700 1 --2- 172.21.15.32:0/769870169 >> v2:172.21.15.55:3300/0 conn(0x7f49c40af550 0x7f49c40b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c2ffd700 1 -- 172.21.15.32:0/769870169 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271713402 0 0) 0x7f49c40aa630 con 0x7f49c4060330 2022-08-01T23:04:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c2ffd700 1 -- 172.21.15.32:0/769870169 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b0003260 con 0x7f49c4060330 2022-08-01T23:04:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c8b00700 1 -- 172.21.15.32:0/769870169 >> v2:172.21.15.55:3300/0 conn(0x7f49c40af550 msgr2=0x7f49c40b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c8b00700 1 --2- 172.21.15.32:0/769870169 >> v2:172.21.15.55:3300/0 conn(0x7f49c40af550 0x7f49c40b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c8b00700 1 -- 172.21.15.32:0/769870169 >> v1:172.21.15.32:6789/0 conn(0x7f49c4060330 legacy=0x7f49c41445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c8b00700 1 -- 172.21.15.32:0/769870169 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c405fd10 con 0x7f49c405c6f0 2022-08-01T23:04:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c8b00700 1 --2- 172.21.15.32:0/769870169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c405c6f0 0x7f49c40a8d80 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f49bc0032f0 tx=0x7f49bc0091d0).ready entity=mon.0 client_cookie=576255721427692f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c2ffd700 1 -- 172.21.15.32:0/769870169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49bc00b600 con 0x7f49c405c6f0 2022-08-01T23:04:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c2ffd700 1 -- 172.21.15.32:0/769870169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc00b760 con 0x7f49c405c6f0 2022-08-01T23:04:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c2ffd700 1 -- 172.21.15.32:0/769870169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49bc00b8d0 con 0x7f49c405c6f0 2022-08-01T23:04:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 -- 172.21.15.32:0/769870169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c405c6f0 msgr2=0x7f49c40a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 --2- 172.21.15.32:0/769870169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c405c6f0 0x7f49c40a8d80 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f49bc0032f0 tx=0x7f49bc0091d0).stop 2022-08-01T23:04:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 -- 172.21.15.32:0/769870169 shutdown_connections 2022-08-01T23:04:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 --2- 172.21.15.32:0/769870169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c405c6f0 0x7f49c40a8d80 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 --2- 172.21.15.32:0/769870169 >> v2:172.21.15.55:3300/0 conn(0x7f49c40af550 0x7f49c40b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 -- 172.21.15.32:0/769870169 >> 172.21.15.32:0/769870169 conn(0x7f49c40612d0 msgr2=0x7f49c4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 -- 172.21.15.32:0/769870169 shutdown_connections 2022-08-01T23:04:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.796+0000 7f49c9301700 1 -- 172.21.15.32:0/769870169 wait complete. 2022-08-01T23:04:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 Processor -- start 2022-08-01T23:04:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 -- start start 2022-08-01T23:04:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c405c6f0 0x7f49c40ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49c4060330 0x7f49c41403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 0x7f49c409ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40ae010 con 0x7f49c40af550 2022-08-01T23:04:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4153230 con 0x7f49c405c6f0 2022-08-01T23:04:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c9301700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4146c00 con 0x7f49c4060330 2022-08-01T23:04:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 0x7f49c409ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 0x7f49c409ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48548/0 (socket says 172.21.15.32:48548) 2022-08-01T23:04:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 -- 172.21.15.32:0/2733146449 learned_addr learned my addr 172.21.15.32:0/2733146449 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c37fe700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49c4060330 0x7f49c41403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c3fff700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c405c6f0 0x7f49c40ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 -- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49c4060330 msgr2=0x7f49c41403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49c4060330 0x7f49c41403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 -- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c405c6f0 msgr2=0x7f49c40ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c405c6f0 0x7f49c40ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 -- 172.21.15.32:0/2733146449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009470 con 0x7f49c40af550 2022-08-01T23:04:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c8b00700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 0x7f49c409ea40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f49bc0032f0 tx=0x7f49bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.800+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49bc00b8d0 con 0x7f49c40af550 2022-08-01T23:04:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c405f840 con 0x7f49c40af550 2022-08-01T23:04:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc01f3f0 con 0x7f49c40af550 2022-08-01T23:04:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49bc00b600 con 0x7f49c40af550 2022-08-01T23:04:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c40ae2b0 con 0x7f49c40af550 2022-08-01T23:04:33.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c40ae6f0 con 0x7f49c40af550 2022-08-01T23:04:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f49bc026030 con 0x7f49c40af550 2022-08-01T23:04:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c17fa700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f49a4053b00 0x7f49a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f49bc06cbb0 con 0x7f49c40af550 2022-08-01T23:04:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c3fff700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f49a4053b00 0x7f49a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.804+0000 7f49c3fff700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f49a4053b00 0x7f49a4055fc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f49b400b000 tx=0x7f49b4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.808+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49bc043920 con 0x7f49c40af550 2022-08-01T23:04:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.936+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49c40678a0 con 0x7f49c40af550 2022-08-01T23:04:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c17fa700 1 -- 172.21.15.32:0/2733146449 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f49bc031fc0 con 0x7f49c40af550 2022-08-01T23:04:33.945 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:33.945 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":136,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:33.672212+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:04:31.641583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T23:04:32.681709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f49a4053b00 msgr2=0x7f49a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f49a4053b00 0x7f49a4055fc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f49b400b000 tx=0x7f49b4008040).stop 2022-08-01T23:04:33.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 msgr2=0x7f49c409ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:33.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 0x7f49c409ea40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f49bc0032f0 tx=0x7f49bc016000).stop 2022-08-01T23:04:33.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 shutdown_connections 2022-08-01T23:04:33.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f49a4053b00 0x7f49a4055fc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c40af550 0x7f49c409ea40 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f49c4060330 0x7f49c41403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 --2- 172.21.15.32:0/2733146449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c405c6f0 0x7f49c40ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:33.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 >> 172.21.15.32:0/2733146449 conn(0x7f49c40612d0 msgr2=0x7f49c40b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:33.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 shutdown_connections 2022-08-01T23:04:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:33.940+0000 7f49c9301700 1 -- 172.21.15.32:0/2733146449 wait complete. 2022-08-01T23:04:33.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:34.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.060+0000 7f2dc29ee700 1 Processor -- start 2022-08-01T23:04:34.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.060+0000 7f2dc29ee700 1 -- start start 2022-08-01T23:04:34.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2dbc0b1f40 0x7f2dbc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0b04e0 con 0x7f2dbc148530 2022-08-01T23:04:34.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0a9250 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0b0780 con 0x7f2dbc0a7250 2022-08-01T23:04:34.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2dbc0b1f40 0x7f2dbc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbb7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2dbc148530 0x7f2dbc0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57000/0 (socket says 172.21.15.32:57000) 2022-08-01T23:04:34.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2dbc0b1f40 0x7f2dbc0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48554/0 (socket says 172.21.15.32:48554) 2022-08-01T23:04:34.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbb7fe700 1 -- 172.21.15.32:0/1992293648 learned_addr learned my addr 172.21.15.32:0/1992293648 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:34.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc0f8b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbaffd700 1 -- 172.21.15.32:0/1992293648 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196556579 0 0) 0x7f2dbc0b04e0 con 0x7f2dbc148530 2022-08-01T23:04:34.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbaffd700 1 -- 172.21.15.32:0/1992293648 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da8003260 con 0x7f2dbc148530 2022-08-01T23:04:34.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 -- 172.21.15.32:0/1992293648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 msgr2=0x7f2dbc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 --2- 172.21.15.32:0/1992293648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 -- 172.21.15.32:0/1992293648 >> v1:172.21.15.55:6789/0 conn(0x7f2dbc148530 legacy=0x7f2dbc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 -- 172.21.15.32:0/1992293648 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc0b0a20 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbbfff700 1 --2- 172.21.15.32:0/1992293648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc0b1160 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f2dbc1478c0 tx=0x7f2db0006df0).ready entity=mon.0 client_cookie=1ef11799d0c501e8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbaffd700 1 -- 172.21.15.32:0/1992293648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db000d570 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbaffd700 1 -- 172.21.15.32:0/1992293648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db0000e90 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dbaffd700 1 -- 172.21.15.32:0/1992293648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db000d8c0 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- 172.21.15.32:0/1992293648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 msgr2=0x7f2dbc0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/1992293648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc0b1160 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f2dbc1478c0 tx=0x7f2db0006df0).stop 2022-08-01T23:04:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- 172.21.15.32:0/1992293648 shutdown_connections 2022-08-01T23:04:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/1992293648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/1992293648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc0b1160 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- 172.21.15.32:0/1992293648 >> 172.21.15.32:0/1992293648 conn(0x7f2dbc05f840 msgr2=0x7f2dbc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- 172.21.15.32:0/1992293648 shutdown_connections 2022-08-01T23:04:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.064+0000 7f2dc29ee700 1 -- 172.21.15.32:0/1992293648 wait complete. 2022-08-01T23:04:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 Processor -- start 2022-08-01T23:04:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 -- start start 2022-08-01T23:04:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc14d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc148530 0x7f2dbc156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0a9670 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0a9910 con 0x7f2dbc148530 2022-08-01T23:04:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0a9bb0 con 0x7f2dbc0a7250 2022-08-01T23:04:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc14d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc14d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48560/0 (socket says 172.21.15.32:48560) 2022-08-01T23:04:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 -- 172.21.15.32:0/580950554 learned_addr learned my addr 172.21.15.32:0/580950554 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbbfff700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc0f8b700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc148530 0x7f2dbc156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 -- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 msgr2=0x7f2dbc14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 -- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc148530 msgr2=0x7f2dbc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc148530 0x7f2dbc156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 -- 172.21.15.32:0/580950554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db000b180 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dbb7fe700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc14d4b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f2dac001490 tx=0x7f2dac007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dac005c70 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac005dd0 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc14d9a0 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dac0072c0 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.068+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc14af70 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.072+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc0a9f30 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.072+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2dac010070 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.072+0000 7f2db97fa700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d9c053a70 0x7f2d9c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.072+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2dac06bab0 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.072+0000 7f2dbbfff700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d9c053a70 0x7f2d9c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.072+0000 7f2dbbfff700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d9c053a70 0x7f2d9c055f30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f2db001a000 tx=0x7f2db0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.080+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2dac042870 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.204+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dbc0574e0 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.204+0000 7f2db97fa700 1 -- 172.21.15.32:0/580950554 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f2dac045e90 con 0x7f2dbc0b1f40 2022-08-01T23:04:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:34.212 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":136,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:33.672212+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:04:31.641583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T23:04:32.681709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d9c053a70 msgr2=0x7f2d9c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d9c053a70 0x7f2d9c055f30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f2db001a000 tx=0x7f2db0014000).stop 2022-08-01T23:04:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 msgr2=0x7f2dbc14d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc14d4b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f2dac001490 tx=0x7f2dac007de0).stop 2022-08-01T23:04:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 shutdown_connections 2022-08-01T23:04:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d9c053a70 0x7f2d9c055f30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc148530 0x7f2dbc156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc0b1f40 0x7f2dbc14d4b0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 --2- 172.21.15.32:0/580950554 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2dbc0a7250 0x7f2dbc14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 >> 172.21.15.32:0/580950554 conn(0x7f2dbc05f840 msgr2=0x7f2dbc0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 shutdown_connections 2022-08-01T23:04:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.208+0000 7f2dc29ee700 1 -- 172.21.15.32:0/580950554 wait complete. 2022-08-01T23:04:34.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:04:34.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 Processor -- start 2022-08-01T23:04:34.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 -- start start 2022-08-01T23:04:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0614b0 0x7f2d8c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2d8c1405d0 0x7f2d8c140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0aee80 con 0x7f2d8c0ad750 2022-08-01T23:04:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b70b0 con 0x7f2d8c1405d0 2022-08-01T23:04:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d90895700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0af0b0 con 0x7f2d8c0614b0 2022-08-01T23:04:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2d8c1405d0 0x7f2d8c140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0614b0 0x7f2d8c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2d8c1405d0 0x7f2d8c140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58356/0 (socket says 172.21.15.32:58356) 2022-08-01T23:04:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 -- 172.21.15.32:0/146363985 learned_addr learned my addr 172.21.15.32:0/146363985 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 -- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0614b0 msgr2=0x7f2d8c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 --2- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0614b0 0x7f2d8c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 -- 172.21.15.32:0/146363985 >> v1:172.21.15.55:6789/0 conn(0x7f2d8c0ad750 legacy=0x7f2d8c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8affd700 1 -- 172.21.15.32:0/146363985 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901099559 0 0) 0x7f2d8c0aee80 con 0x7f2d8c0ad750 2022-08-01T23:04:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 -- 172.21.15.32:0/146363985 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0af350 con 0x7f2d8c1405d0 2022-08-01T23:04:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.332+0000 7f2d8bfff700 1 --2- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c140a60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f2d840032f0 tx=0x7f2d84009290).ready entity=mon.1 client_cookie=31d20f59e494346b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8affd700 1 -- 172.21.15.32:0/146363985 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8400b8a0 con 0x7f2d8c1405d0 2022-08-01T23:04:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8affd700 1 -- 172.21.15.32:0/146363985 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8400ba00 con 0x7f2d8c1405d0 2022-08-01T23:04:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8affd700 1 -- 172.21.15.32:0/146363985 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8400bb70 con 0x7f2d8c1405d0 2022-08-01T23:04:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c1405d0 msgr2=0x7f2d8c140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 --2- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c140a60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f2d840032f0 tx=0x7f2d84009290).stop 2022-08-01T23:04:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- 172.21.15.32:0/146363985 shutdown_connections 2022-08-01T23:04:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 --2- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c140a60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 --2- 172.21.15.32:0/146363985 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0614b0 0x7f2d8c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- 172.21.15.32:0/146363985 >> 172.21.15.32:0/146363985 conn(0x7f2d8c066f70 msgr2=0x7f2d8c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- 172.21.15.32:0/146363985 shutdown_connections 2022-08-01T23:04:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- 172.21.15.32:0/146363985 wait complete. 2022-08-01T23:04:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 Processor -- start 2022-08-01T23:04:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- start start 2022-08-01T23:04:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c0614b0 0x7f2d8c153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0ad750 0x7f2d8c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0aa930 con 0x7f2d8c1405d0 2022-08-01T23:04:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c153570 con 0x7f2d8c0614b0 2022-08-01T23:04:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d90895700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0aabd0 con 0x7f2d8c0ad750 2022-08-01T23:04:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c0614b0 0x7f2d8c153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48576/0 (socket says 172.21.15.32:48576) 2022-08-01T23:04:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 -- 172.21.15.32:0/522324108 learned_addr learned my addr 172.21.15.32:0/522324108 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:34.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d82ffd700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0ad750 0x7f2d8c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 -- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0ad750 msgr2=0x7f2d8c0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0ad750 0x7f2d8c0b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 -- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c0614b0 msgr2=0x7f2d8c153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c0614b0 0x7f2d8c153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 -- 172.21.15.32:0/522324108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009530 con 0x7f2d8c1405d0 2022-08-01T23:04:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d82ffd700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0ad750 0x7f2d8c0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.336+0000 7f2d8bfff700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c0b3c90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f2d840032f0 tx=0x7f2d84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8400bb70 con 0x7f2d8c1405d0 2022-08-01T23:04:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84021630 con 0x7f2d8c1405d0 2022-08-01T23:04:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8400b8a0 con 0x7f2d8c1405d0 2022-08-01T23:04:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c0aae70 con 0x7f2d8c1405d0 2022-08-01T23:04:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c0ab3b0 con 0x7f2d8c1405d0 2022-08-01T23:04:34.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c0ab570 con 0x7f2d8c1405d0 2022-08-01T23:04:34.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d84027020 con 0x7f2d8c1405d0 2022-08-01T23:04:34.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d897fa700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d70053950 0x7f2d70055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2d8406d410 con 0x7f2d8c1405d0 2022-08-01T23:04:34.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.340+0000 7f2d8b7fe700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d70053950 0x7f2d70055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.344+0000 7f2d8b7fe700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d70053950 0x7f2d70055e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f2d7c00b000 tx=0x7f2d7c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.348+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d840441d0 con 0x7f2d8c1405d0 2022-08-01T23:04:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.512+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2d8c0574e0 con 0x7f2d8c1405d0 2022-08-01T23:04:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.680+0000 7f2d897fa700 1 -- 172.21.15.32:0/522324108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2d84015020 con 0x7f2d8c1405d0 2022-08-01T23:04:34.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.680+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d70053950 msgr2=0x7f2d70055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.680+0000 7f2d90895700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d70053950 0x7f2d70055e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f2d7c00b000 tx=0x7f2d7c006df0).stop 2022-08-01T23:04:34.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.680+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 msgr2=0x7f2d8c0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.680+0000 7f2d90895700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c0b3c90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f2d840032f0 tx=0x7f2d84016000).stop 2022-08-01T23:04:34.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 shutdown_connections 2022-08-01T23:04:34.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d70053950 0x7f2d70055e10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c1405d0 0x7f2d8c0b3c90 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2d8c0ad750 0x7f2d8c0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 --2- 172.21.15.32:0/522324108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d8c0614b0 0x7f2d8c153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 >> 172.21.15.32:0/522324108 conn(0x7f2d8c066f70 msgr2=0x7f2d8c0b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:34.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 shutdown_connections 2022-08-01T23:04:34.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.684+0000 7f2d90895700 1 -- 172.21.15.32:0/522324108 wait complete. 2022-08-01T23:04:34.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:34.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 Processor -- start 2022-08-01T23:04:34.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 -- start start 2022-08-01T23:04:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6880b2710 0x7ff68814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b01a0 0x7ff68814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880a7250 con 0x7ff6880afd60 2022-08-01T23:04:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6880a7480 con 0x7ff6880b2710 2022-08-01T23:04:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880a7720 con 0x7ff6880b01a0 2022-08-01T23:04:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6880b2710 0x7ff68814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6880b2710 0x7ff68814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48580/0 (socket says 172.21.15.32:48580) 2022-08-01T23:04:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff68659c700 1 -- 172.21.15.32:0/3846702544 learned_addr learned my addr 172.21.15.32:0/3846702544 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.800+0000 7ff685d9b700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b01a0 0x7ff68814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68659c700 1 -- 172.21.15.32:0/3846702544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b01a0 msgr2=0x7ff68814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68659c700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b01a0 0x7ff68814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68659c700 1 -- 172.21.15.32:0/3846702544 >> v1:172.21.15.32:6789/0 conn(0x7ff6880afd60 legacy=0x7ff6880a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68659c700 1 -- 172.21.15.32:0/3846702544 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880a79c0 con 0x7ff6880b2710 2022-08-01T23:04:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff685d9b700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b01a0 0x7ff68814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68659c700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880b2710 0x7ff68814afd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7ff6800032f0 tx=0x7ff680009290).ready entity=mon.0 client_cookie=a097e44825bb86bb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68559a700 1 -- 172.21.15.32:0/3846702544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff68000b8a0 con 0x7ff6880b2710 2022-08-01T23:04:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68559a700 1 -- 172.21.15.32:0/3846702544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68000ba00 con 0x7ff6880b2710 2022-08-01T23:04:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68559a700 1 -- 172.21.15.32:0/3846702544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff68000bb70 con 0x7ff6880b2710 2022-08-01T23:04:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 -- 172.21.15.32:0/3846702544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880b2710 msgr2=0x7ff68814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880b2710 0x7ff68814afd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7ff6800032f0 tx=0x7ff680009290).stop 2022-08-01T23:04:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 -- 172.21.15.32:0/3846702544 shutdown_connections 2022-08-01T23:04:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b01a0 0x7ff68814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 --2- 172.21.15.32:0/3846702544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880b2710 0x7ff68814afd0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 -- 172.21.15.32:0/3846702544 >> 172.21.15.32:0/3846702544 conn(0x7ff688060310 msgr2=0x7ff6880b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 -- 172.21.15.32:0/3846702544 shutdown_connections 2022-08-01T23:04:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 -- 172.21.15.32:0/3846702544 wait complete. 2022-08-01T23:04:34.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 Processor -- start 2022-08-01T23:04:34.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.804+0000 7ff68759e700 1 -- start start 2022-08-01T23:04:34.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 0x7ff6880a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b01a0 0x7ff6880af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b2710 0x7ff6881568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815a750 con 0x7ff6880afd60 2022-08-01T23:04:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815a9c0 con 0x7ff6880b01a0 2022-08-01T23:04:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815ac60 con 0x7ff6880b2710 2022-08-01T23:04:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 0x7ff6880a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 0x7ff6880a9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48586/0 (socket says 172.21.15.32:48586) 2022-08-01T23:04:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 -- 172.21.15.32:0/2368097666 learned_addr learned my addr 172.21.15.32:0/2368097666 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff685d9b700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b01a0 0x7ff6880af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff686d9d700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b2710 0x7ff6881568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 -- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b2710 msgr2=0x7ff6881568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b2710 0x7ff6881568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 -- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b01a0 msgr2=0x7ff6880af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b01a0 0x7ff6880af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 -- 172.21.15.32:0/2368097666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680009530 con 0x7ff6880afd60 2022-08-01T23:04:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68659c700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 0x7ff6880a9fc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7ff6800032f0 tx=0x7ff680016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff680021630 con 0x7ff6880afd60 2022-08-01T23:04:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680021790 con 0x7ff6880afd60 2022-08-01T23:04:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880a9880 con 0x7ff6880afd60 2022-08-01T23:04:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff680021af0 con 0x7ff6880afd60 2022-08-01T23:04:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.808+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68814cb10 con 0x7ff6880afd60 2022-08-01T23:04:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.812+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6880574e0 con 0x7ff6880afd60 2022-08-01T23:04:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.812+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff68000f070 con 0x7ff6880afd60 2022-08-01T23:04:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.812+0000 7ff6777fe700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff664053b00 0x7ff664055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.812+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff68006d2d0 con 0x7ff6880afd60 2022-08-01T23:04:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.812+0000 7ff685d9b700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff664053b00 0x7ff664055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.812+0000 7ff685d9b700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff664053b00 0x7ff664055fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7ff67c00b000 tx=0x7ff67c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.820+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff680043f80 con 0x7ff6880afd60 2022-08-01T23:04:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff68806a5e0 con 0x7ff6880afd60 2022-08-01T23:04:34.989 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff6777fe700 1 -- 172.21.15.32:0/2368097666 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2489 (secure 0 0 0) 0x7ff680015020 con 0x7ff6880afd60 2022-08-01T23:04:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:34.990 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:34.675417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8079":{"gid":8079,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/766376033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":766376033},{"type":"v1","addr":"172.21.15.32:6835","nonce":766376033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:34.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff664053b00 msgr2=0x7ff664055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff664053b00 0x7ff664055fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7ff67c00b000 tx=0x7ff67c008040).stop 2022-08-01T23:04:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 msgr2=0x7ff6880a9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 0x7ff6880a9fc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7ff6800032f0 tx=0x7ff680016000).stop 2022-08-01T23:04:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 shutdown_connections 2022-08-01T23:04:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff664053b00 0x7ff664055fc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff6880b2710 0x7ff6881568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b01a0 0x7ff6880af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 --2- 172.21.15.32:0/2368097666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6880afd60 0x7ff6880a9fc0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:34.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 >> 172.21.15.32:0/2368097666 conn(0x7ff688060310 msgr2=0x7ff688148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:34.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 shutdown_connections 2022-08-01T23:04:34.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:34.984+0000 7ff68759e700 1 -- 172.21.15.32:0/2368097666 wait complete. 2022-08-01T23:04:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 68 2022-08-01T23:04:35.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05fffff700 1 Processor -- start 2022-08-01T23:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05fffff700 1 -- start start 2022-08-01T23:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05fffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06000ac880 0x7f0600143710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05fffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0600067a10 con 0x7f06000b2710 2022-08-01T23:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05fffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0600067cb0 con 0x7f06000b0270 2022-08-01T23:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05fffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06000ad740 con 0x7f06000ac880 2022-08-01T23:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05ff7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06000ac880 0x7f0600143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05feffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f06000b2710 0x7f06001401e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36138/0 (socket says 172.21.15.32:36138) 2022-08-01T23:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05feffd700 1 -- 172.21.15.32:0/3523281344 learned_addr learned my addr 172.21.15.32:0/3523281344 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.104+0000 7f05ff7fe700 1 -- 172.21.15.32:0/3523281344 >> v1:172.21.15.55:6789/0 conn(0x7f06000b0270 legacy=0x7f0600143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05ff7fe700 1 -- 172.21.15.32:0/3523281344 >> v1:172.21.15.32:6789/0 conn(0x7f06000b2710 legacy=0x7f06001401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05ff7fe700 1 -- 172.21.15.32:0/3523281344 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0600147160 con 0x7f06000ac880 2022-08-01T23:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fdffb700 1 -- 172.21.15.32:0/3523281344 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269636619 0 0) 0x7f0600067a10 con 0x7f06000b2710 2022-08-01T23:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05ff7fe700 1 --2- 172.21.15.32:0/3523281344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600143710 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f05f8002f70 tx=0x7f05f8006df0).ready entity=mon.0 client_cookie=a093eac71be8646d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fdffb700 1 -- 172.21.15.32:0/3523281344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05f800d610 con 0x7f06000ac880 2022-08-01T23:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fdffb700 1 -- 172.21.15.32:0/3523281344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8000e90 con 0x7f06000ac880 2022-08-01T23:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fdffb700 1 -- 172.21.15.32:0/3523281344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05f800d960 con 0x7f06000ac880 2022-08-01T23:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 -- 172.21.15.32:0/3523281344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 msgr2=0x7f0600143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 --2- 172.21.15.32:0/3523281344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600143710 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f05f8002f70 tx=0x7f05f8006df0).stop 2022-08-01T23:04:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 -- 172.21.15.32:0/3523281344 shutdown_connections 2022-08-01T23:04:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 --2- 172.21.15.32:0/3523281344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600143710 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 -- 172.21.15.32:0/3523281344 >> 172.21.15.32:0/3523281344 conn(0x7f0600060310 msgr2=0x7f06000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 -- 172.21.15.32:0/3523281344 shutdown_connections 2022-08-01T23:04:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 -- 172.21.15.32:0/3523281344 wait complete. 2022-08-01T23:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 Processor -- start 2022-08-01T23:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 -- start start 2022-08-01T23:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.108+0000 7f05fffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06000b0270 0x7f06000ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06000b2710 0x7f060014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000aada0 con 0x7f06000ac880 2022-08-01T23:04:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000a1800 con 0x7f06000b2710 2022-08-01T23:04:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000a1a60 con 0x7f06000b0270 2022-08-01T23:04:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48600/0 (socket says 172.21.15.32:48600) 2022-08-01T23:04:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 -- 172.21.15.32:0/3876661448 learned_addr learned my addr 172.21.15.32:0/3876661448 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05ff7fe700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06000b2710 0x7f060014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fe7fc700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06000b0270 0x7f06000ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 -- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06000b0270 msgr2=0x7f06000ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06000b0270 0x7f06000ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 -- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06000b2710 msgr2=0x7f060014f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06000b2710 0x7f060014f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 -- 172.21.15.32:0/3876661448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f800b180 con 0x7f06000ac880 2022-08-01T23:04:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05ff7fe700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06000b2710 0x7f060014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05feffd700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600152f30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f05e8000bc0 tx=0x7f05e8007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e80070d0 con 0x7f06000ac880 2022-08-01T23:04:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8007230 con 0x7f06000ac880 2022-08-01T23:04:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06000a1d00 con 0x7f06000ac880 2022-08-01T23:04:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e8005610 con 0x7f06000ac880 2022-08-01T23:04:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000ab970 con 0x7f06000ac880 2022-08-01T23:04:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0600143c00 con 0x7f06000ac880 2022-08-01T23:04:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f05e8010070 con 0x7f06000ac880 2022-08-01T23:04:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05f7fff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05e4053a70 0x7f05e4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.112+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f05e806b7f0 con 0x7f06000ac880 2022-08-01T23:04:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.116+0000 7f05fe7fc700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05e4053a70 0x7f05e4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.116+0000 7f05fe7fc700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05e4053a70 0x7f05e4055f30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f05f000b000 tx=0x7f05f0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.120+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05e8042510 con 0x7f06000ac880 2022-08-01T23:04:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.248+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06000574e0 con 0x7f06000ac880 2022-08-01T23:04:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.252+0000 7f05f7fff700 1 -- 172.21.15.32:0/3876661448 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f05e8069e60 con 0x7f06000ac880 2022-08-01T23:04:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:35.259 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":137,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:34.676165+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:04:31.641583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T23:04:32.681709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05e4053a70 msgr2=0x7f05e4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05e4053a70 0x7f05e4055f30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f05f000b000 tx=0x7f05f0009040).stop 2022-08-01T23:04:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 msgr2=0x7f0600152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600152f30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f05e8000bc0 tx=0x7f05e8007c90).stop 2022-08-01T23:04:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 shutdown_connections 2022-08-01T23:04:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05e4053a70 0x7f05e4055f30 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06000b2710 0x7f060014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f06000b0270 0x7f06000ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 --2- 172.21.15.32:0/3876661448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06000ac880 0x7f0600152f30 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 >> 172.21.15.32:0/3876661448 conn(0x7f0600060310 msgr2=0x7f0600146410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 shutdown_connections 2022-08-01T23:04:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.256+0000 7f05fffff700 1 -- 172.21.15.32:0/3876661448 wait complete. 2022-08-01T23:04:35.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:04:35.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.384+0000 7f0553fff700 1 Processor -- start 2022-08-01T23:04:35.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.384+0000 7f0553fff700 1 -- start start 2022-08-01T23:04:35.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0553fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05540614b0 0x7f0554061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0553fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 0x7f05540b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0553fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05540aee80 con 0x7f05541405d0 2022-08-01T23:04:35.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0553fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05540b70b0 con 0x7f05540614b0 2022-08-01T23:04:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0553fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05540af0b0 con 0x7f05540ad750 2022-08-01T23:04:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05537fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05541405d0 0x7f05541466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36154/0 (socket says 172.21.15.32:36154) 2022-08-01T23:04:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05537fe700 1 -- 172.21.15.32:0/3724533374 learned_addr learned my addr 172.21.15.32:0/3724533374 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 0x7f05540b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968708230 0 0) 0x7f05540aee80 con 0x7f05541405d0 2022-08-01T23:04:35.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0552ffd700 1 --2- 172.21.15.32:0/3724533374 >> v2:172.21.15.55:3300/0 conn(0x7f05540614b0 0x7f0554061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0534003260 con 0x7f05541405d0 2022-08-01T23:04:35.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169554518 0 0) 0x7f0534003260 con 0x7f05541405d0 2022-08-01T23:04:35.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 msgr2=0x7f05540b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 0x7f05540b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 >> v2:172.21.15.55:3300/0 conn(0x7f05540614b0 msgr2=0x7f0554061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 --2- 172.21.15.32:0/3724533374 >> v2:172.21.15.55:3300/0 conn(0x7f05540614b0 0x7f0554061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05540af350 con 0x7f05541405d0 2022-08-01T23:04:35.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f054c0019e0 con 0x7f05541405d0 2022-08-01T23:04:35.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0552ffd700 1 --2- 172.21.15.32:0/3724533374 >> v2:172.21.15.55:3300/0 conn(0x7f05540614b0 0x7f0554061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:35.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 >> v1:172.21.15.32:6789/0 conn(0x7f05541405d0 legacy=0x7f05541466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0534003750 0x7f0534003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534004500 0x7f0534004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 0x7f0534004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540b70b0 con 0x7f0534004500 2022-08-01T23:04:35.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540af0b0 con 0x7f0534003750 2022-08-01T23:04:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053400b750 con 0x7f05540ad750 2022-08-01T23:04:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534004500 0x7f0534004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0552ffd700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0534003750 0x7f0534003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05537fe700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 0x7f0534004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 -- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 msgr2=0x7f0534004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05540ad750 0x7f0534004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 -- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0534003750 msgr2=0x7f0534003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0534003750 0x7f0534003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 -- 172.21.15.32:0/3724533374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534003260 con 0x7f0534004500 2022-08-01T23:04:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f05527fc700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534004500 0x7f0534004980 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f054400c000 tx=0x7f0544006df0).ready entity=mon.0 client_cookie=88168d8360f189a6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f054400e830 con 0x7f0534004500 2022-08-01T23:04:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400e990 con 0x7f0534004500 2022-08-01T23:04:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.388+0000 7f0551ffb700 1 -- 172.21.15.32:0/3724533374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f054400ecf0 con 0x7f0534004500 2022-08-01T23:04:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- 172.21.15.32:0/3724533374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534004500 msgr2=0x7f0534004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 --2- 172.21.15.32:0/3724533374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534004500 0x7f0534004980 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f054400c000 tx=0x7f0544006df0).stop 2022-08-01T23:04:35.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05537fe700 1 -- 172.21.15.32:0/3724533374 reap_dead start 2022-08-01T23:04:35.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- 172.21.15.32:0/3724533374 shutdown_connections 2022-08-01T23:04:35.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- 172.21.15.32:0/3724533374 >> 172.21.15.32:0/3724533374 conn(0x7f0554066f70 msgr2=0x7f0554067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:35.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- 172.21.15.32:0/3724533374 shutdown_connections 2022-08-01T23:04:35.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- 172.21.15.32:0/3724533374 wait complete. 2022-08-01T23:04:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 Processor -- start 2022-08-01T23:04:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- start start 2022-08-01T23:04:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05540b4d90 0x7f055413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 0x7f0554155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0554156100 0x7f05540b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540b5170 con 0x7f055413ee90 2022-08-01T23:04:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540acf70 con 0x7f05540b4d90 2022-08-01T23:04:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0553fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05540b3590 con 0x7f0554156100 2022-08-01T23:04:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 0x7f0554155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 0x7f0554155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48622/0 (socket says 172.21.15.32:48622) 2022-08-01T23:04:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 -- 172.21.15.32:0/3384892823 learned_addr learned my addr 172.21.15.32:0/3384892823 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f0552ffd700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05540b4d90 0x7f055413e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05537fe700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0554156100 0x7f05540b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 -- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0554156100 msgr2=0x7f05540b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0554156100 0x7f05540b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 -- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05540b4d90 msgr2=0x7f055413e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05540b4d90 0x7f055413e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05527fc700 1 -- 172.21.15.32:0/3384892823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05540b2960 con 0x7f055413ee90 2022-08-01T23:04:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.392+0000 7f05537fe700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0554156100 0x7f05540b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f05527fc700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 0x7f0554155c10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f054400c000 tx=0x7f0544019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f054401cce0 con 0x7f055413ee90 2022-08-01T23:04:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544000e90 con 0x7f055413ee90 2022-08-01T23:04:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05540b2c00 con 0x7f055413ee90 2022-08-01T23:04:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05440245e0 con 0x7f055413ee90 2022-08-01T23:04:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055413f2f0 con 0x7f055413ee90 2022-08-01T23:04:35.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f054402a020 con 0x7f055413ee90 2022-08-01T23:04:35.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f054bfff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0538053980 0x7f0538055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f054406d320 con 0x7f055413ee90 2022-08-01T23:04:35.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05540b3750 con 0x7f055413ee90 2022-08-01T23:04:35.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f0552ffd700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0538053980 0x7f0538055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.396+0000 7f0552ffd700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0538053980 0x7f0538055e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f053c00c000 tx=0x7f053c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.400+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05440440e0 con 0x7f055413ee90 2022-08-01T23:04:35.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.568+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f05540af5f0 con 0x7f055413ee90 2022-08-01T23:04:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f054bfff700 1 -- 172.21.15.32:0/3384892823 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f05440320e0 con 0x7f055413ee90 2022-08-01T23:04:35.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0538053980 msgr2=0x7f0538055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0538053980 0x7f0538055e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f053c00c000 tx=0x7f053c008040).stop 2022-08-01T23:04:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 msgr2=0x7f0554155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 0x7f0554155c10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f054400c000 tx=0x7f0544019000).stop 2022-08-01T23:04:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 shutdown_connections 2022-08-01T23:04:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0538053980 0x7f0538055e40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f0554156100 0x7f05540b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055413ee90 0x7f0554155c10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 --2- 172.21.15.32:0/3384892823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05540b4d90 0x7f055413e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 >> 172.21.15.32:0/3384892823 conn(0x7f0554066f70 msgr2=0x7f055413fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 shutdown_connections 2022-08-01T23:04:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.576+0000 7f0553fff700 1 -- 172.21.15.32:0/3384892823 wait complete. 2022-08-01T23:04:35.588 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:04:35.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:04:35.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 Processor -- start 2022-08-01T23:04:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 -- start start 2022-08-01T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c40a9da0 0x7fe1c4060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c41402d0 0x7fe1c4140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c4067030 con 0x7fe1c40b31d0 2022-08-01T23:04:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c4064a70 con 0x7fe1c41402d0 2022-08-01T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.712+0000 7fe1c8ab1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4061f60 con 0x7fe1c40a9da0 2022-08-01T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c41402d0 0x7fe1c4140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c2ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c40a9da0 0x7fe1c4060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c37fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1c40b31d0 0x7fe1c4066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57076/0 (socket says 172.21.15.32:57076) 2022-08-01T23:04:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c37fe700 1 -- 172.21.15.32:0/3618654501 learned_addr learned my addr 172.21.15.32:0/3618654501 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c3fff700 1 -- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c40a9da0 msgr2=0x7fe1c4060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c3fff700 1 --2- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c40a9da0 0x7fe1c4060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c3fff700 1 -- 172.21.15.32:0/3618654501 >> v1:172.21.15.55:6789/0 conn(0x7fe1c40b31d0 legacy=0x7fe1c4066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c3fff700 1 -- 172.21.15.32:0/3618654501 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40b35b0 con 0x7fe1c41402d0 2022-08-01T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c3fff700 1 --2- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c41402d0 0x7fe1c4140760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fe1bc0032f0 tx=0x7fe1bc0091d0).ready entity=mon.1 client_cookie=46258a017c7c466 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c27fc700 1 -- 172.21.15.32:0/3618654501 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00b680 con 0x7fe1c41402d0 2022-08-01T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c27fc700 1 -- 172.21.15.32:0/3618654501 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00b7e0 con 0x7fe1c41402d0 2022-08-01T23:04:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c27fc700 1 -- 172.21.15.32:0/3618654501 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00b990 con 0x7fe1c41402d0 2022-08-01T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c41402d0 msgr2=0x7fe1c4140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c41402d0 0x7fe1c4140760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fe1bc0032f0 tx=0x7fe1bc0091d0).stop 2022-08-01T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/3618654501 shutdown_connections 2022-08-01T23:04:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c41402d0 0x7fe1c4140760 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/3618654501 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c40a9da0 0x7fe1c4060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/3618654501 >> 172.21.15.32:0/3618654501 conn(0x7fe1c4063870 msgr2=0x7fe1c4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/3618654501 shutdown_connections 2022-08-01T23:04:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.716+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/3618654501 wait complete. 2022-08-01T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 Processor -- start 2022-08-01T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 -- start start 2022-08-01T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40a9da0 0x7fe1c40abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 0x7fe1c40b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c41402d0 0x7fe1c40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40adba0 con 0x7fe1c40b31d0 2022-08-01T23:04:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40ac6e0 con 0x7fe1c40a9da0 2022-08-01T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c8ab1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40ac980 con 0x7fe1c41402d0 2022-08-01T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.720+0000 7fe1c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 0x7fe1c40b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 0x7fe1c40b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48636/0 (socket says 172.21.15.32:48636) 2022-08-01T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40a9da0 0x7fe1c40abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 -- 172.21.15.32:0/2401296672 learned_addr learned my addr 172.21.15.32:0/2401296672 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c41402d0 0x7fe1c40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 -- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c41402d0 msgr2=0x7fe1c40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c41402d0 0x7fe1c40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 -- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40a9da0 msgr2=0x7fe1c40abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40a9da0 0x7fe1c40abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 -- 172.21.15.32:0/2401296672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc009470 con 0x7fe1c40b31d0 2022-08-01T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c2ffd700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 0x7fe1c40b1fe0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fe1b800c000 tx=0x7fe1b8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1b8015660 con 0x7fe1c40b31d0 2022-08-01T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c40acc20 con 0x7fe1c40b31d0 2022-08-01T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b80157c0 con 0x7fe1c40b31d0 2022-08-01T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1b8015b20 con 0x7fe1c40b31d0 2022-08-01T23:04:35.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c40abb20 con 0x7fe1c40b31d0 2022-08-01T23:04:35.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c4066af0 con 0x7fe1c40b31d0 2022-08-01T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.724+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe1b801c030 con 0x7fe1c40b31d0 2022-08-01T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.728+0000 7fe1c0ff9700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe1a4053a10 0x7fe1a4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.728+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe1b806dfc0 con 0x7fe1c40b31d0 2022-08-01T23:04:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.728+0000 7fe1c37fe700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe1a4053a10 0x7fe1a4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.728+0000 7fe1c37fe700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe1a4053a10 0x7fe1a4055ed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fe1b400d000 tx=0x7fe1b4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.732+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1b8044d90 con 0x7fe1c40b31d0 2022-08-01T23:04:35.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.900+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe1c40574e0 con 0x7fe1c40b31d0 2022-08-01T23:04:35.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c0ff9700 1 -- 172.21.15.32:0/2401296672 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe1b80483b0 con 0x7fe1c40b31d0 2022-08-01T23:04:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe1a4053a10 msgr2=0x7fe1a4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe1a4053a10 0x7fe1a4055ed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fe1b400d000 tx=0x7fe1b4009040).stop 2022-08-01T23:04:35.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 msgr2=0x7fe1c40b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 0x7fe1c40b1fe0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fe1b800c000 tx=0x7fe1b8006df0).stop 2022-08-01T23:04:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 shutdown_connections 2022-08-01T23:04:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe1a4053a10 0x7fe1a4055ed0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe1c41402d0 0x7fe1c40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1c40b31d0 0x7fe1c40b1fe0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 --2- 172.21.15.32:0/2401296672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40a9da0 0x7fe1c40abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 >> 172.21.15.32:0/2401296672 conn(0x7fe1c4063870 msgr2=0x7fe1c40b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:35.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 shutdown_connections 2022-08-01T23:04:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:35.908+0000 7fe1c8ab1700 1 -- 172.21.15.32:0/2401296672 wait complete. 2022-08-01T23:04:35.919 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:04:35.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:36.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 Processor -- start 2022-08-01T23:04:36.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- start start 2022-08-01T23:04:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff0142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff00ad000 0x7faff01433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff00b0c80 con 0x7faff00b3260 2022-08-01T23:04:36.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff005cd30 con 0x7faff00ad000 2022-08-01T23:04:36.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff00ae810 con 0x7faff00b08a0 2022-08-01T23:04:36.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff00ad000 0x7faff01433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff00ad000 0x7faff01433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48646/0 (socket says 172.21.15.32:48646) 2022-08-01T23:04:36.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff552b700 1 --2- 172.21.15.32:0/57589050 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff0142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff4d2a700 1 --1- 172.21.15.32:0/57589050 >> v1:172.21.15.55:6789/0 conn(0x7faff00b3260 0x7faff005c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57092/0 (socket says 172.21.15.32:57092) 2022-08-01T23:04:36.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 -- 172.21.15.32:0/57589050 learned_addr learned my addr 172.21.15.32:0/57589050 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:36.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 -- 172.21.15.32:0/57589050 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 msgr2=0x7faff0142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 --2- 172.21.15.32:0/57589050 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff0142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 -- 172.21.15.32:0/57589050 >> v1:172.21.15.55:6789/0 conn(0x7faff00b3260 legacy=0x7faff005c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7fafeffff700 1 -- 172.21.15.32:0/57589050 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397551665 0 0) 0x7faff00b0c80 con 0x7faff00b3260 2022-08-01T23:04:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 -- 172.21.15.32:0/57589050 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00aeab0 con 0x7faff00ad000 2022-08-01T23:04:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff5d2c700 1 --2- 172.21.15.32:0/57589050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00ad000 0x7faff01433a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fafe0002f70 tx=0x7fafe0006eb0).ready entity=mon.0 client_cookie=8cc1dfd86598da02 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7fafeffff700 1 -- 172.21.15.32:0/57589050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafe000d570 con 0x7faff00ad000 2022-08-01T23:04:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7fafeffff700 1 -- 172.21.15.32:0/57589050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0000e90 con 0x7faff00ad000 2022-08-01T23:04:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7fafeffff700 1 -- 172.21.15.32:0/57589050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafe000d8c0 con 0x7faff00ad000 2022-08-01T23:04:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- 172.21.15.32:0/57589050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00ad000 msgr2=0x7faff01433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 --2- 172.21.15.32:0/57589050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00ad000 0x7faff01433a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fafe0002f70 tx=0x7fafe0006eb0).stop 2022-08-01T23:04:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- 172.21.15.32:0/57589050 shutdown_connections 2022-08-01T23:04:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 --2- 172.21.15.32:0/57589050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00ad000 0x7faff01433a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 --2- 172.21.15.32:0/57589050 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff0142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- 172.21.15.32:0/57589050 >> 172.21.15.32:0/57589050 conn(0x7faff00612d0 msgr2=0x7faff0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- 172.21.15.32:0/57589050 shutdown_connections 2022-08-01T23:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.032+0000 7faff652d700 1 -- 172.21.15.32:0/57589050 wait complete. 2022-08-01T23:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 Processor -- start 2022-08-01T23:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 -- start start 2022-08-01T23:04:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff00ad000 0x7faff0143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff00ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 0x7faff00ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff00ac770 con 0x7faff00b3260 2022-08-01T23:04:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff0067580 con 0x7faff00ad000 2022-08-01T23:04:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff00ab030 con 0x7faff00b08a0 2022-08-01T23:04:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 0x7faff00ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 0x7faff00ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48652/0 (socket says 172.21.15.32:48652) 2022-08-01T23:04:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 -- 172.21.15.32:0/1985457214 learned_addr learned my addr 172.21.15.32:0/1985457214 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff4d2a700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff00ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff552b700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff00ad000 0x7faff0143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 -- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 msgr2=0x7faff00ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff00ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 -- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff00ad000 msgr2=0x7faff0143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff00ad000 0x7faff0143300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 -- 172.21.15.32:0/1985457214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe000b1e0 con 0x7faff00b3260 2022-08-01T23:04:36.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff5d2c700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 0x7faff00ab950 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7fafe0002f70 tx=0x7fafe0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:36.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafe0011760 con 0x7faff00b3260 2022-08-01T23:04:36.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe00118c0 con 0x7faff00b3260 2022-08-01T23:04:36.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff00ac4d0 con 0x7faff00b3260 2022-08-01T23:04:36.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafe0011c20 con 0x7faff00b3260 2022-08-01T23:04:36.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.036+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff00ac0d0 con 0x7faff00b3260 2022-08-01T23:04:36.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.040+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fafe0025030 con 0x7faff00b3260 2022-08-01T23:04:36.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.040+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff00ab170 con 0x7faff00b3260 2022-08-01T23:04:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.040+0000 7fafee7fc700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafd0053a70 0x7fafd0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.040+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fafe006d380 con 0x7faff00b3260 2022-08-01T23:04:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.040+0000 7faff552b700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafd0053a70 0x7fafd0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.040+0000 7faff552b700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafd0053a70 0x7fafd0055f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fafe400b000 tx=0x7fafe4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.044+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fafe00440a0 con 0x7faff00b3260 2022-08-01T23:04:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.208+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faff00574e0 con 0x7faff00b3260 2022-08-01T23:04:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.208+0000 7fafee7fc700 1 -- 172.21.15.32:0/1985457214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2489 (secure 0 0 0) 0x7fafe0023020 con 0x7faff00b3260 2022-08-01T23:04:36.215 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:36.216 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:34.675417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8079":{"gid":8079,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/766376033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":766376033},{"type":"v1","addr":"172.21.15.32:6835","nonce":766376033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafd0053a70 msgr2=0x7fafd0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafd0053a70 0x7fafd0055f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fafe400b000 tx=0x7fafe4006df0).stop 2022-08-01T23:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 msgr2=0x7faff00ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 0x7faff00ab950 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7fafe0002f70 tx=0x7fafe0015000).stop 2022-08-01T23:04:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 shutdown_connections 2022-08-01T23:04:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fafd0053a70 0x7fafd0055f30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff00b3260 0x7faff00ab950 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faff00b08a0 0x7faff00ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 --2- 172.21.15.32:0/1985457214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff00ad000 0x7faff0143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 >> 172.21.15.32:0/1985457214 conn(0x7faff00612d0 msgr2=0x7faff0141260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 shutdown_connections 2022-08-01T23:04:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.212+0000 7faff652d700 1 -- 172.21.15.32:0/1985457214 wait complete. 2022-08-01T23:04:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 68 2022-08-01T23:04:36.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:04:33.672583+0000', 'modified': '2022-08-01T23:04:34.675417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8079': {'gid': 8079, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/766376033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 766376033}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 766376033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:04:36.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:04:36.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:04:36.660 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8079,"want_state":"up:active","state":"up:active","rank_uptime":2.9665229129999999,"mdsmap_epoch":68,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":7.7666657299999997} 2022-08-01T23:04:36.661 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8079, "mdsmap_epoch": 68, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 2.966522913, "state": "up:active", "uptime": 7.76666573, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:04:36.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:36.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.764+0000 7f142e220700 1 Processor -- start 2022-08-01T23:04:36.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.764+0000 7f142e220700 1 -- start start 2022-08-01T23:04:36.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.764+0000 7f142e220700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14280af120 0x7f1428140150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.764+0000 7f142e220700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14280b0c40 con 0x7f14280b0570 2022-08-01T23:04:36.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.764+0000 7f142e220700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14280b0ee0 con 0x7f14280ad670 2022-08-01T23:04:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.764+0000 7f142e220700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14280ae9e0 con 0x7f14280af120 2022-08-01T23:04:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14280af120 0x7f1428140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14280af120 0x7f1428140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48658/0 (socket says 172.21.15.32:48658) 2022-08-01T23:04:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 -- 172.21.15.32:0/1758880263 learned_addr learned my addr 172.21.15.32:0/1758880263 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:36.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142ca1d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14280b0570 0x7f142813fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36202/0 (socket says 172.21.15.32:36202) 2022-08-01T23:04:36.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 -- 172.21.15.32:0/1758880263 >> v1:172.21.15.55:6789/0 conn(0x7f14280ad670 legacy=0x7f14280b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 -- 172.21.15.32:0/1758880263 >> v1:172.21.15.32:6789/0 conn(0x7f14280b0570 legacy=0x7f142813fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f141ffff700 1 -- 172.21.15.32:0/1758880263 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586899552 0 0) 0x7f14280b0ee0 con 0x7f14280ad670 2022-08-01T23:04:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 -- 172.21.15.32:0/1758880263 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280aec80 con 0x7f14280af120 2022-08-01T23:04:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142da1f700 1 --2- 172.21.15.32:0/1758880263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280af120 0x7f1428140150 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f1424002f70 tx=0x7f1424006e40).ready entity=mon.0 client_cookie=a547edd1157f6824 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f141ffff700 1 -- 172.21.15.32:0/1758880263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f142400bd20 con 0x7f14280af120 2022-08-01T23:04:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f141ffff700 1 -- 172.21.15.32:0/1758880263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424000e90 con 0x7f14280af120 2022-08-01T23:04:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f141ffff700 1 -- 172.21.15.32:0/1758880263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14240155e0 con 0x7f14280af120 2022-08-01T23:04:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 -- 172.21.15.32:0/1758880263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280af120 msgr2=0x7f1428140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 --2- 172.21.15.32:0/1758880263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280af120 0x7f1428140150 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f1424002f70 tx=0x7f1424006e40).stop 2022-08-01T23:04:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 -- 172.21.15.32:0/1758880263 shutdown_connections 2022-08-01T23:04:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 --2- 172.21.15.32:0/1758880263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280af120 0x7f1428140150 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 -- 172.21.15.32:0/1758880263 >> 172.21.15.32:0/1758880263 conn(0x7f1428061360 msgr2=0x7f1428062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 -- 172.21.15.32:0/1758880263 shutdown_connections 2022-08-01T23:04:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.768+0000 7f142e220700 1 -- 172.21.15.32:0/1758880263 wait complete. 2022-08-01T23:04:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 Processor -- start 2022-08-01T23:04:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 -- start start 2022-08-01T23:04:36.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14280ad670 0x7f142813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14280af120 0x7f1428065b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 0x7f1428066010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1428066500 con 0x7f14280b0570 2022-08-01T23:04:36.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14280abe90 con 0x7f14280ad670 2022-08-01T23:04:36.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14280ac130 con 0x7f14280af120 2022-08-01T23:04:36.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 0x7f1428066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 0x7f1428066010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48666/0 (socket says 172.21.15.32:48666) 2022-08-01T23:04:36.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 -- 172.21.15.32:0/1412618448 learned_addr learned my addr 172.21.15.32:0/1412618448 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:36.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142ca1d700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14280af120 0x7f1428065b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142d21e700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14280ad670 0x7f142813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 -- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14280af120 msgr2=0x7f1428065b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14280af120 0x7f1428065b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 -- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14280ad670 msgr2=0x7f142813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14280ad670 0x7f142813cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 -- 172.21.15.32:0/1412618448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142400d4d0 con 0x7f14280b0570 2022-08-01T23:04:36.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142da1f700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 0x7f1428066010 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f1424002f70 tx=0x7f1424018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:36.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1424009070 con 0x7f14280b0570 2022-08-01T23:04:36.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424015980 con 0x7f14280b0570 2022-08-01T23:04:36.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14280ac3d0 con 0x7f14280b0570 2022-08-01T23:04:36.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.772+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1424015ce0 con 0x7f14280b0570 2022-08-01T23:04:36.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142813d0f0 con 0x7f14280b0570 2022-08-01T23:04:36.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14280b34e0 con 0x7f14280b0570 2022-08-01T23:04:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f142400c3e0 con 0x7f14280b0570 2022-08-01T23:04:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f141e7fc700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1410053a70 0x7f1410055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f142406ce00 con 0x7f14280b0570 2022-08-01T23:04:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f142d21e700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1410053a70 0x7f1410055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.776+0000 7f142d21e700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1410053a70 0x7f1410055f30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f142000d000 tx=0x7f1420007d30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.784+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1424043ba0 con 0x7f14280b0570 2022-08-01T23:04:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.908+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14280574e0 con 0x7f14280b0570 2022-08-01T23:04:36.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.908+0000 7f141e7fc700 1 -- 172.21.15.32:0/1412618448 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f1424023020 con 0x7f14280b0570 2022-08-01T23:04:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":137,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:34.676165+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:04:31.641583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T23:04:32.681709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1410053a70 msgr2=0x7f1410055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1410053a70 0x7f1410055f30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f142000d000 tx=0x7f1420007d30).stop 2022-08-01T23:04:36.917 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 msgr2=0x7f1428066010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 0x7f1428066010 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f1424002f70 tx=0x7f1424018000).stop 2022-08-01T23:04:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 shutdown_connections 2022-08-01T23:04:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1410053a70 0x7f1410055f30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14280b0570 0x7f1428066010 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14280af120 0x7f1428065b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 --2- 172.21.15.32:0/1412618448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14280ad670 0x7f142813cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 >> 172.21.15.32:0/1412618448 conn(0x7f1428061360 msgr2=0x7f142813f030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 shutdown_connections 2022-08-01T23:04:36.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:36.912+0000 7f142e220700 1 -- 172.21.15.32:0/1412618448 wait complete. 2022-08-01T23:04:36.929 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:04:31.641583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:04:32.681709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:04:36.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:04:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 Processor -- start 2022-08-01T23:04:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 -- start start 2022-08-01T23:04:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14e80b2050 0x7f14e80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80b0140 0x7f14e80b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e80b05b0 con 0x7f14e80a7250 2022-08-01T23:04:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e80b2920 con 0x7f14e80b2050 2022-08-01T23:04:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14eddcc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80b07e0 con 0x7f14e80b0140 2022-08-01T23:04:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14e6ffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80b0140 0x7f14e80b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14e77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14e80b2050 0x7f14e80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14e7fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f14e80a7250 0x7f14e81413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57118/0 (socket says 172.21.15.32:57118) 2022-08-01T23:04:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14e7fff700 1 -- 172.21.15.32:0/1009394900 learned_addr learned my addr 172.21.15.32:0/1009394900 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.036+0000 7f14e67fc700 1 -- 172.21.15.32:0/1009394900 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101183447 0 0) 0x7f14e80b05b0 con 0x7f14e80a7250 2022-08-01T23:04:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e67fc700 1 -- 172.21.15.32:0/1009394900 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d4003260 con 0x7f14e80a7250 2022-08-01T23:04:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e6ffd700 1 -- 172.21.15.32:0/1009394900 >> v2:172.21.15.55:3300/0 conn(0x7f14e80b2050 msgr2=0x7f14e80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e6ffd700 1 --2- 172.21.15.32:0/1009394900 >> v2:172.21.15.55:3300/0 conn(0x7f14e80b2050 0x7f14e80b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e6ffd700 1 -- 172.21.15.32:0/1009394900 >> v1:172.21.15.55:6789/0 conn(0x7f14e80a7250 legacy=0x7f14e81413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e6ffd700 1 -- 172.21.15.32:0/1009394900 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e80b0a80 con 0x7f14e80b0140 2022-08-01T23:04:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e6ffd700 1 --2- 172.21.15.32:0/1009394900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80b0140 0x7f14e80b2430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f14d8002f70 tx=0x7f14d8006df0).ready entity=mon.2 client_cookie=ef0b24d04d131d1b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e67fc700 1 -- 172.21.15.32:0/1009394900 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d800d4a0 con 0x7f14e80b0140 2022-08-01T23:04:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e67fc700 1 -- 172.21.15.32:0/1009394900 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d8000e90 con 0x7f14e80b0140 2022-08-01T23:04:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14e67fc700 1 -- 172.21.15.32:0/1009394900 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d800d7f0 con 0x7f14e80b0140 2022-08-01T23:04:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 -- 172.21.15.32:0/1009394900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80b0140 msgr2=0x7f14e80b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 --2- 172.21.15.32:0/1009394900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80b0140 0x7f14e80b2430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f14d8002f70 tx=0x7f14d8006df0).stop 2022-08-01T23:04:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 -- 172.21.15.32:0/1009394900 shutdown_connections 2022-08-01T23:04:37.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 --2- 172.21.15.32:0/1009394900 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80b0140 0x7f14e80b2430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 --2- 172.21.15.32:0/1009394900 >> v2:172.21.15.55:3300/0 conn(0x7f14e80b2050 0x7f14e80b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 -- 172.21.15.32:0/1009394900 >> 172.21.15.32:0/1009394900 conn(0x7f14e805f840 msgr2=0x7f14e805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 -- 172.21.15.32:0/1009394900 shutdown_connections 2022-08-01T23:04:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 -- 172.21.15.32:0/1009394900 wait complete. 2022-08-01T23:04:37.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 Processor -- start 2022-08-01T23:04:37.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.040+0000 7f14eddcc700 1 -- start start 2022-08-01T23:04:37.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80a7250 0x7f14e80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e80b0140 0x7f14e80b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14e80b2050 0x7f14e8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e8156fc0 con 0x7f14e80b0140 2022-08-01T23:04:37.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e8141fd0 con 0x7f14e80b2050 2022-08-01T23:04:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e814aa20 con 0x7f14e80a7250 2022-08-01T23:04:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e80b0140 0x7f14e80b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e80b0140 0x7f14e80b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48678/0 (socket says 172.21.15.32:48678) 2022-08-01T23:04:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e6ffd700 1 -- 172.21.15.32:0/269106981 learned_addr learned my addr 172.21.15.32:0/269106981 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e7fff700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14e80b2050 0x7f14e8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80a7250 0x7f14e80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 -- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14e80b2050 msgr2=0x7f14e8156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14e80b2050 0x7f14e8156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 -- 172.21.15.32:0/269106981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e80b0140 msgr2=0x7f14e80b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e80b0140 0x7f14e80b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 -- 172.21.15.32:0/269106981 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d800b180 con 0x7f14e80a7250 2022-08-01T23:04:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e7fff700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14e80b2050 0x7f14e8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e77fe700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80a7250 0x7f14e80b6220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f14e80b6170 tx=0x7f14dc006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14dc011750 con 0x7f14e80a7250 2022-08-01T23:04:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc0118b0 con 0x7f14e80a7250 2022-08-01T23:04:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e815abc0 con 0x7f14e80a7250 2022-08-01T23:04:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14dc011c10 con 0x7f14e80a7250 2022-08-01T23:04:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e8142230 con 0x7f14e80a7250 2022-08-01T23:04:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.044+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e80b0d20 con 0x7f14e80a7250 2022-08-01T23:04:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.048+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f14dc011d70 con 0x7f14e80a7250 2022-08-01T23:04:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.048+0000 7f14e4ff9700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14c80539e0 0x7f14c8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.048+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f14dc06c5b0 con 0x7f14e80a7250 2022-08-01T23:04:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.048+0000 7f14e6ffd700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14c80539e0 0x7f14c8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.048+0000 7f14e6ffd700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14c80539e0 0x7f14c8055ea0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f14d801a000 tx=0x7f14d8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.056+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14dc043180 con 0x7f14e80a7250 2022-08-01T23:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.184+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f14e80574e0 con 0x7f14e80a7250 2022-08-01T23:04:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.184+0000 7f14e4ff9700 1 -- 172.21.15.32:0/269106981 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f14dc0467a0 con 0x7f14e80a7250 2022-08-01T23:04:37.191 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:04:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14c80539e0 msgr2=0x7f14c8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14c80539e0 0x7f14c8055ea0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f14d801a000 tx=0x7f14d8014000).stop 2022-08-01T23:04:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80a7250 msgr2=0x7f14e80b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80a7250 0x7f14e80b6220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f14e80b6170 tx=0x7f14dc006df0).stop 2022-08-01T23:04:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 shutdown_connections 2022-08-01T23:04:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14c80539e0 0x7f14c8055ea0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14e80b2050 0x7f14e8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14e80b0140 0x7f14e80b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 --2- 172.21.15.32:0/269106981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14e80a7250 0x7f14e80b6220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 >> 172.21.15.32:0/269106981 conn(0x7f14e805f840 msgr2=0x7f14e80af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 shutdown_connections 2022-08-01T23:04:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.188+0000 7f14eddcc700 1 -- 172.21.15.32:0/269106981 wait complete. 2022-08-01T23:04:37.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:04:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.312+0000 7fa2de4c4700 1 Processor -- start 2022-08-01T23:04:37.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.312+0000 7fa2de4c4700 1 -- start start 2022-08-01T23:04:37.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2de4c4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2d81445d0 0x7fa2d8144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2de4c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2d80655b0 0x7fa2d8144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2de4c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d81454e0 con 0x7fa2d8060290 2022-08-01T23:04:37.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2de4c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d8065e10 con 0x7fa2d80655b0 2022-08-01T23:04:37.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2de4c4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d8060670 con 0x7fa2d81445d0 2022-08-01T23:04:37.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2d80655b0 0x7fa2d8144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2d80655b0 0x7fa2d8144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48688/0 (socket says 172.21.15.32:48688) 2022-08-01T23:04:37.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 -- 172.21.15.32:0/2761041950 learned_addr learned my addr 172.21.15.32:0/2761041950 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:37.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2cf7fe700 1 --2- 172.21.15.32:0/2761041950 >> v2:172.21.15.55:3300/0 conn(0x7fa2d81445d0 0x7fa2d8144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 -- 172.21.15.32:0/2761041950 >> v2:172.21.15.55:3300/0 conn(0x7fa2d81445d0 msgr2=0x7fa2d8144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:37.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 --2- 172.21.15.32:0/2761041950 >> v2:172.21.15.55:3300/0 conn(0x7fa2d81445d0 0x7fa2d8144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 -- 172.21.15.32:0/2761041950 >> v1:172.21.15.32:6789/0 conn(0x7fa2d8060290 legacy=0x7fa2d8064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2d77fe700 1 -- 172.21.15.32:0/2761041950 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335405773 0 0) 0x7fa2d81454e0 con 0x7fa2d8060290 2022-08-01T23:04:37.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 -- 172.21.15.32:0/2761041950 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80b2540 con 0x7fa2d80655b0 2022-08-01T23:04:37.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2dca61700 1 --2- 172.21.15.32:0/2761041950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d80655b0 0x7fa2d8144ff0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fa2d00032f0 tx=0x7fa2d0009290).ready entity=mon.0 client_cookie=473c2d18e279e23f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2d77fe700 1 -- 172.21.15.32:0/2761041950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2d000b870 con 0x7fa2d80655b0 2022-08-01T23:04:37.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2d77fe700 1 -- 172.21.15.32:0/2761041950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000b9d0 con 0x7fa2d80655b0 2022-08-01T23:04:37.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.316+0000 7fa2d77fe700 1 -- 172.21.15.32:0/2761041950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2d000bb40 con 0x7fa2d80655b0 2022-08-01T23:04:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2761041950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d80655b0 msgr2=0x7fa2d8144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2761041950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d80655b0 0x7fa2d8144ff0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fa2d00032f0 tx=0x7fa2d0009290).stop 2022-08-01T23:04:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2761041950 shutdown_connections 2022-08-01T23:04:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2761041950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d80655b0 0x7fa2d8144ff0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2761041950 >> v2:172.21.15.55:3300/0 conn(0x7fa2d81445d0 0x7fa2d8144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2761041950 >> 172.21.15.32:0/2761041950 conn(0x7fa2d805d0f0 msgr2=0x7fa2d805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2761041950 shutdown_connections 2022-08-01T23:04:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2761041950 wait complete. 2022-08-01T23:04:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 Processor -- start 2022-08-01T23:04:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- start start 2022-08-01T23:04:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 0x7fa2d81413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2d80655b0 0x7fa2d8150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d80ada00 0x7fa2d8150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8154900 con 0x7fa2d8060290 2022-08-01T23:04:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80b0d20 con 0x7fa2d80ada00 2022-08-01T23:04:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2de4c4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80b0fc0 con 0x7fa2d80655b0 2022-08-01T23:04:37.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 0x7fa2d81413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 0x7fa2d81413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48690/0 (socket says 172.21.15.32:48690) 2022-08-01T23:04:37.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2d7fff700 1 -- 172.21.15.32:0/2055006254 learned_addr learned my addr 172.21.15.32:0/2055006254 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2dca61700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d80ada00 0x7fa2d8150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2cf7fe700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2d80655b0 0x7fa2d8150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2d7fff700 1 -- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2d80655b0 msgr2=0x7fa2d8150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.320+0000 7fa2d7fff700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2d80655b0 0x7fa2d8150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d7fff700 1 -- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d80ada00 msgr2=0x7fa2d8150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d7fff700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d80ada00 0x7fa2d8150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d7fff700 1 -- 172.21.15.32:0/2055006254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009530 con 0x7fa2d8060290 2022-08-01T23:04:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2dca61700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d80ada00 0x7fa2d8150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:37.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2cf7fe700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2d80655b0 0x7fa2d8150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:37.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d7fff700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 0x7fa2d81413b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fa2c8000bc0 tx=0x7fa2c8007a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2c800ce00 con 0x7fa2d8060290 2022-08-01T23:04:37.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c800f470 con 0x7fa2d8060290 2022-08-01T23:04:37.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d80b1260 con 0x7fa2d8060290 2022-08-01T23:04:37.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2c800f7d0 con 0x7fa2d8060290 2022-08-01T23:04:37.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80b1700 con 0x7fa2d8060290 2022-08-01T23:04:37.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d80b13f0 con 0x7fa2d8060290 2022-08-01T23:04:37.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa2c8019030 con 0x7fa2d8060290 2022-08-01T23:04:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d5ffb700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2b0053a70 0x7fa2b0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa2c806b3c0 con 0x7fa2d8060290 2022-08-01T23:04:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2cf7fe700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2b0053a70 0x7fa2b0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.324+0000 7fa2cf7fe700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2b0053a70 0x7fa2b0055f30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fa2c000b000 tx=0x7fa2c0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.332+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2c8042230 con 0x7fa2d8060290 2022-08-01T23:04:37.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2d80574e0 con 0x7fa2d8060290 2022-08-01T23:04:37.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2d5ffb700 1 -- 172.21.15.32:0/2055006254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2c8045850 con 0x7fa2d8060290 2022-08-01T23:04:37.462 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:37.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2b0053a70 msgr2=0x7fa2b0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2b0053a70 0x7fa2b0055f30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fa2c000b000 tx=0x7fa2c0008040).stop 2022-08-01T23:04:37.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 msgr2=0x7fa2d81413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 0x7fa2d81413b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fa2c8000bc0 tx=0x7fa2c8007a70).stop 2022-08-01T23:04:37.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 shutdown_connections 2022-08-01T23:04:37.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa2b0053a70 0x7fa2b0055f30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d80ada00 0x7fa2d8150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa2d80655b0 0x7fa2d8150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 --2- 172.21.15.32:0/2055006254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d8060290 0x7fa2d81413b0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 >> 172.21.15.32:0/2055006254 conn(0x7fa2d805d0f0 msgr2=0x7fa2d80adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 shutdown_connections 2022-08-01T23:04:37.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.456+0000 7fa2de4c4700 1 -- 172.21.15.32:0/2055006254 wait complete. 2022-08-01T23:04:37.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.576+0000 7fbf93fff700 1 Processor -- start 2022-08-01T23:04:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.576+0000 7fbf93fff700 1 -- start start 2022-08-01T23:04:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b2730 0x7fbf940666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf940654c0 con 0x7fbf94066ba0 2022-08-01T23:04:37.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf940b3570 con 0x7fbf940b3190 2022-08-01T23:04:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf94062190 con 0x7fbf940b2730 2022-08-01T23:04:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf937fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf94066ba0 0x7fbf9414dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36244/0 (socket says 172.21.15.32:36244) 2022-08-01T23:04:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf937fe700 1 -- 172.21.15.32:0/85949530 learned_addr learned my addr 172.21.15.32:0/85949530 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:37.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf8a7fc700 1 --2- 172.21.15.32:0/85949530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b2730 0x7fbf940666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf927fc700 1 -- 172.21.15.32:0/85949530 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014907086 0 0) 0x7fbf940654c0 con 0x7fbf94066ba0 2022-08-01T23:04:37.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf927fc700 1 -- 172.21.15.32:0/85949530 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf78003260 con 0x7fbf94066ba0 2022-08-01T23:04:37.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf927fc700 1 -- 172.21.15.32:0/85949530 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526959250 0 0) 0x7fbf940b3570 con 0x7fbf940b3190 2022-08-01T23:04:37.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/85949530 >> v1:172.21.15.55:6789/0 conn(0x7fbf940b3190 legacy=0x7fbf94060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/85949530 >> v1:172.21.15.32:6789/0 conn(0x7fbf94066ba0 legacy=0x7fbf9414dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/85949530 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9405fbf0 con 0x7fbf940b2730 2022-08-01T23:04:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf8a7fc700 1 --2- 172.21.15.32:0/85949530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b2730 0x7fbf940666b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fbf7c002f70 tx=0x7fbf7c006df0).ready entity=mon.2 client_cookie=2fe53df426f878f5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf927fc700 1 -- 172.21.15.32:0/85949530 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7c00d5c0 con 0x7fbf940b2730 2022-08-01T23:04:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf927fc700 1 -- 172.21.15.32:0/85949530 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c000e90 con 0x7fbf940b2730 2022-08-01T23:04:37.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf927fc700 1 -- 172.21.15.32:0/85949530 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7c00d910 con 0x7fbf940b2730 2022-08-01T23:04:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- 172.21.15.32:0/85949530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b2730 msgr2=0x7fbf940666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 --2- 172.21.15.32:0/85949530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b2730 0x7fbf940666b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fbf7c002f70 tx=0x7fbf7c006df0).stop 2022-08-01T23:04:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- 172.21.15.32:0/85949530 shutdown_connections 2022-08-01T23:04:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 --2- 172.21.15.32:0/85949530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b2730 0x7fbf940666b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- 172.21.15.32:0/85949530 >> 172.21.15.32:0/85949530 conn(0x7fbf94063870 msgr2=0x7fbf94063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- 172.21.15.32:0/85949530 shutdown_connections 2022-08-01T23:04:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.580+0000 7fbf93fff700 1 -- 172.21.15.32:0/85949530 wait complete. 2022-08-01T23:04:37.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 Processor -- start 2022-08-01T23:04:37.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 -- start start 2022-08-01T23:04:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf94066ba0 0x7fbf94148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 0x7fbf941436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b3190 0x7fbf94143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940a2f30 con 0x7fbf940b2730 2022-08-01T23:04:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940ac980 con 0x7fbf94066ba0 2022-08-01T23:04:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf93fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940ad0e0 con 0x7fbf940b3190 2022-08-01T23:04:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 0x7fbf941436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 0x7fbf941436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48706/0 (socket says 172.21.15.32:48706) 2022-08-01T23:04:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/3285940042 learned_addr learned my addr 172.21.15.32:0/3285940042 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf937fe700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b3190 0x7fbf94143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf92ffd700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf94066ba0 0x7fbf94148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b3190 msgr2=0x7fbf94143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b3190 0x7fbf94143be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf94066ba0 msgr2=0x7fbf94148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf94066ba0 0x7fbf94148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf8a7fc700 1 -- 172.21.15.32:0/3285940042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c00b180 con 0x7fbf940b2730 2022-08-01T23:04:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.584+0000 7fbf92ffd700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf94066ba0 0x7fbf94148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf8a7fc700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 0x7fbf941436f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fbf7c002f70 tx=0x7fbf7c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7c016070 con 0x7fbf940b2730 2022-08-01T23:04:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00de40 con 0x7fbf940b2730 2022-08-01T23:04:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf940ad380 con 0x7fbf940b2730 2022-08-01T23:04:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7c013dd0 con 0x7fbf940b2730 2022-08-01T23:04:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf940ab880 con 0x7fbf940b2730 2022-08-01T23:04:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf940b0b20 con 0x7fbf940b2730 2022-08-01T23:04:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf7c024030 con 0x7fbf940b2730 2022-08-01T23:04:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf90ff9700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf6c053a70 0x7fbf6c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbf7c06cbd0 con 0x7fbf940b2730 2022-08-01T23:04:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.588+0000 7fbf92ffd700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf6c053a70 0x7fbf6c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.592+0000 7fbf92ffd700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf6c053a70 0x7fbf6c055f30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fbf8400b000 tx=0x7fbf84009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.596+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf7c043910 con 0x7fbf940b2730 2022-08-01T23:04:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf940ade40 con 0x7fbf940b2730 2022-08-01T23:04:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf90ff9700 1 -- 172.21.15.32:0/3285940042 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbf7c0220a0 con 0x7fbf940b2730 2022-08-01T23:04:37.730 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf6c053a70 msgr2=0x7fbf6c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf6c053a70 0x7fbf6c055f30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fbf8400b000 tx=0x7fbf84009040).stop 2022-08-01T23:04:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 msgr2=0x7fbf941436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 0x7fbf941436f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fbf7c002f70 tx=0x7fbf7c014000).stop 2022-08-01T23:04:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 shutdown_connections 2022-08-01T23:04:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbf6c053a70 0x7fbf6c055f30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbf940b3190 0x7fbf94143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf940b2730 0x7fbf941436f0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 --2- 172.21.15.32:0/3285940042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf94066ba0 0x7fbf94148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 >> 172.21.15.32:0/3285940042 conn(0x7fbf94063870 msgr2=0x7fbf940b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 shutdown_connections 2022-08-01T23:04:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.724+0000 7fbf93fff700 1 -- 172.21.15.32:0/3285940042 wait complete. 2022-08-01T23:04:37.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:04:37.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b8d27700 1 Processor -- start 2022-08-01T23:04:37.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b8d27700 1 -- start start 2022-08-01T23:04:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b8d27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b4066b60 0x7f83b413e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b8d27700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b405f8e0 con 0x7f83b40b1bc0 2022-08-01T23:04:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b8d27700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40b1630 con 0x7f83b4066780 2022-08-01T23:04:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b8d27700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b405fb10 con 0x7f83b4066b60 2022-08-01T23:04:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b4066b60 0x7f83b413e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83b40b1bc0 0x7f83b4140e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36256/0 (socket says 172.21.15.32:36256) 2022-08-01T23:04:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b2ffd700 1 -- 172.21.15.32:0/3764166795 learned_addr learned my addr 172.21.15.32:0/3764166795 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b27fc700 1 -- 172.21.15.32:0/3764166795 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841305508 0 0) 0x7f83b405f8e0 con 0x7f83b40b1bc0 2022-08-01T23:04:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b27fc700 1 -- 172.21.15.32:0/3764166795 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a0003260 con 0x7f83b40b1bc0 2022-08-01T23:04:37.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b3fff700 1 -- 172.21.15.32:0/3764166795 >> v1:172.21.15.55:6789/0 conn(0x7f83b4066780 legacy=0x7f83b405ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b3fff700 1 -- 172.21.15.32:0/3764166795 >> v1:172.21.15.32:6789/0 conn(0x7f83b40b1bc0 legacy=0x7f83b4140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b3fff700 1 -- 172.21.15.32:0/3764166795 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40b18d0 con 0x7f83b4066b60 2022-08-01T23:04:37.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.852+0000 7f83b3fff700 1 --2- 172.21.15.32:0/3764166795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066b60 0x7f83b413e2b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f83ac0032f0 tx=0x7f83ac0091d0).ready entity=mon.0 client_cookie=c7a4a2e2f4f49bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b27fc700 1 -- 172.21.15.32:0/3764166795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ac00b6c0 con 0x7f83b4066b60 2022-08-01T23:04:37.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b27fc700 1 -- 172.21.15.32:0/3764166795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac00b820 con 0x7f83b4066b60 2022-08-01T23:04:37.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b27fc700 1 -- 172.21.15.32:0/3764166795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ac00b990 con 0x7f83b4066b60 2022-08-01T23:04:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 -- 172.21.15.32:0/3764166795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066b60 msgr2=0x7f83b413e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 --2- 172.21.15.32:0/3764166795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066b60 0x7f83b413e2b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f83ac0032f0 tx=0x7f83ac0091d0).stop 2022-08-01T23:04:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 -- 172.21.15.32:0/3764166795 shutdown_connections 2022-08-01T23:04:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 --2- 172.21.15.32:0/3764166795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066b60 0x7f83b413e2b0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 -- 172.21.15.32:0/3764166795 >> 172.21.15.32:0/3764166795 conn(0x7f83b4063870 msgr2=0x7f83b4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 -- 172.21.15.32:0/3764166795 shutdown_connections 2022-08-01T23:04:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 -- 172.21.15.32:0/3764166795 wait complete. 2022-08-01T23:04:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 Processor -- start 2022-08-01T23:04:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.856+0000 7f83b8d27700 1 -- start start 2022-08-01T23:04:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 0x7f83b4140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83b4066b60 0x7f83b4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b40b1bc0 0x7f83b4154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4144da0 con 0x7f83b4066780 2022-08-01T23:04:37.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4142380 con 0x7f83b40b1bc0 2022-08-01T23:04:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41425b0 con 0x7f83b4066b60 2022-08-01T23:04:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 0x7f83b4140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 0x7f83b4140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48718/0 (socket says 172.21.15.32:48718) 2022-08-01T23:04:37.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 -- 172.21.15.32:0/1331787861 learned_addr learned my addr 172.21.15.32:0/1331787861 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b2ffd700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83b4066b60 0x7f83b4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b3fff700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b40b1bc0 0x7f83b4154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 -- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83b4066b60 msgr2=0x7f83b4153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83b4066b60 0x7f83b4153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 -- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b40b1bc0 msgr2=0x7f83b4154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b40b1bc0 0x7f83b4154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 -- 172.21.15.32:0/1331787861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac009470 con 0x7f83b4066780 2022-08-01T23:04:37.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b37fe700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 0x7f83b4140f20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f83b40b3750 tx=0x7f83a8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83a8007030 con 0x7f83b4066780 2022-08-01T23:04:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8007190 con 0x7f83b4066780 2022-08-01T23:04:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83a8006510 con 0x7f83b4066780 2022-08-01T23:04:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b4142850 con 0x7f83b4066780 2022-08-01T23:04:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b40abc60 con 0x7f83b4066780 2022-08-01T23:04:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.860+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b405f440 con 0x7f83b4066780 2022-08-01T23:04:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.864+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f83a8010030 con 0x7f83b4066780 2022-08-01T23:04:37.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.864+0000 7f83b0ff9700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8394053950 0x7f8394055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.864+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f83a806bb60 con 0x7f83b4066780 2022-08-01T23:04:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.864+0000 7f83b2ffd700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8394053950 0x7f8394055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.864+0000 7f83b2ffd700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8394053950 0x7f8394055e10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f83a400d000 tx=0x7f83a4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.868+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83a8042920 con 0x7f83b4066780 2022-08-01T23:04:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.988+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83b40b12b0 con 0x7f83b4066780 2022-08-01T23:04:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b0ff9700 1 -- 172.21.15.32:0/1331787861 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+10 (secure 0 0 0) 0x7f83a8045f40 con 0x7f83b4066780 2022-08-01T23:04:37.997 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8394053950 msgr2=0x7f8394055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8394053950 0x7f8394055e10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f83a400d000 tx=0x7f83a4009040).stop 2022-08-01T23:04:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 msgr2=0x7f83b4140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 0x7f83b4140f20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f83b40b3750 tx=0x7f83a8007c10).stop 2022-08-01T23:04:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 shutdown_connections 2022-08-01T23:04:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8394053950 0x7f8394055e10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b40b1bc0 0x7f83b4154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f83b4066b60 0x7f83b4153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 --2- 172.21.15.32:0/1331787861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4066780 0x7f83b4140f20 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 >> 172.21.15.32:0/1331787861 conn(0x7f83b4063870 msgr2=0x7f83b4145d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 shutdown_connections 2022-08-01T23:04:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:37.992+0000 7f83b8d27700 1 -- 172.21.15.32:0/1331787861 wait complete. 2022-08-01T23:04:38.010 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:04:38.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:38.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.104+0000 7f1393fff700 1 Processor -- start 2022-08-01T23:04:38.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.104+0000 7f1393fff700 1 -- start start 2022-08-01T23:04:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13940b28b0 0x7f139413e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13940b1180 0x7f139413e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940aac20 con 0x7f1394141210 2022-08-01T23:04:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1394141db0 con 0x7f13940b28b0 2022-08-01T23:04:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1394140e20 con 0x7f13940b1180 2022-08-01T23:04:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13940b28b0 0x7f139413e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f138bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13940b1180 0x7f139413e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13940b28b0 0x7f139413e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48730/0 (socket says 172.21.15.32:48730) 2022-08-01T23:04:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f13937fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1394141210 0x7f13941416a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57176/0 (socket says 172.21.15.32:57176) 2022-08-01T23:04:38.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 -- 172.21.15.32:0/1066711377 learned_addr learned my addr 172.21.15.32:0/1066711377 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:38.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 -- 172.21.15.32:0/1066711377 >> v2:172.21.15.55:3300/0 conn(0x7f13940b1180 msgr2=0x7f139413e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:38.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 --2- 172.21.15.32:0/1066711377 >> v2:172.21.15.55:3300/0 conn(0x7f13940b1180 0x7f139413e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 -- 172.21.15.32:0/1066711377 >> v1:172.21.15.55:6789/0 conn(0x7f1394141210 legacy=0x7f13941416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 -- 172.21.15.32:0/1066711377 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13940b2c90 con 0x7f13940b28b0 2022-08-01T23:04:38.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f138bfff700 1 --2- 172.21.15.32:0/1066711377 >> v2:172.21.15.55:3300/0 conn(0x7f13940b1180 0x7f139413e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:38.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1392ffd700 1 --2- 172.21.15.32:0/1066711377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f139413e1c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f1384002f70 tx=0x7f1384006eb0).ready entity=mon.0 client_cookie=f32798780c7b463d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:38.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f13927fc700 1 -- 172.21.15.32:0/1066711377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f138400d5c0 con 0x7f13940b28b0 2022-08-01T23:04:38.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f13927fc700 1 -- 172.21.15.32:0/1066711377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384000e90 con 0x7f13940b28b0 2022-08-01T23:04:38.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f13927fc700 1 -- 172.21.15.32:0/1066711377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f138400d950 con 0x7f13940b28b0 2022-08-01T23:04:38.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- 172.21.15.32:0/1066711377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 msgr2=0x7f139413e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 --2- 172.21.15.32:0/1066711377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f139413e1c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f1384002f70 tx=0x7f1384006eb0).stop 2022-08-01T23:04:38.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- 172.21.15.32:0/1066711377 shutdown_connections 2022-08-01T23:04:38.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 --2- 172.21.15.32:0/1066711377 >> v2:172.21.15.55:3300/0 conn(0x7f13940b1180 0x7f139413e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 --2- 172.21.15.32:0/1066711377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f139413e1c0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- 172.21.15.32:0/1066711377 >> 172.21.15.32:0/1066711377 conn(0x7f1394060310 msgr2=0x7f13940b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:38.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- 172.21.15.32:0/1066711377 shutdown_connections 2022-08-01T23:04:38.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.108+0000 7f1393fff700 1 -- 172.21.15.32:0/1066711377 wait complete. 2022-08-01T23:04:38.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 Processor -- start 2022-08-01T23:04:38.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- start start 2022-08-01T23:04:38.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13940b1180 0x7f1394144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:38.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f13941524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:38.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1394141210 0x7f13940b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:38.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940ae770 con 0x7f13940b28b0 2022-08-01T23:04:38.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394144d60 con 0x7f1394141210 2022-08-01T23:04:38.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1394142c60 con 0x7f13940b1180 2022-08-01T23:04:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f13941524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f13941524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48732/0 (socket says 172.21.15.32:48732) 2022-08-01T23:04:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 -- 172.21.15.32:0/945701530 learned_addr learned my addr 172.21.15.32:0/945701530 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f13937fe700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1394141210 0x7f13940b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:38.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1392ffd700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13940b1180 0x7f1394144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 -- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13940b1180 msgr2=0x7f1394144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13940b1180 0x7f1394144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 -- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1394141210 msgr2=0x7f13940b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.128 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1394141210 0x7f13940b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 -- 172.21.15.32:0/945701530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c00b180 con 0x7f13940b28b0 2022-08-01T23:04:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1392ffd700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13940b1180 0x7f1394144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f138bfff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f13941524a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f1394144ef0 tx=0x7f137c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:38.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f137c00ca40 con 0x7f13940b28b0 2022-08-01T23:04:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00cba0 con 0x7f13940b28b0 2022-08-01T23:04:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138400b1e0 con 0x7f13940b28b0 2022-08-01T23:04:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f137c0123f0 con 0x7f13940b28b0 2022-08-01T23:04:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13940b3a50 con 0x7f13940b28b0 2022-08-01T23:04:38.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.112+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394152b30 con 0x7f13940b28b0 2022-08-01T23:04:38.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.116+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f137c012550 con 0x7f13940b28b0 2022-08-01T23:04:38.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.116+0000 7f1390ff9700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1378053a10 0x7f1378055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:38.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.116+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f137c06c690 con 0x7f13940b28b0 2022-08-01T23:04:38.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.116+0000 7f1392ffd700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1378053a10 0x7f1378055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:38.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.116+0000 7f1392ffd700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1378053a10 0x7f1378055ed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f1384017000 tx=0x7f1384015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:38.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.120+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f137c0433f0 con 0x7f13940b28b0 2022-08-01T23:04:38.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.284+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13940574e0 con 0x7f13940b28b0 2022-08-01T23:04:38.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.284+0000 7f1390ff9700 1 -- 172.21.15.32:0/945701530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f137c046a10 con 0x7f13940b28b0 2022-08-01T23:04:38.291 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:38.291 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:34.675417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8079":{"gid":8079,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/766376033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":766376033},{"type":"v1","addr":"172.21.15.32:6835","nonce":766376033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:38.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1378053a10 msgr2=0x7f1378055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1378053a10 0x7f1378055ed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f1384017000 tx=0x7f1384015000).stop 2022-08-01T23:04:38.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 msgr2=0x7f13941524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:38.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f13941524a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f1394144ef0 tx=0x7f137c006df0).stop 2022-08-01T23:04:38.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 shutdown_connections 2022-08-01T23:04:38.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1378053a10 0x7f1378055ed0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1394141210 0x7f13940b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13940b28b0 0x7f13941524a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 --2- 172.21.15.32:0/945701530 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f13940b1180 0x7f1394144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:38.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 >> 172.21.15.32:0/945701530 conn(0x7f1394060310 msgr2=0x7f139413dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:38.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 shutdown_connections 2022-08-01T23:04:38.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:38.288+0000 7f1393fff700 1 -- 172.21.15.32:0/945701530 wait complete. 2022-08-01T23:04:38.296 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 68 2022-08-01T23:04:38.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:04:33.672583+0000', 'modified': '2022-08-01T23:04:34.675417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8079': {'gid': 8079, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/766376033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 766376033}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 766376033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:04:38.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:04:38.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:04:38.729 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8079,"want_state":"up:active","state":"up:active","rank_uptime":5.0421134910000003,"mdsmap_epoch":68,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":9.8422498849999993} 2022-08-01T23:04:38.730 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8079, "mdsmap_epoch": 68, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 5.042113491, "state": "up:active", "uptime": 9.842249885, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:04:38.730 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:04:38.731 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:04:38.731 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:04:38.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:38.732 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:38.740 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:04:38.758 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:38.759 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:38.771 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:38.773 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:38.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:38.774 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:04:38.779 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:04:38.779 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:38.782 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:04:38.799 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:38.799 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:38.858 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:38.859 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:38.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:38.861 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:04:38.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:04:38.854+0000 7f3c9acd2080 -1 init, newargv = 0x559b478e3450 newargc=9 2022-08-01T23:04:38.862 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23523]: starting ceph client 2022-08-01T23:04:38.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23523]: starting fuse 2022-08-01T23:04:39.970 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:04:39.988 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:39.989 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:40.047 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:40.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:40.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:40.049 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:04:40.057 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:04:40.057 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:04:40.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:40.058 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:04:40.059 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:04:40.063 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:04:40.064 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:04:40.064 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:04:40.065 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:40.065 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:04:40.065 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:04:40.066 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:04:40.066 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:40.067 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:04:40.067 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:04:40.067 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:04:40.068 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:04:40.068 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:40.069 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:04:40.069 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:04:40.070 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:04:40.070 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:04:40.070 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:04:40.071 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:40.071 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:04:40.072 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:04:40.115 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23523.asok 2022-08-01T23:04:40.120 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23523.asok 2022-08-01T23:04:40.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:40.121 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23523.asok status 2022-08-01T23:04:40.209 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:04:40.209 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:04:40.210 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:04:40.210 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:04:40.211 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:04:40.211 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:04:40.211 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:04:40.211 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23523", 2022-08-01T23:04:40.212 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:04:40.212 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:40.212 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:04:40.212 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:04:40.212 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 7064, 2022-08-01T23:04:40.213 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:04:40.213 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:04:40.213 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:04:40.214 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 7064 2022-08-01T23:04:40.214 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:40.215 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:04:40.215 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:04:40.216 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:04:40.216 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2467862196 2022-08-01T23:04:40.217 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:04:40.217 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:40.217 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:04:40.217 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:04:40.218 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:04:40.218 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2467862196 2022-08-01T23:04:40.218 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:04:40.218 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.7064 172.21.15.130:0/2467862196", 2022-08-01T23:04:40.219 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/2467862196", 2022-08-01T23:04:40.219 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:04:40.219 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 68, 2022-08-01T23:04:40.219 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 137, 2022-08-01T23:04:40.219 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:04:40.220 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:04:40.220 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:04:40.220 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:04:40.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:40.222 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:40.222 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:40.231 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:04:40.232 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:40.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:40.233 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:40.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.352+0000 7f76e5f0d700 1 Processor -- start 2022-08-01T23:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.352+0000 7f76e5f0d700 1 -- start start 2022-08-01T23:04:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e5f0d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76e00b31d0 0x7f76e0066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e5f0d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e0067030 con 0x7f76e00a9da0 2022-08-01T23:04:40.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e5f0d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e00b35b0 con 0x7f76e01402d0 2022-08-01T23:04:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e5f0d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e0064a70 con 0x7f76e00b31d0 2022-08-01T23:04:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76dffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76e00a9da0 0x7f76e0060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36284/0 (socket says 172.21.15.32:36284) 2022-08-01T23:04:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76dffff700 1 -- 172.21.15.32:0/236884317 learned_addr learned my addr 172.21.15.32:0/236884317 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/236884317 >> v2:172.21.15.55:3300/0 conn(0x7f76e00b31d0 0x7f76e0066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105666254 0 0) 0x7f76e0067030 con 0x7f76e00a9da0 2022-08-01T23:04:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c4003260 con 0x7f76e00a9da0 2022-08-01T23:04:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769860325 0 0) 0x7f76e00b35b0 con 0x7f76e01402d0 2022-08-01T23:04:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e0067030 con 0x7f76e01402d0 2022-08-01T23:04:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576098392 0 0) 0x7f76c4003260 con 0x7f76e00a9da0 2022-08-01T23:04:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 >> v2:172.21.15.55:3300/0 conn(0x7f76e00b31d0 msgr2=0x7f76e0066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:40.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 --2- 172.21.15.32:0/236884317 >> v2:172.21.15.55:3300/0 conn(0x7f76e00b31d0 0x7f76e0066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 >> v1:172.21.15.55:6789/0 conn(0x7f76e01402d0 legacy=0x7f76e0140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0066af0 con 0x7f76e00a9da0 2022-08-01T23:04:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f76cc001680 con 0x7f76e00a9da0 2022-08-01T23:04:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/236884317 >> v2:172.21.15.55:3300/0 conn(0x7f76e00b31d0 0x7f76e0066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 >> v1:172.21.15.32:6789/0 conn(0x7f76e00a9da0 legacy=0x7f76e0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76c4003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76c4004500 0x7f76c4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76c4004e70 0x7f76c400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c400bcf0 con 0x7f76c40038e0 2022-08-01T23:04:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c400be30 con 0x7f76c4004500 2022-08-01T23:04:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c400bff0 con 0x7f76c4004e70 2022-08-01T23:04:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76c4003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76dffff700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76c4004500 0x7f76c4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e570c700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76c4004e70 0x7f76c400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 -- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76c4004e70 msgr2=0x7f76c400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76c4004e70 0x7f76c400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 -- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76c4004500 msgr2=0x7f76c4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76c4004500 0x7f76c4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 -- 172.21.15.32:0/236884317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4003260 con 0x7f76c40038e0 2022-08-01T23:04:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e570c700 1 -- 172.21.15.32:0/236884317 reap_dead start 2022-08-01T23:04:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76c4003ce0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f76d400c000 tx=0x7f76d4006df0).ready entity=mon.0 client_cookie=54395eaa3407f9c7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d400e800 con 0x7f76c40038e0 2022-08-01T23:04:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d400e960 con 0x7f76c40038e0 2022-08-01T23:04:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.356+0000 7f76df7fe700 1 -- 172.21.15.32:0/236884317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d400ecc0 con 0x7f76c40038e0 2022-08-01T23:04:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 -- 172.21.15.32:0/236884317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 msgr2=0x7f76c4003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76c4003ce0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f76d400c000 tx=0x7f76d4006df0).stop 2022-08-01T23:04:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 -- 172.21.15.32:0/236884317 shutdown_connections 2022-08-01T23:04:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/236884317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76c4003ce0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 -- 172.21.15.32:0/236884317 >> 172.21.15.32:0/236884317 conn(0x7f76e0063870 msgr2=0x7f76e0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 -- 172.21.15.32:0/236884317 shutdown_connections 2022-08-01T23:04:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 -- 172.21.15.32:0/236884317 wait complete. 2022-08-01T23:04:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 Processor -- start 2022-08-01T23:04:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.360+0000 7f76e5f0d700 1 -- start start 2022-08-01T23:04:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76e00b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e00b5590 0x7f76e00ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76e0146cc0 0x7f76e0147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0147630 con 0x7f76c40038e0 2022-08-01T23:04:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0152de0 con 0x7f76e00b5590 2022-08-01T23:04:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0153010 con 0x7f76e0146cc0 2022-08-01T23:04:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76e00b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76e00b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48752/0 (socket says 172.21.15.32:48752) 2022-08-01T23:04:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 -- 172.21.15.32:0/2487912951 learned_addr learned my addr 172.21.15.32:0/2487912951 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76dffff700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e00b5590 0x7f76e00ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e570c700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76e0146cc0 0x7f76e0147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 -- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76e0146cc0 msgr2=0x7f76e0147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76e0146cc0 0x7f76e0147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 -- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e00b5590 msgr2=0x7f76e00ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e00b5590 0x7f76e00ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 -- 172.21.15.32:0/2487912951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0146820 con 0x7f76c40038e0 2022-08-01T23:04:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e570c700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76e0146cc0 0x7f76e0147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e4f0b700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76e00b0070 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f76e00b2bc0 tx=0x7f76d4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d401ca50 con 0x7f76c40038e0 2022-08-01T23:04:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d401cbb0 con 0x7f76c40038e0 2022-08-01T23:04:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e00b0820 con 0x7f76c40038e0 2022-08-01T23:04:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76d40243f0 con 0x7f76c40038e0 2022-08-01T23:04:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.364+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e013f4e0 con 0x7f76c40038e0 2022-08-01T23:04:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.368+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e01531d0 con 0x7f76c40038e0 2022-08-01T23:04:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.368+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f76d402b030 con 0x7f76c40038e0 2022-08-01T23:04:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.368+0000 7f76ddffb700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76c00539e0 0x7f76c0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.368+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f76d406d850 con 0x7f76c40038e0 2022-08-01T23:04:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.368+0000 7f76dffff700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76c00539e0 0x7f76c0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.368+0000 7f76dffff700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76c00539e0 0x7f76c0055ea0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f76cc005000 tx=0x7f76e0065990).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.376+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76d4047fa0 con 0x7f76c40038e0 2022-08-01T23:04:40.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.504+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e0066d90 con 0x7f76c40038e0 2022-08-01T23:04:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76ddffb700 1 -- 172.21.15.32:0/2487912951 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f76d4029020 con 0x7f76c40038e0 2022-08-01T23:04:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:40.515 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":137,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:34.676165+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:04:31.641583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T23:04:32.681709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:40.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76c00539e0 msgr2=0x7f76c0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76c00539e0 0x7f76c0055ea0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f76cc005000 tx=0x7f76e0065990).stop 2022-08-01T23:04:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 msgr2=0x7f76e00b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76e00b0070 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f76e00b2bc0 tx=0x7f76d4019000).stop 2022-08-01T23:04:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 shutdown_connections 2022-08-01T23:04:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76c00539e0 0x7f76c0055ea0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f76e0146cc0 0x7f76e0147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e00b5590 0x7f76e00ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 --2- 172.21.15.32:0/2487912951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40038e0 0x7f76e00b0070 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 >> 172.21.15.32:0/2487912951 conn(0x7f76e0063870 msgr2=0x7f76e005fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:40.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 shutdown_connections 2022-08-01T23:04:40.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.508+0000 7f76e5f0d700 1 -- 172.21.15.32:0/2487912951 wait complete. 2022-08-01T23:04:40.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:04:31.641583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:04:32.681709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:04:40.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:04:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7703fff700 1 Processor -- start 2022-08-01T23:04:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7703fff700 1 -- start start 2022-08-01T23:04:40.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7703fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77040a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7703fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77040a7250 con 0x7f77040afd60 2022-08-01T23:04:40.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7703fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77040a7480 con 0x7f77040b2710 2022-08-01T23:04:40.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7703fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77040a7720 con 0x7f77040b01a0 2022-08-01T23:04:40.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77037fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f77040afd60 0x7f7704151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36306/0 (socket says 172.21.15.32:36306) 2022-08-01T23:04:40.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77037fe700 1 -- 172.21.15.32:0/1885144305 learned_addr learned my addr 172.21.15.32:0/1885144305 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:40.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77027fc700 1 --2- 172.21.15.32:0/1885144305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77040a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001024175 0 0) 0x7f77040a7250 con 0x7f77040afd60 2022-08-01T23:04:40.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e4003260 con 0x7f77040afd60 2022-08-01T23:04:40.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255778120 0 0) 0x7f77040a7480 con 0x7f77040b2710 2022-08-01T23:04:40.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77040a7250 con 0x7f77040b2710 2022-08-01T23:04:40.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77027fc700 1 -- 172.21.15.32:0/1885144305 >> v1:172.21.15.55:6789/0 conn(0x7f77040b2710 legacy=0x7f770414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77027fc700 1 -- 172.21.15.32:0/1885144305 >> v1:172.21.15.32:6789/0 conn(0x7f77040afd60 legacy=0x7f7704151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77027fc700 1 -- 172.21.15.32:0/1885144305 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040a79c0 con 0x7f77040b01a0 2022-08-01T23:04:40.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f77027fc700 1 --2- 172.21.15.32:0/1885144305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77040a8540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f76ec002f70 tx=0x7f76ec006df0).ready entity=mon.2 client_cookie=270047b9eac99512 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.632+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76ec00d5c0 con 0x7f77040b01a0 2022-08-01T23:04:40.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec000e90 con 0x7f77040b01a0 2022-08-01T23:04:40.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7701ffb700 1 -- 172.21.15.32:0/1885144305 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76ec00d910 con 0x7f77040b01a0 2022-08-01T23:04:40.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 -- 172.21.15.32:0/1885144305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 msgr2=0x7f77040a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 --2- 172.21.15.32:0/1885144305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77040a8540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f76ec002f70 tx=0x7f76ec006df0).stop 2022-08-01T23:04:40.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 -- 172.21.15.32:0/1885144305 shutdown_connections 2022-08-01T23:04:40.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 --2- 172.21.15.32:0/1885144305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77040a8540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 -- 172.21.15.32:0/1885144305 >> 172.21.15.32:0/1885144305 conn(0x7f7704060310 msgr2=0x7f77040b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:40.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 -- 172.21.15.32:0/1885144305 shutdown_connections 2022-08-01T23:04:40.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 -- 172.21.15.32:0/1885144305 wait complete. 2022-08-01T23:04:40.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 Processor -- start 2022-08-01T23:04:40.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.636+0000 7f7703fff700 1 -- start start 2022-08-01T23:04:40.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 0x7f77040b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77041423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77040b2710 0x7f77041497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040b5e00 con 0x7f77040afd60 2022-08-01T23:04:40.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704142890 con 0x7f77040b2710 2022-08-01T23:04:40.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77040b2f60 con 0x7f77040b01a0 2022-08-01T23:04:40.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 0x7f77040b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 0x7f77040b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48766/0 (socket says 172.21.15.32:48766) 2022-08-01T23:04:40.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 -- 172.21.15.32:0/1448305356 learned_addr learned my addr 172.21.15.32:0/1448305356 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:40.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f77037fe700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77040b2710 0x7f77041497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f77027fc700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77041423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 -- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 msgr2=0x7f77041423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77041423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 -- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77040b2710 msgr2=0x7f77041497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77040b2710 0x7f77041497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 -- 172.21.15.32:0/1448305356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec00b180 con 0x7f77040afd60 2022-08-01T23:04:40.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7702ffd700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 0x7f77040b5480 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f7704156610 tx=0x7f76f4007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f400cdd0 con 0x7f77040afd60 2022-08-01T23:04:40.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4012400 con 0x7f77040afd60 2022-08-01T23:04:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77040b3720 con 0x7f77040afd60 2022-08-01T23:04:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f4012760 con 0x7f77040afd60 2022-08-01T23:04:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.640+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77040b3980 con 0x7f77040afd60 2022-08-01T23:04:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.644+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77040574e0 con 0x7f77040afd60 2022-08-01T23:04:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.644+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f76f4019030 con 0x7f77040afd60 2022-08-01T23:04:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.644+0000 7f76fbfff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76e8053950 0x7f76e8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.644+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f76f406b450 con 0x7f77040afd60 2022-08-01T23:04:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.644+0000 7f77027fc700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76e8053950 0x7f76e8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.644+0000 7f77027fc700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76e8053950 0x7f76e8055e10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f76ec018000 tx=0x7f76ec014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.648+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76f4042080 con 0x7f77040afd60 2022-08-01T23:04:40.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.772+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7704069a70 con 0x7f77040afd60 2022-08-01T23:04:40.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f76fbfff700 1 -- 172.21.15.32:0/1448305356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f76f40456a0 con 0x7f77040afd60 2022-08-01T23:04:40.781 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:04:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76e8053950 msgr2=0x7f76e8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76e8053950 0x7f76e8055e10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f76ec018000 tx=0x7f76ec014000).stop 2022-08-01T23:04:40.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 msgr2=0x7f77040b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 0x7f77040b5480 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f7704156610 tx=0x7f76f4007a90).stop 2022-08-01T23:04:40.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 shutdown_connections 2022-08-01T23:04:40.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f76e8053950 0x7f76e8055e10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77040b2710 0x7f77041497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f77040b01a0 0x7f77041423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 --2- 172.21.15.32:0/1448305356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77040afd60 0x7f77040b5480 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 >> 172.21.15.32:0/1448305356 conn(0x7f7704060310 msgr2=0x7f7704145310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:40.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 shutdown_connections 2022-08-01T23:04:40.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.776+0000 7f7703fff700 1 -- 172.21.15.32:0/1448305356 wait complete. 2022-08-01T23:04:40.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:04:40.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 Processor -- start 2022-08-01T23:04:40.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 -- start start 2022-08-01T23:04:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c146ae0 0x7f133c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f133c0b4310 0x7f133c1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f133c0b4820 0x7f133c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c1476a0 con 0x7f133c0b4310 2022-08-01T23:04:40.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c143cf0 con 0x7f133c0b4820 2022-08-01T23:04:40.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f13433f9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c143f90 con 0x7f133c146ae0 2022-08-01T23:04:40.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f133c0b4310 0x7f133c1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f133c0b4310 0x7f133c1408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48778/0 (socket says 172.21.15.32:48778) 2022-08-01T23:04:40.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 -- 172.21.15.32:0/3365004797 learned_addr learned my addr 172.21.15.32:0/3365004797 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:40.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1341195700 1 --2- 172.21.15.32:0/3365004797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c146ae0 0x7f133c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1341996700 1 --2- 172.21.15.32:0/3365004797 >> v2:172.21.15.55:3300/0 conn(0x7f133c0b4820 0x7f133c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 -- 172.21.15.32:0/3365004797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c146ae0 msgr2=0x7f133c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:40.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 --2- 172.21.15.32:0/3365004797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c146ae0 0x7f133c1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 -- 172.21.15.32:0/3365004797 >> v2:172.21.15.55:3300/0 conn(0x7f133c0b4820 msgr2=0x7f133c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:40.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 --2- 172.21.15.32:0/3365004797 >> v2:172.21.15.55:3300/0 conn(0x7f133c0b4820 0x7f133c0b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 -- 172.21.15.32:0/3365004797 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0b0e20 con 0x7f133c0b4310 2022-08-01T23:04:40.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1341996700 1 --2- 172.21.15.32:0/3365004797 >> v2:172.21.15.55:3300/0 conn(0x7f133c0b4820 0x7f133c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:40.916 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.904+0000 7f1340994700 1 --2- 172.21.15.32:0/3365004797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c1408b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f132c002f70 tx=0x7f132c006eb0).ready entity=mon.0 client_cookie=cffdc24779c82501 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f132bfff700 1 -- 172.21.15.32:0/3365004797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c00d570 con 0x7f133c0b4310 2022-08-01T23:04:40.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f132bfff700 1 -- 172.21.15.32:0/3365004797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c000e90 con 0x7f133c0b4310 2022-08-01T23:04:40.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f132bfff700 1 -- 172.21.15.32:0/3365004797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c00d900 con 0x7f133c0b4310 2022-08-01T23:04:40.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 -- 172.21.15.32:0/3365004797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 msgr2=0x7f133c1408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 --2- 172.21.15.32:0/3365004797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c1408b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f132c002f70 tx=0x7f132c006eb0).stop 2022-08-01T23:04:40.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 -- 172.21.15.32:0/3365004797 shutdown_connections 2022-08-01T23:04:40.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 --2- 172.21.15.32:0/3365004797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c1408b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 --2- 172.21.15.32:0/3365004797 >> v2:172.21.15.55:3300/0 conn(0x7f133c0b4820 0x7f133c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 --2- 172.21.15.32:0/3365004797 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c146ae0 0x7f133c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 -- 172.21.15.32:0/3365004797 >> 172.21.15.32:0/3365004797 conn(0x7f133c05c620 msgr2=0x7f133c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 -- 172.21.15.32:0/3365004797 shutdown_connections 2022-08-01T23:04:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.908+0000 7f13433f9700 1 -- 172.21.15.32:0/3365004797 wait complete. 2022-08-01T23:04:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 Processor -- start 2022-08-01T23:04:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 -- start start 2022-08-01T23:04:40.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c0ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c0b4820 0x7f133c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133c146ae0 0x7f133c1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c1429e0 con 0x7f133c0b4310 2022-08-01T23:04:40.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0aff70 con 0x7f133c146ae0 2022-08-01T23:04:40.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0b0210 con 0x7f133c0b4820 2022-08-01T23:04:40.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c0ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c0ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48780/0 (socket says 172.21.15.32:48780) 2022-08-01T23:04:40.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 -- 172.21.15.32:0/2100422805 learned_addr learned my addr 172.21.15.32:0/2100422805 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:40.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1340994700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c0b4820 0x7f133c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341996700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133c146ae0 0x7f133c1424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 -- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c0b4820 msgr2=0x7f133c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c0b4820 0x7f133c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 -- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133c146ae0 msgr2=0x7f133c1424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133c146ae0 0x7f133c1424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 -- 172.21.15.32:0/2100422805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c00b1e0 con 0x7f133c0b4310 2022-08-01T23:04:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1340994700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c0b4820 0x7f133c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f1341195700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c0ae5c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f1338006e00 tx=0x7f13380091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f133800c850 con 0x7f133c0b4310 2022-08-01T23:04:40.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133800c9b0 con 0x7f133c0b4310 2022-08-01T23:04:40.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c0b04b0 con 0x7f133c0b4310 2022-08-01T23:04:40.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f133800cd10 con 0x7f133c0b4310 2022-08-01T23:04:40.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.912+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c142f50 con 0x7f133c0b4310 2022-08-01T23:04:40.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.916+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c0574e0 con 0x7f133c0b4310 2022-08-01T23:04:40.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.916+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1338022070 con 0x7f133c0b4310 2022-08-01T23:04:40.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.916+0000 7f132a7fc700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f131c0539e0 0x7f131c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:40.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.916+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f133806c820 con 0x7f133c0b4310 2022-08-01T23:04:40.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.916+0000 7f1340994700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f131c0539e0 0x7f131c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:40.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.916+0000 7f1340994700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f131c0539e0 0x7f131c055ea0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f132c01a000 tx=0x7f132c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:40.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:40.924+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13380435e0 con 0x7f133c0b4310 2022-08-01T23:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f133c13fed0 con 0x7f133c0b4310 2022-08-01T23:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f132a7fc700 1 -- 172.21.15.32:0/2100422805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f13380160a0 con 0x7f133c0b4310 2022-08-01T23:04:41.056 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f131c0539e0 msgr2=0x7f131c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f131c0539e0 0x7f131c055ea0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f132c01a000 tx=0x7f132c015000).stop 2022-08-01T23:04:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 msgr2=0x7f133c0ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c0ae5c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f1338006e00 tx=0x7f13380091d0).stop 2022-08-01T23:04:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 shutdown_connections 2022-08-01T23:04:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f131c0539e0 0x7f131c055ea0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133c146ae0 0x7f133c1424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f133c0b4820 0x7f133c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 --2- 172.21.15.32:0/2100422805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c0b4310 0x7f133c0ae5c0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 >> 172.21.15.32:0/2100422805 conn(0x7f133c05c620 msgr2=0x7f133c145cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 shutdown_connections 2022-08-01T23:04:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.048+0000 7f13433f9700 1 -- 172.21.15.32:0/2100422805 wait complete. 2022-08-01T23:04:41.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:04:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 Processor -- start 2022-08-01T23:04:41.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- start start 2022-08-01T23:04:41.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f99f00ad750 0x7f99f00b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99f01405d0 0x7f99f0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00aee80 con 0x7f99f01405d0 2022-08-01T23:04:41.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00b70b0 con 0x7f99f00ad750 2022-08-01T23:04:41.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00af0b0 con 0x7f99f00614b0 2022-08-01T23:04:41.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99f01405d0 0x7f99f0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99f01405d0 0x7f99f0140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48790/0 (socket says 172.21.15.32:48790) 2022-08-01T23:04:41.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 -- 172.21.15.32:0/3589509073 learned_addr learned my addr 172.21.15.32:0/3589509073 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:41.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99effff700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99ef7fe700 1 --2- 172.21.15.32:0/3589509073 >> v2:172.21.15.55:3300/0 conn(0x7f99f00ad750 0x7f99f00b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 -- 172.21.15.32:0/3589509073 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 msgr2=0x7f99f0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:41.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 -- 172.21.15.32:0/3589509073 >> v2:172.21.15.55:3300/0 conn(0x7f99f00ad750 msgr2=0x7f99f00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:41.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 --2- 172.21.15.32:0/3589509073 >> v2:172.21.15.55:3300/0 conn(0x7f99f00ad750 0x7f99f00b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 -- 172.21.15.32:0/3589509073 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00af350 con 0x7f99f01405d0 2022-08-01T23:04:41.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99effff700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:41.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99ef7fe700 1 --2- 172.21.15.32:0/3589509073 >> v2:172.21.15.55:3300/0 conn(0x7f99f00ad750 0x7f99f00b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:41.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f4f00700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f0140a80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f99e80032f0 tx=0x7f99e8009290).ready entity=mon.0 client_cookie=94e765cdf3fe3bd4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99eeffd700 1 -- 172.21.15.32:0/3589509073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99e800b870 con 0x7f99f01405d0 2022-08-01T23:04:41.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99eeffd700 1 -- 172.21.15.32:0/3589509073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e800b9d0 con 0x7f99f01405d0 2022-08-01T23:04:41.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99eeffd700 1 -- 172.21.15.32:0/3589509073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99e800bb40 con 0x7f99f01405d0 2022-08-01T23:04:41.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- 172.21.15.32:0/3589509073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 msgr2=0x7f99f0140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f0140a80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f99e80032f0 tx=0x7f99e8009290).stop 2022-08-01T23:04:41.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- 172.21.15.32:0/3589509073 shutdown_connections 2022-08-01T23:04:41.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f0140a80 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- 172.21.15.32:0/3589509073 >> v2:172.21.15.55:3300/0 conn(0x7f99f00ad750 0x7f99f00b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 --2- 172.21.15.32:0/3589509073 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- 172.21.15.32:0/3589509073 >> 172.21.15.32:0/3589509073 conn(0x7f99f0066f70 msgr2=0x7f99f0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- 172.21.15.32:0/3589509073 shutdown_connections 2022-08-01T23:04:41.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.172+0000 7f99f5701700 1 -- 172.21.15.32:0/3589509073 wait complete. 2022-08-01T23:04:41.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 Processor -- start 2022-08-01T23:04:41.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- start start 2022-08-01T23:04:41.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99f00ad750 0x7f99f00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f00ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0142da0 con 0x7f99f01405d0 2022-08-01T23:04:41.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0142b40 con 0x7f99f00ad750 2022-08-01T23:04:41.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00ad270 con 0x7f99f00614b0 2022-08-01T23:04:41.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f00ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f00ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48798/0 (socket says 172.21.15.32:48798) 2022-08-01T23:04:41.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 -- 172.21.15.32:0/540596192 learned_addr learned my addr 172.21.15.32:0/540596192 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:41.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99ef7fe700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99f00ad750 0x7f99f00aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99effff700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 -- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 msgr2=0x7f99f0142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 -- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99f00ad750 msgr2=0x7f99f00aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99f00ad750 0x7f99f00aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 -- 172.21.15.32:0/540596192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8009530 con 0x7f99f01405d0 2022-08-01T23:04:41.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f4f00700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f00ab490 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f99e80032f0 tx=0x7f99e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99e8021480 con 0x7f99f01405d0 2022-08-01T23:04:41.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80215e0 con 0x7f99f01405d0 2022-08-01T23:04:41.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f00aab00 con 0x7f99f01405d0 2022-08-01T23:04:41.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f00ad510 con 0x7f99f01405d0 2022-08-01T23:04:41.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99e8021940 con 0x7f99f01405d0 2022-08-01T23:04:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.176+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0142d00 con 0x7f99f01405d0 2022-08-01T23:04:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.180+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f99e800f070 con 0x7f99f01405d0 2022-08-01T23:04:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.180+0000 7f99ed7fa700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99d00539e0 0x7f99d0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.180+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f99e806d270 con 0x7f99f01405d0 2022-08-01T23:04:41.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.180+0000 7f99effff700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99d00539e0 0x7f99d0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.180+0000 7f99effff700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99d00539e0 0x7f99d0055ea0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f99e000d000 tx=0x7f99e0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.184+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f99e8044030 con 0x7f99f01405d0 2022-08-01T23:04:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.312+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99f00aac90 con 0x7f99f01405d0 2022-08-01T23:04:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.312+0000 7f99ed7fa700 1 -- 172.21.15.32:0/540596192 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+10 (secure 0 0 0) 0x7f99e8015020 con 0x7f99f01405d0 2022-08-01T23:04:41.319 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99d00539e0 msgr2=0x7f99d0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99d00539e0 0x7f99d0055ea0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f99e000d000 tx=0x7f99e0008040).stop 2022-08-01T23:04:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 msgr2=0x7f99f00ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f00ab490 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f99e80032f0 tx=0x7f99e8016000).stop 2022-08-01T23:04:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 shutdown_connections 2022-08-01T23:04:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f99d00539e0 0x7f99d0055ea0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99f01405d0 0x7f99f00ab490 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99f00ad750 0x7f99f00aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 --2- 172.21.15.32:0/540596192 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f99f00614b0 0x7f99f0142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 >> 172.21.15.32:0/540596192 conn(0x7f99f0066f70 msgr2=0x7f99f01467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 shutdown_connections 2022-08-01T23:04:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.316+0000 7f99f5701700 1 -- 172.21.15.32:0/540596192 wait complete. 2022-08-01T23:04:41.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:04:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 Processor -- start 2022-08-01T23:04:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 -- start start 2022-08-01T23:04:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff2980a9760 0x7ff2980a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 0x7ff2980b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff298139d50 con 0x7ff2980acbe0 2022-08-01T23:04:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff298140690 con 0x7ff2980a9760 2022-08-01T23:04:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29edc4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980b02c0 con 0x7ff2980b2d90 2022-08-01T23:04:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 0x7ff2980b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29cb60700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff2980a9760 0x7ff2980a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff297fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff2980acbe0 0x7ff2980a8bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57252/0 (socket says 172.21.15.32:57252) 2022-08-01T23:04:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 0x7ff2980b0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60544/0 (socket says 172.21.15.32:60544) 2022-08-01T23:04:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff297fff700 1 -- 172.21.15.32:0/1886886335 learned_addr learned my addr 172.21.15.32:0/1886886335 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 -- 172.21.15.32:0/1886886335 >> v2:172.21.15.55:3300/0 conn(0x7ff2980a9760 msgr2=0x7ff2980a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 --2- 172.21.15.32:0/1886886335 >> v2:172.21.15.55:3300/0 conn(0x7ff2980a9760 0x7ff2980a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 -- 172.21.15.32:0/1886886335 >> v1:172.21.15.55:6789/0 conn(0x7ff2980acbe0 legacy=0x7ff2980a8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 -- 172.21.15.32:0/1886886335 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980b0560 con 0x7ff2980b2d90 2022-08-01T23:04:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.444+0000 7ff29d361700 1 --2- 172.21.15.32:0/1886886335 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 0x7ff2980b0a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff2900032f0 tx=0x7ff29000b4b0).ready entity=mon.2 client_cookie=cb0fd8804e6175c7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff2977fe700 1 -- 172.21.15.32:0/1886886335 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff29000de70 con 0x7ff2980b2d90 2022-08-01T23:04:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff2977fe700 1 -- 172.21.15.32:0/1886886335 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2900153f0 con 0x7ff2980b2d90 2022-08-01T23:04:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff2977fe700 1 -- 172.21.15.32:0/1886886335 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff290015750 con 0x7ff2980b2d90 2022-08-01T23:04:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 -- 172.21.15.32:0/1886886335 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 msgr2=0x7ff2980b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 --2- 172.21.15.32:0/1886886335 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 0x7ff2980b0a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff2900032f0 tx=0x7ff29000b4b0).stop 2022-08-01T23:04:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 -- 172.21.15.32:0/1886886335 shutdown_connections 2022-08-01T23:04:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 --2- 172.21.15.32:0/1886886335 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980b2d90 0x7ff2980b0a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 --2- 172.21.15.32:0/1886886335 >> v2:172.21.15.55:3300/0 conn(0x7ff2980a9760 0x7ff2980a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 -- 172.21.15.32:0/1886886335 >> 172.21.15.32:0/1886886335 conn(0x7ff29805f840 msgr2=0x7ff29805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 -- 172.21.15.32:0/1886886335 shutdown_connections 2022-08-01T23:04:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.448+0000 7ff29edc4700 1 -- 172.21.15.32:0/1886886335 wait complete. 2022-08-01T23:04:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 Processor -- start 2022-08-01T23:04:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 -- start start 2022-08-01T23:04:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 0x7ff2980ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980acbe0 0x7ff2981338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2980b2d90 0x7ff298133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ac4b0 con 0x7ff2980a9760 2022-08-01T23:04:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ab820 con 0x7ff2980b2d90 2022-08-01T23:04:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980abac0 con 0x7ff2980acbe0 2022-08-01T23:04:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 0x7ff2980ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 0x7ff2980ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48808/0 (socket says 172.21.15.32:48808) 2022-08-01T23:04:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 -- 172.21.15.32:0/3895198509 learned_addr learned my addr 172.21.15.32:0/3895198509 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff297fff700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980acbe0 0x7ff2981338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29d361700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2980b2d90 0x7ff298133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 -- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980acbe0 msgr2=0x7ff2981338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980acbe0 0x7ff2981338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 -- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2980b2d90 msgr2=0x7ff298133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2980b2d90 0x7ff298133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 -- 172.21.15.32:0/3895198509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29000b710 con 0x7ff2980a9760 2022-08-01T23:04:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29cb60700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 0x7ff2980ad350 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7ff28c00c000 tx=0x7ff28c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff28c010780 con 0x7ff2980a9760 2022-08-01T23:04:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c0108e0 con 0x7ff2980a9760 2022-08-01T23:04:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2980abd60 con 0x7ff2980a9760 2022-08-01T23:04:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff28c010c40 con 0x7ff2980a9760 2022-08-01T23:04:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.452+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2980ac130 con 0x7ff2980a9760 2022-08-01T23:04:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.456+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2980abef0 con 0x7ff2980a9760 2022-08-01T23:04:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.456+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff28c01c460 con 0x7ff2980a9760 2022-08-01T23:04:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.456+0000 7ff295ffb700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff278053aa0 0x7ff278055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.456+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff28c06c500 con 0x7ff2980a9760 2022-08-01T23:04:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.456+0000 7ff297fff700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff278053aa0 0x7ff278055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.456+0000 7ff297fff700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff278053aa0 0x7ff278055f60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7ff28400d000 tx=0x7ff284009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.464+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff28c043150 con 0x7ff2980a9760 2022-08-01T23:04:41.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.592+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2980574e0 con 0x7ff2980a9760 2022-08-01T23:04:41.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.592+0000 7ff295ffb700 1 -- 172.21.15.32:0/3895198509 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff28c046770 con 0x7ff2980a9760 2022-08-01T23:04:41.599 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:41.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.592+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff278053aa0 msgr2=0x7ff278055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.592+0000 7ff29edc4700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff278053aa0 0x7ff278055f60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7ff28400d000 tx=0x7ff284009040).stop 2022-08-01T23:04:41.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.592+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 msgr2=0x7ff2980ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.592+0000 7ff29edc4700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 0x7ff2980ad350 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7ff28c00c000 tx=0x7ff28c006df0).stop 2022-08-01T23:04:41.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 shutdown_connections 2022-08-01T23:04:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff278053aa0 0x7ff278055f60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2980b2d90 0x7ff298133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff2980acbe0 0x7ff2981338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 --2- 172.21.15.32:0/3895198509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2980a9760 0x7ff2980ad350 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 >> 172.21.15.32:0/3895198509 conn(0x7ff29805f840 msgr2=0x7ff2981368f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 shutdown_connections 2022-08-01T23:04:41.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.596+0000 7ff29edc4700 1 -- 172.21.15.32:0/3895198509 wait complete. 2022-08-01T23:04:41.612 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:04:41.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.704+0000 7f5b172e2700 1 Processor -- start 2022-08-01T23:04:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.704+0000 7f5b172e2700 1 -- start start 2022-08-01T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b100614b0 0x7f5b10061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100aee80 con 0x7f5b101405d0 2022-08-01T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b100b70b0 con 0x7f5b100614b0 2022-08-01T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100af0b0 con 0x7f5b100ad750 2022-08-01T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b162e0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b100614b0 0x7f5b10061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60554/0 (socket says 172.21.15.32:60554) 2022-08-01T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 -- 172.21.15.32:0/1431217367 learned_addr learned my addr 172.21.15.32:0/1431217367 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b16ae1700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5b101405d0 0x7f5b101466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57268/0 (socket says 172.21.15.32:57268) 2022-08-01T23:04:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 -- 172.21.15.32:0/1431217367 >> v2:172.21.15.55:3300/0 conn(0x7f5b100614b0 msgr2=0x7f5b10061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 --2- 172.21.15.32:0/1431217367 >> v2:172.21.15.55:3300/0 conn(0x7f5b100614b0 0x7f5b10061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 -- 172.21.15.32:0/1431217367 >> v1:172.21.15.55:6789/0 conn(0x7f5b101405d0 legacy=0x7f5b101466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 -- 172.21.15.32:0/1431217367 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100af350 con 0x7f5b100ad750 2022-08-01T23:04:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b15adf700 1 --2- 172.21.15.32:0/1431217367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100b9ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5b0c002f70 tx=0x7f5b0c006eb0).ready entity=mon.2 client_cookie=92de4cb535856f29 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b152de700 1 -- 172.21.15.32:0/1431217367 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b0c00d570 con 0x7f5b100ad750 2022-08-01T23:04:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b152de700 1 -- 172.21.15.32:0/1431217367 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c000e90 con 0x7f5b100ad750 2022-08-01T23:04:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b152de700 1 -- 172.21.15.32:0/1431217367 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b0c00d900 con 0x7f5b100ad750 2022-08-01T23:04:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- 172.21.15.32:0/1431217367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 msgr2=0x7f5b100b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 --2- 172.21.15.32:0/1431217367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100b9ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5b0c002f70 tx=0x7f5b0c006eb0).stop 2022-08-01T23:04:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- 172.21.15.32:0/1431217367 shutdown_connections 2022-08-01T23:04:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 --2- 172.21.15.32:0/1431217367 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100b9ff0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 --2- 172.21.15.32:0/1431217367 >> v2:172.21.15.55:3300/0 conn(0x7f5b100614b0 0x7f5b10061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- 172.21.15.32:0/1431217367 >> 172.21.15.32:0/1431217367 conn(0x7f5b10066f70 msgr2=0x7f5b10067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- 172.21.15.32:0/1431217367 shutdown_connections 2022-08-01T23:04:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.708+0000 7f5b172e2700 1 -- 172.21.15.32:0/1431217367 wait complete. 2022-08-01T23:04:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 Processor -- start 2022-08-01T23:04:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 -- start start 2022-08-01T23:04:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 0x7f5b100b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b101405d0 0x7f5b1009fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009f300 con 0x7f5b100614b0 2022-08-01T23:04:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100a2a90 con 0x7f5b101405d0 2022-08-01T23:04:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009f060 con 0x7f5b100ad750 2022-08-01T23:04:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 0x7f5b100b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 0x7f5b100b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48822/0 (socket says 172.21.15.32:48822) 2022-08-01T23:04:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 -- 172.21.15.32:0/455162329 learned_addr learned my addr 172.21.15.32:0/455162329 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b16ae1700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b101405d0 0x7f5b1009fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b15adf700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 -- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 msgr2=0x7f5b100a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 -- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b101405d0 msgr2=0x7f5b1009fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b101405d0 0x7f5b1009fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 -- 172.21.15.32:0/455162329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c00b1e0 con 0x7f5b100614b0 2022-08-01T23:04:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b162e0700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 0x7f5b100b36c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f5b0400c000 tx=0x7f5b04006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b04010780 con 0x7f5b100614b0 2022-08-01T23:04:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040108e0 con 0x7f5b100614b0 2022-08-01T23:04:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b04010c40 con 0x7f5b100614b0 2022-08-01T23:04:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b10156770 con 0x7f5b100614b0 2022-08-01T23:04:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.712+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b100a2680 con 0x7f5b100614b0 2022-08-01T23:04:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.716+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b0401c460 con 0x7f5b100614b0 2022-08-01T23:04:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.716+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b100af5f0 con 0x7f5b100614b0 2022-08-01T23:04:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.716+0000 7f5b037fe700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af0053980 0x7f5af0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.716+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b0406c3f0 con 0x7f5b100614b0 2022-08-01T23:04:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.716+0000 7f5b15adf700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af0053980 0x7f5af0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.716+0000 7f5b15adf700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af0053980 0x7f5af0055e40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f5b0c01c000 tx=0x7f5b0c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.724+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b04043150 con 0x7f5b100614b0 2022-08-01T23:04:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b100574e0 con 0x7f5b100614b0 2022-08-01T23:04:41.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b037fe700 1 -- 172.21.15.32:0/455162329 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f5b04046770 con 0x7f5b100614b0 2022-08-01T23:04:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:41.894 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:34.675417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8079":{"gid":8079,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/766376033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":766376033},{"type":"v1","addr":"172.21.15.32:6835","nonce":766376033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:41.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af0053980 msgr2=0x7f5af0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af0053980 0x7f5af0055e40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f5b0c01c000 tx=0x7f5b0c015000).stop 2022-08-01T23:04:41.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 msgr2=0x7f5b100b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:41.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 0x7f5b100b36c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f5b0400c000 tx=0x7f5b04006df0).stop 2022-08-01T23:04:41.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 shutdown_connections 2022-08-01T23:04:41.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5af0053980 0x7f5af0055e40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b101405d0 0x7f5b1009fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5b100ad750 0x7f5b100a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 --2- 172.21.15.32:0/455162329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b100614b0 0x7f5b100b36c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 >> 172.21.15.32:0/455162329 conn(0x7f5b10066f70 msgr2=0x7f5b100b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 shutdown_connections 2022-08-01T23:04:41.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:41.888+0000 7f5b172e2700 1 -- 172.21.15.32:0/455162329 wait complete. 2022-08-01T23:04:41.899 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 68 2022-08-01T23:04:42.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:04:33.672583+0000', 'modified': '2022-08-01T23:04:34.675417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8079': {'gid': 8079, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/766376033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 766376033}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 766376033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:04:42.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:04:42.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:04:42.330 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8079,"want_state":"up:active","state":"up:active","rank_uptime":8.6431113899999996,"mdsmap_epoch":68,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":13.443247951} 2022-08-01T23:04:42.331 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8079, "mdsmap_epoch": 68, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 8.64311139, "state": "up:active", "uptime": 13.443247951, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:04:42.331 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:04:42.331 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:04:42.332 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:04:42.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:42.333 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:42.341 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:42.356 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:42.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:42.412 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:42.413 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:42.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:42.414 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:42.421 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:04:42.421 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:42.424 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:42.440 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:42.440 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:42.495 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:42.496 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:42.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:42.498 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:42.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:04:42.492+0000 7f6007e04080 -1 init, newargv = 0x564b64688880 newargc=9 2022-08-01T23:04:42.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23010]: starting ceph client 2022-08-01T23:04:42.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23010]: starting fuse 2022-08-01T23:04:43.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:43.623 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:43.623 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:43.680 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:43.681 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:43.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.682 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:43.687 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:04:43.687 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:04:43.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:04:43.689 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:04:43.689 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:04:43.690 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:04:43.690 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:04:43.690 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:43.691 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:04:43.691 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:04:43.692 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:04:43.692 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:43.693 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:04:43.693 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:04:43.694 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:04:43.694 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:04:43.694 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:43.695 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:04:43.695 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:04:43.696 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:04:43.696 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:04:43.696 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:04:43.697 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:43.697 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:04:43.698 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:04:43.772 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.23010.asok 2022-08-01T23:04:43.777 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23010.asok 2022-08-01T23:04:43.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23010.asok status 2022-08-01T23:04:43.868 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:04:43.869 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:04:43.869 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:04:43.870 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:04:43.870 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:04:43.870 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:04:43.870 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:04:43.871 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "23010", 2022-08-01T23:04:43.871 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:04:43.871 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:43.871 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:04:43.871 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:04:43.872 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7100, 2022-08-01T23:04:43.872 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:04:43.872 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:04:43.872 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:04:43.873 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7100 2022-08-01T23:04:43.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:43.873 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:04:43.873 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:04:43.873 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:04:43.874 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 744902423 2022-08-01T23:04:43.874 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:04:43.874 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:43.874 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:04:43.875 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:04:43.875 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:04:43.875 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 744902423 2022-08-01T23:04:43.875 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:43.875 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7100 172.21.15.157:0/744902423", 2022-08-01T23:04:43.876 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/744902423", 2022-08-01T23:04:43.876 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:04:43.876 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 68, 2022-08-01T23:04:43.876 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 137, 2022-08-01T23:04:43.877 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:04:43.877 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:04:43.877 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:04:43.877 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:04:43.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.879 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:43.879 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:43.886 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:43.887 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:43.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.888 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:43.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.947 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:43.948 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:43.955 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:43.956 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:43.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:43.957 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:43.958 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:43.966 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:43.967 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:43.967 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:04:43.968 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:04:43.968 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:44.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23010]: fuse finished with error 0 and tester_r 0 2022-08-01T23:04:44.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:44.030 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:44.030 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:44.036 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:04:44.037 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:04:44.038 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T23:04:44.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:44.091 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:44.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:44.103 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:44.106 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:44.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:44.137 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2022-08-01T23:04:44.138 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2022-08-01T23:04:44.138 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:44.138 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2022-08-01T23:04:44.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.139 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2022-08-01T23:04:44.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.139 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:44.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.140 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:44.140 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2022-08-01T23:04:44.140 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.140 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2022-08-01T23:04:44.140 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2022-08-01T23:04:44.141 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.141 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:44.141 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:44.141 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.141 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:44.142 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:44.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.142 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2022-08-01T23:04:44.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.143 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:44.143 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2022-08-01T23:04:44.143 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:44.143 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:44.143 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:44.144 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:44.144 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2022-08-01T23:04:44.144 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:44.144 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:44.144 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.145 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:44.145 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.145 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2022-08-01T23:04:44.145 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.145 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2022-08-01T23:04:44.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2022-08-01T23:04:44.146 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2022-08-01T23:04:44.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2022-08-01T23:04:44.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:44.147 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:44.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.147 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2022-08-01T23:04:44.147 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2022-08-01T23:04:44.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.148 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:44.148 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2022-08-01T23:04:44.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2022-08-01T23:04:44.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:44.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.150 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2022-08-01T23:04:44.150 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2022-08-01T23:04:44.150 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:44.150 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2022-08-01T23:04:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2022-08-01T23:04:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2022-08-01T23:04:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2022-08-01T23:04:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2022-08-01T23:04:44.154 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2022-08-01T23:04:44.154 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2022-08-01T23:04:44.154 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:44.154 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2022-08-01T23:04:44.154 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2022-08-01T23:04:44.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2022-08-01T23:04:44.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:44.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2022-08-01T23:04:44.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2022-08-01T23:04:44.156 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:44.156 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:44.156 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:44.157 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:44.157 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2022-08-01T23:04:44.157 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2022-08-01T23:04:44.157 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.157 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2022-08-01T23:04:44.158 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:44.158 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2022-08-01T23:04:44.158 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2022-08-01T23:04:44.158 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2022-08-01T23:04:44.158 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.159 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.159 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:44.159 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.159 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2022-08-01T23:04:44.159 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:44.160 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2022-08-01T23:04:44.160 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2022-08-01T23:04:44.160 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2022-08-01T23:04:44.160 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2022-08-01T23:04:44.160 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:44.161 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.161 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:44.161 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2022-08-01T23:04:44.161 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.161 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2022-08-01T23:04:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.164 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2022-08-01T23:04:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2022-08-01T23:04:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2022-08-01T23:04:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2022-08-01T23:04:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:44.165 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:44.165 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2022-08-01T23:04:44.165 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2022-08-01T23:04:44.165 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:44.166 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.166 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.166 INFO:teuthology.orchestra.run.smithi032.stdout: client debug getattr caps = true 2022-08-01T23:04:44.166 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2022-08-01T23:04:44.166 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2022-08-01T23:04:44.167 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2022-08-01T23:04:44.167 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2022-08-01T23:04:44.167 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2022-08-01T23:04:44.167 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2022-08-01T23:04:44.167 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2022-08-01T23:04:44.168 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2022-08-01T23:04:44.168 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:44.168 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal frag = True 2022-08-01T23:04:44.169 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:44.169 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal merge size = 5 2022-08-01T23:04:44.169 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split bits = 3 2022-08-01T23:04:44.169 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split size = 100 2022-08-01T23:04:44.169 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2022-08-01T23:04:44.170 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2022-08-01T23:04:44.170 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2022-08-01T23:04:44.170 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2022-08-01T23:04:44.170 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2022-08-01T23:04:44.171 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.171 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.174 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2022-08-01T23:04:44.174 INFO:teuthology.orchestra.run.smithi130.stdout: chdir = "" 2022-08-01T23:04:44.175 INFO:teuthology.orchestra.run.smithi130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:44.175 INFO:teuthology.orchestra.run.smithi130.stdout: auth supported = cephx 2022-08-01T23:04:44.175 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.175 INFO:teuthology.orchestra.run.smithi130.stdout: filestore xattr use omap = true 2022-08-01T23:04:44.175 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.176 INFO:teuthology.orchestra.run.smithi130.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:44.176 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.176 INFO:teuthology.orchestra.run.smithi130.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:44.176 INFO:teuthology.orchestra.run.smithi130.stdout: auth debug = true 2022-08-01T23:04:44.176 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.177 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on old message = true 2022-08-01T23:04:44.177 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on bug = true 2022-08-01T23:04:44.177 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.177 INFO:teuthology.orchestra.run.smithi130.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:44.178 INFO:teuthology.orchestra.run.smithi130.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:44.178 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.178 INFO:teuthology.orchestra.run.smithi130.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:44.178 INFO:teuthology.orchestra.run.smithi130.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:44.179 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.179 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default size = 2 2022-08-01T23:04:44.179 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.179 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:44.180 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow pg remap = true 2022-08-01T23:04:44.180 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:44.180 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:44.180 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:44.180 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:44.181 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on too few osds = false 2022-08-01T23:04:44.181 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:44.181 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:44.181 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.182 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:44.182 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.182 INFO:teuthology.orchestra.run.smithi130.stdout: osd default data pool replay window = 5 2022-08-01T23:04:44.182 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.182 INFO:teuthology.orchestra.run.smithi130.stdout: mon allow pool delete = true 2022-08-01T23:04:44.183 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.183 INFO:teuthology.orchestra.run.smithi130.stdout: mon cluster log file level = debug 2022-08-01T23:04:44.183 INFO:teuthology.orchestra.run.smithi130.stdout: debug asserts on shutdown = true 2022-08-01T23:04:44.183 INFO:teuthology.orchestra.run.smithi130.stdout: mon health detail to clog = false 2022-08-01T23:04:44.184 INFO:teuthology.orchestra.run.smithi130.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:44.184 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:44.184 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.184 INFO:teuthology.orchestra.run.smithi130.stdout:[osd] 2022-08-01T23:04:44.185 INFO:teuthology.orchestra.run.smithi130.stdout: osd journal size = 100 2022-08-01T23:04:44.185 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.185 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:44.185 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub max interval = 600 2022-08-01T23:04:44.185 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.186 INFO:teuthology.orchestra.run.smithi130.stdout: osd recover clone overlap = true 2022-08-01T23:04:44.186 INFO:teuthology.orchestra.run.smithi130.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:44.186 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.186 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug shutdown = true 2022-08-01T23:04:44.187 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug op order = true 2022-08-01T23:04:44.187 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:44.187 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.187 INFO:teuthology.orchestra.run.smithi130.stdout: osd open classes on start = true 2022-08-01T23:04:44.187 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug pg log writeout = true 2022-08-01T23:04:44.188 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.188 INFO:teuthology.orchestra.run.smithi130.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:44.188 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.188 INFO:teuthology.orchestra.run.smithi130.stdout: osd map max advance = 10 2022-08-01T23:04:44.189 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.189 INFO:teuthology.orchestra.run.smithi130.stdout: journal zero on create = true 2022-08-01T23:04:44.189 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.189 INFO:teuthology.orchestra.run.smithi130.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:44.190 INFO:teuthology.orchestra.run.smithi130.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:44.190 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.190 INFO:teuthology.orchestra.run.smithi130.stdout: bdev debug aio = true 2022-08-01T23:04:44.190 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug misdirected ops = true 2022-08-01T23:04:44.191 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore block size = 96636764160 2022-08-01T23:04:44.191 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:44.191 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore fsck on mount = True 2022-08-01T23:04:44.191 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluefs = 20 2022-08-01T23:04:44.191 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluestore = 20 2022-08-01T23:04:44.192 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:44.192 INFO:teuthology.orchestra.run.smithi130.stdout: debug osd = 20 2022-08-01T23:04:44.192 INFO:teuthology.orchestra.run.smithi130.stdout: debug rocksdb = 10 2022-08-01T23:04:44.192 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:44.193 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:44.193 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:44.193 INFO:teuthology.orchestra.run.smithi130.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:44.193 INFO:teuthology.orchestra.run.smithi130.stdout: osd objectstore = bluestore 2022-08-01T23:04:44.193 INFO:teuthology.orchestra.run.smithi130.stdout: osd op complaint time = 180 2022-08-01T23:04:44.194 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.194 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr] 2022-08-01T23:04:44.195 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:44.195 INFO:teuthology.orchestra.run.smithi130.stdout: debug mgr = 20 2022-08-01T23:04:44.195 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2022-08-01T23:04:44.196 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2022-08-01T23:04:44.196 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.197 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.197 INFO:teuthology.orchestra.run.smithi130.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:44.197 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.198 INFO:teuthology.orchestra.run.smithi130.stdout:[mon] 2022-08-01T23:04:44.198 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:44.198 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2022-08-01T23:04:44.199 INFO:teuthology.orchestra.run.smithi130.stdout: debug paxos = 20 2022-08-01T23:04:44.199 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2022-08-01T23:04:44.199 INFO:teuthology.orchestra.run.smithi130.stdout: mon data avail warn = 5 2022-08-01T23:04:44.199 INFO:teuthology.orchestra.run.smithi130.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:44.200 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.200 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:44.200 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd prime pg temp = true 2022-08-01T23:04:44.200 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.201 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.201 INFO:teuthology.orchestra.run.smithi130.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:44.201 INFO:teuthology.orchestra.run.smithi130.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:44.201 INFO:teuthology.orchestra.run.smithi130.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:44.202 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.202 INFO:teuthology.orchestra.run.smithi130.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:44.203 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:44.203 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:44.203 INFO:teuthology.orchestra.run.smithi130.stdout: mon op complaint time = 120 2022-08-01T23:04:44.203 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:44.204 INFO:teuthology.orchestra.run.smithi130.stdout:[client] 2022-08-01T23:04:44.204 INFO:teuthology.orchestra.run.smithi130.stdout: rgw cache enabled = true 2022-08-01T23:04:44.204 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable ops log = true 2022-08-01T23:04:44.204 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable usage log = true 2022-08-01T23:04:44.204 INFO:teuthology.orchestra.run.smithi130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:44.205 INFO:teuthology.orchestra.run.smithi130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:44.205 INFO:teuthology.orchestra.run.smithi130.stdout: client mount timeout = 600 2022-08-01T23:04:44.205 INFO:teuthology.orchestra.run.smithi130.stdout: debug client = 20 2022-08-01T23:04:44.205 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:44.205 INFO:teuthology.orchestra.run.smithi130.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.206 INFO:teuthology.orchestra.run.smithi130.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.206 INFO:teuthology.orchestra.run.smithi130.stdout: client debug getattr caps = true 2022-08-01T23:04:44.206 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.a] 2022-08-01T23:04:44.206 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.b] 2022-08-01T23:04:44.207 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.c] 2022-08-01T23:04:44.207 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.a] 2022-08-01T23:04:44.207 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.c] 2022-08-01T23:04:44.207 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.b] 2022-08-01T23:04:44.207 INFO:teuthology.orchestra.run.smithi130.stdout:[mds] 2022-08-01T23:04:44.208 INFO:teuthology.orchestra.run.smithi130.stdout: debug mds = 20 2022-08-01T23:04:44.208 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:44.208 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal frag = True 2022-08-01T23:04:44.208 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:44.208 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal merge size = 5 2022-08-01T23:04:44.209 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal split bits = 3 2022-08-01T23:04:44.209 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal split size = 100 2022-08-01T23:04:44.209 INFO:teuthology.orchestra.run.smithi130.stdout: mds debug frag = True 2022-08-01T23:04:44.209 INFO:teuthology.orchestra.run.smithi130.stdout: mds debug scatterstat = True 2022-08-01T23:04:44.210 INFO:teuthology.orchestra.run.smithi130.stdout: mds op complaint time = 180 2022-08-01T23:04:44.210 INFO:teuthology.orchestra.run.smithi130.stdout: mds verify scatter = True 2022-08-01T23:04:44.210 INFO:teuthology.orchestra.run.smithi130.stdout: osd op complaint time = 180 2022-08-01T23:04:44.210 INFO:teuthology.orchestra.run.smithi130.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.211 INFO:teuthology.orchestra.run.smithi130.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.211 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2022-08-01T23:04:44.211 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2022-08-01T23:04:44.211 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:44.212 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2022-08-01T23:04:44.212 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.212 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2022-08-01T23:04:44.212 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.212 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:44.213 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.213 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:44.213 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2022-08-01T23:04:44.213 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.214 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2022-08-01T23:04:44.214 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2022-08-01T23:04:44.214 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.214 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:44.215 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:44.215 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.215 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:44.215 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:44.216 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.216 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2022-08-01T23:04:44.216 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.216 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:44.217 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2022-08-01T23:04:44.217 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:44.217 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:44.217 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:44.217 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:44.218 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2022-08-01T23:04:44.218 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:44.218 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:44.218 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.219 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:44.219 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.219 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2022-08-01T23:04:44.219 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.219 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2022-08-01T23:04:44.220 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.220 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2022-08-01T23:04:44.220 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2022-08-01T23:04:44.220 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2022-08-01T23:04:44.221 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:44.221 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:44.221 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.221 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2022-08-01T23:04:44.222 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2022-08-01T23:04:44.222 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.222 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:44.222 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2022-08-01T23:04:44.223 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.223 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2022-08-01T23:04:44.223 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:44.223 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.223 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2022-08-01T23:04:44.224 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2022-08-01T23:04:44.224 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:44.224 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.224 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2022-08-01T23:04:44.224 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2022-08-01T23:04:44.225 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.225 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:44.225 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.225 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2022-08-01T23:04:44.226 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.226 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2022-08-01T23:04:44.226 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.226 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:44.227 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:44.227 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.227 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2022-08-01T23:04:44.227 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2022-08-01T23:04:44.227 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2022-08-01T23:04:44.228 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:44.228 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2022-08-01T23:04:44.228 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2022-08-01T23:04:44.228 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2022-08-01T23:04:44.228 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:44.229 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2022-08-01T23:04:44.229 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2022-08-01T23:04:44.229 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:44.229 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:44.230 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:44.230 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:44.230 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2022-08-01T23:04:44.230 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2022-08-01T23:04:44.230 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.231 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2022-08-01T23:04:44.231 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:44.231 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2022-08-01T23:04:44.231 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2022-08-01T23:04:44.232 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2022-08-01T23:04:44.232 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.232 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.232 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:44.233 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.233 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2022-08-01T23:04:44.233 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:44.233 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2022-08-01T23:04:44.234 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2022-08-01T23:04:44.234 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2022-08-01T23:04:44.234 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2022-08-01T23:04:44.234 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:44.234 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.235 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:44.235 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2022-08-01T23:04:44.235 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.235 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.235 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:44.236 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:44.236 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:44.236 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.236 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:44.236 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:44.237 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:44.237 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2022-08-01T23:04:44.237 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:44.237 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2022-08-01T23:04:44.238 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2022-08-01T23:04:44.238 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2022-08-01T23:04:44.238 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2022-08-01T23:04:44.238 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:44.239 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:44.239 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2022-08-01T23:04:44.239 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2022-08-01T23:04:44.239 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:44.239 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.240 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.240 INFO:teuthology.orchestra.run.smithi157.stdout: client debug getattr caps = true 2022-08-01T23:04:44.240 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2022-08-01T23:04:44.240 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2022-08-01T23:04:44.240 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2022-08-01T23:04:44.241 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2022-08-01T23:04:44.241 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2022-08-01T23:04:44.241 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2022-08-01T23:04:44.241 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2022-08-01T23:04:44.241 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2022-08-01T23:04:44.242 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:44.242 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal frag = True 2022-08-01T23:04:44.242 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:44.242 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal merge size = 5 2022-08-01T23:04:44.243 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split bits = 3 2022-08-01T23:04:44.243 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split size = 100 2022-08-01T23:04:44.243 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2022-08-01T23:04:44.243 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2022-08-01T23:04:44.244 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2022-08-01T23:04:44.244 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2022-08-01T23:04:44.244 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2022-08-01T23:04:44.244 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.244 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.245 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2022-08-01T23:04:44.245 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2022-08-01T23:04:44.245 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:44.245 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2022-08-01T23:04:44.246 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.246 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2022-08-01T23:04:44.246 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.246 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:44.247 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.247 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:44.247 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2022-08-01T23:04:44.247 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.247 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2022-08-01T23:04:44.248 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2022-08-01T23:04:44.248 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.248 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:44.248 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:44.249 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.249 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:44.249 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:44.249 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.250 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2022-08-01T23:04:44.250 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.250 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:44.250 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2022-08-01T23:04:44.250 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:44.251 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:44.251 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:44.251 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:44.251 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2022-08-01T23:04:44.252 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:44.252 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:44.252 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.252 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:44.252 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.253 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2022-08-01T23:04:44.253 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.253 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2022-08-01T23:04:44.253 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.254 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2022-08-01T23:04:44.254 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2022-08-01T23:04:44.254 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2022-08-01T23:04:44.254 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:44.254 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:44.255 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.255 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2022-08-01T23:04:44.255 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2022-08-01T23:04:44.255 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.256 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:44.256 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2022-08-01T23:04:44.256 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.256 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2022-08-01T23:04:44.257 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:44.257 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.257 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2022-08-01T23:04:44.257 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2022-08-01T23:04:44.257 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:44.258 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.258 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2022-08-01T23:04:44.258 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2022-08-01T23:04:44.258 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.259 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:44.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.259 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2022-08-01T23:04:44.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.259 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2022-08-01T23:04:44.260 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.260 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:44.260 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:44.260 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.260 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2022-08-01T23:04:44.261 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2022-08-01T23:04:44.261 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2022-08-01T23:04:44.261 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:44.261 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2022-08-01T23:04:44.261 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2022-08-01T23:04:44.262 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2022-08-01T23:04:44.262 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:44.262 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2022-08-01T23:04:44.262 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2022-08-01T23:04:44.263 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:44.263 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:44.263 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:44.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:44.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2022-08-01T23:04:44.264 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2022-08-01T23:04:44.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.264 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2022-08-01T23:04:44.264 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:44.264 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2022-08-01T23:04:44.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2022-08-01T23:04:44.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2022-08-01T23:04:44.265 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.265 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.265 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:44.266 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.266 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2022-08-01T23:04:44.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:44.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2022-08-01T23:04:44.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2022-08-01T23:04:44.267 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2022-08-01T23:04:44.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2022-08-01T23:04:44.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:44.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:44.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:44.268 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2022-08-01T23:04:44.268 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:44.268 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.268 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:44.268 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:44.269 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:44.269 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.269 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:44.269 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:44.269 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:44.270 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2022-08-01T23:04:44.270 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:44.270 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2022-08-01T23:04:44.270 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2022-08-01T23:04:44.270 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2022-08-01T23:04:44.271 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2022-08-01T23:04:44.271 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:44.271 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:44.271 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2022-08-01T23:04:44.271 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2022-08-01T23:04:44.272 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:44.272 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.272 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.272 INFO:teuthology.orchestra.run.smithi055.stdout: client debug getattr caps = true 2022-08-01T23:04:44.272 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2022-08-01T23:04:44.273 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2022-08-01T23:04:44.273 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2022-08-01T23:04:44.273 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2022-08-01T23:04:44.273 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2022-08-01T23:04:44.274 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2022-08-01T23:04:44.274 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2022-08-01T23:04:44.274 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2022-08-01T23:04:44.274 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:44.275 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal frag = True 2022-08-01T23:04:44.275 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:44.275 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal merge size = 5 2022-08-01T23:04:44.275 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal split bits = 3 2022-08-01T23:04:44.275 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal split size = 100 2022-08-01T23:04:44.276 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2022-08-01T23:04:44.276 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2022-08-01T23:04:44.276 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2022-08-01T23:04:44.276 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2022-08-01T23:04:44.276 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2022-08-01T23:04:44.277 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 15m 2022-08-01T23:04:44.277 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 15m 2022-08-01T23:04:44.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 Processor -- start 2022-08-01T23:04:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 -- start start 2022-08-01T23:04:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee3c0b26f0 0x7fee3c0a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c147730 0x7fee3c0ab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c14ada0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0a8e50 con 0x7fee3c0b26f0 2022-08-01T23:04:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee42d3b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c147ba0 con 0x7fee3c147730 2022-08-01T23:04:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee3c0b26f0 0x7fee3c0a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee3c0b26f0 0x7fee3c0a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58616/0 (socket says 172.21.15.32:58616) 2022-08-01T23:04:44.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee3bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fee3c0b1ae0 0x7fee3c14b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57278/0 (socket says 172.21.15.32:57278) 2022-08-01T23:04:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 -- 172.21.15.32:0/2035134330 learned_addr learned my addr 172.21.15.32:0/2035134330 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee412d8700 1 --2- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c147730 0x7fee3c0ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 -- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c147730 msgr2=0x7fee3c0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:44.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 --2- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c147730 0x7fee3c0ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 -- 172.21.15.32:0/2035134330 >> v1:172.21.15.55:6789/0 conn(0x7fee3c0b1ae0 legacy=0x7fee3c14b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 -- 172.21.15.32:0/2035134330 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0b2ad0 con 0x7fee3c0b26f0 2022-08-01T23:04:44.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee40ad7700 1 --2- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c0b26f0 0x7fee3c0a7e90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fee24002f70 tx=0x7fee24006eb0).ready entity=mon.1 client_cookie=3721d357beeddb4a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee3b7fe700 1 -- 172.21.15.32:0/2035134330 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2400d570 con 0x7fee3c0b26f0 2022-08-01T23:04:44.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee3b7fe700 1 -- 172.21.15.32:0/2035134330 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24000e90 con 0x7fee3c0b26f0 2022-08-01T23:04:44.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.384+0000 7fee3b7fe700 1 -- 172.21.15.32:0/2035134330 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2400d900 con 0x7fee3c0b26f0 2022-08-01T23:04:44.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c0b26f0 msgr2=0x7fee3c0a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 --2- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c0b26f0 0x7fee3c0a7e90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fee24002f70 tx=0x7fee24006eb0).stop 2022-08-01T23:04:44.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- 172.21.15.32:0/2035134330 shutdown_connections 2022-08-01T23:04:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 --2- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c147730 0x7fee3c0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 --2- 172.21.15.32:0/2035134330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c0b26f0 0x7fee3c0a7e90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- 172.21.15.32:0/2035134330 >> 172.21.15.32:0/2035134330 conn(0x7fee3c05cbb0 msgr2=0x7fee3c05cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- 172.21.15.32:0/2035134330 shutdown_connections 2022-08-01T23:04:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- 172.21.15.32:0/2035134330 wait complete. 2022-08-01T23:04:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 Processor -- start 2022-08-01T23:04:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- start start 2022-08-01T23:04:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 0x7fee3c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c147730 0x7fee3c154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c0b61b0 0x7fee3c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0b5180 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c141770 con 0x7fee3c147730 2022-08-01T23:04:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee42d3b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c062a20 con 0x7fee3c0b61b0 2022-08-01T23:04:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 0x7fee3c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 0x7fee3c0af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48836/0 (socket says 172.21.15.32:48836) 2022-08-01T23:04:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 -- 172.21.15.32:0/3427794393 learned_addr learned my addr 172.21.15.32:0/3427794393 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee3bfff700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c147730 0x7fee3c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee412d8700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c0b61b0 0x7fee3c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 -- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c0b61b0 msgr2=0x7fee3c154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c0b61b0 0x7fee3c154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 -- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c147730 msgr2=0x7fee3c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c147730 0x7fee3c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 -- 172.21.15.32:0/3427794393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2400b1e0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee412d8700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c0b61b0 0x7fee3c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee3bfff700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c147730 0x7fee3c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.388+0000 7fee40ad7700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 0x7fee3c0af6a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fee3c0aea60 tx=0x7fee24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee24011790 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee240118f0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c154a50 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee24011c50 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c0b53b0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c1475a0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee24025030 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee39ffb700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee14053a70 0x7fee14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee2406d1e0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.392+0000 7fee3bfff700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee14053a70 0x7fee14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.396+0000 7fee3bfff700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee14053a70 0x7fee14055f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fee2c00b000 tx=0x7fee2c007e20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.400+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee24043fa0 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.528+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee3c0b5540 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.528+0000 7fee39ffb700 1 -- 172.21.15.32:0/3427794393 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fee24023020 con 0x7fee3c0b1ae0 2022-08-01T23:04:44.536 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":137,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:34.676165+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:04:31.641583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T23:04:32.681709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee14053a70 msgr2=0x7fee14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee14053a70 0x7fee14055f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fee2c00b000 tx=0x7fee2c007e20).stop 2022-08-01T23:04:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 msgr2=0x7fee3c0af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 0x7fee3c0af6a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fee3c0aea60 tx=0x7fee24015000).stop 2022-08-01T23:04:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 shutdown_connections 2022-08-01T23:04:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fee14053a70 0x7fee14055f30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fee3c0b61b0 0x7fee3c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee3c147730 0x7fee3c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 --2- 172.21.15.32:0/3427794393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee3c0b1ae0 0x7fee3c0af6a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 >> 172.21.15.32:0/3427794393 conn(0x7fee3c05cbb0 msgr2=0x7fee3c149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 shutdown_connections 2022-08-01T23:04:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.532+0000 7fee42d3b700 1 -- 172.21.15.32:0/3427794393 wait complete. 2022-08-01T23:04:44.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:04:31.641583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:04:32.681709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:04:44.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:04:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.652+0000 7f3f1709a700 1 Processor -- start 2022-08-01T23:04:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.652+0000 7f3f1709a700 1 -- start start 2022-08-01T23:04:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f100b9aa0 0x7f3f100b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f10146a70 con 0x7f3f100b4500 2022-08-01T23:04:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f100b4b20 con 0x7f3f100b72f0 2022-08-01T23:04:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f100b4dc0 con 0x7f3f100b9aa0 2022-08-01T23:04:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f14e36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f100b9aa0 0x7f3f100b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f100b4500 0x7f3f10147170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36386/0 (socket says 172.21.15.32:36386) 2022-08-01T23:04:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0ffff700 1 -- 172.21.15.32:0/4258349235 learned_addr learned my addr 172.21.15.32:0/4258349235 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0f7fe700 1 -- 172.21.15.32:0/4258349235 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86220002 0 0) 0x7f3f10146a70 con 0x7f3f100b4500 2022-08-01T23:04:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0f7fe700 1 -- 172.21.15.32:0/4258349235 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef4003260 con 0x7f3f100b4500 2022-08-01T23:04:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f14e36700 1 -- 172.21.15.32:0/4258349235 >> v1:172.21.15.55:6789/0 conn(0x7f3f100b72f0 legacy=0x7f3f1013cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0f7fe700 1 -- 172.21.15.32:0/4258349235 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636116710 0 0) 0x7f3f100b4b20 con 0x7f3f100b72f0 2022-08-01T23:04:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f14e36700 1 -- 172.21.15.32:0/4258349235 >> v1:172.21.15.32:6789/0 conn(0x7f3f100b4500 legacy=0x7f3f10147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f14e36700 1 -- 172.21.15.32:0/4258349235 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f10146cd0 con 0x7f3f100b9aa0 2022-08-01T23:04:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f14e36700 1 --2- 172.21.15.32:0/4258349235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b9aa0 0x7f3f100b9e80 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f3efc001800 tx=0x7f3f04006df0).ready entity=mon.0 client_cookie=aaeee3a5278df938 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0f7fe700 1 -- 172.21.15.32:0/4258349235 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3efc001660 con 0x7f3f100b9aa0 2022-08-01T23:04:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0f7fe700 1 -- 172.21.15.32:0/4258349235 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04000e90 con 0x7f3f100b9aa0 2022-08-01T23:04:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f0f7fe700 1 -- 172.21.15.32:0/4258349235 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f04011400 con 0x7f3f100b9aa0 2022-08-01T23:04:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- 172.21.15.32:0/4258349235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b9aa0 msgr2=0x7f3f100b9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 --2- 172.21.15.32:0/4258349235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b9aa0 0x7f3f100b9e80 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f3efc001800 tx=0x7f3f04006df0).stop 2022-08-01T23:04:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- 172.21.15.32:0/4258349235 shutdown_connections 2022-08-01T23:04:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 --2- 172.21.15.32:0/4258349235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b9aa0 0x7f3f100b9e80 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- 172.21.15.32:0/4258349235 >> 172.21.15.32:0/4258349235 conn(0x7f3f100600f0 msgr2=0x7f3f10061830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- 172.21.15.32:0/4258349235 shutdown_connections 2022-08-01T23:04:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.656+0000 7f3f1709a700 1 -- 172.21.15.32:0/4258349235 wait complete. 2022-08-01T23:04:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 Processor -- start 2022-08-01T23:04:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 -- start start 2022-08-01T23:04:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f100b4500 0x7f3f100adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 0x7f3f10149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f100b9aa0 0x7f3f101523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10146110 con 0x7f3f100b72f0 2022-08-01T23:04:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f101455b0 con 0x7f3f100b4500 2022-08-01T23:04:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f1709a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1014a960 con 0x7f3f100b9aa0 2022-08-01T23:04:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 0x7f3f10149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 0x7f3f10149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48852/0 (socket says 172.21.15.32:48852) 2022-08-01T23:04:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 -- 172.21.15.32:0/1521979591 learned_addr learned my addr 172.21.15.32:0/1521979591 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f14e36700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f100b4500 0x7f3f100adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f15637700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f100b9aa0 0x7f3f101523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 -- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f100b9aa0 msgr2=0x7f3f101523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f100b9aa0 0x7f3f101523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 -- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f100b4500 msgr2=0x7f3f100adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f100b4500 0x7f3f100adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.660+0000 7f3f0ffff700 1 -- 172.21.15.32:0/1521979591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0400b180 con 0x7f3f100b72f0 2022-08-01T23:04:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0ffff700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 0x7f3f10149ed0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f3efc0011d0 tx=0x7f3efc001ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3efc007170 con 0x7f3f100b72f0 2022-08-01T23:04:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc0072d0 con 0x7f3f100b72f0 2022-08-01T23:04:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f101450b0 con 0x7f3f100b72f0 2022-08-01T23:04:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3efc004920 con 0x7f3f100b72f0 2022-08-01T23:04:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f101466e0 con 0x7f3f100b72f0 2022-08-01T23:04:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10140dd0 con 0x7f3f100b72f0 2022-08-01T23:04:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3efc01b070 con 0x7f3f100b72f0 2022-08-01T23:04:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0dffb700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ef0053a10 0x7f3ef0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3efc06b720 con 0x7f3f100b72f0 2022-08-01T23:04:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.664+0000 7f3f14e36700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ef0053a10 0x7f3ef0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.668+0000 7f3f14e36700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ef0053a10 0x7f3ef0055ed0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f3f04019000 tx=0x7f3f100b22b0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.672+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f10140dd0 con 0x7f3f100b72f0 2022-08-01T23:04:44.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3f100ad9c0 con 0x7f3f100b72f0 2022-08-01T23:04:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f0dffb700 1 -- 172.21.15.32:0/1521979591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3f100ad9c0 con 0x7f3f100b72f0 2022-08-01T23:04:44.806 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:04:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ef0053a10 msgr2=0x7f3ef0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ef0053a10 0x7f3ef0055ed0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f3f04019000 tx=0x7f3f100b22b0).stop 2022-08-01T23:04:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 msgr2=0x7f3f10149ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 0x7f3f10149ed0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f3efc0011d0 tx=0x7f3efc001ae0).stop 2022-08-01T23:04:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 shutdown_connections 2022-08-01T23:04:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ef0053a10 0x7f3ef0055ed0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f100b9aa0 0x7f3f101523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f100b72f0 0x7f3f10149ed0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 --2- 172.21.15.32:0/1521979591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f100b4500 0x7f3f100adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 >> 172.21.15.32:0/1521979591 conn(0x7f3f100600f0 msgr2=0x7f3f100b8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 shutdown_connections 2022-08-01T23:04:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.800+0000 7f3f1709a700 1 -- 172.21.15.32:0/1521979591 wait complete. 2022-08-01T23:04:44.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:04:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.920+0000 7f8aec981700 1 Processor -- start 2022-08-01T23:04:44.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.920+0000 7f8aec981700 1 -- start start 2022-08-01T23:04:44.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.920+0000 7f8aec981700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ae8143c50 0x7f8ae80b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.920+0000 7f8aec981700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80b6a80 con 0x7f8ae8140b90 2022-08-01T23:04:44.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.920+0000 7f8aec981700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80ae7f0 con 0x7f8ae80b66a0 2022-08-01T23:04:44.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.920+0000 7f8aec981700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80aea90 con 0x7f8ae8143c50 2022-08-01T23:04:44.930 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ae8140b90 0x7f8ae80b4cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36402/0 (socket says 172.21.15.32:36402) 2022-08-01T23:04:44.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae6ffd700 1 -- 172.21.15.32:0/3319001554 learned_addr learned my addr 172.21.15.32:0/3319001554 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:44.931 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae7fff700 1 --2- 172.21.15.32:0/3319001554 >> v2:172.21.15.55:3300/0 conn(0x7f8ae8143c50 0x7f8ae80b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae67fc700 1 -- 172.21.15.32:0/3319001554 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373259497 0 0) 0x7f8ae80b6a80 con 0x7f8ae8140b90 2022-08-01T23:04:44.932 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae67fc700 1 -- 172.21.15.32:0/3319001554 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac8003260 con 0x7f8ae8140b90 2022-08-01T23:04:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae67fc700 1 -- 172.21.15.32:0/3319001554 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 406655855 0 0) 0x7f8ac8003260 con 0x7f8ae8140b90 2022-08-01T23:04:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae7fff700 1 -- 172.21.15.32:0/3319001554 >> v1:172.21.15.55:6789/0 conn(0x7f8ae80b66a0 legacy=0x7f8ae8140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae7fff700 1 -- 172.21.15.32:0/3319001554 >> v1:172.21.15.32:6789/0 conn(0x7f8ae8140b90 legacy=0x7f8ae80b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae7fff700 1 -- 172.21.15.32:0/3319001554 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80aed30 con 0x7f8ae8143c50 2022-08-01T23:04:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae7fff700 1 --2- 172.21.15.32:0/3319001554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b53e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f8ad0001820 tx=0x7f8ae00091d0).ready entity=mon.1 client_cookie=fe816fe04dae0f97 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae67fc700 1 -- 172.21.15.32:0/3319001554 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad0001680 con 0x7f8ae8143c50 2022-08-01T23:04:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae67fc700 1 -- 172.21.15.32:0/3319001554 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000b580 con 0x7f8ae8143c50 2022-08-01T23:04:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8ae67fc700 1 -- 172.21.15.32:0/3319001554 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ae000b6f0 con 0x7f8ae8143c50 2022-08-01T23:04:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 -- 172.21.15.32:0/3319001554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 msgr2=0x7f8ae80b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 --2- 172.21.15.32:0/3319001554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b53e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f8ad0001820 tx=0x7f8ae00091d0).stop 2022-08-01T23:04:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 -- 172.21.15.32:0/3319001554 shutdown_connections 2022-08-01T23:04:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 --2- 172.21.15.32:0/3319001554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b53e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 -- 172.21.15.32:0/3319001554 >> 172.21.15.32:0/3319001554 conn(0x7f8ae805fa50 msgr2=0x7f8ae8061640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 -- 172.21.15.32:0/3319001554 shutdown_connections 2022-08-01T23:04:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.924+0000 7f8aec981700 1 -- 172.21.15.32:0/3319001554 wait complete. 2022-08-01T23:04:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 Processor -- start 2022-08-01T23:04:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 -- start start 2022-08-01T23:04:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 0x7f8ae805c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ae8140b90 0x7f8ae80b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8145c50 con 0x7f8ae80b66a0 2022-08-01T23:04:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8144c20 con 0x7f8ae8143c50 2022-08-01T23:04:44.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8146010 con 0x7f8ae8140b90 2022-08-01T23:04:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 0x7f8ae805c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 0x7f8ae805c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48866/0 (socket says 172.21.15.32:48866) 2022-08-01T23:04:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 -- 172.21.15.32:0/2715122483 learned_addr learned my addr 172.21.15.32:0/2715122483 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae6ffd700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ae8140b90 0x7f8ae80b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae7fff700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 -- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ae8140b90 msgr2=0x7f8ae80b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ae8140b90 0x7f8ae80b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 -- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 msgr2=0x7f8ae80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 -- 172.21.15.32:0/2715122483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0009470 con 0x7f8ae80b66a0 2022-08-01T23:04:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae6ffd700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ae8140b90 0x7f8ae80b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae7fff700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:44.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae77fe700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 0x7f8ae805c330 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f8ad8001150 tx=0x7f8ad8007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad800f470 con 0x7f8ae80b66a0 2022-08-01T23:04:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800f5d0 con 0x7f8ae80b66a0 2022-08-01T23:04:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad800f930 con 0x7f8ae80b66a0 2022-08-01T23:04:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae81462b0 con 0x7f8ae80b66a0 2022-08-01T23:04:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.928+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae8144730 con 0x7f8ae80b66a0 2022-08-01T23:04:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.932+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae8145e70 con 0x7f8ae80b66a0 2022-08-01T23:04:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.932+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ad8019030 con 0x7f8ae80b66a0 2022-08-01T23:04:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.932+0000 7f8ae4ff9700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8acc053a10 0x7f8acc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.932+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8ad806b6d0 con 0x7f8ae80b66a0 2022-08-01T23:04:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.932+0000 7f8ae6ffd700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8acc053a10 0x7f8acc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.936+0000 7f8ae6ffd700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8acc053a10 0x7f8acc055ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f8ad000b000 tx=0x7f8ad0000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:44.940+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ae8145e70 con 0x7f8ae80b66a0 2022-08-01T23:04:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.064+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8ae8145e70 con 0x7f8ae80b66a0 2022-08-01T23:04:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8ae4ff9700 1 -- 172.21.15.32:0/2715122483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8ae8145e70 con 0x7f8ae80b66a0 2022-08-01T23:04:45.073 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8acc053a10 msgr2=0x7f8acc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8acc053a10 0x7f8acc055ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f8ad000b000 tx=0x7f8ad0000bc0).stop 2022-08-01T23:04:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 msgr2=0x7f8ae805c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 0x7f8ae805c330 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f8ad8001150 tx=0x7f8ad8007bb0).stop 2022-08-01T23:04:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 shutdown_connections 2022-08-01T23:04:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8acc053a10 0x7f8acc055ed0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ae8143c50 0x7f8ae80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8ae8140b90 0x7f8ae80b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 --2- 172.21.15.32:0/2715122483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae80b66a0 0x7f8ae805c330 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 >> 172.21.15.32:0/2715122483 conn(0x7f8ae805fa50 msgr2=0x7f8ae813eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 shutdown_connections 2022-08-01T23:04:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.068+0000 7f8aec981700 1 -- 172.21.15.32:0/2715122483 wait complete. 2022-08-01T23:04:45.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:04:45.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 Processor -- start 2022-08-01T23:04:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- start start 2022-08-01T23:04:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b9aa0 0x7f573c0b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f573c0b72f0 0x7f573c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c146a70 con 0x7f573c0b4500 2022-08-01T23:04:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c0b4dc0 con 0x7f573c0b72f0 2022-08-01T23:04:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0b4b20 con 0x7f573c0b9aa0 2022-08-01T23:04:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f573c0b72f0 0x7f573c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f573c0b72f0 0x7f573c0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48876/0 (socket says 172.21.15.32:48876) 2022-08-01T23:04:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 -- 172.21.15.32:0/3516523480 learned_addr learned my addr 172.21.15.32:0/3516523480 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f573bfff700 1 --2- 172.21.15.32:0/3516523480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b9aa0 0x7f573c0b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 -- 172.21.15.32:0/3516523480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b9aa0 msgr2=0x7f573c0b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 --2- 172.21.15.32:0/3516523480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b9aa0 0x7f573c0b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 -- 172.21.15.32:0/3516523480 >> v1:172.21.15.55:6789/0 conn(0x7f573c0b4500 legacy=0x7f573c147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 -- 172.21.15.32:0/3516523480 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c146cd0 con 0x7f573c0b72f0 2022-08-01T23:04:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f574094b700 1 --2- 172.21.15.32:0/3516523480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b72f0 0x7f573c0b54c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f57340032f0 tx=0x7f5734009290).ready entity=mon.0 client_cookie=a10adf411d657e2b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f573affd700 1 -- 172.21.15.32:0/3516523480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f573400b8a0 con 0x7f573c0b72f0 2022-08-01T23:04:45.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f573affd700 1 -- 172.21.15.32:0/3516523480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400ba00 con 0x7f573c0b72f0 2022-08-01T23:04:45.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f573affd700 1 -- 172.21.15.32:0/3516523480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f573400bb70 con 0x7f573c0b72f0 2022-08-01T23:04:45.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- 172.21.15.32:0/3516523480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b72f0 msgr2=0x7f573c0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 --2- 172.21.15.32:0/3516523480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b72f0 0x7f573c0b54c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f57340032f0 tx=0x7f5734009290).stop 2022-08-01T23:04:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- 172.21.15.32:0/3516523480 shutdown_connections 2022-08-01T23:04:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 --2- 172.21.15.32:0/3516523480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b72f0 0x7f573c0b54c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 --2- 172.21.15.32:0/3516523480 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b9aa0 0x7f573c0b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- 172.21.15.32:0/3516523480 >> 172.21.15.32:0/3516523480 conn(0x7f573c0600f0 msgr2=0x7f573c061830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- 172.21.15.32:0/3516523480 shutdown_connections 2022-08-01T23:04:45.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.192+0000 7f57423ae700 1 -- 172.21.15.32:0/3516523480 wait complete. 2022-08-01T23:04:45.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 Processor -- start 2022-08-01T23:04:45.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 -- start start 2022-08-01T23:04:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f573c0b4500 0x7f573c14a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 0x7f573c156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b9aa0 0x7f573c1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c143d80 con 0x7f573c0b9aa0 2022-08-01T23:04:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1426c0 con 0x7f573c0b4500 2022-08-01T23:04:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c155b20 con 0x7f573c0b72f0 2022-08-01T23:04:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f574094b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b9aa0 0x7f573c1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f573c0b4500 0x7f573c14a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 0x7f573c156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 0x7f573c156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60618/0 (socket says 172.21.15.32:60618) 2022-08-01T23:04:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f574094b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b9aa0 0x7f573c1411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48882/0 (socket says 172.21.15.32:48882) 2022-08-01T23:04:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 -- 172.21.15.32:0/3913922583 learned_addr learned my addr 172.21.15.32:0/3913922583 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f573c0b4500 0x7f573c14a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58664/0 (socket says 172.21.15.32:58664) 2022-08-01T23:04:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 -- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f573c0b4500 msgr2=0x7f573c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f573c0b4500 0x7f573c14a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 -- 172.21.15.32:0/3913922583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b9aa0 msgr2=0x7f573c1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b9aa0 0x7f573c1411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 -- 172.21.15.32:0/3913922583 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734009530 con 0x7f573c0b72f0 2022-08-01T23:04:45.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f573b7fe700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 0x7f573c156220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f573c0b43f0 tx=0x7f5724007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5724012400 con 0x7f573c0b72f0 2022-08-01T23:04:45.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724012560 con 0x7f573c0b72f0 2022-08-01T23:04:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c143fb0 con 0x7f573c0b72f0 2022-08-01T23:04:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57240128c0 con 0x7f573c0b72f0 2022-08-01T23:04:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.196+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c0ad970 con 0x7f573c0b72f0 2022-08-01T23:04:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.200+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c0574e0 con 0x7f573c0b72f0 2022-08-01T23:04:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.200+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5724019030 con 0x7f573c0b72f0 2022-08-01T23:04:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.200+0000 7f57397fa700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5720053a10 0x7f5720055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.200+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f572406b430 con 0x7f573c0b72f0 2022-08-01T23:04:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.200+0000 7f573bfff700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5720053a10 0x7f5720055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.200+0000 7f573bfff700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5720053a10 0x7f5720055ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f572c00c000 tx=0x7f572c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.208+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57240420e0 con 0x7f573c0b72f0 2022-08-01T23:04:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.336+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f573c155ef0 con 0x7f573c0b72f0 2022-08-01T23:04:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.340+0000 7f57397fa700 1 -- 172.21.15.32:0/3913922583 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5724045700 con 0x7f573c0b72f0 2022-08-01T23:04:45.345 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.340+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5720053a10 msgr2=0x7f5720055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.340+0000 7f57423ae700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5720053a10 0x7f5720055ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f572c00c000 tx=0x7f572c008040).stop 2022-08-01T23:04:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.340+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 msgr2=0x7f573c156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.340+0000 7f57423ae700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 0x7f573c156220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f573c0b43f0 tx=0x7f5724007c20).stop 2022-08-01T23:04:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.340+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 shutdown_connections 2022-08-01T23:04:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5720053a10 0x7f5720055ed0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f573c0b9aa0 0x7f573c1411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f573c0b72f0 0x7f573c156220 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 --2- 172.21.15.32:0/3913922583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f573c0b4500 0x7f573c14a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 >> 172.21.15.32:0/3913922583 conn(0x7f573c0600f0 msgr2=0x7f573c1452e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 shutdown_connections 2022-08-01T23:04:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.344+0000 7f57423ae700 1 -- 172.21.15.32:0/3913922583 wait complete. 2022-08-01T23:04:45.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:04:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.468+0000 7f9bd33cf700 1 Processor -- start 2022-08-01T23:04:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.468+0000 7f9bd33cf700 1 -- start start 2022-08-01T23:04:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bcc0b26f0 0x7f9bcc0a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bcc0b1ae0 0x7f9bcc147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc14ada0 con 0x7f9bcc14b470 2022-08-01T23:04:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0a8e50 con 0x7f9bcc0b26f0 2022-08-01T23:04:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b2ad0 con 0x7f9bcc0b1ae0 2022-08-01T23:04:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bcc0b26f0 0x7f9bcc0a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bcc0b26f0 0x7f9bcc0a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48888/0 (socket says 172.21.15.32:48888) 2022-08-01T23:04:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 -- 172.21.15.32:0/29904412 learned_addr learned my addr 172.21.15.32:0/29904412 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:45.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd096a700 1 --2- 172.21.15.32:0/29904412 >> v2:172.21.15.55:3300/0 conn(0x7f9bcc0b1ae0 0x7f9bcc147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 -- 172.21.15.32:0/29904412 >> v2:172.21.15.55:3300/0 conn(0x7f9bcc0b1ae0 msgr2=0x7f9bcc147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:45.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bc3fff700 1 -- 172.21.15.32:0/29904412 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537429338 0 0) 0x7f9bcc14ada0 con 0x7f9bcc14b470 2022-08-01T23:04:45.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 --2- 172.21.15.32:0/29904412 >> v2:172.21.15.55:3300/0 conn(0x7f9bcc0b1ae0 0x7f9bcc147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 -- 172.21.15.32:0/29904412 >> v1:172.21.15.55:6789/0 conn(0x7f9bcc14b470 legacy=0x7f9bcc0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 -- 172.21.15.32:0/29904412 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0b0aa0 con 0x7f9bcc0b26f0 2022-08-01T23:04:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd116b700 1 --2- 172.21.15.32:0/29904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc0b26f0 0x7f9bcc0a7e90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f9bbc002f70 tx=0x7f9bbc006eb0).ready entity=mon.0 client_cookie=3eeecf38e512d554 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bc3fff700 1 -- 172.21.15.32:0/29904412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bbc00d5c0 con 0x7f9bcc0b26f0 2022-08-01T23:04:45.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bc3fff700 1 -- 172.21.15.32:0/29904412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc000e90 con 0x7f9bcc0b26f0 2022-08-01T23:04:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bc3fff700 1 -- 172.21.15.32:0/29904412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bbc00d910 con 0x7f9bcc0b26f0 2022-08-01T23:04:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- 172.21.15.32:0/29904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc0b26f0 msgr2=0x7f9bcc0a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/29904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc0b26f0 0x7f9bcc0a7e90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f9bbc002f70 tx=0x7f9bbc006eb0).stop 2022-08-01T23:04:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- 172.21.15.32:0/29904412 shutdown_connections 2022-08-01T23:04:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/29904412 >> v2:172.21.15.55:3300/0 conn(0x7f9bcc0b1ae0 0x7f9bcc147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/29904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc0b26f0 0x7f9bcc0a7e90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- 172.21.15.32:0/29904412 >> 172.21.15.32:0/29904412 conn(0x7f9bcc05cbb0 msgr2=0x7f9bcc05cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- 172.21.15.32:0/29904412 shutdown_connections 2022-08-01T23:04:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.472+0000 7f9bd33cf700 1 -- 172.21.15.32:0/29904412 wait complete. 2022-08-01T23:04:45.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 Processor -- start 2022-08-01T23:04:45.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 -- start start 2022-08-01T23:04:45.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bcc0b1ae0 0x7f9bcc0adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bcc0b26f0 0x7f9bcc143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 0x7f9bcc143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc144410 con 0x7f9bcc14b470 2022-08-01T23:04:45.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0aef40 con 0x7f9bcc0b1ae0 2022-08-01T23:04:45.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0af1e0 con 0x7f9bcc0b26f0 2022-08-01T23:04:45.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 0x7f9bcc143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 0x7f9bcc143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48896/0 (socket says 172.21.15.32:48896) 2022-08-01T23:04:45.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 -- 172.21.15.32:0/2284632946 learned_addr learned my addr 172.21.15.32:0/2284632946 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:45.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd096a700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bcc0b26f0 0x7f9bcc143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd116b700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bcc0b1ae0 0x7f9bcc0adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 -- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bcc0b26f0 msgr2=0x7f9bcc143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bcc0b26f0 0x7f9bcc143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 -- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bcc0b1ae0 msgr2=0x7f9bcc0adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bcc0b1ae0 0x7f9bcc0adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 -- 172.21.15.32:0/2284632946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc00b1e0 con 0x7f9bcc14b470 2022-08-01T23:04:45.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd116b700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bcc0b1ae0 0x7f9bcc0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:45.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd196c700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 0x7f9bcc143f20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f9bc4001250 tx=0x7f9bc400c880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bc4006aa0 con 0x7f9bcc14b470 2022-08-01T23:04:45.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0af480 con 0x7f9bcc14b470 2022-08-01T23:04:45.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc4006c00 con 0x7f9bcc14b470 2022-08-01T23:04:45.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc0b3340 con 0x7f9bcc14b470 2022-08-01T23:04:45.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.476+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bc4007030 con 0x7f9bcc14b470 2022-08-01T23:04:45.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.480+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc0aeda0 con 0x7f9bcc14b470 2022-08-01T23:04:45.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.480+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9bc4010030 con 0x7f9bcc14b470 2022-08-01T23:04:45.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.480+0000 7f9bc27fc700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bb4053950 0x7f9bb4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.480+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9bc406c120 con 0x7f9bcc14b470 2022-08-01T23:04:45.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.480+0000 7f9bd116b700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bb4053950 0x7f9bb4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.480+0000 7f9bd116b700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bb4053950 0x7f9bb4055e10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f9bbc01a000 tx=0x7f9bbc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.488+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9bcc0aeda0 con 0x7f9bcc14b470 2022-08-01T23:04:45.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.608+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bcc0aeda0 con 0x7f9bcc14b470 2022-08-01T23:04:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bc27fc700 1 -- 172.21.15.32:0/2284632946 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9bcc0aeda0 con 0x7f9bcc14b470 2022-08-01T23:04:45.617 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:04:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bb4053950 msgr2=0x7f9bb4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bb4053950 0x7f9bb4055e10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f9bbc01a000 tx=0x7f9bbc015000).stop 2022-08-01T23:04:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 msgr2=0x7f9bcc143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 0x7f9bcc143f20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f9bc4001250 tx=0x7f9bc400c880).stop 2022-08-01T23:04:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 shutdown_connections 2022-08-01T23:04:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bb4053950 0x7f9bb4055e10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bcc14b470 0x7f9bcc143f20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bcc0b26f0 0x7f9bcc143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 --2- 172.21.15.32:0/2284632946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bcc0b1ae0 0x7f9bcc0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 >> 172.21.15.32:0/2284632946 conn(0x7f9bcc05cbb0 msgr2=0x7f9bcc149cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 shutdown_connections 2022-08-01T23:04:45.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.612+0000 7f9bd33cf700 1 -- 172.21.15.32:0/2284632946 wait complete. 2022-08-01T23:04:45.630 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:04:45.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:45.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.728+0000 7f742e1be700 1 Processor -- start 2022-08-01T23:04:45.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.728+0000 7f742e1be700 1 -- start start 2022-08-01T23:04:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f742805d1b0 0x7f7428140e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74280b1e80 0x7f742813e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7428144430 con 0x7f74280af4a0 2022-08-01T23:04:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74280afbc0 con 0x7f74280b1e80 2022-08-01T23:04:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742805d590 con 0x7f742805d1b0 2022-08-01T23:04:45.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74280b1e80 0x7f742813e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74280b1e80 0x7f742813e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48904/0 (socket says 172.21.15.32:48904) 2022-08-01T23:04:45.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 -- 172.21.15.32:0/184654220 learned_addr learned my addr 172.21.15.32:0/184654220 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:45.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d1bc700 1 --2- 172.21.15.32:0/184654220 >> v2:172.21.15.55:3300/0 conn(0x7f742805d1b0 0x7f7428140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f741ffff700 1 -- 172.21.15.32:0/184654220 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406584379 0 0) 0x7f7428144430 con 0x7f74280af4a0 2022-08-01T23:04:45.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f741ffff700 1 -- 172.21.15.32:0/184654220 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f740c003260 con 0x7f74280af4a0 2022-08-01T23:04:45.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 -- 172.21.15.32:0/184654220 >> v2:172.21.15.55:3300/0 conn(0x7f742805d1b0 msgr2=0x7f7428140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:45.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 --2- 172.21.15.32:0/184654220 >> v2:172.21.15.55:3300/0 conn(0x7f742805d1b0 0x7f7428140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 -- 172.21.15.32:0/184654220 >> v1:172.21.15.55:6789/0 conn(0x7f74280af4a0 legacy=0x7f742813de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 -- 172.21.15.32:0/184654220 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428144660 con 0x7f74280b1e80 2022-08-01T23:04:45.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742d9bd700 1 --2- 172.21.15.32:0/184654220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f742813e530 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f74200032f0 tx=0x7f7420009290).ready entity=mon.0 client_cookie=43c520b976bbae3c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f741ffff700 1 -- 172.21.15.32:0/184654220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f742000b680 con 0x7f74280b1e80 2022-08-01T23:04:45.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f741ffff700 1 -- 172.21.15.32:0/184654220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000b7e0 con 0x7f74280b1e80 2022-08-01T23:04:45.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f741ffff700 1 -- 172.21.15.32:0/184654220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f742000b950 con 0x7f74280b1e80 2022-08-01T23:04:45.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- 172.21.15.32:0/184654220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 msgr2=0x7f742813e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 --2- 172.21.15.32:0/184654220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f742813e530 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f74200032f0 tx=0x7f7420009290).stop 2022-08-01T23:04:45.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- 172.21.15.32:0/184654220 shutdown_connections 2022-08-01T23:04:45.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 --2- 172.21.15.32:0/184654220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f742813e530 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 --2- 172.21.15.32:0/184654220 >> v2:172.21.15.55:3300/0 conn(0x7f742805d1b0 0x7f7428140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- 172.21.15.32:0/184654220 >> 172.21.15.32:0/184654220 conn(0x7f7428060310 msgr2=0x7f742805c760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- 172.21.15.32:0/184654220 shutdown_connections 2022-08-01T23:04:45.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.732+0000 7f742e1be700 1 -- 172.21.15.32:0/184654220 wait complete. 2022-08-01T23:04:45.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 Processor -- start 2022-08-01T23:04:45.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 -- start start 2022-08-01T23:04:45.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f742805d1b0 0x7f7428146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74280af4a0 0x7f7428150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f7428150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280665b0 con 0x7f74280b1e80 2022-08-01T23:04:45.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74281470f0 con 0x7f74280af4a0 2022-08-01T23:04:45.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7428151010 con 0x7f742805d1b0 2022-08-01T23:04:45.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f7428150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f7428150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48910/0 (socket says 172.21.15.32:48910) 2022-08-01T23:04:45.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742c9bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74280af4a0 0x7f7428150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 -- 172.21.15.32:0/281227124 learned_addr learned my addr 172.21.15.32:0/281227124 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:45.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d1bc700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f742805d1b0 0x7f7428146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 -- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f742805d1b0 msgr2=0x7f7428146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f742805d1b0 0x7f7428146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 -- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74280af4a0 msgr2=0x7f7428150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74280af4a0 0x7f7428150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 -- 172.21.15.32:0/281227124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009530 con 0x7f74280b1e80 2022-08-01T23:04:45.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742d9bd700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f7428150b20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f74200032f0 tx=0x7f7420017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f742001f070 con 0x7f74280b1e80 2022-08-01T23:04:45.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742001b440 con 0x7f74280b1e80 2022-08-01T23:04:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74281501d0 con 0x7f74280b1e80 2022-08-01T23:04:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f742000b950 con 0x7f74280b1e80 2022-08-01T23:04:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.736+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7428067ae0 con 0x7f74280b1e80 2022-08-01T23:04:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.740+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7428144900 con 0x7f74280b1e80 2022-08-01T23:04:45.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.740+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f742000f070 con 0x7f74280b1e80 2022-08-01T23:04:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.740+0000 7f741e7fc700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f74100539e0 0x7f7410055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.740+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f742006d720 con 0x7f74280b1e80 2022-08-01T23:04:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.740+0000 7f742d1bc700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f74100539e0 0x7f7410055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.744+0000 7f742d1bc700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f74100539e0 0x7f7410055ea0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f742400b000 tx=0x7f7424006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.748+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74200443d0 con 0x7f74280b1e80 2022-08-01T23:04:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.912+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7428150360 con 0x7f74280b1e80 2022-08-01T23:04:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.912+0000 7f741e7fc700 1 -- 172.21.15.32:0/281227124 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f74200479f0 con 0x7f74280b1e80 2022-08-01T23:04:45.920 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:45.921 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:34.675417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8079":{"gid":8079,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/766376033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":766376033},{"type":"v1","addr":"172.21.15.32:6835","nonce":766376033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f74100539e0 msgr2=0x7f7410055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f74100539e0 0x7f7410055ea0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f742400b000 tx=0x7f7424006df0).stop 2022-08-01T23:04:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 msgr2=0x7f7428150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f7428150b20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f74200032f0 tx=0x7f7420017000).stop 2022-08-01T23:04:45.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 shutdown_connections 2022-08-01T23:04:45.926 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f74100539e0 0x7f7410055ea0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74280b1e80 0x7f7428150b20 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74280af4a0 0x7f7428150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 --2- 172.21.15.32:0/281227124 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f742805d1b0 0x7f7428146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 >> 172.21.15.32:0/281227124 conn(0x7f7428060310 msgr2=0x7f7428142a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 shutdown_connections 2022-08-01T23:04:45.929 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:45.916+0000 7f742e1be700 1 -- 172.21.15.32:0/281227124 wait complete. 2022-08-01T23:04:45.929 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 68 2022-08-01T23:04:46.241 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:04:33.672583+0000', 'modified': '2022-08-01T23:04:34.675417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8079}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8079': {'gid': 8079, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/766376033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 766376033}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 766376033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:04:46.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:04:46.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:04:46.357 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8079,"want_state":"up:active","state":"up:active","rank_uptime":12.669757576,"mdsmap_epoch":68,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":17.469903550000002} 2022-08-01T23:04:46.358 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8079, "mdsmap_epoch": 68, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 12.669757576, "state": "up:active", "uptime": 17.46990355, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:04:46.359 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:04:46.360 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:04:46.360 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:04:46.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:46.362 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:46.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:46.427 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:46.428 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:46.439 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:46.440 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:46.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:46.442 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:46.448 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:04:46.449 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:46.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:46.509 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:46.510 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:46.524 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:46.525 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:46.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:46.526 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:46.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:04:46.552+0000 7fe944039080 -1 init, newargv = 0x55774b80ab20 newargc=9 2022-08-01T23:04:46.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23094]: starting ceph client 2022-08-01T23:04:46.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23094]: starting fuse 2022-08-01T23:04:47.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:04:47.686 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:04:47.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:04:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:04:47.747 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:04:47.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:47.748 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:04:47.754 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:04:47.755 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:04:47.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:47.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:04:47.756 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:04:47.757 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:04:47.757 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:04:47.757 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:04:47.758 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:47.758 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:04:47.759 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:04:47.759 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:04:47.759 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:47.760 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:04:47.760 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:04:47.761 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:04:47.761 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:04:47.762 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:47.762 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:04:47.762 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:04:47.763 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:04:47.763 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:04:47.764 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:04:47.764 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:04:47.764 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:04:47.765 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:04:47.809 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.23094.asok 2022-08-01T23:04:47.814 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23094.asok 2022-08-01T23:04:47.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:47.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23094.asok status 2022-08-01T23:04:47.905 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:04:47.906 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:04:47.906 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:04:47.906 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:04:47.907 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:04:47.907 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:04:47.907 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:04:47.907 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "23094", 2022-08-01T23:04:47.908 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:04:47.908 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:47.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:04:47.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:04:47.908 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7130, 2022-08-01T23:04:47.909 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:04:47.909 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:04:47.909 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:04:47.909 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7130 2022-08-01T23:04:47.910 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:47.910 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:04:47.910 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:04:47.910 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:04:47.911 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2032006382 2022-08-01T23:04:47.911 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:04:47.911 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:47.911 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:04:47.911 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:04:47.912 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:04:47.912 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2032006382 2022-08-01T23:04:47.912 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:04:47.912 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7130 172.21.15.157:0/2032006382", 2022-08-01T23:04:47.912 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/2032006382", 2022-08-01T23:04:47.913 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:04:47.913 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 68, 2022-08-01T23:04:47.913 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 137, 2022-08-01T23:04:47.913 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:04:47.914 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:04:47.914 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:04:47.914 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:04:47.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:47.916 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:47.916 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:47.924 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:47.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:47.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:47.926 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:47.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:47.943 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:47.943 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:47.952 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:04:47.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:47.953 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T23:04:47.954 DEBUG:teuthology.orchestra.run.smithi130:> import time 2022-08-01T23:04:47.954 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:04:47.955 DEBUG:teuthology.orchestra.run.smithi130:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2022-08-01T23:04:47.955 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content'"'"') 2022-08-01T23:04:47.955 DEBUG:teuthology.orchestra.run.smithi130:> f.flush() 2022-08-01T23:04:47.956 DEBUG:teuthology.orchestra.run.smithi130:> f.write('"'"'content2'"'"') 2022-08-01T23:04:47.956 DEBUG:teuthology.orchestra.run.smithi130:> while True: 2022-08-01T23:04:47.956 DEBUG:teuthology.orchestra.run.smithi130:> time.sleep(1) 2022-08-01T23:04:47.957 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:04:47.998 DEBUG:teuthology.orchestra.run.smithi130:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-08-01T23:04:48.020 INFO:teuthology.orchestra.run.smithi130.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2022-08-01T23:04:48.021 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-08-01T23:04:49.023 DEBUG:teuthology.orchestra.run.smithi130:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-08-01T23:04:49.039 INFO:teuthology.orchestra.run.smithi130.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2022-08-01T23:04:49.041 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2022-08-01T23:04:49.041 DEBUG:teuthology.orchestra.run.smithi157:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2022-08-01T23:04:49.077 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2022-08-01T23:04:49.078 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2022-08-01T23:04:49.079 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T23:04:49.079 DEBUG:teuthology.orchestra.run.smithi157:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2022-08-01T23:04:49.124 INFO:teuthology.orchestra.run.smithi157.stdout:content 2022-08-01T23:04:49.140 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:04:49.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:49.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:49.182 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:49.203 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:49.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:04:49.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:49.224 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:49.225 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:49.233 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:04:49.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:49.235 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:04:49.235 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:04:49.236 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:49.291 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23523]: fuse finished with error 0 and tester_r 0 2022-08-01T23:04:49.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:49.294 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:49.294 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:49.301 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:04:49.302 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:04:49.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:49.419 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:49.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:49.468 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:49.468 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:49.477 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:04:49.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:49.478 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:04:49.479 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:04:49.479 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:49.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23094]: fuse finished with error 0 and tester_r 0 2022-08-01T23:04:49.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:49.497 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:49.498 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:49.504 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:04:49.505 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:04:49.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:04:49.557 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:49.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:04:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fb59e700 1 Processor -- start 2022-08-01T23:04:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fb59e700 1 -- start start 2022-08-01T23:04:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fb59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48fc0ac930 0x7f48fc0b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc0aee60 con 0x7f48fc1444d0 2022-08-01T23:04:49.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fb59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc0b0be0 con 0x7f48fc0b0800 2022-08-01T23:04:49.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fb59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc0ae480 con 0x7f48fc0ac930 2022-08-01T23:04:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f48fc1444d0 0x7f48fc144920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36458/0 (socket says 172.21.15.32:36458) 2022-08-01T23:04:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48fad9d700 1 -- 172.21.15.32:0/4074172756 learned_addr learned my addr 172.21.15.32:0/4074172756 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f9d9b700 1 --2- 172.21.15.32:0/4074172756 >> v2:172.21.15.55:3300/0 conn(0x7f48fc0ac930 0x7f48fc0b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:49.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619765414 0 0) 0x7f48f4001110 con 0x7f48fc1444d0 2022-08-01T23:04:49.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e0003260 con 0x7f48fc1444d0 2022-08-01T23:04:49.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228836122 0 0) 0x7f48fc0b0be0 con 0x7f48fc0b0800 2022-08-01T23:04:49.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f4001110 con 0x7f48fc0b0800 2022-08-01T23:04:49.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f9d9b700 1 -- 172.21.15.32:0/4074172756 >> v1:172.21.15.55:6789/0 conn(0x7f48fc0b0800 legacy=0x7f48fc0b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:49.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f9d9b700 1 -- 172.21.15.32:0/4074172756 >> v1:172.21.15.32:6789/0 conn(0x7f48fc1444d0 legacy=0x7f48fc144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:49.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f9d9b700 1 -- 172.21.15.32:0/4074172756 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0ae720 con 0x7f48fc0ac930 2022-08-01T23:04:49.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.672+0000 7f48f9d9b700 1 --2- 172.21.15.32:0/4074172756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc0b1500 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f48ec0032f0 tx=0x7f48ec0091d0).ready entity=mon.1 client_cookie=48b2e6ff0fec93e1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:49.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ec00b600 con 0x7f48fc0ac930 2022-08-01T23:04:49.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00b760 con 0x7f48fc0ac930 2022-08-01T23:04:49.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48f959a700 1 -- 172.21.15.32:0/4074172756 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ec00b8d0 con 0x7f48fc0ac930 2022-08-01T23:04:49.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- 172.21.15.32:0/4074172756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 msgr2=0x7f48fc0b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:49.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 --2- 172.21.15.32:0/4074172756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc0b1500 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f48ec0032f0 tx=0x7f48ec0091d0).stop 2022-08-01T23:04:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- 172.21.15.32:0/4074172756 shutdown_connections 2022-08-01T23:04:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 --2- 172.21.15.32:0/4074172756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc0b1500 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- 172.21.15.32:0/4074172756 >> 172.21.15.32:0/4074172756 conn(0x7f48fc05bed0 msgr2=0x7f48fc0626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- 172.21.15.32:0/4074172756 shutdown_connections 2022-08-01T23:04:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- 172.21.15.32:0/4074172756 wait complete. 2022-08-01T23:04:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 Processor -- start 2022-08-01T23:04:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- start start 2022-08-01T23:04:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48fc0b0800 0x7f48fc063560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 0x7f48fc063a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0650c0 con 0x7f48fc1444d0 2022-08-01T23:04:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc156a70 con 0x7f48fc0ac930 2022-08-01T23:04:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc156d10 con 0x7f48fc0b0800 2022-08-01T23:04:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 0x7f48fc063a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 0x7f48fc063a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48922/0 (socket says 172.21.15.32:48922) 2022-08-01T23:04:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fad9d700 1 -- 172.21.15.32:0/1998800290 learned_addr learned my addr 172.21.15.32:0/1998800290 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48f9d9b700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48fc0b0800 0x7f48fc063560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.676+0000 7f48fa59c700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fad9d700 1 -- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48fc0b0800 msgr2=0x7f48fc063560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fad9d700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48fc0b0800 0x7f48fc063560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fad9d700 1 -- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 msgr2=0x7f48fc066ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fad9d700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc066ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fad9d700 1 -- 172.21.15.32:0/1998800290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009470 con 0x7f48fc1444d0 2022-08-01T23:04:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fad9d700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 0x7f48fc063a50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f48f40012a0 tx=0x7f48f4007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48f40070d0 con 0x7f48fc1444d0 2022-08-01T23:04:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f4007230 con 0x7f48fc1444d0 2022-08-01T23:04:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48f4005610 con 0x7f48fc1444d0 2022-08-01T23:04:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc156fb0 con 0x7f48fc1444d0 2022-08-01T23:04:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc0671d0 con 0x7f48fc1444d0 2022-08-01T23:04:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc0ae9c0 con 0x7f48fc1444d0 2022-08-01T23:04:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.680+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f48f401a060 con 0x7f48fc1444d0 2022-08-01T23:04:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.684+0000 7f48eb7fe700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48d8053af0 0x7f48d8055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.684+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f48f406b6f0 con 0x7f48fc1444d0 2022-08-01T23:04:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.684+0000 7f48fa59c700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48d8053af0 0x7f48d8055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.684+0000 7f48fa59c700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48d8053af0 0x7f48d8055fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f48f000d000 tx=0x7f48f0007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.688+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48f40424b0 con 0x7f48fc1444d0 2022-08-01T23:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.812+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f48fc0aee60 con 0x7f48fc1444d0 2022-08-01T23:04:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48eb7fe700 1 -- 172.21.15.32:0/1998800290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f48fc0aee60 con 0x7f48fc1444d0 2022-08-01T23:04:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:49.822 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 137, 2022-08-01T23:04:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:04:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:04:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:04:34.676165+0000", 2022-08-01T23:04:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:04:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:04:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:04:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:04:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:04:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:04:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:04:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:04:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:04:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:04:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:04:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:04:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 26, 2022-08-01T23:04:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:04:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:04:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:04:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:04:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:04:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:04:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:04:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:04:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:04:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:04:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:04:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:04:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:04:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:04:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:04:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:04:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:04:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:04:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:04:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:04:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:04:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:04:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:04:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 25, 2022-08-01T23:04:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:04:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:04:31.641583+0000", 2022-08-01T23:04:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "136", 2022-08-01T23:04:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:04:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:04:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:04:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:04:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:04:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:04:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 26, 2022-08-01T23:04:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:04:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:04:32.681709+0000", 2022-08-01T23:04:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "137", 2022-08-01T23:04:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.888 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:49.889 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:49.890 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:04:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:49.891 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:04:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:04:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.892 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:04:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:04:49.893 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2022-08-01T23:04:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:04:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:04:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:04:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:04:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:04:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:04:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:49.905 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:04:49.906 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:04:49.907 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:04:49.908 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:04:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:04:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:04:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.910 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:04:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:49.912 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:04:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:04:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:04:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:04:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:04:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:04:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:04:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:04:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.924 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:04:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:04:49.925 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:04:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:04:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:49.926 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:49.927 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:04:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.928 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:49.929 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:04:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:04:49.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.933 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.934 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:04:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:04:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:04:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:04:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:04:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:04:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:49.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:04:49.943 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:04:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:04:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:04:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:49.944 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.945 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:04:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.946 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2022-08-01T23:04:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:49.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:04:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:04:49.950 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.951 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:04:49.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:04:49.953 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.954 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:04:49.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:04:49.956 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:49.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:04:49.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:04:49.959 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.960 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:04:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:04:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:04:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:04:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:49.961 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:04:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.963 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.964 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:49.965 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:04:49.966 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:04:49.967 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:49.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:04:49.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:04:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.972 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:04:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:04:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:04:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:04:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:04:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:04:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:04:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:04:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:49.980 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:04:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.981 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:49.982 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:04:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:04:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:04:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:04:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:04:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:04:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:49.992 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:04:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:04:49.994 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:04:49.995 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:04:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:04:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:04:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:49.996 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:49.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:04:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:49.998 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:49.999 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:50.000 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:04:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:04:50.002 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:50.003 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:04:50.004 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:04:50.005 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.006 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:04:50.007 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.008 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:04:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.010 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:04:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:04:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:04:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:04:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:04:50.013 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:04:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:50.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:04:50.015 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:50.016 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:50.017 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.018 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:04:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:04:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:04:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:04:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.023 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:04:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:04:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:50.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:04:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:04:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:50.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:04:50.030 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:04:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:04:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:04:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:50.031 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:50.032 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:04:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.033 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:04:50.034 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.035 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.036 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:04:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:50.037 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.038 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:04:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.039 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.040 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:04:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.042 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:04:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:04:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.046 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:04:50.047 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:50.048 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:50.049 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:04:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.050 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:04:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:04:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:04:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:04:50.051 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:04:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:04:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:04:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:04:50.052 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:04:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:04:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:04:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:04:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:04:50.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:04:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:50.055 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:04:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:04:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:50.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:04:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:04:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:04:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:04:50.057 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:04:50.058 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48d8053af0 msgr2=0x7f48d8055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48d8053af0 0x7f48d8055fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f48f000d000 tx=0x7f48f0007a20).stop 2022-08-01T23:04:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 msgr2=0x7f48fc063a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 0x7f48fc063a50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f48f40012a0 tx=0x7f48f4007a80).stop 2022-08-01T23:04:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 shutdown_connections 2022-08-01T23:04:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f48d8053af0 0x7f48d8055fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48fc1444d0 0x7f48fc063a50 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48fc0b0800 0x7f48fc063560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 --2- 172.21.15.32:0/1998800290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48fc0ac930 0x7f48fc066ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 >> 172.21.15.32:0/1998800290 conn(0x7f48fc05bed0 msgr2=0x7f48fc062610 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 shutdown_connections 2022-08-01T23:04:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:49.816+0000 7f48fb59e700 1 -- 172.21.15.32:0/1998800290 wait complete. 2022-08-01T23:04:50.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.164+0000 7f29cba42700 1 Processor -- start 2022-08-01T23:04:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.164+0000 7f29cba42700 1 -- start start 2022-08-01T23:04:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29cba42700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29c40b8440 0x7f29c41410c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29cba42700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c4067210 con 0x7f29c40b8060 2022-08-01T23:04:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29cba42700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40674b0 con 0x7f29c4064fe0 2022-08-01T23:04:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29cba42700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c4067750 con 0x7f29c40b8440 2022-08-01T23:04:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c97de700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29c40b8060 0x7f29c40b2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36468/0 (socket says 172.21.15.32:36468) 2022-08-01T23:04:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c97de700 1 -- 172.21.15.32:0/3896449262 learned_addr learned my addr 172.21.15.32:0/3896449262 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c9fdf700 1 --2- 172.21.15.32:0/3896449262 >> v2:172.21.15.55:3300/0 conn(0x7f29c40b8440 0x7f29c41410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809302256 0 0) 0x7f29c4067210 con 0x7f29c40b8060 2022-08-01T23:04:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a8003260 con 0x7f29c40b8060 2022-08-01T23:04:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232760717 0 0) 0x7f29c40674b0 con 0x7f29c4064fe0 2022-08-01T23:04:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c4067210 con 0x7f29c4064fe0 2022-08-01T23:04:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2039047747 0 0) 0x7f29a8003260 con 0x7f29c40b8060 2022-08-01T23:04:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 >> v2:172.21.15.55:3300/0 conn(0x7f29c40b8440 msgr2=0x7f29c41410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 --2- 172.21.15.32:0/3896449262 >> v2:172.21.15.55:3300/0 conn(0x7f29c40b8440 0x7f29c41410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 >> v1:172.21.15.55:6789/0 conn(0x7f29c4064fe0 legacy=0x7f29c4141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40b26b0 con 0x7f29c40b8060 2022-08-01T23:04:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f29c00019e0 con 0x7f29c40b8060 2022-08-01T23:04:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 >> v1:172.21.15.32:6789/0 conn(0x7f29c40b8060 legacy=0x7f29c40b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29a80038e0 0x7f29a8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a8004500 0x7f29a8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004e70 0x7f29a800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4067750 con 0x7f29a8004e70 2022-08-01T23:04:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a800bcf0 con 0x7f29a8004500 2022-08-01T23:04:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a800beb0 con 0x7f29a80038e0 2022-08-01T23:04:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a8004500 0x7f29a8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c97de700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29a80038e0 0x7f29a8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c9fdf700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004e70 0x7f29a800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 -- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29a80038e0 msgr2=0x7f29a8003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29a80038e0 0x7f29a8003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 -- 172.21.15.32:0/3896449262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004e70 msgr2=0x7f29a800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004e70 0x7f29a800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 -- 172.21.15.32:0/3896449262 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8003260 con 0x7f29a8004500 2022-08-01T23:04:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c9fdf700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004e70 0x7f29a800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c9fdf700 1 -- 172.21.15.32:0/3896449262 reap_dead start 2022-08-01T23:04:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a8004500 0x7f29a8004980 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f29a80054d0 tx=0x7f29b4007a00).ready entity=mon.1 client_cookie=7fc5aa3f1546940 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b4007d10 con 0x7f29a8004500 2022-08-01T23:04:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4007030 con 0x7f29a8004500 2022-08-01T23:04:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.168+0000 7f29bbfff700 1 -- 172.21.15.32:0/3896449262 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b4007390 con 0x7f29a8004500 2022-08-01T23:04:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a8004500 msgr2=0x7f29a8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a8004500 0x7f29a8004980 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f29a80054d0 tx=0x7f29b4007a00).stop 2022-08-01T23:04:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- 172.21.15.32:0/3896449262 shutdown_connections 2022-08-01T23:04:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 --2- 172.21.15.32:0/3896449262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a8004500 0x7f29a8004980 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- 172.21.15.32:0/3896449262 >> 172.21.15.32:0/3896449262 conn(0x7f29c4060310 msgr2=0x7f29c40b7950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- 172.21.15.32:0/3896449262 shutdown_connections 2022-08-01T23:04:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- 172.21.15.32:0/3896449262 wait complete. 2022-08-01T23:04:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 Processor -- start 2022-08-01T23:04:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- start start 2022-08-01T23:04:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 0x7f29c40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c4154870 0x7f29c4150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c4151270 0x7f29c40aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40aead0 con 0x7f29a8004500 2022-08-01T23:04:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4150990 con 0x7f29c4151270 2022-08-01T23:04:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29cba42700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40ac870 con 0x7f29c4154870 2022-08-01T23:04:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29c97de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 0x7f29c40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29c97de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 0x7f29c40b7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48938/0 (socket says 172.21.15.32:48938) 2022-08-01T23:04:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29c97de700 1 -- 172.21.15.32:0/3786643910 learned_addr learned my addr 172.21.15.32:0/3786643910 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c4154870 0x7f29c4150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.172+0000 7f29c9fdf700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c4151270 0x7f29c40aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c97de700 1 -- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c4154870 msgr2=0x7f29c4150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c97de700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c4154870 0x7f29c4150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c97de700 1 -- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c4151270 msgr2=0x7f29c40aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c97de700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c4151270 0x7f29c40aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c97de700 1 -- 172.21.15.32:0/3786643910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40acb10 con 0x7f29a8004500 2022-08-01T23:04:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c4154870 0x7f29c4150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c9fdf700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c4151270 0x7f29c40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c97de700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 0x7f29c40b7870 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f29c000d000 tx=0x7f29c40673a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c000fa60 con 0x7f29a8004500 2022-08-01T23:04:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000fbc0 con 0x7f29a8004500 2022-08-01T23:04:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40acdb0 con 0x7f29a8004500 2022-08-01T23:04:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c0012350 con 0x7f29a8004500 2022-08-01T23:04:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c4154cd0 con 0x7f29a8004500 2022-08-01T23:04:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c4151890 con 0x7f29a8004500 2022-08-01T23:04:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29c001b030 con 0x7f29a8004500 2022-08-01T23:04:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29ba7fc700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29ac053b30 0x7f29ac055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f29c006bf20 con 0x7f29a8004500 2022-08-01T23:04:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.176+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29ac053b30 0x7f29ac055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.180+0000 7f29c8fdd700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29ac053b30 0x7f29ac055ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f29b4018000 tx=0x7f29b4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.184+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29c0042ce0 con 0x7f29a8004500 2022-08-01T23:04:50.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c40574e0 con 0x7f29a8004500 2022-08-01T23:04:50.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29ba7fc700 1 -- 172.21.15.32:0/3786643910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f29c0019030 con 0x7f29a8004500 2022-08-01T23:04:50.355 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:50.356 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:34.675417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8079":{"gid":8079,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/766376033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":766376033},{"type":"v1","addr":"172.21.15.32:6835","nonce":766376033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:50.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29ac053b30 msgr2=0x7f29ac055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29ac053b30 0x7f29ac055ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f29b4018000 tx=0x7f29b4015000).stop 2022-08-01T23:04:50.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 msgr2=0x7f29c40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 0x7f29c40b7870 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f29c000d000 tx=0x7f29c40673a0).stop 2022-08-01T23:04:50.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 shutdown_connections 2022-08-01T23:04:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f29ac053b30 0x7f29ac055ff0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.362 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29c4151270 0x7f29c40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f29c4154870 0x7f29c4150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 --2- 172.21.15.32:0/3786643910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a8004500 0x7f29c40b7870 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 >> 172.21.15.32:0/3786643910 conn(0x7f29c4060310 msgr2=0x7f29c41416a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 shutdown_connections 2022-08-01T23:04:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.348+0000 7f29cba42700 1 -- 172.21.15.32:0/3786643910 wait complete. 2022-08-01T23:04:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 68 2022-08-01T23:04:50.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:04:50.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d68d4700 1 Processor -- start 2022-08-01T23:04:50.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d68d4700 1 -- start start 2022-08-01T23:04:50.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d68d4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d00b48e0 0x7f72d00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d68d4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d0141030 con 0x7f72d0144f00 2022-08-01T23:04:50.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d68d4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d00adf60 con 0x7f72d00b2740 2022-08-01T23:04:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d68d4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0141260 con 0x7f72d00b48e0 2022-08-01T23:04:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d60d3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72d0144f00 0x7f72d013ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36490/0 (socket says 172.21.15.32:36490) 2022-08-01T23:04:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d60d3700 1 -- 172.21.15.32:0/4269670090 learned_addr learned my addr 172.21.15.32:0/4269670090 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d58d2700 1 --2- 172.21.15.32:0/4269670090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d00b48e0 0x7f72d00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202511443 0 0) 0x7f72d0141030 con 0x7f72d0144f00 2022-08-01T23:04:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72bc003260 con 0x7f72d0144f00 2022-08-01T23:04:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218138843 0 0) 0x7f72d00adf60 con 0x7f72d00b2740 2022-08-01T23:04:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.468+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d0141030 con 0x7f72d00b2740 2022-08-01T23:04:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d58d2700 1 -- 172.21.15.32:0/4269670090 >> v1:172.21.15.55:6789/0 conn(0x7f72d00b2740 legacy=0x7f72d0141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d58d2700 1 -- 172.21.15.32:0/4269670090 >> v1:172.21.15.32:6789/0 conn(0x7f72d0144f00 legacy=0x7f72d013ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d58d2700 1 -- 172.21.15.32:0/4269670090 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0145330 con 0x7f72d00b48e0 2022-08-01T23:04:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d58d2700 1 --2- 172.21.15.32:0/4269670090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d00b48e0 0x7f72d00b4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f72c00032f0 tx=0x7f72c00091d0).ready entity=mon.2 client_cookie=fb53ebd717f546bb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c000b640 con 0x7f72d00b48e0 2022-08-01T23:04:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c000b7a0 con 0x7f72d00b48e0 2022-08-01T23:04:50.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d48d0700 1 -- 172.21.15.32:0/4269670090 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c000b910 con 0x7f72d00b48e0 2022-08-01T23:04:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- 172.21.15.32:0/4269670090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d00b48e0 msgr2=0x7f72d00b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 --2- 172.21.15.32:0/4269670090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d00b48e0 0x7f72d00b4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f72c00032f0 tx=0x7f72c00091d0).stop 2022-08-01T23:04:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- 172.21.15.32:0/4269670090 shutdown_connections 2022-08-01T23:04:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 --2- 172.21.15.32:0/4269670090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d00b48e0 0x7f72d00b4cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- 172.21.15.32:0/4269670090 >> 172.21.15.32:0/4269670090 conn(0x7f72d0062b60 msgr2=0x7f72d0062f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- 172.21.15.32:0/4269670090 shutdown_connections 2022-08-01T23:04:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- 172.21.15.32:0/4269670090 wait complete. 2022-08-01T23:04:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 Processor -- start 2022-08-01T23:04:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- start start 2022-08-01T23:04:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b2740 0x7f72d014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 0x7f72d014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d0144f00 0x7f72d0115000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00af000 con 0x7f72d00b48e0 2022-08-01T23:04:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0152f80 con 0x7f72d00b2740 2022-08-01T23:04:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d68d4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0144920 con 0x7f72d0144f00 2022-08-01T23:04:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 0x7f72d014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 0x7f72d014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48952/0 (socket says 172.21.15.32:48952) 2022-08-01T23:04:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 -- 172.21.15.32:0/3718817332 learned_addr learned my addr 172.21.15.32:0/3718817332 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d60d3700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d0144f00 0x7f72d0115000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d58d2700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b2740 0x7f72d014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 -- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d0144f00 msgr2=0x7f72d0115000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d0144f00 0x7f72d0115000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 -- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b2740 msgr2=0x7f72d014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b2740 0x7f72d014f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.472+0000 7f72d50d1700 1 -- 172.21.15.32:0/3718817332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009470 con 0x7f72d00b48e0 2022-08-01T23:04:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72d50d1700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 0x7f72d014fe60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f72d0152530 tx=0x7f72c4007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c4012400 con 0x7f72d00b48e0 2022-08-01T23:04:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4012560 con 0x7f72d00b48e0 2022-08-01T23:04:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c40128c0 con 0x7f72d00b48e0 2022-08-01T23:04:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d00af1f0 con 0x7f72d00b48e0 2022-08-01T23:04:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d01533b0 con 0x7f72d00b48e0 2022-08-01T23:04:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d0141e10 con 0x7f72d00b48e0 2022-08-01T23:04:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72c4019030 con 0x7f72d00b48e0 2022-08-01T23:04:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72ceffd700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72b0053aa0 0x7f72b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72c406b3f0 con 0x7f72d00b48e0 2022-08-01T23:04:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.476+0000 7f72d58d2700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72b0053aa0 0x7f72b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.480+0000 7f72d58d2700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72b0053aa0 0x7f72b0055f60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f72c0019000 tx=0x7f72c0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:50.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.484+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72c4042110 con 0x7f72d00b48e0 2022-08-01T23:04:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:50.640+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f72d00574e0 con 0x7f72d00b48e0 2022-08-01T23:04:51.454 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:04:51.448+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:04:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.464+0000 7f72ceffd700 1 -- 172.21.15.32:0/3718817332 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72c4045730 con 0x7f72d00b48e0 2022-08-01T23:04:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72b0053aa0 msgr2=0x7f72b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72b0053aa0 0x7f72b0055f60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f72c0019000 tx=0x7f72c0016000).stop 2022-08-01T23:04:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 msgr2=0x7f72d014fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 0x7f72d014fe60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f72d0152530 tx=0x7f72c4007a90).stop 2022-08-01T23:04:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 shutdown_connections 2022-08-01T23:04:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f72b0053aa0 0x7f72b0055f60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f72d0144f00 0x7f72d0115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72d00b48e0 0x7f72d014fe60 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 --2- 172.21.15.32:0/3718817332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b2740 0x7f72d014f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 >> 172.21.15.32:0/3718817332 conn(0x7f72d0062b60 msgr2=0x7f72d013e450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 shutdown_connections 2022-08-01T23:04:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.468+0000 7f72d68d4700 1 -- 172.21.15.32:0/3718817332 wait complete. 2022-08-01T23:04:51.478 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:04:51.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:51.504 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:04:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 Processor -- start 2022-08-01T23:04:51.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 -- start start 2022-08-01T23:04:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f588405ce70 0x7f588405d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b1cd0 0x7f58840b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588413d310 con 0x7f58840b17a0 2022-08-01T23:04:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840675a0 con 0x7f588405ce70 2022-08-01T23:04:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f588a3be700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5884062bd0 con 0x7f58840b1cd0 2022-08-01T23:04:51.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5888bbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b1cd0 0x7f58840b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f588405ce70 0x7f588405d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5888bbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b1cd0 0x7f58840b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60712/0 (socket says 172.21.15.32:60712) 2022-08-01T23:04:51.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f588405ce70 0x7f588405d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58758/0 (socket says 172.21.15.32:58758) 2022-08-01T23:04:51.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5889bbd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f58840b17a0 0x7f588413fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57422/0 (socket says 172.21.15.32:57422) 2022-08-01T23:04:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5888bbb700 1 -- 172.21.15.32:0/2158648858 learned_addr learned my addr 172.21.15.32:0/2158648858 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 -- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b1cd0 msgr2=0x7f58840b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 --2- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b1cd0 0x7f58840b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 -- 172.21.15.32:0/2158648858 >> v1:172.21.15.55:6789/0 conn(0x7f58840b17a0 legacy=0x7f588413fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 -- 172.21.15.32:0/2158648858 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840b26b0 con 0x7f588405ce70 2022-08-01T23:04:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f58893bc700 1 --2- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f588405d250 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f58800032f0 tx=0x7f58800091d0).ready entity=mon.1 client_cookie=a6d9311a5eeb37d9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5877fff700 1 -- 172.21.15.32:0/2158648858 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f588000b6c0 con 0x7f588405ce70 2022-08-01T23:04:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5877fff700 1 -- 172.21.15.32:0/2158648858 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588000b820 con 0x7f588405ce70 2022-08-01T23:04:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.588+0000 7f5877fff700 1 -- 172.21.15.32:0/2158648858 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f588000b990 con 0x7f588405ce70 2022-08-01T23:04:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 msgr2=0x7f588405d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 --2- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f588405d250 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f58800032f0 tx=0x7f58800091d0).stop 2022-08-01T23:04:51.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- 172.21.15.32:0/2158648858 shutdown_connections 2022-08-01T23:04:51.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 --2- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b1cd0 0x7f58840b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 --2- 172.21.15.32:0/2158648858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f588405d250 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- 172.21.15.32:0/2158648858 >> 172.21.15.32:0/2158648858 conn(0x7f5884060310 msgr2=0x7f588405c760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- 172.21.15.32:0/2158648858 shutdown_connections 2022-08-01T23:04:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- 172.21.15.32:0/2158648858 wait complete. 2022-08-01T23:04:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 Processor -- start 2022-08-01T23:04:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- start start 2022-08-01T23:04:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f5884156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b17a0 0x7f58840b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 0x7f58840b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840b6790 con 0x7f58840b1cd0 2022-08-01T23:04:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884063170 con 0x7f588405ce70 2022-08-01T23:04:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f588a3be700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5884062ed0 con 0x7f58840b17a0 2022-08-01T23:04:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 0x7f58840b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f58893bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f5884156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 0x7f58840b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48982/0 (socket says 172.21.15.32:48982) 2022-08-01T23:04:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 -- 172.21.15.32:0/690198239 learned_addr learned my addr 172.21.15.32:0/690198239 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f58893bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f5884156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58764/0 (socket says 172.21.15.32:58764) 2022-08-01T23:04:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5888bbb700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b17a0 0x7f58840b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 -- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b17a0 msgr2=0x7f58840b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b17a0 0x7f58840b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 -- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 msgr2=0x7f5884156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f5884156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 -- 172.21.15.32:0/690198239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5880009470 con 0x7f58840b1cd0 2022-08-01T23:04:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f58893bc700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f5884156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.592+0000 7f5889bbd700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 0x7f58840b4570 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f5884060810 tx=0x7f587c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c0070d0 con 0x7f58840b1cd0 2022-08-01T23:04:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c007230 con 0x7f58840b1cd0 2022-08-01T23:04:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58840b69f0 con 0x7f58840b1cd0 2022-08-01T23:04:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c005740 con 0x7f58840b1cd0 2022-08-01T23:04:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58840b3e90 con 0x7f58840b1cd0 2022-08-01T23:04:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884067790 con 0x7f58840b1cd0 2022-08-01T23:04:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f587c01a070 con 0x7f58840b1cd0 2022-08-01T23:04:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58767fc700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f586c053980 0x7f586c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f587c06b2f0 con 0x7f58840b1cd0 2022-08-01T23:04:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.596+0000 7f58893bc700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f586c053980 0x7f586c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.600+0000 7f58893bc700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f586c053980 0x7f586c055e40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f588001b000 tx=0x7f5880016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.604+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f587c041f60 con 0x7f58840b1cd0 2022-08-01T23:04:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58840574e0 con 0x7f58840b1cd0 2022-08-01T23:04:51.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f58767fc700 1 -- 172.21.15.32:0/690198239 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f587c045580 con 0x7f58840b1cd0 2022-08-01T23:04:51.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:51.771 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:04:33.672583+0000","modified":"2022-08-01T23:04:51.452158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T23:04:51.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f586c053980 msgr2=0x7f586c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f588a3be700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f586c053980 0x7f586c055e40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f588001b000 tx=0x7f5880016000).stop 2022-08-01T23:04:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 msgr2=0x7f58840b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f588a3be700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 0x7f58840b4570 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f5884060810 tx=0x7f587c007bc0).stop 2022-08-01T23:04:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.764+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 shutdown_connections 2022-08-01T23:04:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f586c053980 0x7f586c055e40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58840b1cd0 0x7f58840b4570 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58840b17a0 0x7f58840b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 --2- 172.21.15.32:0/690198239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f588405ce70 0x7f5884156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 >> 172.21.15.32:0/690198239 conn(0x7f5884060310 msgr2=0x7f588405c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:51.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 shutdown_connections 2022-08-01T23:04:51.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.768+0000 7f588a3be700 1 -- 172.21.15.32:0/690198239 wait complete. 2022-08-01T23:04:51.776 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 69 2022-08-01T23:04:51.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:04:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 Processor -- start 2022-08-01T23:04:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 -- start start 2022-08-01T23:04:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f040ad920 0x7f3f040b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f040b2100 0x7f3f040add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f041417c0 0x7f3f04129d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f040ae1f0 con 0x7f3f040b2100 2022-08-01T23:04:51.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f041453f0 con 0x7f3f040ad920 2022-08-01T23:04:51.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0b2ef700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04140e90 con 0x7f3f041417c0 2022-08-01T23:04:51.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f040b2100 0x7f3f040add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f040b2100 0x7f3f040add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48990/0 (socket says 172.21.15.32:48990) 2022-08-01T23:04:51.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 -- 172.21.15.32:0/1495211147 learned_addr learned my addr 172.21.15.32:0/1495211147 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:51.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0a2ed700 1 --2- 172.21.15.32:0/1495211147 >> v2:172.21.15.55:3300/0 conn(0x7f3f040ad920 0x7f3f040b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0aaee700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f041417c0 0x7f3f04129d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 -- 172.21.15.32:0/1495211147 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f041417c0 msgr2=0x7f3f04129d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:51.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f041417c0 0x7f3f04129d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 -- 172.21.15.32:0/1495211147 >> v2:172.21.15.55:3300/0 conn(0x7f3f040ad920 msgr2=0x7f3f040b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:51.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 --2- 172.21.15.32:0/1495211147 >> v2:172.21.15.55:3300/0 conn(0x7f3f040ad920 0x7f3f040b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 -- 172.21.15.32:0/1495211147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040b2540 con 0x7f3f040b2100 2022-08-01T23:04:51.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f0aaee700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f041417c0 0x7f3f04129d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:51.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.888+0000 7f3f09aec700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040b2100 0x7f3f040add00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f3f040654c0 tx=0x7f3f00006eb0).ready entity=mon.0 client_cookie=f4375afb6b92cbd7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:51.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f092eb700 1 -- 172.21.15.32:0/1495211147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f0000d570 con 0x7f3f040b2100 2022-08-01T23:04:51.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f092eb700 1 -- 172.21.15.32:0/1495211147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f00000e90 con 0x7f3f040b2100 2022-08-01T23:04:51.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f092eb700 1 -- 172.21.15.32:0/1495211147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f0000d900 con 0x7f3f040b2100 2022-08-01T23:04:51.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/1495211147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040b2100 msgr2=0x7f3f040add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040b2100 0x7f3f040add00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f3f040654c0 tx=0x7f3f00006eb0).stop 2022-08-01T23:04:51.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/1495211147 shutdown_connections 2022-08-01T23:04:51.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040b2100 0x7f3f040add00 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/1495211147 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f041417c0 0x7f3f04129d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/1495211147 >> v2:172.21.15.55:3300/0 conn(0x7f3f040ad920 0x7f3f040b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/1495211147 >> 172.21.15.32:0/1495211147 conn(0x7f3f040b7760 msgr2=0x7f3f040b7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:51.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/1495211147 shutdown_connections 2022-08-01T23:04:51.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/1495211147 wait complete. 2022-08-01T23:04:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 Processor -- start 2022-08-01T23:04:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- start start 2022-08-01T23:04:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 0x7f3f04142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f040b2100 0x7f3f040b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f041417c0 0x7f3f040b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040624e0 con 0x7f3f040ad920 2022-08-01T23:04:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040b0e00 con 0x7f3f041417c0 2022-08-01T23:04:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.892+0000 7f3f0b2ef700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040b10a0 con 0x7f3f040b2100 2022-08-01T23:04:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 0x7f3f04142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 0x7f3f04142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48992/0 (socket says 172.21.15.32:48992) 2022-08-01T23:04:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 -- 172.21.15.32:0/2505859254 learned_addr learned my addr 172.21.15.32:0/2505859254 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0aaee700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f041417c0 0x7f3f040b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f09aec700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f040b2100 0x7f3f040b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 -- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f040b2100 msgr2=0x7f3f040b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f040b2100 0x7f3f040b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 -- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f041417c0 msgr2=0x7f3f040b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f041417c0 0x7f3f040b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 -- 172.21.15.32:0/2505859254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0000b1e0 con 0x7f3f040ad920 2022-08-01T23:04:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0a2ed700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 0x7f3f04142ef0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f3ef400c000 tx=0x7f3ef4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef4010780 con 0x7f3f040ad920 2022-08-01T23:04:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40108e0 con 0x7f3f040ad920 2022-08-01T23:04:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f04062900 con 0x7f3f040ad920 2022-08-01T23:04:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ef4010c40 con 0x7f3f040ad920 2022-08-01T23:04:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f040b5910 con 0x7f3f040ad920 2022-08-01T23:04:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.896+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f040626e0 con 0x7f3f040ad920 2022-08-01T23:04:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.900+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ef401c460 con 0x7f3f040ad920 2022-08-01T23:04:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.900+0000 7f3ef37fe700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ee4053a10 0x7f3ee4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.900+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f3ef406c420 con 0x7f3f040ad920 2022-08-01T23:04:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.900+0000 7f3f09aec700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ee4053a10 0x7f3ee4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.900+0000 7f3f09aec700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ee4053a10 0x7f3ee4055ed0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f3f0001b000 tx=0x7f3f00015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:51.904+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ef4043130 con 0x7f3f040ad920 2022-08-01T23:04:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.064+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3f041445e0 con 0x7f3f040ad920 2022-08-01T23:04:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.476+0000 7f3ef37fe700 1 -- 172.21.15.32:0/2505859254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ef4046750 con 0x7f3f040ad920 2022-08-01T23:04:52.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ee4053a10 msgr2=0x7f3ee4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ee4053a10 0x7f3ee4055ed0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f3f0001b000 tx=0x7f3f00015000).stop 2022-08-01T23:04:52.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 msgr2=0x7f3f04142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 0x7f3f04142ef0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f3ef400c000 tx=0x7f3ef4006df0).stop 2022-08-01T23:04:52.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 shutdown_connections 2022-08-01T23:04:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3ee4053a10 0x7f3ee4055ed0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f041417c0 0x7f3f040b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f040b2100 0x7f3f040b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 --2- 172.21.15.32:0/2505859254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f040ad920 0x7f3f04142ef0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 >> 172.21.15.32:0/2505859254 conn(0x7f3f040b7760 msgr2=0x7f3f04146f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 shutdown_connections 2022-08-01T23:04:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.480+0000 7f3f0b2ef700 1 -- 172.21.15.32:0/2505859254 wait complete. 2022-08-01T23:04:52.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:04:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.608+0000 7f68ee553700 1 Processor -- start 2022-08-01T23:04:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.608+0000 7f68ee553700 1 -- start start 2022-08-01T23:04:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68e80b9aa0 0x7f68e80b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e8146a70 con 0x7f68e80b4500 2022-08-01T23:04:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80b4b20 con 0x7f68e80b72f0 2022-08-01T23:04:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e80b4dc0 con 0x7f68e80b9aa0 2022-08-01T23:04:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68e80b4500 0x7f68e8147170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36542/0 (socket says 172.21.15.32:36542) 2022-08-01T23:04:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e77fe700 1 -- 172.21.15.32:0/2237124129 learned_addr learned my addr 172.21.15.32:0/2237124129 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e7fff700 1 --2- 172.21.15.32:0/2237124129 >> v2:172.21.15.55:3300/0 conn(0x7f68e80b9aa0 0x7f68e80b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203975984 0 0) 0x7f68e8146a70 con 0x7f68e80b4500 2022-08-01T23:04:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d4003260 con 0x7f68e80b4500 2022-08-01T23:04:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110614920 0 0) 0x7f68e80b4b20 con 0x7f68e80b72f0 2022-08-01T23:04:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e8146a70 con 0x7f68e80b72f0 2022-08-01T23:04:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 840186219 0 0) 0x7f68d4003260 con 0x7f68e80b4500 2022-08-01T23:04:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e7fff700 1 -- 172.21.15.32:0/2237124129 >> v1:172.21.15.55:6789/0 conn(0x7f68e80b72f0 legacy=0x7f68e813cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e7fff700 1 -- 172.21.15.32:0/2237124129 >> v1:172.21.15.32:6789/0 conn(0x7f68e80b4500 legacy=0x7f68e8147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e7fff700 1 -- 172.21.15.32:0/2237124129 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8146cd0 con 0x7f68e80b9aa0 2022-08-01T23:04:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e7fff700 1 --2- 172.21.15.32:0/2237124129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b9aa0 0x7f68e80b9e80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f68dc001110 tx=0x7f68d8006df0).ready entity=mon.1 client_cookie=f26b643b8759367f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68dc001660 con 0x7f68e80b9aa0 2022-08-01T23:04:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d8000e90 con 0x7f68e80b9aa0 2022-08-01T23:04:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68e6ffd700 1 -- 172.21.15.32:0/2237124129 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68d8012460 con 0x7f68e80b9aa0 2022-08-01T23:04:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- 172.21.15.32:0/2237124129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b9aa0 msgr2=0x7f68e80b9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 --2- 172.21.15.32:0/2237124129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b9aa0 0x7f68e80b9e80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f68dc001110 tx=0x7f68d8006df0).stop 2022-08-01T23:04:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- 172.21.15.32:0/2237124129 shutdown_connections 2022-08-01T23:04:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 --2- 172.21.15.32:0/2237124129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b9aa0 0x7f68e80b9e80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- 172.21.15.32:0/2237124129 >> 172.21.15.32:0/2237124129 conn(0x7f68e80600f0 msgr2=0x7f68e8061830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- 172.21.15.32:0/2237124129 shutdown_connections 2022-08-01T23:04:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.612+0000 7f68ee553700 1 -- 172.21.15.32:0/2237124129 wait complete. 2022-08-01T23:04:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 Processor -- start 2022-08-01T23:04:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 -- start start 2022-08-01T23:04:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 0x7f68e80b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b72f0 0x7f68e80b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f68e80b9aa0 0x7f68e813c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8144880 con 0x7f68e80b4500 2022-08-01T23:04:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8146110 con 0x7f68e80b72f0 2022-08-01T23:04:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ee553700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80ae730 con 0x7f68e80b9aa0 2022-08-01T23:04:52.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 0x7f68e80b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:52.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 0x7f68e80b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49006/0 (socket says 172.21.15.32:49006) 2022-08-01T23:04:52.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b72f0 0x7f68e80b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:52.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68e7fff700 1 -- 172.21.15.32:0/2914422150 learned_addr learned my addr 172.21.15.32:0/2914422150 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.616+0000 7f68ecaf0700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f68e80b9aa0 0x7f68e813c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e7fff700 1 -- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f68e80b9aa0 msgr2=0x7f68e813c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e7fff700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f68e80b9aa0 0x7f68e813c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e7fff700 1 -- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b72f0 msgr2=0x7f68e80b8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e7fff700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b72f0 0x7f68e80b8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e7fff700 1 -- 172.21.15.32:0/2914422150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d800b180 con 0x7f68e80b4500 2022-08-01T23:04:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e7fff700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 0x7f68e80b0350 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f68e8144b10 tx=0x7f68d8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68d8016070 con 0x7f68e80b4500 2022-08-01T23:04:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d800bec0 con 0x7f68e80b4500 2022-08-01T23:04:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e81559e0 con 0x7f68e80b4500 2022-08-01T23:04:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68d8012460 con 0x7f68e80b4500 2022-08-01T23:04:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8145380 con 0x7f68e80b4500 2022-08-01T23:04:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e80574e0 con 0x7f68e80b4500 2022-08-01T23:04:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f68d8025030 con 0x7f68e80b4500 2022-08-01T23:04:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e57fa700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68c80539e0 0x7f68c8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f68d806cdc0 con 0x7f68e80b4500 2022-08-01T23:04:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.620+0000 7f68e77fe700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68c80539e0 0x7f68c8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:52.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.624+0000 7f68e77fe700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68c80539e0 0x7f68c8055ea0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f68dc00d000 tx=0x7f68dc000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:52.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.628+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68d8043b30 con 0x7f68e80b4500 2022-08-01T23:04:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:52.756+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68e8069b90 con 0x7f68e80b4500 2022-08-01T23:04:53.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.484+0000 7f68e57fa700 1 -- 172.21.15.32:0/2914422150 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) v1 ==== 185+0+0 (secure 0 0 0) 0x7f68d8023020 con 0x7f68e80b4500 2022-08-01T23:04:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68c80539e0 msgr2=0x7f68c8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68c80539e0 0x7f68c8055ea0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f68dc00d000 tx=0x7f68dc000bc0).stop 2022-08-01T23:04:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 msgr2=0x7f68e80b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 0x7f68e80b0350 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f68e8144b10 tx=0x7f68d8014000).stop 2022-08-01T23:04:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 shutdown_connections 2022-08-01T23:04:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f68c80539e0 0x7f68c8055ea0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f68e80b9aa0 0x7f68e813c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68e80b72f0 0x7f68e80b8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 --2- 172.21.15.32:0/2914422150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e80b4500 0x7f68e80b0350 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 >> 172.21.15.32:0/2914422150 conn(0x7f68e80600f0 msgr2=0x7f68e80b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 shutdown_connections 2022-08-01T23:04:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.488+0000 7f68ee553700 1 -- 172.21.15.32:0/2914422150 wait complete. 2022-08-01T23:04:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:04:53.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:04:53.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 Processor -- start 2022-08-01T23:04:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 -- start start 2022-08-01T23:04:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0580b6660 0x7fc0580b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc058143de0 0x7fc058147020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0580b4590 0x7fc0580b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580b16e0 con 0x7fc0580b4590 2022-08-01T23:04:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580b6f30 con 0x7fc058143de0 2022-08-01T23:04:53.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05f418700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581431a0 con 0x7fc0580b6660 2022-08-01T23:04:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05ec17700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0580b4590 0x7fc0580b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05ec17700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0580b4590 0x7fc0580b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49018/0 (socket says 172.21.15.32:49018) 2022-08-01T23:04:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.620+0000 7fc05ec17700 1 -- 172.21.15.32:0/2107298041 learned_addr learned my addr 172.21.15.32:0/2107298041 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:53.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05e416700 1 --2- 172.21.15.32:0/2107298041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0580b6660 0x7fc0580b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05dc15700 1 --2- 172.21.15.32:0/2107298041 >> v2:172.21.15.55:3300/0 conn(0x7fc058143de0 0x7fc058147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05ec17700 1 -- 172.21.15.32:0/2107298041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0580b6660 msgr2=0x7fc0580b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05ec17700 1 --2- 172.21.15.32:0/2107298041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0580b6660 0x7fc0580b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05ec17700 1 -- 172.21.15.32:0/2107298041 >> v2:172.21.15.55:3300/0 conn(0x7fc058143de0 msgr2=0x7fc058147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05ec17700 1 --2- 172.21.15.32:0/2107298041 >> v2:172.21.15.55:3300/0 conn(0x7fc058143de0 0x7fc058147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05ec17700 1 -- 172.21.15.32:0/2107298041 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058143400 con 0x7fc0580b4590 2022-08-01T23:04:53.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05ec17700 1 --2- 172.21.15.32:0/2107298041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b4590 0x7fc0580b49e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7fc0500032f0 tx=0x7fc050009290).ready entity=mon.0 client_cookie=92a16fc37eddbaf6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05d414700 1 -- 172.21.15.32:0/2107298041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05000b680 con 0x7fc0580b4590 2022-08-01T23:04:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05d414700 1 -- 172.21.15.32:0/2107298041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05000b7e0 con 0x7fc0580b4590 2022-08-01T23:04:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05d414700 1 -- 172.21.15.32:0/2107298041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05000b950 con 0x7fc0580b4590 2022-08-01T23:04:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 -- 172.21.15.32:0/2107298041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b4590 msgr2=0x7fc0580b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 --2- 172.21.15.32:0/2107298041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b4590 0x7fc0580b49e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7fc0500032f0 tx=0x7fc050009290).stop 2022-08-01T23:04:53.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 -- 172.21.15.32:0/2107298041 shutdown_connections 2022-08-01T23:04:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 --2- 172.21.15.32:0/2107298041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b4590 0x7fc0580b49e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 --2- 172.21.15.32:0/2107298041 >> v2:172.21.15.55:3300/0 conn(0x7fc058143de0 0x7fc058147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 --2- 172.21.15.32:0/2107298041 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc0580b6660 0x7fc0580b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 -- 172.21.15.32:0/2107298041 >> 172.21.15.32:0/2107298041 conn(0x7fc05805f840 msgr2=0x7fc058061590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 -- 172.21.15.32:0/2107298041 shutdown_connections 2022-08-01T23:04:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.624+0000 7fc05f418700 1 -- 172.21.15.32:0/2107298041 wait complete. 2022-08-01T23:04:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 Processor -- start 2022-08-01T23:04:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 -- start start 2022-08-01T23:04:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0580b4590 0x7fc058148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 0x7fc05812c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc058143de0 0x7fc05812c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580aec00 con 0x7fc0580b6660 2022-08-01T23:04:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05813d920 con 0x7fc0580b4590 2022-08-01T23:04:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05813dbc0 con 0x7fc058143de0 2022-08-01T23:04:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05dc15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 0x7fc05812c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05e416700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0580b4590 0x7fc058148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05dc15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 0x7fc05812c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49022/0 (socket says 172.21.15.32:49022) 2022-08-01T23:04:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc058143de0 0x7fc05812c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05dc15700 1 -- 172.21.15.32:0/1448937535 learned_addr learned my addr 172.21.15.32:0/1448937535 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 -- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0580b4590 msgr2=0x7fc058148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0580b4590 0x7fc058148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 -- 172.21.15.32:0/1448937535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 msgr2=0x7fc05812c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 0x7fc05812c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 -- 172.21.15.32:0/1448937535 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc050009530 con 0x7fc058143de0 2022-08-01T23:04:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05e416700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0580b4590 0x7fc058148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05dc15700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 0x7fc05812c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05ec17700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc058143de0 0x7fc05812c550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc0500032f0 tx=0x7fc050017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05000b680 con 0x7fc058143de0 2022-08-01T23:04:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc058146540 con 0x7fc058143de0 2022-08-01T23:04:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0580b0250 con 0x7fc058143de0 2022-08-01T23:04:53.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05000bea0 con 0x7fc058143de0 2022-08-01T23:04:53.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.628+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05001b600 con 0x7fc058143de0 2022-08-01T23:04:53.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.632+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05813d780 con 0x7fc058143de0 2022-08-01T23:04:53.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.632+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc05000f070 con 0x7fc058143de0 2022-08-01T23:04:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.632+0000 7fc0477fe700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc038053980 0x7fc038055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.632+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fc05006d520 con 0x7fc058143de0 2022-08-01T23:04:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.632+0000 7fc05e416700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc038053980 0x7fc038055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.632+0000 7fc05e416700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc038053980 0x7fc038055e40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fc04800b000 tx=0x7fc048006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.640+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc050044330 con 0x7fc058143de0 2022-08-01T23:04:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:53.772+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc05813d780 con 0x7fc058143de0 2022-08-01T23:04:54.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.496+0000 7fc0477fe700 1 -- 172.21.15.32:0/1448937535 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc05813d780 con 0x7fc058143de0 2022-08-01T23:04:54.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc038053980 msgr2=0x7fc038055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc038053980 0x7fc038055e40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fc04800b000 tx=0x7fc048006df0).stop 2022-08-01T23:04:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc058143de0 msgr2=0x7fc05812c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc058143de0 0x7fc05812c550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc0500032f0 tx=0x7fc050017000).stop 2022-08-01T23:04:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 shutdown_connections 2022-08-01T23:04:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc038053980 0x7fc038055e40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc058143de0 0x7fc05812c550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0580b6660 0x7fc05812c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 --2- 172.21.15.32:0/1448937535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0580b4590 0x7fc058148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 >> 172.21.15.32:0/1448937535 conn(0x7fc05805f840 msgr2=0x7fc0580b9840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 shutdown_connections 2022-08-01T23:04:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.500+0000 7fc05f418700 1 -- 172.21.15.32:0/1448937535 wait complete. 2022-08-01T23:04:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:04:54.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:54.522 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:54.525 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:54.527 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T23:04:54.547 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2022-08-01T23:04:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2022-08-01T23:04:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2022-08-01T23:04:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2022-08-01T23:04:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2022-08-01T23:04:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2022-08-01T23:04:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2022-08-01T23:04:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2022-08-01T23:04:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2022-08-01T23:04:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2022-08-01T23:04:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2022-08-01T23:04:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2022-08-01T23:04:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2022-08-01T23:04:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2022-08-01T23:04:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2022-08-01T23:04:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.557 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2022-08-01T23:04:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2022-08-01T23:04:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2022-08-01T23:04:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2022-08-01T23:04:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2022-08-01T23:04:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2022-08-01T23:04:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2022-08-01T23:04:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2022-08-01T23:04:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2022-08-01T23:04:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2022-08-01T23:04:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2022-08-01T23:04:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2022-08-01T23:04:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2022-08-01T23:04:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2022-08-01T23:04:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2022-08-01T23:04:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2022-08-01T23:04:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2022-08-01T23:04:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2022-08-01T23:04:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2022-08-01T23:04:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2022-08-01T23:04:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.567 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2022-08-01T23:04:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2022-08-01T23:04:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2022-08-01T23:04:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2022-08-01T23:04:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.569 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2022-08-01T23:04:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2022-08-01T23:04:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2022-08-01T23:04:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2022-08-01T23:04:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2022-08-01T23:04:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2022-08-01T23:04:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2022-08-01T23:04:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:54.574 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2022-08-01T23:04:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2022-08-01T23:04:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2022-08-01T23:04:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2022-08-01T23:04:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2022-08-01T23:04:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2022-08-01T23:04:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.576 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2022-08-01T23:04:54.576 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2022-08-01T23:04:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2022-08-01T23:04:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2022-08-01T23:04:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2022-08-01T23:04:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2022-08-01T23:04:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2022-08-01T23:04:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2022-08-01T23:04:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2022-08-01T23:04:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal frag = True 2022-08-01T23:04:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal merge size = 5 2022-08-01T23:04:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split bits = 3 2022-08-01T23:04:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: mds bal split size = 100 2022-08-01T23:04:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2022-08-01T23:04:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2022-08-01T23:04:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2022-08-01T23:04:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2022-08-01T23:04:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2022-08-01T23:04:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.584 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2022-08-01T23:04:54.584 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2022-08-01T23:04:54.584 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:54.585 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2022-08-01T23:04:54.585 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.585 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2022-08-01T23:04:54.585 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.585 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:54.586 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.586 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:54.586 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2022-08-01T23:04:54.586 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.587 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2022-08-01T23:04:54.587 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2022-08-01T23:04:54.587 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.587 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:54.587 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:54.588 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.588 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:54.588 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:54.588 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.589 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2022-08-01T23:04:54.589 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.589 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:54.589 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2022-08-01T23:04:54.589 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:54.590 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:54.590 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:54.590 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:54.590 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2022-08-01T23:04:54.591 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:54.591 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:54.591 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.591 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:54.591 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.592 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2022-08-01T23:04:54.592 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.592 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2022-08-01T23:04:54.592 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2022-08-01T23:04:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2022-08-01T23:04:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2022-08-01T23:04:54.593 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.594 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2022-08-01T23:04:54.594 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2022-08-01T23:04:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2022-08-01T23:04:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.595 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2022-08-01T23:04:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2022-08-01T23:04:54.596 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2022-08-01T23:04:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2022-08-01T23:04:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2022-08-01T23:04:54.597 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2022-08-01T23:04:54.598 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2022-08-01T23:04:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2022-08-01T23:04:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2022-08-01T23:04:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2022-08-01T23:04:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2022-08-01T23:04:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2022-08-01T23:04:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2022-08-01T23:04:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2022-08-01T23:04:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2022-08-01T23:04:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:54.602 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2022-08-01T23:04:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2022-08-01T23:04:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.603 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2022-08-01T23:04:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2022-08-01T23:04:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2022-08-01T23:04:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2022-08-01T23:04:54.605 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.605 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.605 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:54.605 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.605 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2022-08-01T23:04:54.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:54.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2022-08-01T23:04:54.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2022-08-01T23:04:54.606 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2022-08-01T23:04:54.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2022-08-01T23:04:54.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:54.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:54.607 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2022-08-01T23:04:54.608 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.608 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.608 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:54.608 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:54.609 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:54.609 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.609 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:54.609 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:54.609 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:54.610 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2022-08-01T23:04:54.610 INFO:teuthology.orchestra.run.smithi157.stdout: 2022-08-01T23:04:54.610 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2022-08-01T23:04:54.610 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2022-08-01T23:04:54.611 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2022-08-01T23:04:54.611 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2022-08-01T23:04:54.611 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:54.611 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:54.612 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2022-08-01T23:04:54.612 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2022-08-01T23:04:54.612 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:54.612 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.612 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.613 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2022-08-01T23:04:54.613 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2022-08-01T23:04:54.613 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2022-08-01T23:04:54.613 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2022-08-01T23:04:54.613 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2022-08-01T23:04:54.614 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2022-08-01T23:04:54.614 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2022-08-01T23:04:54.614 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2022-08-01T23:04:54.614 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2022-08-01T23:04:54.615 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal frag = True 2022-08-01T23:04:54.615 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:54.615 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal merge size = 5 2022-08-01T23:04:54.615 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split bits = 3 2022-08-01T23:04:54.616 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split size = 100 2022-08-01T23:04:54.616 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2022-08-01T23:04:54.616 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2022-08-01T23:04:54.616 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2022-08-01T23:04:54.616 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2022-08-01T23:04:54.617 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2022-08-01T23:04:54.617 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.617 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.617 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2022-08-01T23:04:54.618 INFO:teuthology.orchestra.run.smithi130.stdout: chdir = "" 2022-08-01T23:04:54.618 INFO:teuthology.orchestra.run.smithi130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:54.618 INFO:teuthology.orchestra.run.smithi130.stdout: auth supported = cephx 2022-08-01T23:04:54.618 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.618 INFO:teuthology.orchestra.run.smithi130.stdout: filestore xattr use omap = true 2022-08-01T23:04:54.619 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.619 INFO:teuthology.orchestra.run.smithi130.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:54.619 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.619 INFO:teuthology.orchestra.run.smithi130.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:54.620 INFO:teuthology.orchestra.run.smithi130.stdout: auth debug = true 2022-08-01T23:04:54.620 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.620 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on old message = true 2022-08-01T23:04:54.620 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on bug = true 2022-08-01T23:04:54.620 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.621 INFO:teuthology.orchestra.run.smithi130.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:54.621 INFO:teuthology.orchestra.run.smithi130.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:54.621 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.621 INFO:teuthology.orchestra.run.smithi130.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:54.622 INFO:teuthology.orchestra.run.smithi130.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:54.622 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.622 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default size = 2 2022-08-01T23:04:54.622 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.622 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:54.623 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow pg remap = true 2022-08-01T23:04:54.623 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:54.623 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:54.623 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:54.624 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:54.624 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on too few osds = false 2022-08-01T23:04:54.624 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:54.624 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:54.624 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.625 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:54.625 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.625 INFO:teuthology.orchestra.run.smithi130.stdout: osd default data pool replay window = 5 2022-08-01T23:04:54.625 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.625 INFO:teuthology.orchestra.run.smithi130.stdout: mon allow pool delete = true 2022-08-01T23:04:54.626 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.626 INFO:teuthology.orchestra.run.smithi130.stdout: mon cluster log file level = debug 2022-08-01T23:04:54.626 INFO:teuthology.orchestra.run.smithi130.stdout: debug asserts on shutdown = true 2022-08-01T23:04:54.626 INFO:teuthology.orchestra.run.smithi130.stdout: mon health detail to clog = false 2022-08-01T23:04:54.627 INFO:teuthology.orchestra.run.smithi130.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:54.627 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:54.627 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.627 INFO:teuthology.orchestra.run.smithi130.stdout:[osd] 2022-08-01T23:04:54.627 INFO:teuthology.orchestra.run.smithi130.stdout: osd journal size = 100 2022-08-01T23:04:54.628 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.628 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:54.628 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub max interval = 600 2022-08-01T23:04:54.628 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.629 INFO:teuthology.orchestra.run.smithi130.stdout: osd recover clone overlap = true 2022-08-01T23:04:54.629 INFO:teuthology.orchestra.run.smithi130.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:54.629 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.629 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug shutdown = true 2022-08-01T23:04:54.630 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug op order = true 2022-08-01T23:04:54.630 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:54.630 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.630 INFO:teuthology.orchestra.run.smithi130.stdout: osd open classes on start = true 2022-08-01T23:04:54.631 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug pg log writeout = true 2022-08-01T23:04:54.631 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.631 INFO:teuthology.orchestra.run.smithi130.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:54.631 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.631 INFO:teuthology.orchestra.run.smithi130.stdout: osd map max advance = 10 2022-08-01T23:04:54.632 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.632 INFO:teuthology.orchestra.run.smithi130.stdout: journal zero on create = true 2022-08-01T23:04:54.632 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.632 INFO:teuthology.orchestra.run.smithi130.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:54.633 INFO:teuthology.orchestra.run.smithi130.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:54.633 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.633 INFO:teuthology.orchestra.run.smithi130.stdout: bdev debug aio = true 2022-08-01T23:04:54.633 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug misdirected ops = true 2022-08-01T23:04:54.633 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore block size = 96636764160 2022-08-01T23:04:54.634 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:54.634 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore fsck on mount = True 2022-08-01T23:04:54.634 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluefs = 20 2022-08-01T23:04:54.634 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluestore = 20 2022-08-01T23:04:54.634 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:54.635 INFO:teuthology.orchestra.run.smithi130.stdout: debug osd = 20 2022-08-01T23:04:54.635 INFO:teuthology.orchestra.run.smithi130.stdout: debug rocksdb = 10 2022-08-01T23:04:54.635 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:54.635 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:54.635 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:54.636 INFO:teuthology.orchestra.run.smithi130.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:54.636 INFO:teuthology.orchestra.run.smithi130.stdout: osd objectstore = bluestore 2022-08-01T23:04:54.636 INFO:teuthology.orchestra.run.smithi130.stdout: osd op complaint time = 180 2022-08-01T23:04:54.636 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.637 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr] 2022-08-01T23:04:54.637 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:54.637 INFO:teuthology.orchestra.run.smithi130.stdout: debug mgr = 20 2022-08-01T23:04:54.637 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2022-08-01T23:04:54.638 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2022-08-01T23:04:54.638 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.638 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.638 INFO:teuthology.orchestra.run.smithi130.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:54.638 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.639 INFO:teuthology.orchestra.run.smithi130.stdout:[mon] 2022-08-01T23:04:54.639 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:54.639 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2022-08-01T23:04:54.639 INFO:teuthology.orchestra.run.smithi130.stdout: debug paxos = 20 2022-08-01T23:04:54.639 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2022-08-01T23:04:54.640 INFO:teuthology.orchestra.run.smithi130.stdout: mon data avail warn = 5 2022-08-01T23:04:54.640 INFO:teuthology.orchestra.run.smithi130.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:54.640 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.640 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:54.641 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd prime pg temp = true 2022-08-01T23:04:54.641 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.641 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.641 INFO:teuthology.orchestra.run.smithi130.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:54.641 INFO:teuthology.orchestra.run.smithi130.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:54.642 INFO:teuthology.orchestra.run.smithi130.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:54.642 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.642 INFO:teuthology.orchestra.run.smithi130.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:54.642 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:54.643 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:54.643 INFO:teuthology.orchestra.run.smithi130.stdout: mon op complaint time = 120 2022-08-01T23:04:54.643 INFO:teuthology.orchestra.run.smithi130.stdout: 2022-08-01T23:04:54.643 INFO:teuthology.orchestra.run.smithi130.stdout:[client] 2022-08-01T23:04:54.643 INFO:teuthology.orchestra.run.smithi130.stdout: rgw cache enabled = true 2022-08-01T23:04:54.644 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable ops log = true 2022-08-01T23:04:54.644 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable usage log = true 2022-08-01T23:04:54.644 INFO:teuthology.orchestra.run.smithi130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:54.644 INFO:teuthology.orchestra.run.smithi130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:54.645 INFO:teuthology.orchestra.run.smithi130.stdout: client mount timeout = 600 2022-08-01T23:04:54.645 INFO:teuthology.orchestra.run.smithi130.stdout: debug client = 20 2022-08-01T23:04:54.645 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:54.645 INFO:teuthology.orchestra.run.smithi130.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.646 INFO:teuthology.orchestra.run.smithi130.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.646 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.a] 2022-08-01T23:04:54.646 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.b] 2022-08-01T23:04:54.646 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.c] 2022-08-01T23:04:54.646 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.a] 2022-08-01T23:04:54.647 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.c] 2022-08-01T23:04:54.647 INFO:teuthology.orchestra.run.smithi130.stdout:[mds.b] 2022-08-01T23:04:54.647 INFO:teuthology.orchestra.run.smithi130.stdout:[mds] 2022-08-01T23:04:54.647 INFO:teuthology.orchestra.run.smithi130.stdout: debug mds = 20 2022-08-01T23:04:54.648 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2022-08-01T23:04:54.648 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal frag = True 2022-08-01T23:04:54.648 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:54.648 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal merge size = 5 2022-08-01T23:04:54.648 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal split bits = 3 2022-08-01T23:04:54.649 INFO:teuthology.orchestra.run.smithi130.stdout: mds bal split size = 100 2022-08-01T23:04:54.649 INFO:teuthology.orchestra.run.smithi130.stdout: mds debug frag = True 2022-08-01T23:04:54.649 INFO:teuthology.orchestra.run.smithi130.stdout: mds debug scatterstat = True 2022-08-01T23:04:54.649 INFO:teuthology.orchestra.run.smithi130.stdout: mds op complaint time = 180 2022-08-01T23:04:54.650 INFO:teuthology.orchestra.run.smithi130.stdout: mds verify scatter = True 2022-08-01T23:04:54.650 INFO:teuthology.orchestra.run.smithi130.stdout: osd op complaint time = 180 2022-08-01T23:04:54.650 INFO:teuthology.orchestra.run.smithi130.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.650 INFO:teuthology.orchestra.run.smithi130.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.651 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2022-08-01T23:04:54.651 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2022-08-01T23:04:54.651 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T23:04:54.651 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2022-08-01T23:04:54.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.652 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2022-08-01T23:04:54.652 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.652 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2022-08-01T23:04:54.652 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.653 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2022-08-01T23:04:54.653 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2022-08-01T23:04:54.653 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.653 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2022-08-01T23:04:54.654 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2022-08-01T23:04:54.654 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.654 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T23:04:54.654 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2022-08-01T23:04:54.654 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.655 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T23:04:54.655 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T23:04:54.655 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.655 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2022-08-01T23:04:54.656 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.656 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2022-08-01T23:04:54.656 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2022-08-01T23:04:54.656 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2022-08-01T23:04:54.656 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2022-08-01T23:04:54.657 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2022-08-01T23:04:54.657 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2022-08-01T23:04:54.657 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2022-08-01T23:04:54.657 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T23:04:54.658 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T23:04:54.658 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.658 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T23:04:54.658 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.659 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2022-08-01T23:04:54.659 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.659 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2022-08-01T23:04:54.659 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.659 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2022-08-01T23:04:54.660 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2022-08-01T23:04:54.660 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2022-08-01T23:04:54.660 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.32,172.21.15.55,[v2:172.21.15.55:3301,v1:172.21.15.55:6790]" 2022-08-01T23:04:54.660 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e 2022-08-01T23:04:54.661 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.661 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2022-08-01T23:04:54.661 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2022-08-01T23:04:54.661 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.661 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2022-08-01T23:04:54.662 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2022-08-01T23:04:54.662 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.662 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2022-08-01T23:04:54.662 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2022-08-01T23:04:54.663 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.663 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2022-08-01T23:04:54.663 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2022-08-01T23:04:54.663 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2022-08-01T23:04:54.663 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.664 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2022-08-01T23:04:54.664 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2022-08-01T23:04:54.664 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.664 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2022-08-01T23:04:54.664 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.665 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2022-08-01T23:04:54.665 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.665 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2022-08-01T23:04:54.665 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.666 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2022-08-01T23:04:54.666 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2022-08-01T23:04:54.666 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.666 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2022-08-01T23:04:54.666 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2022-08-01T23:04:54.667 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2022-08-01T23:04:54.667 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore compression mode = aggressive 2022-08-01T23:04:54.667 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2022-08-01T23:04:54.667 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2022-08-01T23:04:54.668 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2022-08-01T23:04:54.668 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:54.668 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2022-08-01T23:04:54.668 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2022-08-01T23:04:54.668 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T23:04:54.669 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2022-08-01T23:04:54.669 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2022-08-01T23:04:54.669 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2022-08-01T23:04:54.669 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2022-08-01T23:04:54.669 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2022-08-01T23:04:54.670 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.670 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2022-08-01T23:04:54.670 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:54.670 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2022-08-01T23:04:54.671 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2022-08-01T23:04:54.671 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2022-08-01T23:04:54.671 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.671 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.672 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2022-08-01T23:04:54.672 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.672 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2022-08-01T23:04:54.672 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:54.672 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2022-08-01T23:04:54.673 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2022-08-01T23:04:54.673 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2022-08-01T23:04:54.673 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2022-08-01T23:04:54.673 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2022-08-01T23:04:54.674 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2022-08-01T23:04:54.674 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2022-08-01T23:04:54.674 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2022-08-01T23:04:54.674 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2022-08-01T23:04:54.674 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.675 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T23:04:54.675 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T23:04:54.675 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T23:04:54.675 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.676 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T23:04:54.676 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T23:04:54.676 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T23:04:54.676 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2022-08-01T23:04:54.676 INFO:teuthology.orchestra.run.smithi055.stdout: 2022-08-01T23:04:54.677 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2022-08-01T23:04:54.677 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2022-08-01T23:04:54.677 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2022-08-01T23:04:54.677 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2022-08-01T23:04:54.678 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T23:04:54.678 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T23:04:54.678 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2022-08-01T23:04:54.678 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2022-08-01T23:04:54.678 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:54.679 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.679 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.679 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2022-08-01T23:04:54.679 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2022-08-01T23:04:54.680 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2022-08-01T23:04:54.680 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2022-08-01T23:04:54.680 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2022-08-01T23:04:54.680 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2022-08-01T23:04:54.680 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2022-08-01T23:04:54.681 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2022-08-01T23:04:54.681 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2022-08-01T23:04:54.681 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal frag = True 2022-08-01T23:04:54.681 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal fragment size max = 10000 2022-08-01T23:04:54.682 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal merge size = 5 2022-08-01T23:04:54.682 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal split bits = 3 2022-08-01T23:04:54.682 INFO:teuthology.orchestra.run.smithi055.stdout: mds bal split size = 100 2022-08-01T23:04:54.682 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2022-08-01T23:04:54.682 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2022-08-01T23:04:54.683 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2022-08-01T23:04:54.683 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2022-08-01T23:04:54.683 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2022-08-01T23:04:54.683 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 15m 2022-08-01T23:04:54.684 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 15m 2022-08-01T23:04:54.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:04:54.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:54.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:54.747 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:54.750 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:04:54.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 Processor -- start 2022-08-01T23:04:54.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 -- start start 2022-08-01T23:04:54.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2db00614b0 0x7f2db0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 0x7f2db0140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00aee80 con 0x7f2db00ad750 2022-08-01T23:04:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db00b70b0 con 0x7f2db00614b0 2022-08-01T23:04:54.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db5a5c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00af0b0 con 0x7f2db01405d0 2022-08-01T23:04:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db525b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 0x7f2db0140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2db00614b0 0x7f2db0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2db00ad750 0x7f2db00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57478/0 (socket says 172.21.15.32:57478) 2022-08-01T23:04:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db525b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 0x7f2db0140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:60770/0 (socket says 172.21.15.32:60770) 2022-08-01T23:04:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2db00614b0 0x7f2db0061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:58814/0 (socket says 172.21.15.32:58814) 2022-08-01T23:04:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daffff700 1 -- 172.21.15.32:0/3779896499 learned_addr learned my addr 172.21.15.32:0/3779896499 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daf7fe700 1 -- 172.21.15.32:0/3779896499 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266242489 0 0) 0x7f2db00aee80 con 0x7f2db00ad750 2022-08-01T23:04:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daf7fe700 1 -- 172.21.15.32:0/3779896499 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d90003260 con 0x7f2db00ad750 2022-08-01T23:04:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 -- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 msgr2=0x7f2db0140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 --2- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 0x7f2db0140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 -- 172.21.15.32:0/3779896499 >> v1:172.21.15.55:6789/0 conn(0x7f2db00ad750 legacy=0x7f2db00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 -- 172.21.15.32:0/3779896499 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00af350 con 0x7f2db00614b0 2022-08-01T23:04:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db525b700 1 --2- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 0x7f2db0140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2db4a5a700 1 --2- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db00614b0 0x7f2db0061890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f2db00aed30 tx=0x7f2d98006df0).ready entity=mon.1 client_cookie=bdc0032c9b23c1d6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daf7fe700 1 -- 172.21.15.32:0/3779896499 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800d570 con 0x7f2db00614b0 2022-08-01T23:04:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daf7fe700 1 -- 172.21.15.32:0/3779896499 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98000e90 con 0x7f2db00614b0 2022-08-01T23:04:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.800+0000 7f2daf7fe700 1 -- 172.21.15.32:0/3779896499 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800d8c0 con 0x7f2db00614b0 2022-08-01T23:04:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db00614b0 msgr2=0x7f2db0061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db00614b0 0x7f2db0061890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f2db00aed30 tx=0x7f2d98006df0).stop 2022-08-01T23:04:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- 172.21.15.32:0/3779896499 shutdown_connections 2022-08-01T23:04:54.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db01405d0 0x7f2db0140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/3779896499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db00614b0 0x7f2db0061890 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- 172.21.15.32:0/3779896499 >> 172.21.15.32:0/3779896499 conn(0x7f2db0066f70 msgr2=0x7f2db0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- 172.21.15.32:0/3779896499 shutdown_connections 2022-08-01T23:04:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- 172.21.15.32:0/3779896499 wait complete. 2022-08-01T23:04:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 Processor -- start 2022-08-01T23:04:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- start start 2022-08-01T23:04:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db00614b0 0x7f2db00b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 0x7f2db00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db01405d0 0x7f2db0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b1aa0 con 0x7f2db00ad750 2022-08-01T23:04:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0140b80 con 0x7f2db01405d0 2022-08-01T23:04:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2db5a5c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00ad3b0 con 0x7f2db00614b0 2022-08-01T23:04:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.804+0000 7f2daffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 0x7f2db00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2db4a5a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db00614b0 0x7f2db00b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 0x7f2db00b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49036/0 (socket says 172.21.15.32:49036) 2022-08-01T23:04:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 -- 172.21.15.32:0/1554686090 learned_addr learned my addr 172.21.15.32:0/1554686090 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2db525b700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db01405d0 0x7f2db0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:54.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 -- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db00614b0 msgr2=0x7f2db00b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db00614b0 0x7f2db00b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 -- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db01405d0 msgr2=0x7f2db0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db01405d0 0x7f2db0155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 -- 172.21.15.32:0/1554686090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da800b180 con 0x7f2db00ad750 2022-08-01T23:04:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2daffff700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 0x7f2db00b15b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f2db00b1be0 tx=0x7f2da0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2da0005c70 con 0x7f2db00ad750 2022-08-01T23:04:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0005dd0 con 0x7f2db00ad750 2022-08-01T23:04:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9800b180 con 0x7f2db00ad750 2022-08-01T23:04:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2da00072c0 con 0x7f2db00ad750 2022-08-01T23:04:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db01531d0 con 0x7f2db00ad750 2022-08-01T23:04:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db0153040 con 0x7f2db00ad750 2022-08-01T23:04:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.808+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2da0010070 con 0x7f2db00ad750 2022-08-01T23:04:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.812+0000 7f2dadffb700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d94053a70 0x7f2d94055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.812+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2da006b7a0 con 0x7f2db00ad750 2022-08-01T23:04:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.812+0000 7f2db4a5a700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d94053a70 0x7f2d94055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.812+0000 7f2db4a5a700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d94053a70 0x7f2d94055f30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f2d9801a000 tx=0x7f2d98014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.816+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2da0042730 con 0x7f2db00ad750 2022-08-01T23:04:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.944+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2db00af5f0 con 0x7f2db00ad750 2022-08-01T23:04:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.952+0000 7f2dadffb700 1 -- 172.21.15.32:0/1554686090 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2da0045d50 con 0x7f2db00ad750 2022-08-01T23:04:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d94053a70 msgr2=0x7f2d94055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d94053a70 0x7f2d94055f30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f2d9801a000 tx=0x7f2d98014000).stop 2022-08-01T23:04:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 msgr2=0x7f2db00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 0x7f2db00b15b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f2db00b1be0 tx=0x7f2da0007dc0).stop 2022-08-01T23:04:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 shutdown_connections 2022-08-01T23:04:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2d94053a70 0x7f2d94055f30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2db01405d0 0x7f2db0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2db00ad750 0x7f2db00b15b0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 --2- 172.21.15.32:0/1554686090 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2db00614b0 0x7f2db00b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 >> 172.21.15.32:0/1554686090 conn(0x7f2db0066f70 msgr2=0x7f2db0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 shutdown_connections 2022-08-01T23:04:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:54.956+0000 7f2db5a5c700 1 -- 172.21.15.32:0/1554686090 wait complete. 2022-08-01T23:04:54.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-08-01T23:04:55.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4acc97f700 1 Processor -- start 2022-08-01T23:04:55.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4acc97f700 1 -- start start 2022-08-01T23:04:55.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4acc97f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ac80af6e0 0x7f4ac8147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4acc97f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80b5a10 con 0x7f4ac80b07a0 2022-08-01T23:04:55.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4acc97f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80ac990 con 0x7f4ac80b5630 2022-08-01T23:04:55.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4acc97f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac8145470 con 0x7f4ac80af6e0 2022-08-01T23:04:55.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ac80b07a0 0x7f4ac80acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36584/0 (socket says 172.21.15.32:36584) 2022-08-01T23:04:55.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/2562371241 learned_addr learned my addr 172.21.15.32:0/2562371241 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:55.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac7fff700 1 --2- 172.21.15.32:0/2562371241 >> v2:172.21.15.55:3300/0 conn(0x7f4ac80af6e0 0x7f4ac8147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917530869 0 0) 0x7f4ac80b5a10 con 0x7f4ac80b07a0 2022-08-01T23:04:55.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab4003260 con 0x7f4ac80b07a0 2022-08-01T23:04:55.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3513062882 0 0) 0x7f4ab4003260 con 0x7f4ac80b07a0 2022-08-01T23:04:55.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 >> v2:172.21.15.55:3300/0 conn(0x7f4ac80af6e0 msgr2=0x7f4ac8147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:55.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 --2- 172.21.15.32:0/2562371241 >> v2:172.21.15.55:3300/0 conn(0x7f4ac80af6e0 0x7f4ac8147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 >> v1:172.21.15.55:6789/0 conn(0x7f4ac80b5630 legacy=0x7f4ac8067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80b1bc0 con 0x7f4ac80b07a0 2022-08-01T23:04:55.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.072+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f4ab8001660 con 0x7f4ac80b07a0 2022-08-01T23:04:55.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 >> v1:172.21.15.32:6789/0 conn(0x7f4ac80b07a0 legacy=0x7f4ac80acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ab4003750 0x7f4ab4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ab4004500 0x7f4ab4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004e70 0x7f4ab400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8145470 con 0x7f4ab4004500 2022-08-01T23:04:55.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab400bd20 con 0x7f4ab4004e70 2022-08-01T23:04:55.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab400bee0 con 0x7f4ab4003750 2022-08-01T23:04:55.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ab4004500 0x7f4ab4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac7fff700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004e70 0x7f4ab400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ab4003750 msgr2=0x7f4ab4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac77fe700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ab4003750 0x7f4ab4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ab4003750 0x7f4ab4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004e70 msgr2=0x7f4ab400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004e70 0x7f4ab400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/2562371241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4003260 con 0x7f4ab4004500 2022-08-01T23:04:55.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac7fff700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004e70 0x7f4ab400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:55.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac7fff700 1 -- 172.21.15.32:0/2562371241 reap_dead start 2022-08-01T23:04:55.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac77fe700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ab4003750 0x7f4ab4003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:04:55.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ab4004500 0x7f4ab4004980 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f4ab8001370 tx=0x7f4ab8001e10).ready entity=mon.0 client_cookie=2ce1bffefe81ae0c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab80071a0 con 0x7f4ab4004500 2022-08-01T23:04:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8007300 con 0x7f4ab4004500 2022-08-01T23:04:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac67fc700 1 -- 172.21.15.32:0/2562371241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab800c8f0 con 0x7f4ab4004500 2022-08-01T23:04:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- 172.21.15.32:0/2562371241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ab4004500 msgr2=0x7f4ab4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ab4004500 0x7f4ab4004980 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f4ab8001370 tx=0x7f4ab8001e10).stop 2022-08-01T23:04:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- 172.21.15.32:0/2562371241 shutdown_connections 2022-08-01T23:04:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 --2- 172.21.15.32:0/2562371241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ab4004500 0x7f4ab4004980 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- 172.21.15.32:0/2562371241 >> 172.21.15.32:0/2562371241 conn(0x7f4ac80626f0 msgr2=0x7f4ac8062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- 172.21.15.32:0/2562371241 shutdown_connections 2022-08-01T23:04:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- 172.21.15.32:0/2562371241 wait complete. 2022-08-01T23:04:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 Processor -- start 2022-08-01T23:04:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- start start 2022-08-01T23:04:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004500 0x7f4ac8066d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 0x7f4ac8157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ac8157800 0x7f4ac815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80ae180 con 0x7f4ac8156e90 2022-08-01T23:04:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80654b0 con 0x7f4ab4004500 2022-08-01T23:04:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4acc97f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b1d90 con 0x7f4ac8157800 2022-08-01T23:04:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 0x7f4ac8157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 0x7f4ac8157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49056/0 (socket says 172.21.15.32:49056) 2022-08-01T23:04:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.076+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/3399550180 learned_addr learned my addr 172.21.15.32:0/3399550180 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac7fff700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ac8157800 0x7f4ac815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac77fe700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004500 0x7f4ac8066d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ac8157800 msgr2=0x7f4ac815a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ac8157800 0x7f4ac815a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004500 msgr2=0x7f4ac8066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004500 0x7f4ac8066d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac6ffd700 1 -- 172.21.15.32:0/3399550180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80b2030 con 0x7f4ac8156e90 2022-08-01T23:04:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac6ffd700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 0x7f4ac8157310 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f4ab8001370 tx=0x7f4ab8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab8004520 con 0x7f4ac8156e90 2022-08-01T23:04:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8004680 con 0x7f4ac8156e90 2022-08-01T23:04:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac80b22d0 con 0x7f4ac8156e90 2022-08-01T23:04:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab80049e0 con 0x7f4ac8156e90 2022-08-01T23:04:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac8066760 con 0x7f4ac8156e90 2022-08-01T23:04:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac80b5150 con 0x7f4ac8156e90 2022-08-01T23:04:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ab8010070 con 0x7f4ac8156e90 2022-08-01T23:04:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac4ff9700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4aa8053b00 0x7f4aa8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4ab806cb80 con 0x7f4ac8156e90 2022-08-01T23:04:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.080+0000 7f4ac77fe700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4aa8053b00 0x7f4aa8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.084+0000 7f4ac77fe700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4aa8053b00 0x7f4aa8055fc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f4abc00c000 tx=0x7f4abc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.088+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ac80b5150 con 0x7f4ac8156e90 2022-08-01T23:04:55.259 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.252+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f4ac80b5150 con 0x7f4ac8156e90 2022-08-01T23:04:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.496+0000 7f4ac4ff9700 1 -- 172.21.15.32:0/3399550180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v417) v1 ==== 128+0+0 (secure 0 0 0) 0x7f4ac80b5150 con 0x7f4ac8156e90 2022-08-01T23:04:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4aa8053b00 msgr2=0x7f4aa8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4aa8053b00 0x7f4aa8055fc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f4abc00c000 tx=0x7f4abc009040).stop 2022-08-01T23:04:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 msgr2=0x7f4ac8157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 0x7f4ac8157310 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f4ab8001370 tx=0x7f4ab8015000).stop 2022-08-01T23:04:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 shutdown_connections 2022-08-01T23:04:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4aa8053b00 0x7f4aa8055fc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4ac8157800 0x7f4ac815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac8156e90 0x7f4ac8157310 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 --2- 172.21.15.32:0/3399550180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab4004500 0x7f4ac8066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 >> 172.21.15.32:0/3399550180 conn(0x7f4ac80626f0 msgr2=0x7f4ac814a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 shutdown_connections 2022-08-01T23:04:55.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.500+0000 7f4acc97f700 1 -- 172.21.15.32:0/3399550180 wait complete. 2022-08-01T23:04:55.521 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:04:55.522 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:04:55.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-08-01T23:04:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 Processor -- start 2022-08-01T23:04:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 -- start start 2022-08-01T23:04:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f2c0b1f40 0x7f1f2c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0b04e0 con 0x7f1f2c0a7250 2022-08-01T23:04:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0b0780 con 0x7f1f2c148530 2022-08-01T23:04:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c0a9250 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f2c0b1f40 0x7f1f2c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f2c0a7250 0x7f1f2c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36608/0 (socket says 172.21.15.32:36608) 2022-08-01T23:04:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2b7fe700 1 -- 172.21.15.32:0/2526002797 learned_addr learned my addr 172.21.15.32:0/2526002797 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f29ffb700 1 -- 172.21.15.32:0/2526002797 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105897708 0 0) 0x7f1f2c0b04e0 con 0x7f1f2c0a7250 2022-08-01T23:04:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f29ffb700 1 -- 172.21.15.32:0/2526002797 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f10003260 con 0x7f1f2c0a7250 2022-08-01T23:04:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2affd700 1 -- 172.21.15.32:0/2526002797 >> v1:172.21.15.55:6789/0 conn(0x7f1f2c148530 legacy=0x7f1f2c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2affd700 1 -- 172.21.15.32:0/2526002797 >> v1:172.21.15.32:6789/0 conn(0x7f1f2c0a7250 legacy=0x7f1f2c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2affd700 1 -- 172.21.15.32:0/2526002797 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0b0a20 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f2affd700 1 --2- 172.21.15.32:0/2526002797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c0b1160 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f1f2c1478c0 tx=0x7f1f14006df0).ready entity=mon.0 client_cookie=1940237a9eb28e1c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f29ffb700 1 -- 172.21.15.32:0/2526002797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f1400d610 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f29ffb700 1 -- 172.21.15.32:0/2526002797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14000e90 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f29ffb700 1 -- 172.21.15.32:0/2526002797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f1400d960 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 -- 172.21.15.32:0/2526002797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 msgr2=0x7f1f2c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.632+0000 7f1f317fe700 1 --2- 172.21.15.32:0/2526002797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c0b1160 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f1f2c1478c0 tx=0x7f1f14006df0).stop 2022-08-01T23:04:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- 172.21.15.32:0/2526002797 shutdown_connections 2022-08-01T23:04:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 --2- 172.21.15.32:0/2526002797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c0b1160 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- 172.21.15.32:0/2526002797 >> 172.21.15.32:0/2526002797 conn(0x7f1f2c05f840 msgr2=0x7f1f2c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- 172.21.15.32:0/2526002797 shutdown_connections 2022-08-01T23:04:55.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- 172.21.15.32:0/2526002797 wait complete. 2022-08-01T23:04:55.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 Processor -- start 2022-08-01T23:04:55.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- start start 2022-08-01T23:04:55.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f2c0a7250 0x7f1f2c0a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1f2c148530 0x7f1f2c143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c14bd50 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c14a400 con 0x7f1f2c0a7250 2022-08-01T23:04:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f317fe700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c14a630 con 0x7f1f2c148530 2022-08-01T23:04:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f2a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f2a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49070/0 (socket says 172.21.15.32:49070) 2022-08-01T23:04:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.636+0000 7f1f2a7fc700 1 -- 172.21.15.32:0/3729506702 learned_addr learned my addr 172.21.15.32:0/3729506702 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2b7fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1f2c148530 0x7f1f2c143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2affd700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f2c0a7250 0x7f1f2c0a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2a7fc700 1 -- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1f2c148530 msgr2=0x7f1f2c143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2a7fc700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1f2c148530 0x7f1f2c143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2a7fc700 1 -- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f2c0a7250 msgr2=0x7f1f2c0a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2a7fc700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f2c0a7250 0x7f1f2c0a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2a7fc700 1 -- 172.21.15.32:0/3729506702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1400b180 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2a7fc700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c143a60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f1f2c0b3300 tx=0x7f1f24001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f2400e4a0 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400e600 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c14a8d0 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f2400e960 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c144440 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c0b0cc0 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1f24019030 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f23fff700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f04053b00 0x7f1f04055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f1f2406b6a0 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.640+0000 7f1f2affd700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f04053b00 0x7f1f04055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.644+0000 7f1f2affd700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f04053b00 0x7f1f04055fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f1f14018000 tx=0x7f1f14014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:55.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.652+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1f240426a0 con 0x7f1f2c0b1f40 2022-08-01T23:04:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:55.820+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f1f2c0574e0 con 0x7f1f2c0b1f40 2022-08-01T23:04:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.500+0000 7f1f23fff700 1 -- 172.21.15.32:0/3729506702 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v418) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1f24045cc0 con 0x7f1f2c0b1f40 2022-08-01T23:04:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f04053b00 msgr2=0x7f1f04055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f04053b00 0x7f1f04055fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f1f14018000 tx=0x7f1f14014000).stop 2022-08-01T23:04:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 msgr2=0x7f1f2c143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c143a60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f1f2c0b3300 tx=0x7f1f24001400).stop 2022-08-01T23:04:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 shutdown_connections 2022-08-01T23:04:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f04053b00 0x7f1f04055fc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1f2c148530 0x7f1f2c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2c0b1f40 0x7f1f2c143a60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 --2- 172.21.15.32:0/3729506702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f2c0a7250 0x7f1f2c0a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 >> 172.21.15.32:0/3729506702 conn(0x7f1f2c05f840 msgr2=0x7f1f2c147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:56.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 shutdown_connections 2022-08-01T23:04:56.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.504+0000 7f1f317fe700 1 -- 172.21.15.32:0/3729506702 wait complete. 2022-08-01T23:04:56.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:04:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.640+0000 7f46a4949700 1 Processor -- start 2022-08-01T23:04:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.640+0000 7f46a4949700 1 -- start start 2022-08-01T23:04:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00655b0 0x7f46a014c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a0060670 con 0x7f46a0060290 2022-08-01T23:04:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00b2540 con 0x7f46a01445d0 2022-08-01T23:04:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a0065e10 con 0x7f46a00655b0 2022-08-01T23:04:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46a0060290 0x7f46a0064ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36620/0 (socket says 172.21.15.32:36620) 2022-08-01T23:04:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00655b0 0x7f46a014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469e59c700 1 -- 172.21.15.32:0/2231037921 learned_addr learned my addr 172.21.15.32:0/2231037921 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469ed9d700 1 -- 172.21.15.32:0/2231037921 >> v1:172.21.15.55:6789/0 conn(0x7f46a01445d0 legacy=0x7f46a0144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469ed9d700 1 -- 172.21.15.32:0/2231037921 >> v1:172.21.15.32:6789/0 conn(0x7f46a0060290 legacy=0x7f46a0064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469ed9d700 1 -- 172.21.15.32:0/2231037921 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00b27e0 con 0x7f46a00655b0 2022-08-01T23:04:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469d59a700 1 -- 172.21.15.32:0/2231037921 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447547296 0 0) 0x7f46a0060670 con 0x7f46a0060290 2022-08-01T23:04:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469ed9d700 1 --2- 172.21.15.32:0/2231037921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a014c980 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f46980032f0 tx=0x7f46980091d0).ready entity=mon.0 client_cookie=9c0de5f738b48ba9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469d59a700 1 -- 172.21.15.32:0/2231037921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4698009ea0 con 0x7f46a00655b0 2022-08-01T23:04:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469d59a700 1 -- 172.21.15.32:0/2231037921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800b8a0 con 0x7f46a00655b0 2022-08-01T23:04:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f469d59a700 1 -- 172.21.15.32:0/2231037921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f469800ba10 con 0x7f46a00655b0 2022-08-01T23:04:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- 172.21.15.32:0/2231037921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 msgr2=0x7f46a014c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 --2- 172.21.15.32:0/2231037921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a014c980 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f46980032f0 tx=0x7f46980091d0).stop 2022-08-01T23:04:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- 172.21.15.32:0/2231037921 shutdown_connections 2022-08-01T23:04:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 --2- 172.21.15.32:0/2231037921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a014c980 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- 172.21.15.32:0/2231037921 >> 172.21.15.32:0/2231037921 conn(0x7f46a005d0f0 msgr2=0x7f46a005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- 172.21.15.32:0/2231037921 shutdown_connections 2022-08-01T23:04:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.644+0000 7f46a4949700 1 -- 172.21.15.32:0/2231037921 wait complete. 2022-08-01T23:04:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 Processor -- start 2022-08-01T23:04:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 -- start start 2022-08-01T23:04:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a0060290 0x7f46a0060980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a013d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a0060e70 0x7f46a013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00b13e0 con 0x7f46a00655b0 2022-08-01T23:04:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00affd0 con 0x7f46a0060e70 2022-08-01T23:04:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00b15d0 con 0x7f46a0060290 2022-08-01T23:04:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a013d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a013d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49088/0 (socket says 172.21.15.32:49088) 2022-08-01T23:04:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 -- 172.21.15.32:0/4021205718 learned_addr learned my addr 172.21.15.32:0/4021205718 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469e59c700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a0060290 0x7f46a0060980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469ed9d700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a0060e70 0x7f46a013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 -- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a0060290 msgr2=0x7f46a0060980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a0060290 0x7f46a0060980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 -- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a0060e70 msgr2=0x7f46a013d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a0060e70 0x7f46a013d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 -- 172.21.15.32:0/4021205718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009470 con 0x7f46a00655b0 2022-08-01T23:04:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f469dd9b700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a013d390 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f46a005f2f0 tx=0x7f4694007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4694012400 con 0x7f46a00655b0 2022-08-01T23:04:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4694012560 con 0x7f46a00655b0 2022-08-01T23:04:56.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46940128c0 con 0x7f46a00655b0 2022-08-01T23:04:56.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a00aa730 con 0x7f46a00655b0 2022-08-01T23:04:56.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.648+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a00b0bb0 con 0x7f46a00655b0 2022-08-01T23:04:56.664 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.652+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0064a00 con 0x7f46a00655b0 2022-08-01T23:04:56.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.652+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4694019030 con 0x7f46a00655b0 2022-08-01T23:04:56.665 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.652+0000 7f468f7fe700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f467c053b00 0x7f467c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.652+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f469406b0f0 con 0x7f46a00655b0 2022-08-01T23:04:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.652+0000 7f469e59c700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f467c053b00 0x7f467c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.652+0000 7f469e59c700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f467c053b00 0x7f467c055fc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f469000d000 tx=0x7f4690007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.660+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4694042080 con 0x7f46a00655b0 2022-08-01T23:04:56.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.788+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f46a00574e0 con 0x7f46a00655b0 2022-08-01T23:04:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.796+0000 7f468f7fe700 1 -- 172.21.15.32:0/4021205718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f46940456a0 con 0x7f46a00655b0 2022-08-01T23:04:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f467c053b00 msgr2=0x7f467c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f467c053b00 0x7f467c055fc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f469000d000 tx=0x7f4690007a00).stop 2022-08-01T23:04:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 msgr2=0x7f46a013d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a013d390 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f46a005f2f0 tx=0x7f4694007c50).stop 2022-08-01T23:04:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 shutdown_connections 2022-08-01T23:04:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f467c053b00 0x7f467c055fc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a0060e70 0x7f46a013d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00655b0 0x7f46a013d390 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 --2- 172.21.15.32:0/4021205718 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f46a0060290 0x7f46a0060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:56.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 >> 172.21.15.32:0/4021205718 conn(0x7f46a005d0f0 msgr2=0x7f46a0140520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:56.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 shutdown_connections 2022-08-01T23:04:56.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:56.800+0000 7f46a4949700 1 -- 172.21.15.32:0/4021205718 wait complete. 2022-08-01T23:04:56.819 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:04:56.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:56.839 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:56.861 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:56.882 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:04:56.903 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:04:56.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:56.904 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:56.904 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:04:56.930 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:04:56.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:56.932 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:04:56.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:04:56.933 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:04:56.934 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:04:56.951 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:04:56.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:04:56.953 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:04:56.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:04:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.060+0000 7f4e21578700 1 Processor -- start 2022-08-01T23:04:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.060+0000 7f4e21578700 1 -- start start 2022-08-01T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e1c0b1f40 0x7f4e1c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e1c148530 0x7f4e1c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1c0b04e0 con 0x7f4e1c0a7250 2022-08-01T23:04:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c0a9250 con 0x7f4e1c0b1f40 2022-08-01T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c0b0780 con 0x7f4e1c148530 2022-08-01T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e1c0b1f40 0x7f4e1c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e1c0b1f40 0x7f4e1c0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49094/0 (socket says 172.21.15.32:49094) 2022-08-01T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 -- 172.21.15.32:0/419787806 learned_addr learned my addr 172.21.15.32:0/419787806 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1a7fc700 1 --2- 172.21.15.32:0/419787806 >> v2:172.21.15.55:3300/0 conn(0x7f4e1c148530 0x7f4e1c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 -- 172.21.15.32:0/419787806 >> v2:172.21.15.55:3300/0 conn(0x7f4e1c148530 msgr2=0x7f4e1c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 --2- 172.21.15.32:0/419787806 >> v2:172.21.15.55:3300/0 conn(0x7f4e1c148530 0x7f4e1c0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 -- 172.21.15.32:0/419787806 >> v1:172.21.15.55:6789/0 conn(0x7f4e1c0a7250 legacy=0x7f4e1c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 -- 172.21.15.32:0/419787806 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0b0a20 con 0x7f4e1c0b1f40 2022-08-01T23:04:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e1affd700 1 --2- 172.21.15.32:0/419787806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0b1160 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f4e1c1478c0 tx=0x7f4e10006eb0).ready entity=mon.0 client_cookie=2f8de24659ce2ff5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e19ffb700 1 -- 172.21.15.32:0/419787806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1000d610 con 0x7f4e1c0b1f40 2022-08-01T23:04:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e19ffb700 1 -- 172.21.15.32:0/419787806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10000e90 con 0x7f4e1c0b1f40 2022-08-01T23:04:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e19ffb700 1 -- 172.21.15.32:0/419787806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1000d9a0 con 0x7f4e1c0b1f40 2022-08-01T23:04:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- 172.21.15.32:0/419787806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0b1f40 msgr2=0x7f4e1c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 --2- 172.21.15.32:0/419787806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0b1160 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f4e1c1478c0 tx=0x7f4e10006eb0).stop 2022-08-01T23:04:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- 172.21.15.32:0/419787806 shutdown_connections 2022-08-01T23:04:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 --2- 172.21.15.32:0/419787806 >> v2:172.21.15.55:3300/0 conn(0x7f4e1c148530 0x7f4e1c0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 --2- 172.21.15.32:0/419787806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0b1160 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- 172.21.15.32:0/419787806 >> 172.21.15.32:0/419787806 conn(0x7f4e1c05f840 msgr2=0x7f4e1c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- 172.21.15.32:0/419787806 shutdown_connections 2022-08-01T23:04:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.064+0000 7f4e21578700 1 -- 172.21.15.32:0/419787806 wait complete. 2022-08-01T23:04:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 Processor -- start 2022-08-01T23:04:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 -- start start 2022-08-01T23:04:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 0x7f4e1c14a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e1c148530 0x7f4e1c0abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c14bc90 con 0x7f4e1c0a7250 2022-08-01T23:04:57.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1490f0 con 0x7f4e1c0b1f40 2022-08-01T23:04:57.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e21578700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c14b2b0 con 0x7f4e1c148530 2022-08-01T23:04:57.082 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 0x7f4e1c14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 0x7f4e1c14a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49098/0 (socket says 172.21.15.32:49098) 2022-08-01T23:04:57.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 -- 172.21.15.32:0/1947833173 learned_addr learned my addr 172.21.15.32:0/1947833173 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:57.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1a7fc700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.083 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1b7fe700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e1c148530 0x7f4e1c0abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 -- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e1c148530 msgr2=0x7f4e1c0abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e1c148530 0x7f4e1c0abd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 -- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e1c0b1f40 msgr2=0x7f4e1c0ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 -- 172.21.15.32:0/1947833173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1000b1e0 con 0x7f4e1c0a7250 2022-08-01T23:04:57.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e1affd700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 0x7f4e1c14a770 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f4e1c1478c0 tx=0x7f4e10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.068+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e10011860 con 0x7f4e1c0a7250 2022-08-01T23:04:57.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e100119c0 con 0x7f4e1c0a7250 2022-08-01T23:04:57.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c14b550 con 0x7f4e1c0a7250 2022-08-01T23:04:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e10011d20 con 0x7f4e1c0a7250 2022-08-01T23:04:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c0ac250 con 0x7f4e1c0a7250 2022-08-01T23:04:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c0b0cc0 con 0x7f4e1c0a7250 2022-08-01T23:04:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e10025030 con 0x7f4e1c0a7250 2022-08-01T23:04:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e0bfff700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4df8053a70 0x7f4df8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4e1006ce80 con 0x7f4e1c0a7250 2022-08-01T23:04:57.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.072+0000 7f4e1a7fc700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4df8053a70 0x7f4df8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.076+0000 7f4e1a7fc700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4df8053a70 0x7f4df8055f30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f4e0c00b000 tx=0x7f4e0c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.080+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e10043f20 con 0x7f4e1c0a7250 2022-08-01T23:04:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.204+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4e1c0574e0 con 0x7f4e1c0a7250 2022-08-01T23:04:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.204+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1947833173 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+29227 (secure 0 0 0) 0x7f4e10023090 con 0x7f4e1c0a7250 2022-08-01T23:04:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:57.213 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 140, 2022-08-01T23:04:57.213 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:04:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:04:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:04:54.481649+0000", 2022-08-01T23:04:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:57.214 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:04:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:04:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:04:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:04:57.215 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:04:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:04:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:04:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:04:57.216 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:04:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:04:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:04:57.217 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:04:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:04:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 26, 2022-08-01T23:04:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:04:57.218 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:04:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:04:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:04:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:04:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:04:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:04:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:04:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:57.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:57.221 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:04:57.222 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:04:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:04:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:04:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:04:57.223 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:57.224 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:04:57.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:57.226 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:57.227 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:57.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:57.229 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:57.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:57.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:57.232 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:57.233 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:57.234 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:57.234 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:57.234 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:57.234 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:57.235 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:04:57.235 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:04:57.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.235 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:57.236 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:04:57.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.237 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:04:57.237 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:04:57.237 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:04:57.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:04:57.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:04:57.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:04:57.238 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:04:57.238 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:04:57.239 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:04:57.239 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:04:57.239 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:04:57.239 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:04:57.240 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:04:57.240 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:04:57.240 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:04:57.240 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:04:57.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:04:57.241 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:04:57.241 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:04:57.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:04:57.241 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:04:57.242 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:04:57.242 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:04:57.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.242 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:04:57.243 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:04:57.243 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:04:57.243 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:04:57.244 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:04:57.244 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:04:57.244 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:04:57.244 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:04:57.244 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:04:57.245 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:04:57.245 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:04:57.245 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:04:57.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:04:57.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:04:57.246 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:04:57.246 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:04:57.246 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:04:57.247 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:04:57.247 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:04:57.247 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:04:57.247 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:04:57.248 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:04:57.248 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:04:57.248 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:04:57.248 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:04:57.249 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:04:57.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:04:57.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.249 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:04:57.249 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:04:57.250 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:04:57.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:04:57.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:04:57.250 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:04:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:04:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:04:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:04:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:04:57.251 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:04:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:04:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:04:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:04:57.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:04:57.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:04:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.254 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.255 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2022-08-01T23:04:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.256 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.257 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:04:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:04:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.259 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:04:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:04:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.263 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:04:57.264 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.265 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:04:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:04:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:04:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:04:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:04:57.270 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:04:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:04:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.271 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.272 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:04:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.273 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.274 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.275 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:04:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.277 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:04:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:04:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:04:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.281 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:04:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:04:57.284 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:04:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:04:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:04:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:04:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:04:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:04:57.289 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:04:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:04:57.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.292 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:57.293 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:04:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:04:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:04:57.298 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:04:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.300 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:04:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:04:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:04:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:04:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:04:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:04:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:04:57.307 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:04:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:04:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.308 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.309 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:04:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.310 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.311 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2022-08-01T23:04:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:04:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:04:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.315 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:04:57.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:04:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:04:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:04:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:04:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:04:57.324 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:04:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.325 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:04:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:04:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:04:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:04:57.326 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:04:57.328 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.330 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:04:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:04:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:04:57.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:04:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:04:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:04:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:04:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:04:57.343 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:04:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.345 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:04:57.345 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:04:57.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:04:57.345 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:04:57.345 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:04:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 128, 2022-08-01T23:04:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:04:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:04:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:04:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.356 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:04:57.356 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.357 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.358 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:04:57.358 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:04:57.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.360 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.361 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.361 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:04:57.361 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.362 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:04:57.363 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:04:57.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.363 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:04:57.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:04:57.364 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:04:57.364 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:04:57.364 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.365 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.365 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.366 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:57.366 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:04:57.366 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.366 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.367 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.367 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.367 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.367 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:57.368 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.368 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.368 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:04:57.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:04:57.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.371 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.372 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:04:57.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:04:57.374 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.375 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:04:57.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:04:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.380 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:04:57.380 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:04:57.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:04:57.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.382 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:04:57.382 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:04:57.382 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:57.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:04:57.385 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:04:57.385 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:04:57.385 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:04:57.385 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:04:57.386 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 135, 2022-08-01T23:04:57.387 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:04:57.387 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:04:57.387 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:04:57.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:04:57.388 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:04:57.389 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.390 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:04:57.392 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.393 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:04:57.393 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.394 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:04:57.394 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.395 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:04:57.395 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.396 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:04:57.396 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.397 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:04:57.397 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:04:57.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:04:57.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:04:57.398 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:04:57.399 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:04:57.400 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:04:57.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.401 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:04:57.401 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:04:57.401 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:04:57.401 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:04:57.402 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:04:57.402 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:04:57.402 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:04:57.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:57.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.403 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:57.403 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:04:57.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.404 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:04:57.404 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.404 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.404 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.405 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.405 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.405 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:04:57.405 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.406 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:04:57.406 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.407 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.407 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.407 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.407 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.408 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:04:57.408 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.408 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:04:57.409 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.409 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.409 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.409 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.410 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.410 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:04:57.410 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.411 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:04:57.411 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.411 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.412 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.412 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.412 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.412 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:04:57.413 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.413 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:04:57.413 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.414 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.414 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.414 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.414 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.415 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:04:57.415 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:04:57.416 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.416 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.416 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.417 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.417 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:04:57.417 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:04:57.418 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.418 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.419 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.419 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.419 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.419 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:04:57.420 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:57.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:04:57.421 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:04:57.421 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:04:57.421 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:04:57.421 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:04:57.421 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:04:57.422 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:04:57.422 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:04:57.422 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.422 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:04:57.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:04:57.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:04:57.423 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:04:57.423 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:04:57.424 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:04:57.424 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/766376033": "2022-08-02T23:04:50.644948+0000", 2022-08-01T23:04:57.424 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6833/766376033": "2022-08-02T23:04:50.644948+0000" 2022-08-01T23:04:57.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.425 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:04:57.425 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:04:57.425 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:04:57.425 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:57.426 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:04:57.426 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:04:57.426 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:57.426 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:04:57.426 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:57.427 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:57.427 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:04:57.427 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:04:57.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.428 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:04:57.428 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:04:57.428 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:04:57.428 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:04:57.429 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:04:57.429 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:04:57.429 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:04:57.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:57.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:57.430 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:04:57.430 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:04:57.430 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:04:57.430 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:04:57.431 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:04:57.431 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:57.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4df8053a70 msgr2=0x7f4df8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4df8053a70 0x7f4df8055f30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f4e0c00b000 tx=0x7f4e0c006df0).stop 2022-08-01T23:04:57.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 msgr2=0x7f4e1c14a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 0x7f4e1c14a770 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f4e1c1478c0 tx=0x7f4e10015000).stop 2022-08-01T23:04:57.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 shutdown_connections 2022-08-01T23:04:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4df8053a70 0x7f4df8055f30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e1c148530 0x7f4e1c0abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e1c0b1f40 0x7f4e1c0ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 --2- 172.21.15.32:0/1947833173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e1c0a7250 0x7f4e1c14a770 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 >> 172.21.15.32:0/1947833173 conn(0x7f4e1c05f840 msgr2=0x7f4e1c144b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:57.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 shutdown_connections 2022-08-01T23:04:57.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.208+0000 7f4e21578700 1 -- 172.21.15.32:0/1947833173 wait complete. 2022-08-01T23:04:57.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 Processor -- start 2022-08-01T23:04:57.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 -- start start 2022-08-01T23:04:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f493c0a8c50 0x7f493c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c0a7250 con 0x7f493c0b01a0 2022-08-01T23:04:57.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c0a7480 con 0x7f493c0a8c50 2022-08-01T23:04:57.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0a7720 con 0x7f493c0b2710 2022-08-01T23:04:57.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f493c0a8c50 0x7f493c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f493c0a8c50 0x7f493c14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49110/0 (socket says 172.21.15.32:49110) 2022-08-01T23:04:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 -- 172.21.15.32:0/715290911 learned_addr learned my addr 172.21.15.32:0/715290911 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493affd700 1 --2- 172.21.15.32:0/715290911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 -- 172.21.15.32:0/715290911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 msgr2=0x7f493c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 --2- 172.21.15.32:0/715290911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 -- 172.21.15.32:0/715290911 >> v1:172.21.15.55:6789/0 conn(0x7f493c0b01a0 legacy=0x7f493c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.544+0000 7f493b7fe700 1 -- 172.21.15.32:0/715290911 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c0a79c0 con 0x7f493c0a8c50 2022-08-01T23:04:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493b7fe700 1 --2- 172.21.15.32:0/715290911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0a8c50 0x7f493c14b4c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f49340032f0 tx=0x7f4934009290).ready entity=mon.0 client_cookie=296ba3eb8dff925c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f4939ffb700 1 -- 172.21.15.32:0/715290911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493400b8a0 con 0x7f493c0a8c50 2022-08-01T23:04:57.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f4939ffb700 1 -- 172.21.15.32:0/715290911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400ba00 con 0x7f493c0a8c50 2022-08-01T23:04:57.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f4939ffb700 1 -- 172.21.15.32:0/715290911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493400bb70 con 0x7f493c0a8c50 2022-08-01T23:04:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 -- 172.21.15.32:0/715290911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0a8c50 msgr2=0x7f493c14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 --2- 172.21.15.32:0/715290911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0a8c50 0x7f493c14b4c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f49340032f0 tx=0x7f4934009290).stop 2022-08-01T23:04:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 -- 172.21.15.32:0/715290911 shutdown_connections 2022-08-01T23:04:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 --2- 172.21.15.32:0/715290911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0a8c50 0x7f493c14b4c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 --2- 172.21.15.32:0/715290911 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 -- 172.21.15.32:0/715290911 >> 172.21.15.32:0/715290911 conn(0x7f493c060310 msgr2=0x7f493c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 -- 172.21.15.32:0/715290911 shutdown_connections 2022-08-01T23:04:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.548+0000 7f493bfff700 1 -- 172.21.15.32:0/715290911 wait complete. 2022-08-01T23:04:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 Processor -- start 2022-08-01T23:04:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 -- start start 2022-08-01T23:04:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f493c0a8c50 0x7f493c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 0x7f493c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c0655d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c063980 con 0x7f493c0b01a0 2022-08-01T23:04:57.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0634d0 con 0x7f493c0a8c50 2022-08-01T23:04:57.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c063be0 con 0x7f493c0b2710 2022-08-01T23:04:57.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 0x7f493c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 0x7f493c0b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49114/0 (socket says 172.21.15.32:49114) 2022-08-01T23:04:57.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 -- 172.21.15.32:0/3250707242 learned_addr learned my addr 172.21.15.32:0/3250707242 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:57.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493affd700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f493c0a8c50 0x7f493c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493b7fe700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c0655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 -- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 msgr2=0x7f493c0655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c0655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 -- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f493c0a8c50 msgr2=0x7f493c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f493c0a8c50 0x7f493c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 -- 172.21.15.32:0/3250707242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4934009530 con 0x7f493c0b01a0 2022-08-01T23:04:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493affd700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f493c0a8c50 0x7f493c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493a7fc700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 0x7f493c0b4d80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f493c061c10 tx=0x7f492c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c00ce30 con 0x7f493c0b01a0 2022-08-01T23:04:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c007240 con 0x7f493c0b01a0 2022-08-01T23:04:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c0b5270 con 0x7f493c0b01a0 2022-08-01T23:04:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.552+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492c006700 con 0x7f493c0b01a0 2022-08-01T23:04:57.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c1565b0 con 0x7f493c0b01a0 2022-08-01T23:04:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c0574e0 con 0x7f493c0b01a0 2022-08-01T23:04:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f492c010030 con 0x7f493c0b01a0 2022-08-01T23:04:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f492bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4918053b00 0x7f4918055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f492c06bb60 con 0x7f493c0b01a0 2022-08-01T23:04:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f493affd700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4918053b00 0x7f4918055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.556+0000 7f493affd700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4918053b00 0x7f4918055fc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f493000b000 tx=0x7f4930008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.564+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f492c042af0 con 0x7f493c0b01a0 2022-08-01T23:04:57.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f493c069a70 con 0x7f493c0b01a0 2022-08-01T23:04:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f492bfff700 1 -- 172.21.15.32:0/3250707242 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f492c046110 con 0x7f493c0b01a0 2022-08-01T23:04:57.735 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:57.735 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1087791640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1087791640},{"type":"v1","addr":"172.21.15.32:6835","nonce":1087791640}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2022-08-01T23:04:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4918053b00 msgr2=0x7f4918055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4918053b00 0x7f4918055fc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f493000b000 tx=0x7f4930008040).stop 2022-08-01T23:04:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 msgr2=0x7f493c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 0x7f493c0b4d80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f493c061c10 tx=0x7f492c007c10).stop 2022-08-01T23:04:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 shutdown_connections 2022-08-01T23:04:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4918053b00 0x7f4918055fc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f493c0b2710 0x7f493c0655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f493c0b01a0 0x7f493c0b4d80 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 --2- 172.21.15.32:0/3250707242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f493c0a8c50 0x7f493c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 >> 172.21.15.32:0/3250707242 conn(0x7f493c060310 msgr2=0x7f493c14cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:57.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.728+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 shutdown_connections 2022-08-01T23:04:57.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.732+0000 7f493bfff700 1 -- 172.21.15.32:0/3250707242 wait complete. 2022-08-01T23:04:57.739 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2022-08-01T23:04:57.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:04:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.840+0000 7fb49b59e700 1 Processor -- start 2022-08-01T23:04:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.840+0000 7fb49b59e700 1 -- start start 2022-08-01T23:04:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb49c0af450 0x7fb49c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c0aff30 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c05c7a0 con 0x7fb49c0ad9e0 2022-08-01T23:04:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c140990 con 0x7fb49c0af450 2022-08-01T23:04:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb49c0af450 0x7fb49c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb499d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb49c0ad4b0 0x7fb49c1432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36664/0 (socket says 172.21.15.32:36664) 2022-08-01T23:04:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb499d9b700 1 -- 172.21.15.32:0/2247730550 learned_addr learned my addr 172.21.15.32:0/2247730550 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:57.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49a59c700 1 -- 172.21.15.32:0/2247730550 >> v1:172.21.15.55:6789/0 conn(0x7fb49c0ad9e0 legacy=0x7fb49c0a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49a59c700 1 -- 172.21.15.32:0/2247730550 >> v1:172.21.15.32:6789/0 conn(0x7fb49c0ad4b0 legacy=0x7fb49c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49959a700 1 -- 172.21.15.32:0/2247730550 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174023072 0 0) 0x7fb49c0aff30 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49a59c700 1 -- 172.21.15.32:0/2247730550 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c1439f0 con 0x7fb49c0af450 2022-08-01T23:04:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49a59c700 1 --2- 172.21.15.32:0/2247730550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0af450 0x7fb49c0af830 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fb49c0ad900 tx=0x7fb48c006df0).ready entity=mon.0 client_cookie=52975784d189b6c3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49959a700 1 -- 172.21.15.32:0/2247730550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb48c00d610 con 0x7fb49c0af450 2022-08-01T23:04:57.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49959a700 1 -- 172.21.15.32:0/2247730550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c000e90 con 0x7fb49c0af450 2022-08-01T23:04:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.844+0000 7fb49959a700 1 -- 172.21.15.32:0/2247730550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb48c00d960 con 0x7fb49c0af450 2022-08-01T23:04:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- 172.21.15.32:0/2247730550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0af450 msgr2=0x7fb49c0af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2247730550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0af450 0x7fb49c0af830 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fb49c0ad900 tx=0x7fb48c006df0).stop 2022-08-01T23:04:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- 172.21.15.32:0/2247730550 shutdown_connections 2022-08-01T23:04:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2247730550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0af450 0x7fb49c0af830 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- 172.21.15.32:0/2247730550 >> 172.21.15.32:0/2247730550 conn(0x7fb49c0612d0 msgr2=0x7fb49c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- 172.21.15.32:0/2247730550 shutdown_connections 2022-08-01T23:04:57.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- 172.21.15.32:0/2247730550 wait complete. 2022-08-01T23:04:57.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 Processor -- start 2022-08-01T23:04:57.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- start start 2022-08-01T23:04:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 0x7fb49c143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb49c0ad9e0 0x7fb49c0b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb49c0af450 0x7fb49c05f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0aaf40 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0ab410 con 0x7fb49c0af450 2022-08-01T23:04:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.848+0000 7fb49b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0b39f0 con 0x7fb49c0ad9e0 2022-08-01T23:04:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 0x7fb49c143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 0x7fb49c143190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49126/0 (socket says 172.21.15.32:49126) 2022-08-01T23:04:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 -- 172.21.15.32:0/2472037126 learned_addr learned my addr 172.21.15.32:0/2472037126 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb499d9b700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb49c0ad9e0 0x7fb49c0b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49ad9d700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb49c0af450 0x7fb49c05f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 -- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb49c0ad9e0 msgr2=0x7fb49c0b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb49c0ad9e0 0x7fb49c0b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 -- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb49c0af450 msgr2=0x7fb49c05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb49c0af450 0x7fb49c05f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 -- 172.21.15.32:0/2472037126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c00b180 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49a59c700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 0x7fb49c143190 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fb49c0ad900 tx=0x7fb48c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb48c00de70 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c011b80 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c0b3c90 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb48c01e460 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c0aa420 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c0574e0 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb48c025030 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb48b7fe700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb478053a10 0x7fb478055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.852+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb48c06cf80 con 0x7fb49c0ad4b0 2022-08-01T23:04:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.856+0000 7fb499d9b700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb478053a10 0x7fb478055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.856+0000 7fb499d9b700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb478053a10 0x7fb478055ed0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fb49000d000 tx=0x7fb4900011a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:57.860+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb48c044020 con 0x7fb49c0ad4b0 2022-08-01T23:04:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.020+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb49c061b20 con 0x7fb49c0ad4b0 2022-08-01T23:04:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.020+0000 7fb48b7fe700 1 -- 172.21.15.32:0/2472037126 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fb48c023070 con 0x7fb49c0ad4b0 2022-08-01T23:04:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:58.029 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4051576173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4051576173},{"type":"v1","addr":"172.21.15.55:6835","nonce":4051576173}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6834/798182880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":798182880},{"type":"v1","addr":"172.21.15.32:6834","nonce":798182880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/1087791640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1087791640},{"type":"v1","addr":"172.21.15.32:6835","nonce":1087791640}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2022-08-01T23:04:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb478053a10 msgr2=0x7fb478055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb478053a10 0x7fb478055ed0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fb49000d000 tx=0x7fb4900011a0).stop 2022-08-01T23:04:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 msgr2=0x7fb49c143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 0x7fb49c143190 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fb49c0ad900 tx=0x7fb48c015000).stop 2022-08-01T23:04:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 shutdown_connections 2022-08-01T23:04:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb478053a10 0x7fb478055ed0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb49c0af450 0x7fb49c05f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb49c0ad9e0 0x7fb49c0b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 --2- 172.21.15.32:0/2472037126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb49c0ad4b0 0x7fb49c143190 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 >> 172.21.15.32:0/2472037126 conn(0x7fb49c0612d0 msgr2=0x7fb49c13f430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 shutdown_connections 2022-08-01T23:04:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.024+0000 7fb49b59e700 1 -- 172.21.15.32:0/2472037126 wait complete. 2022-08-01T23:04:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2022-08-01T23:04:58.041 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:04:58.041 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:04:58.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:04:58.042 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:04:58.043 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:04:58.043 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:04:58.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:04:58.044 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:04:58.044 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:04:58.044 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:04:58.045 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:04:58.045 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:04:58.053 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:04:58.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:04:58.056 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:04:58.057 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:58.058 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:04:58.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:04:58.060 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:04:58.060 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:04:58.062 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:04:58.063 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:58.103 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:04:58.103 DEBUG:teuthology.parallel:result is None 2022-08-01T23:04:58.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:04:58.125 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:04:58.126 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:04:58.183 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:04:58.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.188+0000 7f3852d38700 1 Processor -- start 2022-08-01T23:04:58.195 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.188+0000 7f3852d38700 1 -- start start 2022-08-01T23:04:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.188+0000 7f3852d38700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f384c0619b0 0x7f384c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.188+0000 7f3852d38700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f384c0b6ee0 0x7f384c143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.188+0000 7f3852d38700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c061d90 con 0x7f384c0b5ed0 2022-08-01T23:04:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c142f60 con 0x7f384c0b6ee0 2022-08-01T23:04:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c0ba5a0 con 0x7f384c0619b0 2022-08-01T23:04:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f384c0b6ee0 0x7f384c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f384c0b6ee0 0x7f384c143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49162/0 (socket says 172.21.15.32:49162) 2022-08-01T23:04:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852537700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f384c0b5ed0 0x7f384c0b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36702/0 (socket says 172.21.15.32:36702) 2022-08-01T23:04:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 -- 172.21.15.32:0/863440701 learned_addr learned my addr 172.21.15.32:0/863440701 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851d36700 1 --2- 172.21.15.32:0/863440701 >> v2:172.21.15.55:3300/0 conn(0x7f384c0619b0 0x7f384c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 -- 172.21.15.32:0/863440701 >> v2:172.21.15.55:3300/0 conn(0x7f384c0619b0 msgr2=0x7f384c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 --2- 172.21.15.32:0/863440701 >> v2:172.21.15.55:3300/0 conn(0x7f384c0619b0 0x7f384c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 -- 172.21.15.32:0/863440701 >> v1:172.21.15.32:6789/0 conn(0x7f384c0b5ed0 legacy=0x7f384c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3850d34700 1 -- 172.21.15.32:0/863440701 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164450972 0 0) 0x7f384c061d90 con 0x7f384c0b5ed0 2022-08-01T23:04:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 -- 172.21.15.32:0/863440701 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c143150 con 0x7f384c0b6ee0 2022-08-01T23:04:58.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851d36700 1 --2- 172.21.15.32:0/863440701 >> v2:172.21.15.55:3300/0 conn(0x7f384c0619b0 0x7f384c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:04:58.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3851535700 1 --2- 172.21.15.32:0/863440701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0b6ee0 0x7f384c143890 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f383c002f70 tx=0x7f383c006eb0).ready entity=mon.0 client_cookie=756e190dc8a728bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:58.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3850d34700 1 -- 172.21.15.32:0/863440701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383c00d5c0 con 0x7f384c0b6ee0 2022-08-01T23:04:58.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3850d34700 1 -- 172.21.15.32:0/863440701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c000e90 con 0x7f384c0b6ee0 2022-08-01T23:04:58.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3850d34700 1 -- 172.21.15.32:0/863440701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383c00d910 con 0x7f384c0b6ee0 2022-08-01T23:04:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- 172.21.15.32:0/863440701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0b6ee0 msgr2=0x7f384c143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 --2- 172.21.15.32:0/863440701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0b6ee0 0x7f384c143890 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f383c002f70 tx=0x7f383c006eb0).stop 2022-08-01T23:04:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- 172.21.15.32:0/863440701 shutdown_connections 2022-08-01T23:04:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 --2- 172.21.15.32:0/863440701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0b6ee0 0x7f384c143890 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 --2- 172.21.15.32:0/863440701 >> v2:172.21.15.55:3300/0 conn(0x7f384c0619b0 0x7f384c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- 172.21.15.32:0/863440701 >> 172.21.15.32:0/863440701 conn(0x7f384c06af80 msgr2=0x7f384c05fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- 172.21.15.32:0/863440701 shutdown_connections 2022-08-01T23:04:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.192+0000 7f3852d38700 1 -- 172.21.15.32:0/863440701 wait complete. 2022-08-01T23:04:58.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 Processor -- start 2022-08-01T23:04:58.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 -- start start 2022-08-01T23:04:58.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 0x7f384c0b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:58.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f384c0b5ed0 0x7f384c13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:58.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f384c0b6ee0 0x7f384c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:58.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c13bed0 con 0x7f384c0619b0 2022-08-01T23:04:58.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c13de30 con 0x7f384c0b5ed0 2022-08-01T23:04:58.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c13bc40 con 0x7f384c0b6ee0 2022-08-01T23:04:58.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 0x7f384c0b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:58.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 0x7f384c0b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49164/0 (socket says 172.21.15.32:49164) 2022-08-01T23:04:58.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 -- 172.21.15.32:0/1591995165 learned_addr learned my addr 172.21.15.32:0/1591995165 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:58.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852537700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f384c0b6ee0 0x7f384c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:58.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851535700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f384c0b5ed0 0x7f384c13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 -- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f384c0b6ee0 msgr2=0x7f384c0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f384c0b6ee0 0x7f384c0b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 -- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f384c0b5ed0 msgr2=0x7f384c13d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f384c0b5ed0 0x7f384c13d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 -- 172.21.15.32:0/1591995165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009470 con 0x7f384c0619b0 2022-08-01T23:04:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851535700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f384c0b5ed0 0x7f384c13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3851d36700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 0x7f384c0b97b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f384c0af150 tx=0x7f38480091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384800c970 con 0x7f384c0619b0 2022-08-01T23:04:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384800cad0 con 0x7f384c0619b0 2022-08-01T23:04:58.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383c00b1e0 con 0x7f384c0619b0 2022-08-01T23:04:58.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384800ce30 con 0x7f384c0619b0 2022-08-01T23:04:58.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.196+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c0b88b0 con 0x7f384c0619b0 2022-08-01T23:04:58.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.200+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f384801b400 con 0x7f384c0619b0 2022-08-01T23:04:58.214 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.200+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c0574e0 con 0x7f384c0619b0 2022-08-01T23:04:58.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.200+0000 7f383affd700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f382c053a10 0x7f382c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:58.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.200+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f384806c980 con 0x7f384c0619b0 2022-08-01T23:04:58.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.200+0000 7f3851535700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f382c053a10 0x7f382c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:58.215 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.200+0000 7f3851535700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f382c053a10 0x7f382c055ed0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f383c01a000 tx=0x7f383c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:58.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.212+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f38480438b0 con 0x7f384c0619b0 2022-08-01T23:04:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:58.344+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f384c13f940 con 0x7f384c0619b0 2022-08-01T23:04:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.156+0000 7f383affd700 1 -- 172.21.15.32:0/1591995165 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v142) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3848046ed0 con 0x7f384c0619b0 2022-08-01T23:04:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f382c053a10 msgr2=0x7f382c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f382c053a10 0x7f382c055ed0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f383c01a000 tx=0x7f383c015000).stop 2022-08-01T23:04:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 msgr2=0x7f384c0b97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 0x7f384c0b97b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f384c0af150 tx=0x7f38480091d0).stop 2022-08-01T23:04:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 shutdown_connections 2022-08-01T23:04:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f382c053a10 0x7f382c055ed0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f384c0b6ee0 0x7f384c0b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f384c0b5ed0 0x7f384c13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 --2- 172.21.15.32:0/1591995165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384c0619b0 0x7f384c0b97b0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 >> 172.21.15.32:0/1591995165 conn(0x7f384c06af80 msgr2=0x7f384c141410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 shutdown_connections 2022-08-01T23:04:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.160+0000 7f3852d38700 1 -- 172.21.15.32:0/1591995165 wait complete. 2022-08-01T23:04:59.170 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:04:59.182 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:04:59.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:04:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f84eaa700 1 Processor -- start 2022-08-01T23:04:59.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f84eaa700 1 -- start start 2022-08-01T23:04:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f84eaa700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f800b1f40 0x7f3f800b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f84eaa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800b04e0 con 0x7f3f800a7250 2022-08-01T23:04:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f84eaa700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800b0780 con 0x7f3f80148530 2022-08-01T23:04:59.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f84eaa700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800a9250 con 0x7f3f800b1f40 2022-08-01T23:04:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f800a7250 0x7f3f801448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36718/0 (socket says 172.21.15.32:36718) 2022-08-01T23:04:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7ed9d700 1 -- 172.21.15.32:0/1932893006 learned_addr learned my addr 172.21.15.32:0/1932893006 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7e59c700 1 --2- 172.21.15.32:0/1932893006 >> v2:172.21.15.55:3300/0 conn(0x7f3f800b1f40 0x7f3f800b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7dd9b700 1 -- 172.21.15.32:0/1932893006 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972086644 0 0) 0x7f3f800b04e0 con 0x7f3f800a7250 2022-08-01T23:04:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7dd9b700 1 -- 172.21.15.32:0/1932893006 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f60003260 con 0x7f3f800a7250 2022-08-01T23:04:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7e59c700 1 -- 172.21.15.32:0/1932893006 >> v1:172.21.15.55:6789/0 conn(0x7f3f80148530 legacy=0x7f3f800b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7e59c700 1 -- 172.21.15.32:0/1932893006 >> v1:172.21.15.32:6789/0 conn(0x7f3f800a7250 legacy=0x7f3f801448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7e59c700 1 -- 172.21.15.32:0/1932893006 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800b0a20 con 0x7f3f800b1f40 2022-08-01T23:04:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7e59c700 1 --2- 172.21.15.32:0/1932893006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f800b1160 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f3f801478c0 tx=0x7f3f70006df0).ready entity=mon.1 client_cookie=399cfdaa7d30512e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7dd9b700 1 -- 172.21.15.32:0/1932893006 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f7000d610 con 0x7f3f800b1f40 2022-08-01T23:04:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7dd9b700 1 -- 172.21.15.32:0/1932893006 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f70000e90 con 0x7f3f800b1f40 2022-08-01T23:04:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.328+0000 7f3f7dd9b700 1 -- 172.21.15.32:0/1932893006 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f7000d960 con 0x7f3f800b1f40 2022-08-01T23:04:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 -- 172.21.15.32:0/1932893006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 msgr2=0x7f3f800b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/1932893006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f800b1160 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f3f801478c0 tx=0x7f3f70006df0).stop 2022-08-01T23:04:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 -- 172.21.15.32:0/1932893006 shutdown_connections 2022-08-01T23:04:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/1932893006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f800b1160 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 -- 172.21.15.32:0/1932893006 >> 172.21.15.32:0/1932893006 conn(0x7f3f8005f840 msgr2=0x7f3f8005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 -- 172.21.15.32:0/1932893006 shutdown_connections 2022-08-01T23:04:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 -- 172.21.15.32:0/1932893006 wait complete. 2022-08-01T23:04:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 Processor -- start 2022-08-01T23:04:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.332+0000 7f3f84eaa700 1 -- start start 2022-08-01T23:04:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f800a7250 0x7f3f80063bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f80153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f80148530 0x7f3f80154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800abe70 con 0x7f3f80148530 2022-08-01T23:04:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800b35d0 con 0x7f3f800b1f40 2022-08-01T23:04:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800b3800 con 0x7f3f800a7250 2022-08-01T23:04:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f80148530 0x7f3f80154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f7e59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f800a7250 0x7f3f80063bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f80153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f80148530 0x7f3f80154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49182/0 (socket says 172.21.15.32:49182) 2022-08-01T23:04:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f7ed9d700 1 -- 172.21.15.32:0/2123177969 learned_addr learned my addr 172.21.15.32:0/2123177969 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 -- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f800a7250 msgr2=0x7f3f80063bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f800a7250 0x7f3f80063bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 -- 172.21.15.32:0/2123177969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f80148530 msgr2=0x7f3f80154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f80148530 0x7f3f80154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 -- 172.21.15.32:0/2123177969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7000b180 con 0x7f3f800b1f40 2022-08-01T23:04:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f75d9b700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f80153fa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f3f800637c0 tx=0x7f3f68007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f6800ce60 con 0x7f3f800b1f40 2022-08-01T23:04:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f68007e00 con 0x7f3f800b1f40 2022-08-01T23:04:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f800aa070 con 0x7f3f800b1f40 2022-08-01T23:04:59.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f68007220 con 0x7f3f800b1f40 2022-08-01T23:04:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.336+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f800aa9f0 con 0x7f3f800b1f40 2022-08-01T23:04:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.340+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f800b0cc0 con 0x7f3f800b1f40 2022-08-01T23:04:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.340+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3f68010030 con 0x7f3f800b1f40 2022-08-01T23:04:59.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.340+0000 7f3f77fff700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3f64053b00 0x7f3f64055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.340+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3f6806b670 con 0x7f3f800b1f40 2022-08-01T23:04:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.340+0000 7f3f7e59c700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3f64053b00 0x7f3f64055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.340+0000 7f3f7e59c700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3f64053b00 0x7f3f64055fc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f3f7001a000 tx=0x7f3f70014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.348+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f68042770 con 0x7f3f800b1f40 2022-08-01T23:04:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.520+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f800574e0 con 0x7f3f800b1f40 2022-08-01T23:04:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.520+0000 7f3f77fff700 1 -- 172.21.15.32:0/2123177969 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f3f68045d90 con 0x7f3f800b1f40 2022-08-01T23:04:59.528 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:59.528 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:04:59.529 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:04:59.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.529 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:04:59.530 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:04:59.530 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:59.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.531 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.532 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:04:59.532 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:04:59.532 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.532 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:59.533 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.533 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.533 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.534 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:04:59.535 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:04:59.535 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.535 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:59.536 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.536 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.537 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:04:59.537 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:04:59.537 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.538 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:04:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:04:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:04:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:04:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:04:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:04:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:04:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:04:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.546 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.547 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:04:59.547 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:04:59.548 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.548 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.549 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:04:59.549 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:04:59.549 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.550 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.551 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:04:59.551 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:04:59.552 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.552 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.552 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.552 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.553 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:04:59.554 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:04:59.554 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.554 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:59.554 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:04:59.555 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:04:59.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.555 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:04:59.556 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:04:59.556 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.556 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:04:59.556 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:04:59.557 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:04:59.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.558 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:04:59.558 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:04:59.558 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.559 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:04:59.559 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:04:59.559 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:04:59.559 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.560 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:04:59.561 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:04:59.561 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.561 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:04:59.561 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:04:59.561 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:04:59.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:04:59.562 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:04:59.563 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:04:59.563 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:04:59.563 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:04:59.563 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:04:59.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:04:59.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:04:59.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:04:59.564 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:04:59.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3f64053b00 msgr2=0x7f3f64055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3f64053b00 0x7f3f64055fc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f3f7001a000 tx=0x7f3f70014000).stop 2022-08-01T23:04:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 msgr2=0x7f3f80153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f80153fa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f3f800637c0 tx=0x7f3f68007b70).stop 2022-08-01T23:04:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 shutdown_connections 2022-08-01T23:04:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3f64053b00 0x7f3f64055fc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f80148530 0x7f3f80154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f800b1f40 0x7f3f80153fa0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 --2- 172.21.15.32:0/2123177969 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3f800a7250 0x7f3f80063bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 >> 172.21.15.32:0/2123177969 conn(0x7f3f8005f840 msgr2=0x7f3f80147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:59.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 shutdown_connections 2022-08-01T23:04:59.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.524+0000 7f3f84eaa700 1 -- 172.21.15.32:0/2123177969 wait complete. 2022-08-01T23:04:59.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:04:59.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.664+0000 7ffaa5a47700 1 Processor -- start 2022-08-01T23:04:59.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.664+0000 7ffaa5a47700 1 -- start start 2022-08-01T23:04:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffaa00b2710 0x7ffaa014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00a7250 con 0x7ffaa00b01a0 2022-08-01T23:04:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00a7480 con 0x7ffaa00a8c50 2022-08-01T23:04:59.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa00a7720 con 0x7ffaa00b2710 2022-08-01T23:04:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffaa00b01a0 0x7ffaa00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36728/0 (socket says 172.21.15.32:36728) 2022-08-01T23:04:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9ffff700 1 -- 172.21.15.32:0/2115989715 learned_addr learned my addr 172.21.15.32:0/2115989715 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa4a45700 1 --2- 172.21.15.32:0/2115989715 >> v2:172.21.15.32:3300/0 conn(0x7ffaa00b2710 0x7ffaa014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9f7fe700 1 -- 172.21.15.32:0/2115989715 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985394708 0 0) 0x7ffaa00a7250 con 0x7ffaa00b01a0 2022-08-01T23:04:59.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9f7fe700 1 -- 172.21.15.32:0/2115989715 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa8c003260 con 0x7ffaa00b01a0 2022-08-01T23:04:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa4a45700 1 -- 172.21.15.32:0/2115989715 >> v1:172.21.15.55:6789/0 conn(0x7ffaa00a8c50 legacy=0x7ffaa0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa4a45700 1 -- 172.21.15.32:0/2115989715 >> v1:172.21.15.32:6789/0 conn(0x7ffaa00b01a0 legacy=0x7ffaa00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa4a45700 1 -- 172.21.15.32:0/2115989715 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00a79c0 con 0x7ffaa00b2710 2022-08-01T23:04:59.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa4a45700 1 --2- 172.21.15.32:0/2115989715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa014afd0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7ffa94002f70 tx=0x7ffa94006df0).ready entity=mon.0 client_cookie=59f2557281c7ad75 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9f7fe700 1 -- 172.21.15.32:0/2115989715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa9400d610 con 0x7ffaa00b2710 2022-08-01T23:04:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9f7fe700 1 -- 172.21.15.32:0/2115989715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa94000e90 con 0x7ffaa00b2710 2022-08-01T23:04:59.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffa9f7fe700 1 -- 172.21.15.32:0/2115989715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa9400d960 con 0x7ffaa00b2710 2022-08-01T23:04:59.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2115989715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 msgr2=0x7ffaa014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2115989715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa014afd0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7ffa94002f70 tx=0x7ffa94006df0).stop 2022-08-01T23:04:59.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2115989715 shutdown_connections 2022-08-01T23:04:59.681 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2115989715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa014afd0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2115989715 >> 172.21.15.32:0/2115989715 conn(0x7ffaa0060310 msgr2=0x7ffaa00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:59.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2115989715 shutdown_connections 2022-08-01T23:04:59.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.668+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2115989715 wait complete. 2022-08-01T23:04:59.682 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 Processor -- start 2022-08-01T23:04:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 -- start start 2022-08-01T23:04:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffaa00a8c50 0x7ffaa00ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffaa00b01a0 0x7ffaa00b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa00b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00b2f60 con 0x7ffaa00b2710 2022-08-01T23:04:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00b3190 con 0x7ffaa00b01a0 2022-08-01T23:04:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00b3430 con 0x7ffaa00a8c50 2022-08-01T23:04:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa00b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa00b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49194/0 (socket says 172.21.15.32:49194) 2022-08-01T23:04:59.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 -- 172.21.15.32:0/2211632648 learned_addr learned my addr 172.21.15.32:0/2211632648 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:59.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa4a45700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffaa00a8c50 0x7ffaa00ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffa9ffff700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffaa00b01a0 0x7ffaa00b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 -- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffaa00a8c50 msgr2=0x7ffaa00ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffaa00a8c50 0x7ffaa00ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 -- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffaa00b01a0 msgr2=0x7ffaa00b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffaa00b01a0 0x7ffaa00b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 -- 172.21.15.32:0/2211632648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9400b180 con 0x7ffaa00b2710 2022-08-01T23:04:59.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffa9ffff700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffaa00b01a0 0x7ffaa00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:04:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5246700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa00b4690 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7ffaa0061c10 tx=0x7ffa98007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa98012400 con 0x7ffaa00b2710 2022-08-01T23:04:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa00af270 con 0x7ffaa00b2710 2022-08-01T23:04:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98012560 con 0x7ffaa00b2710 2022-08-01T23:04:59.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa980128c0 con 0x7ffaa00b2710 2022-08-01T23:04:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.672+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa00b3f00 con 0x7ffaa00b2710 2022-08-01T23:04:59.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.676+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa00b35f0 con 0x7ffaa00b2710 2022-08-01T23:04:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.676+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa98019030 con 0x7ffaa00b2710 2022-08-01T23:04:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.676+0000 7ffa9dffb700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa80053aa0 0x7ffa80055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.676+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ffa9806b090 con 0x7ffaa00b2710 2022-08-01T23:04:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.676+0000 7ffaa4a45700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa80053aa0 0x7ffa80055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.676+0000 7ffaa4a45700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa80053aa0 0x7ffa80055f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7ffa9401a000 tx=0x7ffa94014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.684+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffa980420f0 con 0x7ffaa00b2710 2022-08-01T23:04:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.808+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaa00574e0 con 0x7ffaa00b2710 2022-08-01T23:04:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.808+0000 7ffa9dffb700 1 -- 172.21.15.32:0/2211632648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+13424 (secure 0 0 0) 0x7ffa98045710 con 0x7ffaa00b2710 2022-08-01T23:04:59.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:04:59.817 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":142,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:04:59.141952+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:04:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa80053aa0 msgr2=0x7ffa80055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa80053aa0 0x7ffa80055f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7ffa9401a000 tx=0x7ffa94014000).stop 2022-08-01T23:04:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 msgr2=0x7ffaa00b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa00b4690 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7ffaa0061c10 tx=0x7ffa98007b90).stop 2022-08-01T23:04:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 shutdown_connections 2022-08-01T23:04:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa80053aa0 0x7ffa80055f60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaa00b2710 0x7ffaa00b4690 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffaa00b01a0 0x7ffaa00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 --2- 172.21.15.32:0/2211632648 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffaa00a8c50 0x7ffaa00ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 >> 172.21.15.32:0/2211632648 conn(0x7ffaa0060310 msgr2=0x7ffaa014aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 shutdown_connections 2022-08-01T23:04:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.812+0000 7ffaa5a47700 1 -- 172.21.15.32:0/2211632648 wait complete. 2022-08-01T23:04:59.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:04:59.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:04:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.932+0000 7f412c884700 1 Processor -- start 2022-08-01T23:04:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 -- start start 2022-08-01T23:04:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41280af550 0x7f41280b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f412805c6f0 0x7f41280a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280aa630 con 0x7f4128060330 2022-08-01T23:04:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280b15e0 con 0x7f41280af550 2022-08-01T23:04:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41280af930 con 0x7f412805c6f0 2022-08-01T23:04:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41280af550 0x7f41280b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f4126ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4128060330 0x7f41281445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36742/0 (socket says 172.21.15.32:36742) 2022-08-01T23:04:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f4126ffd700 1 -- 172.21.15.32:0/620451874 learned_addr learned my addr 172.21.15.32:0/620451874 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:04:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f4127fff700 1 --2- 172.21.15.32:0/620451874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f412805c6f0 0x7f41280a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41267fc700 1 -- 172.21.15.32:0/620451874 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221446032 0 0) 0x7f41280aa630 con 0x7f4128060330 2022-08-01T23:04:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41267fc700 1 -- 172.21.15.32:0/620451874 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4108003260 con 0x7f4128060330 2022-08-01T23:04:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41277fe700 1 -- 172.21.15.32:0/620451874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f412805c6f0 msgr2=0x7f41280a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:04:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41277fe700 1 --2- 172.21.15.32:0/620451874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f412805c6f0 0x7f41280a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41277fe700 1 -- 172.21.15.32:0/620451874 >> v1:172.21.15.32:6789/0 conn(0x7f4128060330 legacy=0x7f41281445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41277fe700 1 -- 172.21.15.32:0/620451874 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412805fd10 con 0x7f41280af550 2022-08-01T23:04:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41277fe700 1 --2- 172.21.15.32:0/620451874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41280af550 0x7f41280b0e40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f4118002f70 tx=0x7f4118006df0).ready entity=mon.0 client_cookie=41fce53a7c687cb2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41267fc700 1 -- 172.21.15.32:0/620451874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411800d4a0 con 0x7f41280af550 2022-08-01T23:04:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41267fc700 1 -- 172.21.15.32:0/620451874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118000e90 con 0x7f41280af550 2022-08-01T23:04:59.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f41267fc700 1 -- 172.21.15.32:0/620451874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411800d7f0 con 0x7f41280af550 2022-08-01T23:04:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.936+0000 7f412c884700 1 -- 172.21.15.32:0/620451874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41280af550 msgr2=0x7f41280b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 --2- 172.21.15.32:0/620451874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41280af550 0x7f41280b0e40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f4118002f70 tx=0x7f4118006df0).stop 2022-08-01T23:04:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- 172.21.15.32:0/620451874 shutdown_connections 2022-08-01T23:04:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 --2- 172.21.15.32:0/620451874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f412805c6f0 0x7f41280a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 --2- 172.21.15.32:0/620451874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41280af550 0x7f41280b0e40 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- 172.21.15.32:0/620451874 >> 172.21.15.32:0/620451874 conn(0x7f41280612d0 msgr2=0x7f4128062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:04:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- 172.21.15.32:0/620451874 shutdown_connections 2022-08-01T23:04:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- 172.21.15.32:0/620451874 wait complete. 2022-08-01T23:04:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 Processor -- start 2022-08-01T23:04:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- start start 2022-08-01T23:04:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 0x7f4128150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4128060330 0x7f4128144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41280af550 0x7f412813fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41281432c0 con 0x7f412805c6f0 2022-08-01T23:04:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128142960 con 0x7f41280af550 2022-08-01T23:04:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f412c884700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4128143ab0 con 0x7f4128060330 2022-08-01T23:04:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f41277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 0x7f4128150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f41277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 0x7f4128150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49210/0 (socket says 172.21.15.32:49210) 2022-08-01T23:04:59.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f41277fe700 1 -- 172.21.15.32:0/2872072495 learned_addr learned my addr 172.21.15.32:0/2872072495 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:04:59.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f4126ffd700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4128060330 0x7f4128144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.940+0000 7f4127fff700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41280af550 0x7f412813fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f41277fe700 1 -- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4128060330 msgr2=0x7f4128144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f41277fe700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4128060330 0x7f4128144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f41277fe700 1 -- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41280af550 msgr2=0x7f412813fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:04:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f41277fe700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41280af550 0x7f412813fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:04:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f41277fe700 1 -- 172.21.15.32:0/2872072495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411800b180 con 0x7f412805c6f0 2022-08-01T23:04:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f41277fe700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 0x7f4128150930 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f4118002f70 tx=0x7f4118014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4118012930 con 0x7f412805c6f0 2022-08-01T23:04:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118012a90 con 0x7f412805c6f0 2022-08-01T23:04:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4118012df0 con 0x7f412805c6f0 2022-08-01T23:04:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412805f840 con 0x7f412805c6f0 2022-08-01T23:04:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4128063b70 con 0x7f412805c6f0 2022-08-01T23:04:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412805fa70 con 0x7f412805c6f0 2022-08-01T23:04:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f411801c070 con 0x7f412805c6f0 2022-08-01T23:04:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.944+0000 7f4124ff9700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f410c053950 0x7f410c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:04:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.948+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f411806c860 con 0x7f412805c6f0 2022-08-01T23:04:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.948+0000 7f4126ffd700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f410c053950 0x7f410c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:04:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.948+0000 7f4126ffd700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f410c053950 0x7f410c055e10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f411000d000 tx=0x7f4110007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:04:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:04:59.952+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4118043970 con 0x7f412805c6f0 2022-08-01T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.076+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f41280574e0 con 0x7f412805c6f0 2022-08-01T23:05:00.084 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.076+0000 7f4124ff9700 1 -- 172.21.15.32:0/2872072495 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v142) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4118019070 con 0x7f412805c6f0 2022-08-01T23:05:00.085 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f410c053950 msgr2=0x7f410c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f410c053950 0x7f410c055e10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f411000d000 tx=0x7f4110007a20).stop 2022-08-01T23:05:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 msgr2=0x7f4128150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 0x7f4128150930 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f4118002f70 tx=0x7f4118014000).stop 2022-08-01T23:05:00.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 shutdown_connections 2022-08-01T23:05:00.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f410c053950 0x7f410c055e10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41280af550 0x7f412813fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4128060330 0x7f4128144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 --2- 172.21.15.32:0/2872072495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f412805c6f0 0x7f4128150930 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 >> 172.21.15.32:0/2872072495 conn(0x7f41280612d0 msgr2=0x7f41280b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:00.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 shutdown_connections 2022-08-01T23:05:00.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.080+0000 7f412c884700 1 -- 172.21.15.32:0/2872072495 wait complete. 2022-08-01T23:05:00.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:05:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.196+0000 7f7a3935b700 1 Processor -- start 2022-08-01T23:05:00.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.196+0000 7f7a3935b700 1 -- start start 2022-08-01T23:05:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a340b6510 0x7f7a34140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a340b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a341467f0 con 0x7f7a340b2aa0 2022-08-01T23:05:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a34062140 con 0x7f7a340b6510 2022-08-01T23:05:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340656b0 con 0x7f7a34061480 2022-08-01T23:05:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a337fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7a340b2aa0 0x7f7a340b0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36760/0 (socket says 172.21.15.32:36760) 2022-08-01T23:05:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a337fe700 1 -- 172.21.15.32:0/2322796839 learned_addr learned my addr 172.21.15.32:0/2322796839 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a38b5a700 1 --2- 172.21.15.32:0/2322796839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a340b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a33fff700 1 --2- 172.21.15.32:0/2322796839 >> v2:172.21.15.55:3300/0 conn(0x7f7a340b6510 0x7f7a34140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a32ffd700 1 -- 172.21.15.32:0/2322796839 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171421919 0 0) 0x7f7a341467f0 con 0x7f7a340b2aa0 2022-08-01T23:05:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a32ffd700 1 -- 172.21.15.32:0/2322796839 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a14003260 con 0x7f7a340b2aa0 2022-08-01T23:05:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a32ffd700 1 -- 172.21.15.32:0/2322796839 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1644356655 0 0) 0x7f7a14003260 con 0x7f7a340b2aa0 2022-08-01T23:05:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a38b5a700 1 -- 172.21.15.32:0/2322796839 >> v2:172.21.15.55:3300/0 conn(0x7f7a340b6510 msgr2=0x7f7a34140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a38b5a700 1 --2- 172.21.15.32:0/2322796839 >> v2:172.21.15.55:3300/0 conn(0x7f7a340b6510 0x7f7a34140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a38b5a700 1 -- 172.21.15.32:0/2322796839 >> v1:172.21.15.32:6789/0 conn(0x7f7a340b2aa0 legacy=0x7f7a340b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a38b5a700 1 -- 172.21.15.32:0/2322796839 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a341476a0 con 0x7f7a34061480 2022-08-01T23:05:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a38b5a700 1 --2- 172.21.15.32:0/2322796839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a340b1570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f7a24001800 tx=0x7f7a2c0091d0).ready entity=mon.2 client_cookie=1cb2954b2e05296a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a32ffd700 1 -- 172.21.15.32:0/2322796839 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a24001660 con 0x7f7a34061480 2022-08-01T23:05:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a32ffd700 1 -- 172.21.15.32:0/2322796839 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00f510 con 0x7f7a34061480 2022-08-01T23:05:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a32ffd700 1 -- 172.21.15.32:0/2322796839 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2c00f680 con 0x7f7a34061480 2022-08-01T23:05:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- 172.21.15.32:0/2322796839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 msgr2=0x7f7a340b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 --2- 172.21.15.32:0/2322796839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a340b1570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f7a24001800 tx=0x7f7a2c0091d0).stop 2022-08-01T23:05:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- 172.21.15.32:0/2322796839 shutdown_connections 2022-08-01T23:05:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 --2- 172.21.15.32:0/2322796839 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a340b1570 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 --2- 172.21.15.32:0/2322796839 >> v2:172.21.15.55:3300/0 conn(0x7f7a340b6510 0x7f7a34140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- 172.21.15.32:0/2322796839 >> 172.21.15.32:0/2322796839 conn(0x7f7a34063800 msgr2=0x7f7a34063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- 172.21.15.32:0/2322796839 shutdown_connections 2022-08-01T23:05:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- 172.21.15.32:0/2322796839 wait complete. 2022-08-01T23:05:00.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 Processor -- start 2022-08-01T23:05:00.216 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.200+0000 7f7a3935b700 1 -- start start 2022-08-01T23:05:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a34142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a340b2aa0 0x7f7a34066f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 0x7f7a34155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340ae410 con 0x7f7a340b6510 2022-08-01T23:05:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341522f0 con 0x7f7a340b2aa0 2022-08-01T23:05:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340b0240 con 0x7f7a34061480 2022-08-01T23:05:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 0x7f7a34155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 0x7f7a34155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49222/0 (socket says 172.21.15.32:49222) 2022-08-01T23:05:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 -- 172.21.15.32:0/891182271 learned_addr learned my addr 172.21.15.32:0/891182271 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a337fe700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a340b2aa0 0x7f7a34066f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a33fff700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a34142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 -- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 msgr2=0x7f7a34142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a34142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 -- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a340b2aa0 msgr2=0x7f7a34066f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a340b2aa0 0x7f7a34066f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 -- 172.21.15.32:0/891182271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009470 con 0x7f7a340b6510 2022-08-01T23:05:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a38b5a700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 0x7f7a34155bd0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f7a24001800 tx=0x7f7a2c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2c0032f0 con 0x7f7a340b6510 2022-08-01T23:05:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c011b80 con 0x7f7a340b6510 2022-08-01T23:05:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a340afa60 con 0x7f7a340b6510 2022-08-01T23:05:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2c021480 con 0x7f7a340b6510 2022-08-01T23:05:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a340af720 con 0x7f7a340b6510 2022-08-01T23:05:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.204+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34155910 con 0x7f7a340b6510 2022-08-01T23:05:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.208+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a2c00f070 con 0x7f7a340b6510 2022-08-01T23:05:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.208+0000 7f7a317fa700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7a18053a10 0x7f7a18055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.208+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7a2c06cf70 con 0x7f7a340b6510 2022-08-01T23:05:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.208+0000 7f7a33fff700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7a18053a10 0x7f7a18055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.208+0000 7f7a33fff700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7a18053a10 0x7f7a18055ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f7a1c00b000 tx=0x7f7a1c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.216+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a34155910 con 0x7f7a340b6510 2022-08-01T23:05:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7a34155910 con 0x7f7a340b6510 2022-08-01T23:05:00.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a317fa700 1 -- 172.21.15.32:0/891182271 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v142) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7a34155910 con 0x7f7a340b6510 2022-08-01T23:05:00.347 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:00.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7a18053a10 msgr2=0x7f7a18055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7a18053a10 0x7f7a18055ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f7a1c00b000 tx=0x7f7a1c006df0).stop 2022-08-01T23:05:00.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 msgr2=0x7f7a34155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 0x7f7a34155bd0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f7a24001800 tx=0x7f7a2c016000).stop 2022-08-01T23:05:00.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 shutdown_connections 2022-08-01T23:05:00.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7a18053a10 0x7f7a18055ed0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a340b6510 0x7f7a34155bd0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a340b2aa0 0x7f7a34066f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 --2- 172.21.15.32:0/891182271 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7a34061480 0x7f7a34142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 >> 172.21.15.32:0/891182271 conn(0x7f7a34063800 msgr2=0x7f7a3413fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 shutdown_connections 2022-08-01T23:05:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.340+0000 7f7a3935b700 1 -- 172.21.15.32:0/891182271 wait complete. 2022-08-01T23:05:00.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:05:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 Processor -- start 2022-08-01T23:05:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 -- start start 2022-08-01T23:05:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07440ad3f0 0x7f07440ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07440afff0 0x7f0744147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07440ad7d0 con 0x7f0744148090 2022-08-01T23:05:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07440abb50 con 0x7f07440ad3f0 2022-08-01T23:05:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0749896700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0744147650 con 0x7f07440afff0 2022-08-01T23:05:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0748894700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07440ad3f0 0x7f07440ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0748894700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07440ad3f0 0x7f07440ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49232/0 (socket says 172.21.15.32:49232) 2022-08-01T23:05:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0743fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07440afff0 0x7f0744147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0748894700 1 -- 172.21.15.32:0/918208500 learned_addr learned my addr 172.21.15.32:0/918208500 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0743fff700 1 -- 172.21.15.32:0/918208500 >> v2:172.21.15.32:3300/0 conn(0x7f07440ad3f0 msgr2=0x7f07440ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0743fff700 1 --2- 172.21.15.32:0/918208500 >> v2:172.21.15.32:3300/0 conn(0x7f07440ad3f0 0x7f07440ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0743fff700 1 -- 172.21.15.32:0/918208500 >> v1:172.21.15.55:6789/0 conn(0x7f0744148090 legacy=0x7f07440ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0743fff700 1 -- 172.21.15.32:0/918208500 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07440b0610 con 0x7f07440afff0 2022-08-01T23:05:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f0743fff700 1 --2- 172.21.15.32:0/918208500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07440afff0 0x7f0744147ba0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f07340032f0 tx=0x7f07340091d0).ready entity=mon.1 client_cookie=ff7485b6f81f2411 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f07437fe700 1 -- 172.21.15.32:0/918208500 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f073400b680 con 0x7f07440afff0 2022-08-01T23:05:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.460+0000 7f07437fe700 1 -- 172.21.15.32:0/918208500 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073400b7e0 con 0x7f07440afff0 2022-08-01T23:05:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f07437fe700 1 -- 172.21.15.32:0/918208500 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f073400b950 con 0x7f07440afff0 2022-08-01T23:05:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- 172.21.15.32:0/918208500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07440afff0 msgr2=0x7f0744147ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 --2- 172.21.15.32:0/918208500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07440afff0 0x7f0744147ba0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f07340032f0 tx=0x7f07340091d0).stop 2022-08-01T23:05:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- 172.21.15.32:0/918208500 shutdown_connections 2022-08-01T23:05:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 --2- 172.21.15.32:0/918208500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07440afff0 0x7f0744147ba0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 --2- 172.21.15.32:0/918208500 >> v2:172.21.15.32:3300/0 conn(0x7f07440ad3f0 0x7f07440ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- 172.21.15.32:0/918208500 >> 172.21.15.32:0/918208500 conn(0x7f07440613c0 msgr2=0x7f0744062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- 172.21.15.32:0/918208500 shutdown_connections 2022-08-01T23:05:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- 172.21.15.32:0/918208500 wait complete. 2022-08-01T23:05:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 Processor -- start 2022-08-01T23:05:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- start start 2022-08-01T23:05:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07440ad3f0 0x7f07440b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 0x7f0744143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0744148090 0x7f0744144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744144610 con 0x7f07440afff0 2022-08-01T23:05:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074415a030 con 0x7f0744148090 2022-08-01T23:05:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.464+0000 7f0749896700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074415a2d0 con 0x7f07440ad3f0 2022-08-01T23:05:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 0x7f0744143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 0x7f0744143c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49236/0 (socket says 172.21.15.32:49236) 2022-08-01T23:05:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 -- 172.21.15.32:0/1994292789 learned_addr learned my addr 172.21.15.32:0/1994292789 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0748894700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07440ad3f0 0x7f07440b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0749095700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0744148090 0x7f0744144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 -- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07440ad3f0 msgr2=0x7f07440b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07440ad3f0 0x7f07440b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 -- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0744148090 msgr2=0x7f0744144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0744148090 0x7f0744144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 -- 172.21.15.32:0/1994292789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734009470 con 0x7f07440afff0 2022-08-01T23:05:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0743fff700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 0x7f0744143c30 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f07340032f0 tx=0x7f0734016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0734011e50 con 0x7f07440afff0 2022-08-01T23:05:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0734021440 con 0x7f07440afff0 2022-08-01T23:05:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07340217a0 con 0x7f07440afff0 2022-08-01T23:05:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074415a570 con 0x7f07440afff0 2022-08-01T23:05:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07441438f0 con 0x7f07440afff0 2022-08-01T23:05:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07440b08b0 con 0x7f07440afff0 2022-08-01T23:05:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0734027020 con 0x7f07440afff0 2022-08-01T23:05:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0741ffb700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0724053950 0x7f0724055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.468+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f073406d160 con 0x7f07440afff0 2022-08-01T23:05:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.472+0000 7f0748894700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0724053950 0x7f0724055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.472+0000 7f0748894700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0724053950 0x7f0724055e10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f0738008000 tx=0x7f0738008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.476+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0734044260 con 0x7f07440afff0 2022-08-01T23:05:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f07440574e0 con 0x7f07440afff0 2022-08-01T23:05:00.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0741ffb700 1 -- 172.21.15.32:0/1994292789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) v1 ==== 84+0+143 (secure 0 0 0) 0x7f07340150a0 con 0x7f07440afff0 2022-08-01T23:05:00.614 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:00.614 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:05:00.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:00.615 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:05:00.615 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:05:00.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:00.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:00.615 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:05:00.616 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:05:00.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:00.616 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:05:00.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0724053950 msgr2=0x7f0724055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0724053950 0x7f0724055e10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f0738008000 tx=0x7f0738008040).stop 2022-08-01T23:05:00.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 msgr2=0x7f0744143c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 0x7f0744143c30 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f07340032f0 tx=0x7f0734016000).stop 2022-08-01T23:05:00.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 shutdown_connections 2022-08-01T23:05:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0724053950 0x7f0724055e10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0744148090 0x7f0744144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07440afff0 0x7f0744143c30 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 --2- 172.21.15.32:0/1994292789 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f07440ad3f0 0x7f07440b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 >> 172.21.15.32:0/1994292789 conn(0x7f07440613c0 msgr2=0x7f07440ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 shutdown_connections 2022-08-01T23:05:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.608+0000 7f0749896700 1 -- 172.21.15.32:0/1994292789 wait complete. 2022-08-01T23:05:00.626 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:05:00.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:05:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33b49a4700 1 Processor -- start 2022-08-01T23:05:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33b49a4700 1 -- start start 2022-08-01T23:05:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33b49a4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f33b00ad750 0x7f33b0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33b49a4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b00aee80 con 0x7f33b00614b0 2022-08-01T23:05:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33b49a4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b00b70b0 con 0x7f33b01405d0 2022-08-01T23:05:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33b49a4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b00af0b0 con 0x7f33b00ad750 2022-08-01T23:05:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33aed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33b00614b0 0x7f33b00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36784/0 (socket says 172.21.15.32:36784) 2022-08-01T23:05:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.732+0000 7f33aed9d700 1 -- 172.21.15.32:0/2972618413 learned_addr learned my addr 172.21.15.32:0/2972618413 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33ae59c700 1 --2- 172.21.15.32:0/2972618413 >> v2:172.21.15.55:3300/0 conn(0x7f33b00ad750 0x7f33b0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487782938 0 0) 0x7f33b00aee80 con 0x7f33b00614b0 2022-08-01T23:05:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3394003260 con 0x7f33b00614b0 2022-08-01T23:05:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774581125 0 0) 0x7f33b00b70b0 con 0x7f33b01405d0 2022-08-01T23:05:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b00aee80 con 0x7f33b01405d0 2022-08-01T23:05:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 774571997 0 0) 0x7f3394003260 con 0x7f33b00614b0 2022-08-01T23:05:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 >> v2:172.21.15.55:3300/0 conn(0x7f33b00ad750 msgr2=0x7f33b0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 --2- 172.21.15.32:0/2972618413 >> v2:172.21.15.55:3300/0 conn(0x7f33b00ad750 0x7f33b0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 >> v1:172.21.15.55:6789/0 conn(0x7f33b01405d0 legacy=0x7f33b01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b00af350 con 0x7f33b00614b0 2022-08-01T23:05:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f33a0001660 con 0x7f33b00614b0 2022-08-01T23:05:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 >> v1:172.21.15.32:6789/0 conn(0x7f33b00614b0 legacy=0x7f33b00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3394003930 0x7f3394003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3394004500 0x7f3394004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f339400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00af0b0 con 0x7f3394004e70 2022-08-01T23:05:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339400bd20 con 0x7f3394004500 2022-08-01T23:05:00.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339400bee0 con 0x7f3394003930 2022-08-01T23:05:00.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f339400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33aed9d700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3394003930 0x7f3394003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33ae59c700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3394004500 0x7f3394004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 -- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3394003930 msgr2=0x7f3394003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:00.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3394003930 0x7f3394003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 -- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3394004500 msgr2=0x7f3394004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:00.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3394004500 0x7f3394004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 -- 172.21.15.32:0/2972618413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394003260 con 0x7f3394004e70 2022-08-01T23:05:00.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33ae59c700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3394004500 0x7f3394004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:00.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 -- 172.21.15.32:0/2972618413 reap_dead start 2022-08-01T23:05:00.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33af59e700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f339400b750 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f33a8001490 tx=0x7f33a8007d80).ready entity=mon.0 client_cookie=449b3d67455058e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a8001780 con 0x7f3394004e70 2022-08-01T23:05:00.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a8006890 con 0x7f3394004e70 2022-08-01T23:05:00.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33add9b700 1 -- 172.21.15.32:0/2972618413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a8006bf0 con 0x7f3394004e70 2022-08-01T23:05:00.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 -- 172.21.15.32:0/2972618413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 msgr2=0x7f339400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f339400b750 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f33a8001490 tx=0x7f33a8007d80).stop 2022-08-01T23:05:00.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 -- 172.21.15.32:0/2972618413 shutdown_connections 2022-08-01T23:05:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2972618413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f339400b750 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 -- 172.21.15.32:0/2972618413 >> 172.21.15.32:0/2972618413 conn(0x7f33b0066f70 msgr2=0x7f33b0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 -- 172.21.15.32:0/2972618413 shutdown_connections 2022-08-01T23:05:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.736+0000 7f33b49a4700 1 -- 172.21.15.32:0/2972618413 wait complete. 2022-08-01T23:05:00.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 Processor -- start 2022-08-01T23:05:00.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 -- start start 2022-08-01T23:05:00.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f33b00b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b0142640 0x7f33b0153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33b00b1ed0 0x7f33b00aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00b1860 con 0x7f3394004e70 2022-08-01T23:05:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0152780 con 0x7f33b0142640 2022-08-01T23:05:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0152a20 con 0x7f33b00b1ed0 2022-08-01T23:05:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f33b00b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f33b00b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49254/0 (socket says 172.21.15.32:49254) 2022-08-01T23:05:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 -- 172.21.15.32:0/2446582688 learned_addr learned my addr 172.21.15.32:0/2446582688 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33af59e700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33b00b1ed0 0x7f33b00aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33ae59c700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b0142640 0x7f33b0153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 -- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33b00b1ed0 msgr2=0x7f33b00aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33b00b1ed0 0x7f33b00aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 -- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b0142640 msgr2=0x7f33b0153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:00.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b0142640 0x7f33b0153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:00.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 -- 172.21.15.32:0/2446582688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0152cc0 con 0x7f3394004e70 2022-08-01T23:05:00.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33af59e700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33b00b1ed0 0x7f33b00aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:00.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33aed9d700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f33b00b1230 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f33a000b000 tx=0x7f33b00af4e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a0015630 con 0x7f3394004e70 2022-08-01T23:05:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b0143f00 con 0x7f3394004e70 2022-08-01T23:05:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a0010040 con 0x7f3394004e70 2022-08-01T23:05:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b009f710 con 0x7f3394004e70 2022-08-01T23:05:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.740+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a0015980 con 0x7f3394004e70 2022-08-01T23:05:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.744+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b0142a20 con 0x7f3394004e70 2022-08-01T23:05:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.744+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33a0015ae0 con 0x7f3394004e70 2022-08-01T23:05:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.744+0000 7f339ffff700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f338c053aa0 0x7f338c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.744+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f33a006b8c0 con 0x7f3394004e70 2022-08-01T23:05:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.744+0000 7f33ae59c700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f338c053aa0 0x7f338c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.744+0000 7f33ae59c700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f338c053aa0 0x7f338c055f60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f33a400b000 tx=0x7f33a4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.752+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33a00428c0 con 0x7f3394004e70 2022-08-01T23:05:00.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:00.876+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f33b0152e50 con 0x7f3394004e70 2022-08-01T23:05:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f339ffff700 1 -- 172.21.15.32:0/2446582688 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v143) v1 ==== 133+0+0 (secure 0 0 0) 0x7f33a0045ee0 con 0x7f3394004e70 2022-08-01T23:05:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f338c053aa0 msgr2=0x7f338c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f338c053aa0 0x7f338c055f60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f33a400b000 tx=0x7f33a4008040).stop 2022-08-01T23:05:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 msgr2=0x7f33b00b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f33b00b1230 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f33a000b000 tx=0x7f33b00af4e0).stop 2022-08-01T23:05:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 shutdown_connections 2022-08-01T23:05:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f338c053aa0 0x7f338c055f60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f33b00b1ed0 0x7f33b00aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b0142640 0x7f33b0153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 --2- 172.21.15.32:0/2446582688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3394004e70 0x7f33b00b1230 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.544+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 >> 172.21.15.32:0/2446582688 conn(0x7f33b0066f70 msgr2=0x7f33b00b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.548+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 shutdown_connections 2022-08-01T23:05:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.548+0000 7f33b49a4700 1 -- 172.21.15.32:0/2446582688 wait complete. 2022-08-01T23:05:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:05:01.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:05:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 Processor -- start 2022-08-01T23:05:01.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 -- start start 2022-08-01T23:05:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50040b2710 0x7f500414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f50040a8c50 0x7f500414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:01.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50040a7250 con 0x7f50040b01a0 2022-08-01T23:05:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040a7480 con 0x7f50040b2710 2022-08-01T23:05:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.676+0000 7f500ac46700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040a7720 con 0x7f50040a8c50 2022-08-01T23:05:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009c44700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50040b2710 0x7f500414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:01.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009443700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f50040b01a0 0x7f50040a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36808/0 (socket says 172.21.15.32:36808) 2022-08-01T23:05:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009443700 1 -- 172.21.15.32:0/1660540787 learned_addr learned my addr 172.21.15.32:0/1660540787 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500a445700 1 --2- 172.21.15.32:0/1660540787 >> v2:172.21.15.55:3300/0 conn(0x7f50040a8c50 0x7f500414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5008c42700 1 -- 172.21.15.32:0/1660540787 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265232597 0 0) 0x7f50040a7250 con 0x7f50040b01a0 2022-08-01T23:05:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009c44700 1 -- 172.21.15.32:0/1660540787 >> v2:172.21.15.55:3300/0 conn(0x7f50040a8c50 msgr2=0x7f500414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009c44700 1 --2- 172.21.15.32:0/1660540787 >> v2:172.21.15.55:3300/0 conn(0x7f50040a8c50 0x7f500414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009c44700 1 -- 172.21.15.32:0/1660540787 >> v1:172.21.15.32:6789/0 conn(0x7f50040b01a0 legacy=0x7f50040a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009c44700 1 -- 172.21.15.32:0/1660540787 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50040a79c0 con 0x7f50040b2710 2022-08-01T23:05:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5009c44700 1 --2- 172.21.15.32:0/1660540787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b2710 0x7f500414afd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f4ff4002f70 tx=0x7f4ff4006df0).ready entity=mon.0 client_cookie=7bcf3468d1b8a898 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:01.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5008c42700 1 -- 172.21.15.32:0/1660540787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ff400d570 con 0x7f50040b2710 2022-08-01T23:05:01.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5008c42700 1 -- 172.21.15.32:0/1660540787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4000e90 con 0x7f50040b2710 2022-08-01T23:05:01.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f5008c42700 1 -- 172.21.15.32:0/1660540787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ff400d8c0 con 0x7f50040b2710 2022-08-01T23:05:01.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 -- 172.21.15.32:0/1660540787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b2710 msgr2=0x7f500414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:01.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 --2- 172.21.15.32:0/1660540787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b2710 0x7f500414afd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f4ff4002f70 tx=0x7f4ff4006df0).stop 2022-08-01T23:05:01.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 -- 172.21.15.32:0/1660540787 shutdown_connections 2022-08-01T23:05:01.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 --2- 172.21.15.32:0/1660540787 >> v2:172.21.15.55:3300/0 conn(0x7f50040a8c50 0x7f500414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 --2- 172.21.15.32:0/1660540787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b2710 0x7f500414afd0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 -- 172.21.15.32:0/1660540787 >> 172.21.15.32:0/1660540787 conn(0x7f5004060310 msgr2=0x7f50040b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:01.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 -- 172.21.15.32:0/1660540787 shutdown_connections 2022-08-01T23:05:01.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 -- 172.21.15.32:0/1660540787 wait complete. 2022-08-01T23:05:01.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 Processor -- start 2022-08-01T23:05:01.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.680+0000 7f500ac46700 1 -- start start 2022-08-01T23:05:01.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50040a8c50 0x7f50040b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:01.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 0x7f5004140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:01.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50040b2710 0x7f5004141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:01.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500414d3d0 con 0x7f50040b01a0 2022-08-01T23:05:01.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040ac420 con 0x7f50040b2710 2022-08-01T23:05:01.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040abee0 con 0x7f50040a8c50 2022-08-01T23:05:01.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 0x7f5004140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:01.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 0x7f5004140f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49268/0 (socket says 172.21.15.32:49268) 2022-08-01T23:05:01.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 -- 172.21.15.32:0/1242708305 learned_addr learned my addr 172.21.15.32:0/1242708305 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:01.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500a445700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50040b2710 0x7f5004141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:01.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009c44700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50040a8c50 0x7f50040b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:01.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 -- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50040a8c50 msgr2=0x7f50040b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:01.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50040a8c50 0x7f50040b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 -- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50040b2710 msgr2=0x7f5004141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:01.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50040b2710 0x7f5004141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:01.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 -- 172.21.15.32:0/1242708305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff400b180 con 0x7f50040b01a0 2022-08-01T23:05:01.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009c44700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50040a8c50 0x7f50040b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:01.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f5009443700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 0x7f5004140f90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f5000001250 tx=0x7f5000007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:01.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5000005ba0 con 0x7f50040b01a0 2022-08-01T23:05:01.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5000005d00 con 0x7f50040b01a0 2022-08-01T23:05:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50040ac180 con 0x7f50040b01a0 2022-08-01T23:05:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5000007300 con 0x7f50040b01a0 2022-08-01T23:05:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500414d8e0 con 0x7f50040b01a0 2022-08-01T23:05:01.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.684+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50040574e0 con 0x7f50040b01a0 2022-08-01T23:05:01.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.688+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5000010030 con 0x7f50040b01a0 2022-08-01T23:05:01.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.688+0000 7f4ff2ffd700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4fe4053a70 0x7f4fe4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:01.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.688+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f500006bf10 con 0x7f50040b01a0 2022-08-01T23:05:01.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.688+0000 7f5009c44700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4fe4053a70 0x7f4fe4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:01.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.688+0000 7f5009c44700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4fe4053a70 0x7f4fe4055f30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f4ff4018000 tx=0x7f4ff4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:01.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.696+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5000042ea0 con 0x7f50040b01a0 2022-08-01T23:05:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:01.824+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f50040623b0 con 0x7f50040b01a0 2022-08-01T23:05:02.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.136+0000 7f4ff2ffd700 1 -- 172.21.15.32:0/1242708305 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v144) v1 ==== 125+0+0 (secure 0 0 0) 0x7f50000464c0 con 0x7f50040b01a0 2022-08-01T23:05:02.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4fe4053a70 msgr2=0x7f4fe4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4fe4053a70 0x7f4fe4055f30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f4ff4018000 tx=0x7f4ff4014000).stop 2022-08-01T23:05:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 msgr2=0x7f5004140f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 0x7f5004140f90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f5000001250 tx=0x7f5000007de0).stop 2022-08-01T23:05:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 shutdown_connections 2022-08-01T23:05:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4fe4053a70 0x7f4fe4055f30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50040b2710 0x7f5004141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50040b01a0 0x7f5004140f90 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 --2- 172.21.15.32:0/1242708305 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f50040a8c50 0x7f50040b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 >> 172.21.15.32:0/1242708305 conn(0x7f5004060310 msgr2=0x7f5004146150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 shutdown_connections 2022-08-01T23:05:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.140+0000 7f500ac46700 1 -- 172.21.15.32:0/1242708305 wait complete. 2022-08-01T23:05:02.150 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:05:02.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:05:02.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.260+0000 7f91cb59e700 1 Processor -- start 2022-08-01T23:05:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.260+0000 7f91cb59e700 1 -- start start 2022-08-01T23:05:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91cc0b08a0 0x7f91cc142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b3260 0x7f91cc1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cb59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc0b0c80 con 0x7f91cc0ad000 2022-08-01T23:05:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc0ae810 con 0x7f91cc0b08a0 2022-08-01T23:05:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0aeab0 con 0x7f91cc0b3260 2022-08-01T23:05:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91cc0ad000 0x7f91cc05f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36822/0 (socket says 172.21.15.32:36822) 2022-08-01T23:05:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91cad9d700 1 -- 172.21.15.32:0/4027000845 learned_addr learned my addr 172.21.15.32:0/4027000845 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91ca59c700 1 --2- 172.21.15.32:0/4027000845 >> v2:172.21.15.32:3300/0 conn(0x7f91cc0b08a0 0x7f91cc142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91c3fff700 1 --2- 172.21.15.32:0/4027000845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b3260 0x7f91cc1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91c9d9b700 1 -- 172.21.15.32:0/4027000845 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237136573 0 0) 0x7f91cc0b0c80 con 0x7f91cc0ad000 2022-08-01T23:05:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91c9d9b700 1 -- 172.21.15.32:0/4027000845 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ac003260 con 0x7f91cc0ad000 2022-08-01T23:05:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91ca59c700 1 -- 172.21.15.32:0/4027000845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b3260 msgr2=0x7f91cc1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91ca59c700 1 --2- 172.21.15.32:0/4027000845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b3260 0x7f91cc1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91ca59c700 1 -- 172.21.15.32:0/4027000845 >> v1:172.21.15.32:6789/0 conn(0x7f91cc0ad000 legacy=0x7f91cc05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91ca59c700 1 -- 172.21.15.32:0/4027000845 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0aed50 con 0x7f91cc0b08a0 2022-08-01T23:05:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91ca59c700 1 --2- 172.21.15.32:0/4027000845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b08a0 0x7f91cc142eb0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f91bc002f70 tx=0x7f91bc006df0).ready entity=mon.0 client_cookie=574840b6b401ba07 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91c9d9b700 1 -- 172.21.15.32:0/4027000845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91bc00d4a0 con 0x7f91cc0b08a0 2022-08-01T23:05:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91c9d9b700 1 -- 172.21.15.32:0/4027000845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc000e90 con 0x7f91cc0b08a0 2022-08-01T23:05:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.264+0000 7f91c9d9b700 1 -- 172.21.15.32:0/4027000845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91bc00d7f0 con 0x7f91cc0b08a0 2022-08-01T23:05:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 -- 172.21.15.32:0/4027000845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b08a0 msgr2=0x7f91cc142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 --2- 172.21.15.32:0/4027000845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b08a0 0x7f91cc142eb0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f91bc002f70 tx=0x7f91bc006df0).stop 2022-08-01T23:05:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 -- 172.21.15.32:0/4027000845 shutdown_connections 2022-08-01T23:05:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 --2- 172.21.15.32:0/4027000845 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b3260 0x7f91cc1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 --2- 172.21.15.32:0/4027000845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b08a0 0x7f91cc142eb0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 -- 172.21.15.32:0/4027000845 >> 172.21.15.32:0/4027000845 conn(0x7f91cc0612d0 msgr2=0x7f91cc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 -- 172.21.15.32:0/4027000845 shutdown_connections 2022-08-01T23:05:02.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 -- 172.21.15.32:0/4027000845 wait complete. 2022-08-01T23:05:02.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 Processor -- start 2022-08-01T23:05:02.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.268+0000 7f91cb59e700 1 -- start start 2022-08-01T23:05:02.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91cc0ad000 0x7f91cc13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b08a0 0x7f91cc152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 0x7f91cc156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0674e0 con 0x7f91cc0b3260 2022-08-01T23:05:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc13e4b0 con 0x7f91cc0ad000 2022-08-01T23:05:02.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc155c40 con 0x7f91cc0b08a0 2022-08-01T23:05:02.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 0x7f91cc156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 0x7f91cc156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49286/0 (socket says 172.21.15.32:49286) 2022-08-01T23:05:02.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 -- 172.21.15.32:0/1453113733 learned_addr learned my addr 172.21.15.32:0/1453113733 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:02.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91ca59c700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91cc0ad000 0x7f91cc13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91c3fff700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b08a0 0x7f91cc152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 -- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b08a0 msgr2=0x7f91cc152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b08a0 0x7f91cc152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 -- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91cc0ad000 msgr2=0x7f91cc13f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91cc0ad000 0x7f91cc13f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 -- 172.21.15.32:0/1453113733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc00b180 con 0x7f91cc0b3260 2022-08-01T23:05:02.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cad9d700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 0x7f91cc156380 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f91c4001490 tx=0x7f91c4007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91c4005c70 con 0x7f91cc0b3260 2022-08-01T23:05:02.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4005dd0 con 0x7f91cc0b3260 2022-08-01T23:05:02.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc155ee0 con 0x7f91cc0b3260 2022-08-01T23:05:02.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91c40072c0 con 0x7f91cc0b3260 2022-08-01T23:05:02.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc156870 con 0x7f91cc0b3260 2022-08-01T23:05:02.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.272+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc13e670 con 0x7f91cc0b3260 2022-08-01T23:05:02.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.276+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f91c401a060 con 0x7f91cc0b3260 2022-08-01T23:05:02.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.276+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91b0053a70 0x7f91b0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.276+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f91c406bad0 con 0x7f91cc0b3260 2022-08-01T23:05:02.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.276+0000 7f91ca59c700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91b0053a70 0x7f91b0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.276+0000 7f91ca59c700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91b0053a70 0x7f91b0055f30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f91bc01a000 tx=0x7f91bc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.280+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f91c4042800 con 0x7f91cc0b3260 2022-08-01T23:05:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.444+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f91cc156070 con 0x7f91cc0b3260 2022-08-01T23:05:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.452+0000 7f91c37fe700 1 -- 172.21.15.32:0/1453113733 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v73) v1 ==== 189+0+0 (secure 0 0 0) 0x7f91c4018070 con 0x7f91cc0b3260 2022-08-01T23:05:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91b0053a70 msgr2=0x7f91b0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91b0053a70 0x7f91b0055f30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f91bc01a000 tx=0x7f91bc014000).stop 2022-08-01T23:05:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 msgr2=0x7f91cc156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 0x7f91cc156380 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f91c4001490 tx=0x7f91c4007dc0).stop 2022-08-01T23:05:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 shutdown_connections 2022-08-01T23:05:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f91b0053a70 0x7f91b0055f30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91cc0b3260 0x7f91cc156380 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f91cc0b08a0 0x7f91cc152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 --2- 172.21.15.32:0/1453113733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91cc0ad000 0x7f91cc13f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.456+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 >> 172.21.15.32:0/1453113733 conn(0x7f91cc0612d0 msgr2=0x7f91cc141f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.460+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 shutdown_connections 2022-08-01T23:05:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.460+0000 7f91cb59e700 1 -- 172.21.15.32:0/1453113733 wait complete. 2022-08-01T23:05:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 27 and data pool 28 2022-08-01T23:05:02.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:02.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256b3f8700 1 Processor -- start 2022-08-01T23:05:02.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256b3f8700 1 -- start start 2022-08-01T23:05:02.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256b3f8700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2564066dc0 0x7f256414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256b3f8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640654c0 con 0x7f25640b3190 2022-08-01T23:05:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256b3f8700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640b3570 con 0x7f25640b2730 2022-08-01T23:05:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256b3f8700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2564062190 con 0x7f2564066dc0 2022-08-01T23:05:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256a3f6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25640b3190 0x7f2564060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36844/0 (socket says 172.21.15.32:36844) 2022-08-01T23:05:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.568+0000 7f256a3f6700 1 -- 172.21.15.32:0/1408094893 learned_addr learned my addr 172.21.15.32:0/1408094893 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256abf7700 1 --2- 172.21.15.32:0/1408094893 >> v2:172.21.15.55:3300/0 conn(0x7f2564066dc0 0x7f256414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511373132 0 0) 0x7f25640654c0 con 0x7f25640b3190 2022-08-01T23:05:02.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f254c003260 con 0x7f25640b3190 2022-08-01T23:05:02.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12942969 0 0) 0x7f25640b3570 con 0x7f25640b2730 2022-08-01T23:05:02.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25640654c0 con 0x7f25640b2730 2022-08-01T23:05:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902580462 0 0) 0x7f254c003260 con 0x7f25640b3190 2022-08-01T23:05:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 >> v2:172.21.15.55:3300/0 conn(0x7f2564066dc0 msgr2=0x7f256414eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 --2- 172.21.15.32:0/1408094893 >> v2:172.21.15.55:3300/0 conn(0x7f2564066dc0 0x7f256414eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 >> v1:172.21.15.55:6789/0 conn(0x7f25640b2730 legacy=0x7f25640666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256405fbf0 con 0x7f25640b3190 2022-08-01T23:05:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f2560001660 con 0x7f25640b3190 2022-08-01T23:05:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 >> v1:172.21.15.32:6789/0 conn(0x7f25640b3190 legacy=0x7f2564060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c0038e0 0x7f254c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c004500 0x7f254c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f254c004e70 0x7f254c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2564062190 con 0x7f254c0038e0 2022-08-01T23:05:02.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c00bcf0 con 0x7f254c004e70 2022-08-01T23:05:02.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c00beb0 con 0x7f254c004500 2022-08-01T23:05:02.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256abf7700 1 --2- 172.21.15.32:0/1408094893 >> v2:172.21.15.55:3300/0 conn(0x7f2564066dc0 0x7f256414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:02.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c0038e0 0x7f254c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f2569bf5700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c004500 0x7f254c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256abf7700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f254c004e70 0x7f254c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 -- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c004500 msgr2=0x7f254c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c004500 0x7f254c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 -- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f254c004e70 msgr2=0x7f254c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f254c004e70 0x7f254c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256abf7700 1 -- 172.21.15.32:0/1408094893 reap_dead start 2022-08-01T23:05:02.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 -- 172.21.15.32:0/1408094893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c003260 con 0x7f254c0038e0 2022-08-01T23:05:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c0038e0 0x7f254c003ce0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f2560001430 tx=0x7f256405fd80).ready entity=mon.0 client_cookie=b0d8603ed7ced1cd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25600071a0 con 0x7f254c0038e0 2022-08-01T23:05:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560007300 con 0x7f254c0038e0 2022-08-01T23:05:02.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f25693f4700 1 -- 172.21.15.32:0/1408094893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256000c8f0 con 0x7f254c0038e0 2022-08-01T23:05:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 -- 172.21.15.32:0/1408094893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c0038e0 msgr2=0x7f254c003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c0038e0 0x7f254c003ce0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f2560001430 tx=0x7f256405fd80).stop 2022-08-01T23:05:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 -- 172.21.15.32:0/1408094893 shutdown_connections 2022-08-01T23:05:02.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1408094893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c0038e0 0x7f254c003ce0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 -- 172.21.15.32:0/1408094893 >> 172.21.15.32:0/1408094893 conn(0x7f2564063870 msgr2=0x7f2564063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 -- 172.21.15.32:0/1408094893 shutdown_connections 2022-08-01T23:05:02.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.572+0000 7f256b3f8700 1 -- 172.21.15.32:0/1408094893 wait complete. 2022-08-01T23:05:02.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 Processor -- start 2022-08-01T23:05:02.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 -- start start 2022-08-01T23:05:02.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c0038e0 0x7f25640ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 0x7f2564152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2564152d80 0x7f25641437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2564153200 con 0x7f25640acc50 2022-08-01T23:05:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641486a0 con 0x7f2564152d80 2022-08-01T23:05:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2564148940 con 0x7f254c0038e0 2022-08-01T23:05:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 0x7f2564152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 0x7f2564152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49316/0 (socket says 172.21.15.32:49316) 2022-08-01T23:05:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 -- 172.21.15.32:0/1060766365 learned_addr learned my addr 172.21.15.32:0/1060766365 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c0038e0 0x7f25640ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256abf7700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2564152d80 0x7f25641437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 -- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c0038e0 msgr2=0x7f25640ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c0038e0 0x7f25640ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 -- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2564152d80 msgr2=0x7f25641437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2564152d80 0x7f25641437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 -- 172.21.15.32:0/1060766365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564148be0 con 0x7f25640acc50 2022-08-01T23:05:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256abf7700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2564152d80 0x7f25641437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f2569bf5700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 0x7f2564152890 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f255400d000 tx=0x7f25540079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f255400fac0 con 0x7f25640acc50 2022-08-01T23:05:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255400fc20 con 0x7f25640acc50 2022-08-01T23:05:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25641523f0 con 0x7f25640acc50 2022-08-01T23:05:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2554002e30 con 0x7f25640acc50 2022-08-01T23:05:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.576+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640b0870 con 0x7f25640acc50 2022-08-01T23:05:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.580+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2564152580 con 0x7f25640acc50 2022-08-01T23:05:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.580+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f255401b030 con 0x7f25640acc50 2022-08-01T23:05:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.580+0000 7f25537fe700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2544053a10 0x7f2544055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.580+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f255406c230 con 0x7f25640acc50 2022-08-01T23:05:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.580+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2544053a10 0x7f2544055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.580+0000 7f256a3f6700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2544053a10 0x7f2544055ed0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f2560018000 tx=0x7f2560013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.588+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2554042e80 con 0x7f25640acc50 2022-08-01T23:05:02.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.708+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f256405fe90 con 0x7f25640acc50 2022-08-01T23:05:02.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.708+0000 7f25537fe700 1 -- 172.21.15.32:0/1060766365 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f25540464a0 con 0x7f25640acc50 2022-08-01T23:05:02.715 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:02.716 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":146,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:02.448243+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:00.933410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:01.881408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:02.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2544053a10 msgr2=0x7f2544055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2544053a10 0x7f2544055ed0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f2560018000 tx=0x7f2560013000).stop 2022-08-01T23:05:02.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 msgr2=0x7f2564152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 0x7f2564152890 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f255400d000 tx=0x7f25540079e0).stop 2022-08-01T23:05:02.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 shutdown_connections 2022-08-01T23:05:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2544053a10 0x7f2544055ed0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2564152d80 0x7f25641437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25640acc50 0x7f2564152890 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 --2- 172.21.15.32:0/1060766365 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f254c0038e0 0x7f25640ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 >> 172.21.15.32:0/1060766365 conn(0x7f2564063870 msgr2=0x7f25641462d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 shutdown_connections 2022-08-01T23:05:02.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.712+0000 7f256b3f8700 1 -- 172.21.15.32:0/1060766365 wait complete. 2022-08-01T23:05:02.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 Processor -- start 2022-08-01T23:05:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 -- start start 2022-08-01T23:05:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3640b2710 0x7ff36414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 0x7ff36414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3640a7250 con 0x7ff3640b01a0 2022-08-01T23:05:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3640a7480 con 0x7ff3640b2710 2022-08-01T23:05:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff368b9b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640a7720 con 0x7ff3640a8c50 2022-08-01T23:05:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff3637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3640b2710 0x7ff36414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff363fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 0x7ff36414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff362ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff3640b01a0 0x7ff3640a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57770/0 (socket says 172.21.15.32:57770) 2022-08-01T23:05:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff363fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 0x7ff36414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:32832/0 (socket says 172.21.15.32:32832) 2022-08-01T23:05:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.824+0000 7ff362ffd700 1 -- 172.21.15.32:0/3483253264 learned_addr learned my addr 172.21.15.32:0/3483253264 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff3627fc700 1 -- 172.21.15.32:0/3483253264 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195849051 0 0) 0x7ff3640a7250 con 0x7ff3640b01a0 2022-08-01T23:05:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff3627fc700 1 -- 172.21.15.32:0/3483253264 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff350003260 con 0x7ff3640b01a0 2022-08-01T23:05:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff363fff700 1 -- 172.21.15.32:0/3483253264 >> v2:172.21.15.55:3300/0 conn(0x7ff3640b2710 msgr2=0x7ff36414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff363fff700 1 --2- 172.21.15.32:0/3483253264 >> v2:172.21.15.55:3300/0 conn(0x7ff3640b2710 0x7ff36414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff363fff700 1 -- 172.21.15.32:0/3483253264 >> v1:172.21.15.55:6789/0 conn(0x7ff3640b01a0 legacy=0x7ff3640a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff363fff700 1 -- 172.21.15.32:0/3483253264 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3640a79c0 con 0x7ff3640a8c50 2022-08-01T23:05:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff363fff700 1 --2- 172.21.15.32:0/3483253264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 0x7ff36414b4c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7ff35c0032f0 tx=0x7ff35c0091d0).ready entity=mon.2 client_cookie=627cd3c3d873ee47 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff3627fc700 1 -- 172.21.15.32:0/3483253264 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35c00b600 con 0x7ff3640a8c50 2022-08-01T23:05:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff3627fc700 1 -- 172.21.15.32:0/3483253264 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c00b760 con 0x7ff3640a8c50 2022-08-01T23:05:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff3627fc700 1 -- 172.21.15.32:0/3483253264 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35c00b8d0 con 0x7ff3640a8c50 2022-08-01T23:05:02.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 -- 172.21.15.32:0/3483253264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 msgr2=0x7ff36414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 --2- 172.21.15.32:0/3483253264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 0x7ff36414b4c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7ff35c0032f0 tx=0x7ff35c0091d0).stop 2022-08-01T23:05:02.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 -- 172.21.15.32:0/3483253264 shutdown_connections 2022-08-01T23:05:02.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 --2- 172.21.15.32:0/3483253264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640a8c50 0x7ff36414b4c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 --2- 172.21.15.32:0/3483253264 >> v2:172.21.15.55:3300/0 conn(0x7ff3640b2710 0x7ff36414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 -- 172.21.15.32:0/3483253264 >> 172.21.15.32:0/3483253264 conn(0x7ff364060310 msgr2=0x7ff3640b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 -- 172.21.15.32:0/3483253264 shutdown_connections 2022-08-01T23:05:02.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 -- 172.21.15.32:0/3483253264 wait complete. 2022-08-01T23:05:02.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 Processor -- start 2022-08-01T23:05:02.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.828+0000 7ff368b9b700 1 -- start start 2022-08-01T23:05:02.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3640a8c50 0x7ff3640aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640b01a0 0x7ff3640a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 0x7ff3640a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640b3000 con 0x7ff3640b2710 2022-08-01T23:05:02.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640af990 con 0x7ff3640a8c50 2022-08-01T23:05:02.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36415a800 con 0x7ff3640b01a0 2022-08-01T23:05:02.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 0x7ff3640a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff3637fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3640a8c50 0x7ff3640aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 0x7ff3640a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49332/0 (socket says 172.21.15.32:49332) 2022-08-01T23:05:02.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 -- 172.21.15.32:0/1116611160 learned_addr learned my addr 172.21.15.32:0/1116611160 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:02.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff362ffd700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640b01a0 0x7ff3640a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 -- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640b01a0 msgr2=0x7ff3640a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640b01a0 0x7ff3640a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 -- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3640a8c50 msgr2=0x7ff3640aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3640a8c50 0x7ff3640aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 -- 172.21.15.32:0/1116611160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c009470 con 0x7ff3640b2710 2022-08-01T23:05:02.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff363fff700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 0x7ff3640a9f80 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7ff35c0032f0 tx=0x7ff35c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35c00b8d0 con 0x7ff3640b2710 2022-08-01T23:05:02.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c01f3f0 con 0x7ff3640b2710 2022-08-01T23:05:02.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36415a510 con 0x7ff3640b2710 2022-08-01T23:05:02.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35c00b600 con 0x7ff3640b2710 2022-08-01T23:05:02.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3640a92d0 con 0x7ff3640b2710 2022-08-01T23:05:02.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.832+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff35c026030 con 0x7ff3640b2710 2022-08-01T23:05:02.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.836+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3640b3190 con 0x7ff3640b2710 2022-08-01T23:05:02.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.836+0000 7ff360ff9700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff344053aa0 0x7ff344055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:02.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.836+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff35c06cb70 con 0x7ff3640b2710 2022-08-01T23:05:02.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.836+0000 7ff3637fe700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff344053aa0 0x7ff344055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:02.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.836+0000 7ff3637fe700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff344053aa0 0x7ff344055f60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7ff35800b000 tx=0x7ff358006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:02.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.840+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3640b3190 con 0x7ff3640b2710 2022-08-01T23:05:02.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.968+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3640b3190 con 0x7ff3640b2710 2022-08-01T23:05:02.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.968+0000 7ff360ff9700 1 -- 172.21.15.32:0/1116611160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+16298 (secure 0 0 0) 0x7ff3640b3190 con 0x7ff3640b2710 2022-08-01T23:05:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:02.976 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":146,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:02.448243+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:00.933410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:01.881408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:02.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff344053aa0 msgr2=0x7ff344055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff344053aa0 0x7ff344055f60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7ff35800b000 tx=0x7ff358006df0).stop 2022-08-01T23:05:02.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 msgr2=0x7ff3640a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:02.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 0x7ff3640a9f80 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7ff35c0032f0 tx=0x7ff35c016000).stop 2022-08-01T23:05:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 shutdown_connections 2022-08-01T23:05:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff344053aa0 0x7ff344055f60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3640b2710 0x7ff3640a9f80 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff3640b01a0 0x7ff3640a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 --2- 172.21.15.32:0/1116611160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3640a8c50 0x7ff3640aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 >> 172.21.15.32:0/1116611160 conn(0x7ff364060310 msgr2=0x7ff364148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 shutdown_connections 2022-08-01T23:05:02.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:02.972+0000 7ff368b9b700 1 -- 172.21.15.32:0/1116611160 wait complete. 2022-08-01T23:05:02.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:05:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 Processor -- start 2022-08-01T23:05:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 -- start start 2022-08-01T23:05:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4d40b3070 0x7fa4d4065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4d414b040 0x7fa4d414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d40b2c20 con 0x7fa4d40b7840 2022-08-01T23:05:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d4061830 con 0x7fa4d40b3070 2022-08-01T23:05:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4daaa3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d4065c10 con 0x7fa4d414b040 2022-08-01T23:05:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4d40b3070 0x7fa4d4065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4d40b3070 0x7fa4d4065720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49338/0 (socket says 172.21.15.32:49338) 2022-08-01T23:05:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 -- 172.21.15.32:0/3725871353 learned_addr learned my addr 172.21.15.32:0/3725871353 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:03.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4da2a2700 1 --2- 172.21.15.32:0/3725871353 >> v2:172.21.15.55:3300/0 conn(0x7fa4d414b040 0x7fa4d414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 -- 172.21.15.32:0/3725871353 >> v2:172.21.15.55:3300/0 conn(0x7fa4d414b040 msgr2=0x7fa4d414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:03.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 --2- 172.21.15.32:0/3725871353 >> v2:172.21.15.55:3300/0 conn(0x7fa4d414b040 0x7fa4d414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 -- 172.21.15.32:0/3725871353 >> v1:172.21.15.32:6789/0 conn(0x7fa4d40b7840 legacy=0x7fa4d4061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 -- 172.21.15.32:0/3725871353 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d40b7c20 con 0x7fa4d40b3070 2022-08-01T23:05:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d8a9f700 1 -- 172.21.15.32:0/3725871353 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789091061 0 0) 0x7fa4d40b2c20 con 0x7fa4d40b7840 2022-08-01T23:05:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d92a0700 1 --2- 172.21.15.32:0/3725871353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d4065720 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7fa4c4002f70 tx=0x7fa4c4006eb0).ready entity=mon.0 client_cookie=4dc9aa3c2e693fbb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d8a9f700 1 -- 172.21.15.32:0/3725871353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c400d5c0 con 0x7fa4d40b3070 2022-08-01T23:05:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d8a9f700 1 -- 172.21.15.32:0/3725871353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4000e90 con 0x7fa4d40b3070 2022-08-01T23:05:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.092+0000 7fa4d8a9f700 1 -- 172.21.15.32:0/3725871353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c400d910 con 0x7fa4d40b3070 2022-08-01T23:05:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3725871353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 msgr2=0x7fa4d4065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3725871353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d4065720 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7fa4c4002f70 tx=0x7fa4c4006eb0).stop 2022-08-01T23:05:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3725871353 shutdown_connections 2022-08-01T23:05:03.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3725871353 >> v2:172.21.15.55:3300/0 conn(0x7fa4d414b040 0x7fa4d414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3725871353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d4065720 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3725871353 >> 172.21.15.32:0/3725871353 conn(0x7fa4d405f840 msgr2=0x7fa4d405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3725871353 shutdown_connections 2022-08-01T23:05:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3725871353 wait complete. 2022-08-01T23:05:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 Processor -- start 2022-08-01T23:05:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 -- start start 2022-08-01T23:05:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d40b7840 0x7fa4d40a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.096+0000 7fa4daaa3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4d414b040 0x7fa4d40a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4daaa3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d414e530 con 0x7fa4d40b3070 2022-08-01T23:05:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4daaa3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d414e7d0 con 0x7fa4d40b7840 2022-08-01T23:05:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4daaa3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d414ea70 con 0x7fa4d414b040 2022-08-01T23:05:03.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d409ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49342/0 (socket says 172.21.15.32:49342) 2022-08-01T23:05:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 -- 172.21.15.32:0/3976220874 learned_addr learned my addr 172.21.15.32:0/3976220874 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d92a0700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d40b7840 0x7fa4d40a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4da2a2700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4d414b040 0x7fa4d40a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 -- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4d414b040 msgr2=0x7fa4d40a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4d414b040 0x7fa4d40a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 -- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d40b7840 msgr2=0x7fa4d40a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d40b7840 0x7fa4d40a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 -- 172.21.15.32:0/3976220874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c400b1e0 con 0x7fa4d40b3070 2022-08-01T23:05:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d92a0700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d40b7840 0x7fa4d40a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4d9aa1700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d409ed80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fa4d0000f40 tx=0x7fa4d0007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d0013450 con 0x7fa4d40b3070 2022-08-01T23:05:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d00135b0 con 0x7fa4d40b3070 2022-08-01T23:05:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d414ed10 con 0x7fa4d40b3070 2022-08-01T23:05:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d0013910 con 0x7fa4d40b3070 2022-08-01T23:05:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d40ae390 con 0x7fa4d40b3070 2022-08-01T23:05:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa4d0010070 con 0x7fa4d40b3070 2022-08-01T23:05:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d40b2980 con 0x7fa4d40b3070 2022-08-01T23:05:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.100+0000 7fa4caffd700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4bc053b00 0x7fa4bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.104+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa4d006b910 con 0x7fa4d40b3070 2022-08-01T23:05:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.104+0000 7fa4d92a0700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4bc053b00 0x7fa4bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.104+0000 7fa4d92a0700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4bc053b00 0x7fa4bc055fc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fa4c401c000 tx=0x7fa4c4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.108+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4d0042540 con 0x7fa4d40b3070 2022-08-01T23:05:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.272+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa4d40574e0 con 0x7fa4d40b3070 2022-08-01T23:05:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.464+0000 7fa4caffd700 1 -- 172.21.15.32:0/3976220874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa4d0045b60 con 0x7fa4d40b3070 2022-08-01T23:05:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4bc053b00 msgr2=0x7fa4bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4bc053b00 0x7fa4bc055fc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fa4c401c000 tx=0x7fa4c4015000).stop 2022-08-01T23:05:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 msgr2=0x7fa4d409ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d409ed80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fa4d0000f40 tx=0x7fa4d0007a30).stop 2022-08-01T23:05:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 shutdown_connections 2022-08-01T23:05:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4bc053b00 0x7fa4bc055fc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4d414b040 0x7fa4d40a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4d40b7840 0x7fa4d40a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 --2- 172.21.15.32:0/3976220874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d40b3070 0x7fa4d409ed80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 >> 172.21.15.32:0/3976220874 conn(0x7fa4d405f840 msgr2=0x7fa4d4066550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:03.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 shutdown_connections 2022-08-01T23:05:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.468+0000 7fa4daaa3700 1 -- 172.21.15.32:0/3976220874 wait complete. 2022-08-01T23:05:03.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:03.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.592+0000 7fb0fa85f700 1 Processor -- start 2022-08-01T23:05:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- start start 2022-08-01T23:05:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0f40b2710 0x7fb0f41401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb0f40ac880 0x7fb0f41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f4067a10 con 0x7fb0f40b0270 2022-08-01T23:05:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40ad740 con 0x7fb0f40b2710 2022-08-01T23:05:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f4067cb0 con 0x7fb0f40ac880 2022-08-01T23:05:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0f40b2710 0x7fb0f41401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0f40b2710 0x7fb0f41401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49354/0 (socket says 172.21.15.32:49354) 2022-08-01T23:05:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 -- 172.21.15.32:0/3205168794 learned_addr learned my addr 172.21.15.32:0/3205168794 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa05e700 1 --2- 172.21.15.32:0/3205168794 >> v2:172.21.15.55:3300/0 conn(0x7fb0f40ac880 0x7fb0f41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 -- 172.21.15.32:0/3205168794 >> v2:172.21.15.55:3300/0 conn(0x7fb0f40ac880 msgr2=0x7fb0f41406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 --2- 172.21.15.32:0/3205168794 >> v2:172.21.15.55:3300/0 conn(0x7fb0f40ac880 0x7fb0f41406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 -- 172.21.15.32:0/3205168794 >> v1:172.21.15.55:6789/0 conn(0x7fb0f40b0270 legacy=0x7fb0f4143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 -- 172.21.15.32:0/3205168794 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4147160 con 0x7fb0f40b2710 2022-08-01T23:05:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f985d700 1 --2- 172.21.15.32:0/3205168794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f41401e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fb0e8002f70 tx=0x7fb0e8006eb0).ready entity=mon.0 client_cookie=fdb6eaf7f120aa12 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f885b700 1 -- 172.21.15.32:0/3205168794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0e800d5c0 con 0x7fb0f40b2710 2022-08-01T23:05:03.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f885b700 1 -- 172.21.15.32:0/3205168794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e8000e90 con 0x7fb0f40b2710 2022-08-01T23:05:03.607 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0f885b700 1 -- 172.21.15.32:0/3205168794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0e800d950 con 0x7fb0f40b2710 2022-08-01T23:05:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3205168794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 msgr2=0x7fb0f41401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3205168794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f41401e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fb0e8002f70 tx=0x7fb0e8006eb0).stop 2022-08-01T23:05:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3205168794 shutdown_connections 2022-08-01T23:05:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3205168794 >> v2:172.21.15.55:3300/0 conn(0x7fb0f40ac880 0x7fb0f41406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3205168794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f41401e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3205168794 >> 172.21.15.32:0/3205168794 conn(0x7fb0f4060310 msgr2=0x7fb0f40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3205168794 shutdown_connections 2022-08-01T23:05:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.596+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3205168794 wait complete. 2022-08-01T23:05:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 Processor -- start 2022-08-01T23:05:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 -- start start 2022-08-01T23:05:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0f40ac880 0x7fb0f40ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0f40b0270 0x7fb0f4156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f4157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41568a0 con 0x7fb0f40b2710 2022-08-01T23:05:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40a7250 con 0x7fb0f40ac880 2022-08-01T23:05:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4143160 con 0x7fb0f40b0270 2022-08-01T23:05:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f4157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f4157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49360/0 (socket says 172.21.15.32:49360) 2022-08-01T23:05:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 -- 172.21.15.32:0/3749944863 learned_addr learned my addr 172.21.15.32:0/3749944863 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0f905c700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0f40b0270 0x7fb0f4156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0f985d700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0f40ac880 0x7fb0f40ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 -- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0f40b0270 msgr2=0x7fb0f4156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0f40b0270 0x7fb0f4156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 -- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0f40ac880 msgr2=0x7fb0f40ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0f40ac880 0x7fb0f40ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 -- 172.21.15.32:0/3749944863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e800b1e0 con 0x7fb0f40b2710 2022-08-01T23:05:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa05e700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f4157180 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fb0f405cf00 tx=0x7fb0ec0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0ec00eee0 con 0x7fb0f40b2710 2022-08-01T23:05:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec00c850 con 0x7fb0f40b2710 2022-08-01T23:05:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f4141020 con 0x7fb0f40b2710 2022-08-01T23:05:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0ec00cbb0 con 0x7fb0f40b2710 2022-08-01T23:05:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.600+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f4157670 con 0x7fb0f40b2710 2022-08-01T23:05:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.604+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4143360 con 0x7fb0f40b2710 2022-08-01T23:05:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.604+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb0ec00cd10 con 0x7fb0f40b2710 2022-08-01T23:05:03.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.604+0000 7fb0e6ffd700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0d4053aa0 0x7fb0d4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.604+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb0ec06c4d0 con 0x7fb0f40b2710 2022-08-01T23:05:03.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.604+0000 7fb0f985d700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0d4053aa0 0x7fb0d4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.604+0000 7fb0f985d700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0d4053aa0 0x7fb0d4055f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fb0e801a000 tx=0x7fb0e8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.608+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb0ec043180 con 0x7fb0f40b2710 2022-08-01T23:05:03.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0f41411b0 con 0x7fb0f40b2710 2022-08-01T23:05:03.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0e6ffd700 1 -- 172.21.15.32:0/3749944863 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fb0ec0467a0 con 0x7fb0f40b2710 2022-08-01T23:05:03.778 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:03.778 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:02.448548+0000","modified":"2022-08-01T23:05:03.452680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2229490987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2229490987},{"type":"v1","addr":"172.21.15.32:6835","nonce":2229490987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T23:05:03.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0d4053aa0 msgr2=0x7fb0d4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0d4053aa0 0x7fb0d4055f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fb0e801a000 tx=0x7fb0e8015000).stop 2022-08-01T23:05:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 msgr2=0x7fb0f4157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f4157180 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fb0f405cf00 tx=0x7fb0ec0091d0).stop 2022-08-01T23:05:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 shutdown_connections 2022-08-01T23:05:03.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb0d4053aa0 0x7fb0d4055f60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f40b2710 0x7fb0f4157180 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb0f40b0270 0x7fb0f4156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 --2- 172.21.15.32:0/3749944863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0f40ac880 0x7fb0f40ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 >> 172.21.15.32:0/3749944863 conn(0x7fb0f4060310 msgr2=0x7fb0f4146730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 shutdown_connections 2022-08-01T23:05:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.772+0000 7fb0fa85f700 1 -- 172.21.15.32:0/3749944863 wait complete. 2022-08-01T23:05:03.783 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2022-08-01T23:05:03.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:03.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.884+0000 7f5c0f59e700 1 Processor -- start 2022-08-01T23:05:03.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.884+0000 7f5c0f59e700 1 -- start start 2022-08-01T23:05:03.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c100b08a0 0x7f5c10142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c100b3260 0x7f5c101433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100ad000 0x7f5c1005f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100b0c80 con 0x7f5c100b08a0 2022-08-01T23:05:03.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100ae810 con 0x7f5c100b3260 2022-08-01T23:05:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100aeab0 con 0x7f5c100ad000 2022-08-01T23:05:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c100b08a0 0x7f5c10142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c100b08a0 0x7f5c10142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49364/0 (socket says 172.21.15.32:49364) 2022-08-01T23:05:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 -- 172.21.15.32:0/3181260275 learned_addr learned my addr 172.21.15.32:0/3181260275 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:03.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0ed9d700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100ad000 0x7f5c1005f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c07fff700 1 --2- 172.21.15.32:0/3181260275 >> v2:172.21.15.55:3300/0 conn(0x7f5c100b3260 0x7f5c101433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 -- 172.21.15.32:0/3181260275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100ad000 msgr2=0x7f5c1005f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:03.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100ad000 0x7f5c1005f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 -- 172.21.15.32:0/3181260275 >> v2:172.21.15.55:3300/0 conn(0x7f5c100b3260 msgr2=0x7f5c101433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:03.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/3181260275 >> v2:172.21.15.55:3300/0 conn(0x7f5c100b3260 0x7f5c101433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 -- 172.21.15.32:0/3181260275 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100aed50 con 0x7f5c100b08a0 2022-08-01T23:05:03.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0ed9d700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100ad000 0x7f5c1005f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:03.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b08a0 0x7f5c10142eb0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f5c00002f70 tx=0x7f5c00006eb0).ready entity=mon.0 client_cookie=56c86e36a6576084 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0dd9b700 1 -- 172.21.15.32:0/3181260275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0000d570 con 0x7f5c100b08a0 2022-08-01T23:05:03.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0dd9b700 1 -- 172.21.15.32:0/3181260275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00000e90 con 0x7f5c100b08a0 2022-08-01T23:05:03.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0dd9b700 1 -- 172.21.15.32:0/3181260275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0000d900 con 0x7f5c100b08a0 2022-08-01T23:05:03.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 -- 172.21.15.32:0/3181260275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b08a0 msgr2=0x7f5c10142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b08a0 0x7f5c10142eb0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f5c00002f70 tx=0x7f5c00006eb0).stop 2022-08-01T23:05:03.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 -- 172.21.15.32:0/3181260275 shutdown_connections 2022-08-01T23:05:03.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100ad000 0x7f5c1005f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/3181260275 >> v2:172.21.15.55:3300/0 conn(0x7f5c100b3260 0x7f5c101433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/3181260275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b08a0 0x7f5c10142eb0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.888+0000 7f5c0f59e700 1 -- 172.21.15.32:0/3181260275 >> 172.21.15.32:0/3181260275 conn(0x7f5c100612d0 msgr2=0x7f5c10062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:03.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- 172.21.15.32:0/3181260275 shutdown_connections 2022-08-01T23:05:03.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- 172.21.15.32:0/3181260275 wait complete. 2022-08-01T23:05:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 Processor -- start 2022-08-01T23:05:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- start start 2022-08-01T23:05:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c100ad000 0x7f5c10156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100b08a0 0x7f5c1013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 0x7f5c100a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1009f100 con 0x7f5c100b3260 2022-08-01T23:05:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100abac0 con 0x7f5c100ad000 2022-08-01T23:05:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1009ebc0 con 0x7f5c100b08a0 2022-08-01T23:05:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 0x7f5c100a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 0x7f5c100a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49374/0 (socket says 172.21.15.32:49374) 2022-08-01T23:05:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 -- 172.21.15.32:0/1741220642 learned_addr learned my addr 172.21.15.32:0/1741220642 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c100ad000 0x7f5c10156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c07fff700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100b08a0 0x7f5c1013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 -- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100b08a0 msgr2=0x7f5c1013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100b08a0 0x7f5c1013f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 -- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c100ad000 msgr2=0x7f5c10156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c100ad000 0x7f5c10156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 -- 172.21.15.32:0/1741220642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0000b1e0 con 0x7f5c100b3260 2022-08-01T23:05:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c07fff700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100b08a0 0x7f5c1013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:03.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c100ad000 0x7f5c10156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:03.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0ed9d700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 0x7f5c100a9db0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f5c08006cc0 tx=0x7f5c08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0800c770 con 0x7f5c100b3260 2022-08-01T23:05:03.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800c8d0 con 0x7f5c100b3260 2022-08-01T23:05:03.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1009ee60 con 0x7f5c100b3260 2022-08-01T23:05:03.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0800cc30 con 0x7f5c100b3260 2022-08-01T23:05:03.913 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.892+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c100aabd0 con 0x7f5c100b3260 2022-08-01T23:05:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.896+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c100a9aa0 con 0x7f5c100b3260 2022-08-01T23:05:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.896+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c0801d460 con 0x7f5c100b3260 2022-08-01T23:05:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.896+0000 7f5c077fe700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5be8053950 0x7f5be8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.896+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5c0806c580 con 0x7f5c100b3260 2022-08-01T23:05:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.896+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5be8053950 0x7f5be8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.896+0000 7f5c0e59c700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5be8053950 0x7f5be8055e10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f5c0001a000 tx=0x7f5c00015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:03.904+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c080432e0 con 0x7f5c100b3260 2022-08-01T23:05:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.028+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c100574e0 con 0x7f5c100b3260 2022-08-01T23:05:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c077fe700 1 -- 172.21.15.32:0/1741220642 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f5c0800ee60 con 0x7f5c100b3260 2022-08-01T23:05:04.037 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:04.038 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":147,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:03.451546+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:00.933410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:01.881408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5be8053950 msgr2=0x7f5be8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5be8053950 0x7f5be8055e10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f5c0001a000 tx=0x7f5c00015000).stop 2022-08-01T23:05:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 msgr2=0x7f5c100a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 0x7f5c100a9db0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f5c08006cc0 tx=0x7f5c08008040).stop 2022-08-01T23:05:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 shutdown_connections 2022-08-01T23:05:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5be8053950 0x7f5be8055e10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c100b3260 0x7f5c100a9db0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5c100b08a0 0x7f5c1013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 --2- 172.21.15.32:0/1741220642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c100ad000 0x7f5c10156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 >> 172.21.15.32:0/1741220642 conn(0x7f5c100612d0 msgr2=0x7f5c10142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 shutdown_connections 2022-08-01T23:05:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.032+0000 7f5c0f59e700 1 -- 172.21.15.32:0/1741220642 wait complete. 2022-08-01T23:05:04.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:05:04.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 Processor -- start 2022-08-01T23:05:04.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 -- start start 2022-08-01T23:05:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0614b0 0x7f487c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f487c0ad750 0x7f487c0b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c0aee80 con 0x7f487c1405d0 2022-08-01T23:05:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c0b70b0 con 0x7f487c0ad750 2022-08-01T23:05:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f4880c0b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0af0b0 con 0x7f487c0614b0 2022-08-01T23:05:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f487c0ad750 0x7f487c0b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f487c1405d0 0x7f487c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36920/0 (socket says 172.21.15.32:36920) 2022-08-01T23:05:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487bfff700 1 -- 172.21.15.32:0/2824849853 learned_addr learned my addr 172.21.15.32:0/2824849853 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487b7fe700 1 --2- 172.21.15.32:0/2824849853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0614b0 0x7f487c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487affd700 1 -- 172.21.15.32:0/2824849853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0614b0 msgr2=0x7f487c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487affd700 1 --2- 172.21.15.32:0/2824849853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0614b0 0x7f487c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487affd700 1 -- 172.21.15.32:0/2824849853 >> v1:172.21.15.32:6789/0 conn(0x7f487c1405d0 legacy=0x7f487c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487a7fc700 1 -- 172.21.15.32:0/2824849853 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749028401 0 0) 0x7f487c0aee80 con 0x7f487c1405d0 2022-08-01T23:05:04.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.156+0000 7f487affd700 1 -- 172.21.15.32:0/2824849853 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0af350 con 0x7f487c0ad750 2022-08-01T23:05:04.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f487affd700 1 --2- 172.21.15.32:0/2824849853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0ad750 0x7f487c0b9ff0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f4864002f70 tx=0x7f4864006df0).ready entity=mon.0 client_cookie=4b0d69c60f0238d1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f487a7fc700 1 -- 172.21.15.32:0/2824849853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f486400d570 con 0x7f487c0ad750 2022-08-01T23:05:04.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f487a7fc700 1 -- 172.21.15.32:0/2824849853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864000e90 con 0x7f487c0ad750 2022-08-01T23:05:04.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f487a7fc700 1 -- 172.21.15.32:0/2824849853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f486400d8c0 con 0x7f487c0ad750 2022-08-01T23:05:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 -- 172.21.15.32:0/2824849853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0ad750 msgr2=0x7f487c0b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 --2- 172.21.15.32:0/2824849853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0ad750 0x7f487c0b9ff0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f4864002f70 tx=0x7f4864006df0).stop 2022-08-01T23:05:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 -- 172.21.15.32:0/2824849853 shutdown_connections 2022-08-01T23:05:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 --2- 172.21.15.32:0/2824849853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0ad750 0x7f487c0b9ff0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 --2- 172.21.15.32:0/2824849853 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0614b0 0x7f487c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 -- 172.21.15.32:0/2824849853 >> 172.21.15.32:0/2824849853 conn(0x7f487c066f70 msgr2=0x7f487c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 -- 172.21.15.32:0/2824849853 shutdown_connections 2022-08-01T23:05:04.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 -- 172.21.15.32:0/2824849853 wait complete. 2022-08-01T23:05:04.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 Processor -- start 2022-08-01T23:05:04.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.160+0000 7f4880c0b700 1 -- start start 2022-08-01T23:05:04.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 0x7f487c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0ad750 0x7f487c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f487c1405d0 0x7f487c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1457e0 con 0x7f487c0614b0 2022-08-01T23:05:04.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0b0340 con 0x7f487c1405d0 2022-08-01T23:05:04.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0b05e0 con 0x7f487c0ad750 2022-08-01T23:05:04.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 0x7f487c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 0x7f487c13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49382/0 (socket says 172.21.15.32:49382) 2022-08-01T23:05:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 -- 172.21.15.32:0/3854787527 learned_addr learned my addr 172.21.15.32:0/3854787527 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487affd700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0ad750 0x7f487c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487bfff700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f487c1405d0 0x7f487c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 -- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0ad750 msgr2=0x7f487c0b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0ad750 0x7f487c0b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 -- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f487c1405d0 msgr2=0x7f487c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f487c1405d0 0x7f487c0b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 -- 172.21.15.32:0/3854787527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486400b180 con 0x7f487c0614b0 2022-08-01T23:05:04.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f487b7fe700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 0x7f487c13fc00 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f487400c000 tx=0x7f4874006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4874010780 con 0x7f487c0614b0 2022-08-01T23:05:04.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740108e0 con 0x7f487c0614b0 2022-08-01T23:05:04.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c0b0880 con 0x7f487c0614b0 2022-08-01T23:05:04.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4874010c40 con 0x7f487c0614b0 2022-08-01T23:05:04.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c0b1980 con 0x7f487c0614b0 2022-08-01T23:05:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.164+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c0af5f0 con 0x7f487c0614b0 2022-08-01T23:05:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.168+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f487401c460 con 0x7f487c0614b0 2022-08-01T23:05:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.168+0000 7f4878ff9700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4860053b00 0x7f4860055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.168+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f487406c490 con 0x7f487c0614b0 2022-08-01T23:05:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.168+0000 7f487affd700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4860053b00 0x7f4860055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.168+0000 7f487affd700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4860053b00 0x7f4860055fc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f486401a000 tx=0x7f4864014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.172+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48740430e0 con 0x7f487c0614b0 2022-08-01T23:05:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.336+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f487c13dd50 con 0x7f487c0614b0 2022-08-01T23:05:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.344+0000 7f4878ff9700 1 -- 172.21.15.32:0/3854787527 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4874046700 con 0x7f487c0614b0 2022-08-01T23:05:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4860053b00 msgr2=0x7f4860055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4860053b00 0x7f4860055fc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f486401a000 tx=0x7f4864014000).stop 2022-08-01T23:05:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 msgr2=0x7f487c13fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 0x7f487c13fc00 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f487400c000 tx=0x7f4874006df0).stop 2022-08-01T23:05:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 shutdown_connections 2022-08-01T23:05:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4860053b00 0x7f4860055fc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f487c1405d0 0x7f487c0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f487c0ad750 0x7f487c0b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 --2- 172.21.15.32:0/3854787527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0614b0 0x7f487c13fc00 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 >> 172.21.15.32:0/3854787527 conn(0x7f487c066f70 msgr2=0x7f487c146880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 shutdown_connections 2022-08-01T23:05:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.348+0000 7f4880c0b700 1 -- 172.21.15.32:0/3854787527 wait complete. 2022-08-01T23:05:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:05:04.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:05:04.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 Processor -- start 2022-08-01T23:05:04.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 -- start start 2022-08-01T23:05:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90040b3190 0x7f9004060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9004061190 0x7f9004066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90040654c0 con 0x7f90040b2730 2022-08-01T23:05:04.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9004062190 con 0x7f9004061190 2022-08-01T23:05:04.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9004061600 con 0x7f90040b3190 2022-08-01T23:05:04.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f9002d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9004061190 0x7f9004066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f8ffbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f90040b2730 0x7f90040666b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36934/0 (socket says 172.21.15.32:36934) 2022-08-01T23:05:04.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f8ffbfff700 1 -- 172.21.15.32:0/4093832678 learned_addr learned my addr 172.21.15.32:0/4093832678 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.476+0000 7f900259c700 1 --2- 172.21.15.32:0/4093832678 >> v2:172.21.15.55:3300/0 conn(0x7f90040b3190 0x7f9004060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9002d9d700 1 -- 172.21.15.32:0/4093832678 >> v2:172.21.15.55:3300/0 conn(0x7f90040b3190 msgr2=0x7f9004060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9002d9d700 1 --2- 172.21.15.32:0/4093832678 >> v2:172.21.15.55:3300/0 conn(0x7f90040b3190 0x7f9004060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9002d9d700 1 -- 172.21.15.32:0/4093832678 >> v1:172.21.15.32:6789/0 conn(0x7f90040b2730 legacy=0x7f90040666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9002d9d700 1 -- 172.21.15.32:0/4093832678 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040b3570 con 0x7f9004061190 2022-08-01T23:05:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9002d9d700 1 --2- 172.21.15.32:0/4093832678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9004061190 0x7f9004066dc0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f8ffc0032f0 tx=0x7f8ffc0091d0).ready entity=mon.0 client_cookie=12572339e437313b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9001d9b700 1 -- 172.21.15.32:0/4093832678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ffc00b680 con 0x7f9004061190 2022-08-01T23:05:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9001d9b700 1 -- 172.21.15.32:0/4093832678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00b7e0 con 0x7f9004061190 2022-08-01T23:05:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9001d9b700 1 -- 172.21.15.32:0/4093832678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ffc00b950 con 0x7f9004061190 2022-08-01T23:05:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- 172.21.15.32:0/4093832678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9004061190 msgr2=0x7f9004066dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 --2- 172.21.15.32:0/4093832678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9004061190 0x7f9004066dc0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f8ffc0032f0 tx=0x7f8ffc0091d0).stop 2022-08-01T23:05:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- 172.21.15.32:0/4093832678 shutdown_connections 2022-08-01T23:05:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 --2- 172.21.15.32:0/4093832678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9004061190 0x7f9004066dc0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 --2- 172.21.15.32:0/4093832678 >> v2:172.21.15.55:3300/0 conn(0x7f90040b3190 0x7f9004060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- 172.21.15.32:0/4093832678 >> 172.21.15.32:0/4093832678 conn(0x7f9004063870 msgr2=0x7f9004063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- 172.21.15.32:0/4093832678 shutdown_connections 2022-08-01T23:05:04.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- 172.21.15.32:0/4093832678 wait complete. 2022-08-01T23:05:04.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 Processor -- start 2022-08-01T23:05:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- start start 2022-08-01T23:05:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9004061190 0x7f90040b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 0x7f90041563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90040b3190 0x7f9004152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040b1460 con 0x7f90040b2730 2022-08-01T23:05:04.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040a3e50 con 0x7f90040b3190 2022-08-01T23:05:04.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90040a40f0 con 0x7f9004061190 2022-08-01T23:05:04.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 0x7f90041563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 0x7f90041563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49398/0 (socket says 172.21.15.32:49398) 2022-08-01T23:05:04.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f8ffbfff700 1 -- 172.21.15.32:0/775428795 learned_addr learned my addr 172.21.15.32:0/775428795 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:04.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f9002d9d700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90040b3190 0x7f9004152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.480+0000 7f900259c700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9004061190 0x7f90040b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffbfff700 1 -- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9004061190 msgr2=0x7f90040b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffbfff700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9004061190 0x7f90040b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffbfff700 1 -- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90040b3190 msgr2=0x7f9004152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffbfff700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90040b3190 0x7f9004152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffbfff700 1 -- 172.21.15.32:0/775428795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc009470 con 0x7f90040b2730 2022-08-01T23:05:04.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffbfff700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 0x7f90041563a0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f90040634e0 tx=0x7f8fec007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fec0070d0 con 0x7f90040b2730 2022-08-01T23:05:04.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec007230 con 0x7f90040b2730 2022-08-01T23:05:04.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90040a4390 con 0x7f90040b2730 2022-08-01T23:05:04.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fec005610 con 0x7f90040b2730 2022-08-01T23:05:04.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9004152470 con 0x7f90040b2730 2022-08-01T23:05:04.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900405fbf0 con 0x7f90040b2730 2022-08-01T23:05:04.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8fec010070 con 0x7f90040b2730 2022-08-01T23:05:04.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffb7fe700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fe8053a70 0x7f8fe8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.484+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8fec06b480 con 0x7f90040b2730 2022-08-01T23:05:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.488+0000 7f900259c700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fe8053a70 0x7f8fe8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.488+0000 7f900259c700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fe8053a70 0x7f8fe8055f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8ff400b000 tx=0x7f8ff4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.492+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8fec042240 con 0x7f90040b2730 2022-08-01T23:05:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.656+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f90040574e0 con 0x7f90040b2730 2022-08-01T23:05:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.664+0000 7f8ffb7fe700 1 -- 172.21.15.32:0/775428795 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8fec045860 con 0x7f90040b2730 2022-08-01T23:05:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fe8053a70 msgr2=0x7f8fe8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fe8053a70 0x7f8fe8055f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8ff400b000 tx=0x7f8ff4006df0).stop 2022-08-01T23:05:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 msgr2=0x7f90041563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 0x7f90041563a0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f90040634e0 tx=0x7f8fec007bd0).stop 2022-08-01T23:05:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 shutdown_connections 2022-08-01T23:05:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8fe8053a70 0x7f8fe8055f30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90040b3190 0x7f9004152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90040b2730 0x7f90041563a0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 --2- 172.21.15.32:0/775428795 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9004061190 0x7f90040b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 >> 172.21.15.32:0/775428795 conn(0x7f9004063870 msgr2=0x7f9004142b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 shutdown_connections 2022-08-01T23:05:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.668+0000 7f900359e700 1 -- 172.21.15.32:0/775428795 wait complete. 2022-08-01T23:05:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:05:04.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 Processor -- start 2022-08-01T23:05:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 -- start start 2022-08-01T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b2710 0x7f838c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f838c0b01a0 0x7f838c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c0a7250 con 0x7f838c0afd60 2022-08-01T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0a7480 con 0x7f838c0b01a0 2022-08-01T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.780+0000 7f838b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0a7720 con 0x7f838c0b2710 2022-08-01T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f838c0b01a0 0x7f838c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f838c0b01a0 0x7f838c14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:59190/0 (socket says 172.21.15.32:59190) 2022-08-01T23:05:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 -- 172.21.15.32:0/1468221671 learned_addr learned my addr 172.21.15.32:0/1468221671 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838a59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b2710 0x7f838c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 -- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b2710 msgr2=0x7f838c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 --2- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b2710 0x7f838c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 -- 172.21.15.32:0/1468221671 >> v1:172.21.15.55:6789/0 conn(0x7f838c0afd60 legacy=0x7f838c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 -- 172.21.15.32:0/1468221671 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0a79c0 con 0x7f838c0b01a0 2022-08-01T23:05:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f8389d9b700 1 --2- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b01a0 0x7f838c14b4c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f837c0032f0 tx=0x7f837c009290).ready entity=mon.1 client_cookie=7b8924e76388e861 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838959a700 1 -- 172.21.15.32:0/1468221671 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c00b8a0 con 0x7f838c0b01a0 2022-08-01T23:05:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838959a700 1 -- 172.21.15.32:0/1468221671 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c00ba00 con 0x7f838c0b01a0 2022-08-01T23:05:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838959a700 1 -- 172.21.15.32:0/1468221671 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f837c00bb70 con 0x7f838c0b01a0 2022-08-01T23:05:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 -- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b01a0 msgr2=0x7f838c14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 --2- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b01a0 0x7f838c14b4c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f837c0032f0 tx=0x7f837c009290).stop 2022-08-01T23:05:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 -- 172.21.15.32:0/1468221671 shutdown_connections 2022-08-01T23:05:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 --2- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b01a0 0x7f838c14b4c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 --2- 172.21.15.32:0/1468221671 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b2710 0x7f838c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 -- 172.21.15.32:0/1468221671 >> 172.21.15.32:0/1468221671 conn(0x7f838c060310 msgr2=0x7f838c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 -- 172.21.15.32:0/1468221671 shutdown_connections 2022-08-01T23:05:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.784+0000 7f838b59e700 1 -- 172.21.15.32:0/1468221671 wait complete. 2022-08-01T23:05:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 Processor -- start 2022-08-01T23:05:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 -- start start 2022-08-01T23:05:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0afd60 0x7f838c148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b01a0 0x7f838c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 0x7f838c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0b40c0 con 0x7f838c0b2710 2022-08-01T23:05:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0b42f0 con 0x7f838c0afd60 2022-08-01T23:05:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0b4590 con 0x7f838c0b01a0 2022-08-01T23:05:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 0x7f838c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 0x7f838c15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49414/0 (socket says 172.21.15.32:49414) 2022-08-01T23:05:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 -- 172.21.15.32:0/3179237486 learned_addr learned my addr 172.21.15.32:0/3179237486 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0afd60 0x7f838c148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f8389d9b700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b01a0 0x7f838c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 -- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b01a0 msgr2=0x7f838c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b01a0 0x7f838c0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 -- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0afd60 msgr2=0x7f838c148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0afd60 0x7f838c148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 -- 172.21.15.32:0/3179237486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c009530 con 0x7f838c0b2710 2022-08-01T23:05:04.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838ad9d700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 0x7f838c15a010 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f838c061c10 tx=0x7f8384007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8384012400 con 0x7f838c0b2710 2022-08-01T23:05:04.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384012560 con 0x7f838c0b2710 2022-08-01T23:05:04.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c159b40 con 0x7f838c0b2710 2022-08-01T23:05:04.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83840128c0 con 0x7f838c0b2710 2022-08-01T23:05:04.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.788+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c0b5fd0 con 0x7f838c0b2710 2022-08-01T23:05:04.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.792+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c0b5680 con 0x7f838c0b2710 2022-08-01T23:05:04.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.792+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8384019030 con 0x7f838c0b2710 2022-08-01T23:05:04.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.792+0000 7f837b7fe700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8368053b30 0x7f8368055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:04.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.792+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f838406b360 con 0x7f838c0b2710 2022-08-01T23:05:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.792+0000 7f838a59c700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8368053b30 0x7f8368055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.792+0000 7f838a59c700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8368053b30 0x7f8368055ff0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f838000b000 tx=0x7f8380006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.796+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f838c0b5680 con 0x7f838c0b2710 2022-08-01T23:05:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.960+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f838c0b5680 con 0x7f838c0b2710 2022-08-01T23:05:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.960+0000 7f837b7fe700 1 -- 172.21.15.32:0/3179237486 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f838c0b5680 con 0x7f838c0b2710 2022-08-01T23:05:04.967 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:04.967 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:02.448548+0000","modified":"2022-08-01T23:05:03.452680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2229490987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2229490987},{"type":"v1","addr":"172.21.15.32:6835","nonce":2229490987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T23:05:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8368053b30 msgr2=0x7f8368055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8368053b30 0x7f8368055ff0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f838000b000 tx=0x7f8380006df0).stop 2022-08-01T23:05:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 msgr2=0x7f838c15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 0x7f838c15a010 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f838c061c10 tx=0x7f8384007bc0).stop 2022-08-01T23:05:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 shutdown_connections 2022-08-01T23:05:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8368053b30 0x7f8368055ff0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b2710 0x7f838c15a010 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f838c0b01a0 0x7f838c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 --2- 172.21.15.32:0/3179237486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0afd60 0x7f838c148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 >> 172.21.15.32:0/3179237486 conn(0x7f838c060310 msgr2=0x7f838c14a210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 shutdown_connections 2022-08-01T23:05:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:04.964+0000 7f838b59e700 1 -- 172.21.15.32:0/3179237486 wait complete. 2022-08-01T23:05:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2022-08-01T23:05:05.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:05:02.448548+0000', 'modified': '2022-08-01T23:05:03.452680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2229490987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2229490987}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2229490987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:05:05.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:05:05.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:05:05.401 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8427,"want_state":"up:active","state":"up:active","rank_uptime":2.9382177220000001,"mdsmap_epoch":74,"osdmap_epoch":147,"osdmap_epoch_barrier":147,"uptime":7.2641926610000001} 2022-08-01T23:05:05.402 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8427, "mdsmap_epoch": 74, "osdmap_epoch": 147, "osdmap_epoch_barrier": 147, "rank_uptime": 2.938217722, "state": "up:active", "uptime": 7.264192661, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:05:05.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 Processor -- start 2022-08-01T23:05:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 -- start start 2022-08-01T23:05:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b28b0 0x7f661013e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66100b1180 0x7f661013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6610141210 0x7f66101416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6610140e20 con 0x7f66100b1180 2022-08-01T23:05:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66100aac20 con 0x7f6610141210 2022-08-01T23:05:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f66153f3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100b2c90 con 0x7f66100b28b0 2022-08-01T23:05:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f660f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66100b1180 0x7f661013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f660f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66100b1180 0x7f661013e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49418/0 (socket says 172.21.15.32:49418) 2022-08-01T23:05:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f660f7fe700 1 -- 172.21.15.32:0/1685409884 learned_addr learned my addr 172.21.15.32:0/1685409884 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f6614bf2700 1 --2- 172.21.15.32:0/1685409884 >> v2:172.21.15.55:3300/0 conn(0x7f6610141210 0x7f66101416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.504+0000 7f660ffff700 1 --2- 172.21.15.32:0/1685409884 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b28b0 0x7f661013e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660f7fe700 1 -- 172.21.15.32:0/1685409884 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b28b0 msgr2=0x7f661013e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660f7fe700 1 --2- 172.21.15.32:0/1685409884 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b28b0 0x7f661013e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660f7fe700 1 -- 172.21.15.32:0/1685409884 >> v2:172.21.15.55:3300/0 conn(0x7f6610141210 msgr2=0x7f66101416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660f7fe700 1 --2- 172.21.15.32:0/1685409884 >> v2:172.21.15.55:3300/0 conn(0x7f6610141210 0x7f66101416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660f7fe700 1 -- 172.21.15.32:0/1685409884 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100aa440 con 0x7f66100b1180 2022-08-01T23:05:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660f7fe700 1 --2- 172.21.15.32:0/1685409884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66100b1180 0x7f661013e6b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f6604002f70 tx=0x7f6604006eb0).ready entity=mon.0 client_cookie=eaee8d517b49512d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660effd700 1 -- 172.21.15.32:0/1685409884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f660400d570 con 0x7f66100b1180 2022-08-01T23:05:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660effd700 1 -- 172.21.15.32:0/1685409884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604000e90 con 0x7f66100b1180 2022-08-01T23:05:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f660effd700 1 -- 172.21.15.32:0/1685409884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f660400d900 con 0x7f66100b1180 2022-08-01T23:05:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 -- 172.21.15.32:0/1685409884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66100b1180 msgr2=0x7f661013e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 --2- 172.21.15.32:0/1685409884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66100b1180 0x7f661013e6b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f6604002f70 tx=0x7f6604006eb0).stop 2022-08-01T23:05:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 -- 172.21.15.32:0/1685409884 shutdown_connections 2022-08-01T23:05:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 --2- 172.21.15.32:0/1685409884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66100b1180 0x7f661013e6b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 --2- 172.21.15.32:0/1685409884 >> v2:172.21.15.55:3300/0 conn(0x7f6610141210 0x7f66101416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 --2- 172.21.15.32:0/1685409884 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b28b0 0x7f661013e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 -- 172.21.15.32:0/1685409884 >> 172.21.15.32:0/1685409884 conn(0x7f6610060310 msgr2=0x7f66100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 -- 172.21.15.32:0/1685409884 shutdown_connections 2022-08-01T23:05:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 -- 172.21.15.32:0/1685409884 wait complete. 2022-08-01T23:05:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 Processor -- start 2022-08-01T23:05:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.508+0000 7f66153f3700 1 -- start start 2022-08-01T23:05:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b1180 0x7f66100ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66100b28b0 0x7f661005c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 0x7f66100b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661005ce60 con 0x7f6610141210 2022-08-01T23:05:05.525 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661013c830 con 0x7f66100b28b0 2022-08-01T23:05:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661013d030 con 0x7f66100b1180 2022-08-01T23:05:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 0x7f66100b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f660ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b1180 0x7f66100ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.526 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 0x7f66100b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49426/0 (socket says 172.21.15.32:49426) 2022-08-01T23:05:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f660ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b1180 0x7f66100ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:32930/0 (socket says 172.21.15.32:32930) 2022-08-01T23:05:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 -- 172.21.15.32:0/2354573542 learned_addr learned my addr 172.21.15.32:0/2354573542 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f660f7fe700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66100b28b0 0x7f661005c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.527 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 -- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b1180 msgr2=0x7f66100ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b1180 0x7f66100ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 -- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66100b28b0 msgr2=0x7f661005c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66100b28b0 0x7f661005c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 -- 172.21.15.32:0/2354573542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660400b1e0 con 0x7f6610141210 2022-08-01T23:05:05.528 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f6614bf2700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 0x7f66100b3780 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f6608006e00 tx=0x7f66080091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6608012600 con 0x7f6610141210 2022-08-01T23:05:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608012760 con 0x7f6610141210 2022-08-01T23:05:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661013d2d0 con 0x7f6610141210 2022-08-01T23:05:05.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6608012ac0 con 0x7f6610141210 2022-08-01T23:05:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.512+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6610146730 con 0x7f6610141210 2022-08-01T23:05:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.516+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661013c9a0 con 0x7f6610141210 2022-08-01T23:05:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.516+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6608022070 con 0x7f6610141210 2022-08-01T23:05:05.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.516+0000 7f660d7fa700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65f00539e0 0x7f65f0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.516+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f660806c660 con 0x7f6610141210 2022-08-01T23:05:05.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.516+0000 7f660ffff700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65f00539e0 0x7f65f0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.531 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.516+0000 7f660ffff700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65f00539e0 0x7f65f0055ea0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f660000c000 tx=0x7f6600008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:05.532 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.524+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f661013c9a0 con 0x7f6610141210 2022-08-01T23:05:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.656+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f661013c9a0 con 0x7f6610141210 2022-08-01T23:05:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.656+0000 7f660d7fa700 1 -- 172.21.15.32:0/2354573542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f661013c9a0 con 0x7f6610141210 2022-08-01T23:05:05.664 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:05.665 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":147,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:03.451546+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:00.933410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:01.881408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65f00539e0 msgr2=0x7f65f0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65f00539e0 0x7f65f0055ea0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f660000c000 tx=0x7f6600008040).stop 2022-08-01T23:05:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 msgr2=0x7f66100b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 0x7f66100b3780 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f6608006e00 tx=0x7f66080091d0).stop 2022-08-01T23:05:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 shutdown_connections 2022-08-01T23:05:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f65f00539e0 0x7f65f0055ea0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6610141210 0x7f66100b3780 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66100b28b0 0x7f661005c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 --2- 172.21.15.32:0/2354573542 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f66100b1180 0x7f66100ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 >> 172.21.15.32:0/2354573542 conn(0x7f6610060310 msgr2=0x7f66100af550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 shutdown_connections 2022-08-01T23:05:05.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.660+0000 7f66153f3700 1 -- 172.21.15.32:0/2354573542 wait complete. 2022-08-01T23:05:05.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:05:00.933410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:05:01.881408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:05:05.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:05:05.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2340a90700 1 Processor -- start 2022-08-01T23:05:05.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2340a90700 1 -- start start 2022-08-01T23:05:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2340a90700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1580 0x7f233c0b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2340a90700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0ae7e0 con 0x7f233c0b1ef0 2022-08-01T23:05:05.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2340a90700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0b1960 con 0x7f233c147450 2022-08-01T23:05:05.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2340a90700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c0b2470 con 0x7f233c0b1580 2022-08-01T23:05:05.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2339d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f233c0b1ef0 0x7f233c146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36976/0 (socket says 172.21.15.32:36976) 2022-08-01T23:05:05.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f2339d9b700 1 -- 172.21.15.32:0/3348274295 learned_addr learned my addr 172.21.15.32:0/3348274295 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:05.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233a59c700 1 --2- 172.21.15.32:0/3348274295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1580 0x7f233c0b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078352344 0 0) 0x7f233c0ae7e0 con 0x7f233c0b1ef0 2022-08-01T23:05:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c003260 con 0x7f233c0b1ef0 2022-08-01T23:05:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214138286 0 0) 0x7f233c0b1960 con 0x7f233c147450 2022-08-01T23:05:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f233c0ae7e0 con 0x7f233c147450 2022-08-01T23:05:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2201879694 0 0) 0x7f231c003260 con 0x7f233c0b1ef0 2022-08-01T23:05:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233a59c700 1 -- 172.21.15.32:0/3348274295 >> v1:172.21.15.55:6789/0 conn(0x7f233c147450 legacy=0x7f233c143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233a59c700 1 -- 172.21.15.32:0/3348274295 >> v1:172.21.15.32:6789/0 conn(0x7f233c0b1ef0 legacy=0x7f233c146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.784+0000 7f233a59c700 1 -- 172.21.15.32:0/3348274295 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0b2710 con 0x7f233c0b1580 2022-08-01T23:05:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f233a59c700 1 --2- 172.21.15.32:0/3348274295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1580 0x7f233c0b3210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f2324001110 tx=0x7f232c0091d0).ready entity=mon.2 client_cookie=573ab457c1428983 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2324001660 con 0x7f233c0b1580 2022-08-01T23:05:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00f510 con 0x7f233c0b1580 2022-08-01T23:05:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f233959a700 1 -- 172.21.15.32:0/3348274295 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f232c00f680 con 0x7f233c0b1580 2022-08-01T23:05:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 -- 172.21.15.32:0/3348274295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1580 msgr2=0x7f233c0b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 --2- 172.21.15.32:0/3348274295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1580 0x7f233c0b3210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f2324001110 tx=0x7f232c0091d0).stop 2022-08-01T23:05:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 -- 172.21.15.32:0/3348274295 shutdown_connections 2022-08-01T23:05:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 --2- 172.21.15.32:0/3348274295 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1580 0x7f233c0b3210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 -- 172.21.15.32:0/3348274295 >> 172.21.15.32:0/3348274295 conn(0x7f233c060950 msgr2=0x7f233c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 -- 172.21.15.32:0/3348274295 shutdown_connections 2022-08-01T23:05:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 -- 172.21.15.32:0/3348274295 wait complete. 2022-08-01T23:05:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 Processor -- start 2022-08-01T23:05:05.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.788+0000 7f2340a90700 1 -- start start 2022-08-01T23:05:05.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f233c0b1580 0x7f233c067380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1ef0 0x7f233c064ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 0x7f233c064fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c064660 con 0x7f233c147450 2022-08-01T23:05:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c14ada0 con 0x7f233c0b1580 2022-08-01T23:05:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c065ac0 con 0x7f233c0b1ef0 2022-08-01T23:05:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 0x7f233c064fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 0x7f233c064fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49440/0 (socket says 172.21.15.32:49440) 2022-08-01T23:05:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 -- 172.21.15.32:0/4111936213 learned_addr learned my addr 172.21.15.32:0/4111936213 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2339d9b700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1ef0 0x7f233c064ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233a59c700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f233c0b1580 0x7f233c067380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 -- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1ef0 msgr2=0x7f233c064ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1ef0 0x7f233c064ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 -- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f233c0b1580 msgr2=0x7f233c067380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f233c0b1580 0x7f233c067380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 -- 172.21.15.32:0/4111936213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c009470 con 0x7f233c147450 2022-08-01T23:05:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233a59c700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f233c0b1580 0x7f233c067380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f233ad9d700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 0x7f233c064fb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f2334001110 tx=0x7f2334007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2334012400 con 0x7f233c147450 2022-08-01T23:05:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334012560 con 0x7f233c147450 2022-08-01T23:05:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c152920 con 0x7f233c147450 2022-08-01T23:05:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23340128c0 con 0x7f233c147450 2022-08-01T23:05:05.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c063f70 con 0x7f233c147450 2022-08-01T23:05:05.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2334019030 con 0x7f233c147450 2022-08-01T23:05:05.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.792+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c0b29b0 con 0x7f233c147450 2022-08-01T23:05:05.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.796+0000 7f23337fe700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2320053b30 0x7f2320055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:05.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.796+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f233406b3d0 con 0x7f233c147450 2022-08-01T23:05:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.796+0000 7f233a59c700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2320053b30 0x7f2320055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.796+0000 7f233a59c700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2320053b30 0x7f2320055ff0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f232c019000 tx=0x7f232c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.800+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2334042190 con 0x7f233c147450 2022-08-01T23:05:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f233c0574e0 con 0x7f233c147450 2022-08-01T23:05:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f23337fe700 1 -- 172.21.15.32:0/4111936213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v147) v1 ==== 110+0+10 (secure 0 0 0) 0x7f23340457b0 con 0x7f233c147450 2022-08-01T23:05:05.935 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:05:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2320053b30 msgr2=0x7f2320055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2320053b30 0x7f2320055ff0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f232c019000 tx=0x7f232c016000).stop 2022-08-01T23:05:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 msgr2=0x7f233c064fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 0x7f233c064fb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f2334001110 tx=0x7f2334007a50).stop 2022-08-01T23:05:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 shutdown_connections 2022-08-01T23:05:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2320053b30 0x7f2320055ff0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233c147450 0x7f233c064fb0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.938 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f233c0b1ef0 0x7f233c064ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 --2- 172.21.15.32:0/4111936213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f233c0b1580 0x7f233c067380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 >> 172.21.15.32:0/4111936213 conn(0x7f233c060950 msgr2=0x7f233c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 shutdown_connections 2022-08-01T23:05:05.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:05.928+0000 7f2340a90700 1 -- 172.21.15.32:0/4111936213 wait complete. 2022-08-01T23:05:05.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:05:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 Processor -- start 2022-08-01T23:05:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 -- start start 2022-08-01T23:05:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2fdc0b2710 0x7f2fdc14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc0a7250 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc0a7480 con 0x7f2fdc0b2710 2022-08-01T23:05:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fe09b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0a7720 con 0x7f2fdc0a8c50 2022-08-01T23:05:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2fdc0b2710 0x7f2fdc14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:32950/0 (socket says 172.21.15.32:32950) 2022-08-01T23:05:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 -- 172.21.15.32:0/602739155 learned_addr learned my addr 172.21.15.32:0/602739155 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdaffd700 1 --1- 172.21.15.32:0/602739155 >> v1:172.21.15.55:6789/0 conn(0x7f2fdc0b01a0 0x7f2fdc0a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:57892/0 (socket says 172.21.15.32:57892) 2022-08-01T23:05:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fda7fc700 1 -- 172.21.15.32:0/602739155 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914704985 0 0) 0x7f2fdc0a7250 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 -- 172.21.15.32:0/602739155 >> v2:172.21.15.55:3300/0 conn(0x7f2fdc0b2710 msgr2=0x7f2fdc14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 --2- 172.21.15.32:0/602739155 >> v2:172.21.15.55:3300/0 conn(0x7f2fdc0b2710 0x7f2fdc14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 -- 172.21.15.32:0/602739155 >> v1:172.21.15.55:6789/0 conn(0x7f2fdc0b01a0 legacy=0x7f2fdc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 -- 172.21.15.32:0/602739155 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc0a79c0 con 0x7f2fdc0a8c50 2022-08-01T23:05:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.052+0000 7f2fdbfff700 1 --2- 172.21.15.32:0/602739155 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc14b4c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f2fd40032f0 tx=0x7f2fd4009290).ready entity=mon.2 client_cookie=ec4d5c1c6482fa43 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fda7fc700 1 -- 172.21.15.32:0/602739155 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd400b8a0 con 0x7f2fdc0a8c50 2022-08-01T23:05:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fda7fc700 1 -- 172.21.15.32:0/602739155 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400ba00 con 0x7f2fdc0a8c50 2022-08-01T23:05:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fda7fc700 1 -- 172.21.15.32:0/602739155 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd400bb70 con 0x7f2fdc0a8c50 2022-08-01T23:05:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- 172.21.15.32:0/602739155 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 msgr2=0x7f2fdc14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/602739155 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc14b4c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f2fd40032f0 tx=0x7f2fd4009290).stop 2022-08-01T23:05:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- 172.21.15.32:0/602739155 shutdown_connections 2022-08-01T23:05:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/602739155 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc14b4c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/602739155 >> v2:172.21.15.55:3300/0 conn(0x7f2fdc0b2710 0x7f2fdc14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- 172.21.15.32:0/602739155 >> 172.21.15.32:0/602739155 conn(0x7f2fdc060310 msgr2=0x7f2fdc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- 172.21.15.32:0/602739155 shutdown_connections 2022-08-01T23:05:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- 172.21.15.32:0/602739155 wait complete. 2022-08-01T23:05:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 Processor -- start 2022-08-01T23:05:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- start start 2022-08-01T23:05:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 0x7f2fdc0abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fdc0b2710 0x7f2fdc0ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0657a0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc064dc0 con 0x7f2fdc0b2710 2022-08-01T23:05:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fe09b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc065060 con 0x7f2fdc0a8c50 2022-08-01T23:05:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.056+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 0x7f2fdc0abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 0x7f2fdc0abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49452/0 (socket says 172.21.15.32:49452) 2022-08-01T23:05:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 -- 172.21.15.32:0/66043586 learned_addr learned my addr 172.21.15.32:0/66043586 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdb7fe700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdbfff700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fdc0b2710 0x7f2fdc0ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 -- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 msgr2=0x7f2fdc0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc0ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 -- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fdc0b2710 msgr2=0x7f2fdc0ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fdc0b2710 0x7f2fdc0ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 -- 172.21.15.32:0/66043586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009530 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 0x7f2fdc0abeb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f2fcc000bc0 tx=0x7f2fcc007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fcc00ce80 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0070d0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc065300 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fcc007430 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc0aab90 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc0af1d0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2fcc010070 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fd8ff9700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fc0053b30 0x7f2fc0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2fcc06b5b0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.060+0000 7f2fdb7fe700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fc0053b30 0x7f2fc0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.064+0000 7f2fdb7fe700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fc0053b30 0x7f2fc0055ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f2fc400b000 tx=0x7f2fc4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.068+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fcc0422d0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.192+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2fdc0574e0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fd8ff9700 1 -- 172.21.15.32:0/66043586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v147) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2fcc0458f0 con 0x7f2fdc0b01a0 2022-08-01T23:05:06.201 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fc0053b30 msgr2=0x7f2fc0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fc0053b30 0x7f2fc0055ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f2fc400b000 tx=0x7f2fc4006df0).stop 2022-08-01T23:05:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 msgr2=0x7f2fdc0abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 0x7f2fdc0abeb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f2fcc000bc0 tx=0x7f2fcc007a00).stop 2022-08-01T23:05:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 shutdown_connections 2022-08-01T23:05:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2fc0053b30 0x7f2fc0055ff0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fdc0b2710 0x7f2fdc0ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0b01a0 0x7f2fdc0abeb0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 --2- 172.21.15.32:0/66043586 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2fdc0a8c50 0x7f2fdc0ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 >> 172.21.15.32:0/66043586 conn(0x7f2fdc060310 msgr2=0x7f2fdc148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 shutdown_connections 2022-08-01T23:05:06.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.196+0000 7f2fe09b4700 1 -- 172.21.15.32:0/66043586 wait complete. 2022-08-01T23:05:06.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:05:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.316+0000 7fb228ce3700 1 Processor -- start 2022-08-01T23:05:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.316+0000 7fb228ce3700 1 -- start start 2022-08-01T23:05:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.316+0000 7fb228ce3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb224060330 0x7fb2240b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240aa630 con 0x7fb22405c6f0 2022-08-01T23:05:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22405fd10 con 0x7fb2240af550 2022-08-01T23:05:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240b15e0 con 0x7fb224060330 2022-08-01T23:05:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb223fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb22405c6f0 0x7fb2240a8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37002/0 (socket says 172.21.15.32:37002) 2022-08-01T23:05:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb223fff700 1 -- 172.21.15.32:0/2084083816 learned_addr learned my addr 172.21.15.32:0/2084083816 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb21bfff700 1 --2- 172.21.15.32:0/2084083816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb224060330 0x7fb2240b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979420951 0 0) 0x7fb2240aa630 con 0x7fb22405c6f0 2022-08-01T23:05:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb208003260 con 0x7fb22405c6f0 2022-08-01T23:05:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850425767 0 0) 0x7fb22405fd10 con 0x7fb2240af550 2022-08-01T23:05:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2240aa630 con 0x7fb2240af550 2022-08-01T23:05:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3332408258 0 0) 0x7fb208003260 con 0x7fb22405c6f0 2022-08-01T23:05:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb21bfff700 1 -- 172.21.15.32:0/2084083816 >> v1:172.21.15.55:6789/0 conn(0x7fb2240af550 legacy=0x7fb2241445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb21bfff700 1 -- 172.21.15.32:0/2084083816 >> v1:172.21.15.32:6789/0 conn(0x7fb22405c6f0 legacy=0x7fb2240a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb21bfff700 1 -- 172.21.15.32:0/2084083816 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22405f840 con 0x7fb224060330 2022-08-01T23:05:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb21bfff700 1 --2- 172.21.15.32:0/2084083816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb224060330 0x7fb2240b0e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fb21c001b80 tx=0x7fb20c006df0).ready entity=mon.2 client_cookie=65a59d1021404722 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb21c0019e0 con 0x7fb224060330 2022-08-01T23:05:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c000e90 con 0x7fb224060330 2022-08-01T23:05:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb222ffd700 1 -- 172.21.15.32:0/2084083816 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb20c012460 con 0x7fb224060330 2022-08-01T23:05:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- 172.21.15.32:0/2084083816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb224060330 msgr2=0x7fb2240b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 --2- 172.21.15.32:0/2084083816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb224060330 0x7fb2240b0e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fb21c001b80 tx=0x7fb20c006df0).stop 2022-08-01T23:05:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- 172.21.15.32:0/2084083816 shutdown_connections 2022-08-01T23:05:06.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 --2- 172.21.15.32:0/2084083816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb224060330 0x7fb2240b0e40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- 172.21.15.32:0/2084083816 >> 172.21.15.32:0/2084083816 conn(0x7fb2240612d0 msgr2=0x7fb224062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- 172.21.15.32:0/2084083816 shutdown_connections 2022-08-01T23:05:06.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.320+0000 7fb228ce3700 1 -- 172.21.15.32:0/2084083816 wait complete. 2022-08-01T23:05:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 Processor -- start 2022-08-01T23:05:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 -- start start 2022-08-01T23:05:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb22405c6f0 0x7fb22413f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 0x7fb2240652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2240af550 0x7fb2240657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224143bb0 con 0x7fb224060330 2022-08-01T23:05:06.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224143e50 con 0x7fb2240af550 2022-08-01T23:05:06.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb228ce3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240673a0 con 0x7fb22405c6f0 2022-08-01T23:05:06.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 0x7fb2240652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 0x7fb2240652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49468/0 (socket says 172.21.15.32:49468) 2022-08-01T23:05:06.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 -- 172.21.15.32:0/1629776242 learned_addr learned my addr 172.21.15.32:0/1629776242 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:06.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb223fff700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2240af550 0x7fb2240657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb2237fe700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb22405c6f0 0x7fb22413f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 -- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb22405c6f0 msgr2=0x7fb22413f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb22405c6f0 0x7fb22413f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 -- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2240af550 msgr2=0x7fb2240657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2240af550 0x7fb2240657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.324+0000 7fb21bfff700 1 -- 172.21.15.32:0/1629776242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c00b180 con 0x7fb224060330 2022-08-01T23:05:06.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb21bfff700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 0x7fb2240652e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fb21c001b80 tx=0x7fb20c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb20c00d920 con 0x7fb224060330 2022-08-01T23:05:06.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c00da80 con 0x7fb224060330 2022-08-01T23:05:06.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb224144570 con 0x7fb224060330 2022-08-01T23:05:06.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb20c00dde0 con 0x7fb224060330 2022-08-01T23:05:06.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2241411d0 con 0x7fb224060330 2022-08-01T23:05:06.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb20c01c070 con 0x7fb224060330 2022-08-01T23:05:06.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2217fa700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb1fc053a70 0x7fb1fc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb224067200 con 0x7fb224060330 2022-08-01T23:05:06.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb20c06c8c0 con 0x7fb224060330 2022-08-01T23:05:06.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2237fe700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb1fc053a70 0x7fb1fc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.328+0000 7fb2237fe700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb1fc053a70 0x7fb1fc055f30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fb21400d000 tx=0x7fb214007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.332+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb224067200 con 0x7fb224060330 2022-08-01T23:05:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.456+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb224067200 con 0x7fb224060330 2022-08-01T23:05:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb2217fa700 1 -- 172.21.15.32:0/1629776242 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb224067200 con 0x7fb224060330 2022-08-01T23:05:06.465 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb1fc053a70 msgr2=0x7fb1fc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb1fc053a70 0x7fb1fc055f30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fb21400d000 tx=0x7fb214007a20).stop 2022-08-01T23:05:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 msgr2=0x7fb2240652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 0x7fb2240652e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fb21c001b80 tx=0x7fb20c014000).stop 2022-08-01T23:05:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 shutdown_connections 2022-08-01T23:05:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb1fc053a70 0x7fb1fc055f30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2240af550 0x7fb2240657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb224060330 0x7fb2240652e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 --2- 172.21.15.32:0/1629776242 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb22405c6f0 0x7fb22413f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 >> 172.21.15.32:0/1629776242 conn(0x7fb2240612d0 msgr2=0x7fb2240ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 shutdown_connections 2022-08-01T23:05:06.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.460+0000 7fb228ce3700 1 -- 172.21.15.32:0/1629776242 wait complete. 2022-08-01T23:05:06.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:05:06.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.584+0000 7fb3ee1e6700 1 Processor -- start 2022-08-01T23:05:06.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.584+0000 7fb3ee1e6700 1 -- start start 2022-08-01T23:05:06.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ee1e6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3e805c960 0x7fb3e80b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ee1e6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e805cd40 con 0x7fb3e813f520 2022-08-01T23:05:06.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ee1e6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80b1920 con 0x7fb3e805d240 2022-08-01T23:05:06.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ee1e6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e80ad6d0 con 0x7fb3e805c960 2022-08-01T23:05:06.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed9e5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3e813f520 0x7fb3e80aee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37018/0 (socket says 172.21.15.32:37018) 2022-08-01T23:05:06.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed9e5700 1 -- 172.21.15.32:0/757947285 learned_addr learned my addr 172.21.15.32:0/757947285 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:06.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/757947285 >> v2:172.21.15.55:3300/0 conn(0x7fb3e805c960 0x7fb3e80b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756318657 0 0) 0x7fb3e805cd40 con 0x7fb3e813f520 2022-08-01T23:05:06.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3cc003260 con 0x7fb3e813f520 2022-08-01T23:05:06.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068982451 0 0) 0x7fb3e80b1920 con 0x7fb3e805d240 2022-08-01T23:05:06.613 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e805cd40 con 0x7fb3e805d240 2022-08-01T23:05:06.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886095243 0 0) 0x7fb3cc003260 con 0x7fb3e813f520 2022-08-01T23:05:06.614 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 >> v2:172.21.15.55:3300/0 conn(0x7fb3e805c960 msgr2=0x7fb3e80b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:06.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 --2- 172.21.15.32:0/757947285 >> v2:172.21.15.55:3300/0 conn(0x7fb3e805c960 0x7fb3e80b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 >> v1:172.21.15.55:6789/0 conn(0x7fb3e805d240 legacy=0x7fb3e80acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e80b1bc0 con 0x7fb3e813f520 2022-08-01T23:05:06.615 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fb3e00019e0 con 0x7fb3e813f520 2022-08-01T23:05:06.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 >> v1:172.21.15.32:6789/0 conn(0x7fb3e813f520 legacy=0x7fb3e80aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3cc003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3cc004500 0x7fb3cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.616 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3cc004e70 0x7fb3cc00c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80ad6d0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc00ce00 con 0x7fb3cc004e70 2022-08-01T23:05:06.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc00cfc0 con 0x7fb3cc004500 2022-08-01T23:05:06.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3cc003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed9e5700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3cc004e70 0x7fb3cc00c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ec9e3700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3cc004500 0x7fb3cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3cc004500 msgr2=0x7fb3cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:06.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3cc004500 0x7fb3cc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3cc004e70 msgr2=0x7fb3cc00c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:06.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3cc004e70 0x7fb3cc00c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/757947285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc003260 con 0x7fb3cc0038e0 2022-08-01T23:05:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed9e5700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3cc004e70 0x7fb3cc00c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.588+0000 7fb3ed9e5700 1 -- 172.21.15.32:0/757947285 reap_dead start 2022-08-01T23:05:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3cc003ce0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7fb3dc00c000 tx=0x7fb3dc006df0).ready entity=mon.0 client_cookie=42c7e31516fcae2b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3dc00e800 con 0x7fb3cc0038e0 2022-08-01T23:05:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc00e960 con 0x7fb3cc0038e0 2022-08-01T23:05:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3d3fff700 1 -- 172.21.15.32:0/757947285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3dc00ecc0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/757947285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 msgr2=0x7fb3cc003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3cc003ce0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7fb3dc00c000 tx=0x7fb3dc006df0).stop 2022-08-01T23:05:06.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/757947285 shutdown_connections 2022-08-01T23:05:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/757947285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3cc003ce0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/757947285 >> 172.21.15.32:0/757947285 conn(0x7fb3e8060310 msgr2=0x7fb3e80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/757947285 shutdown_connections 2022-08-01T23:05:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.592+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/757947285 wait complete. 2022-08-01T23:05:06.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 Processor -- start 2022-08-01T23:05:06.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 -- start start 2022-08-01T23:05:06.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3e80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3e80a92d0 0x7fb3e8150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e8142e30 0x7fb3e80aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8150fb0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80a9770 con 0x7fb3e8142e30 2022-08-01T23:05:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8150380 con 0x7fb3e80a92d0 2022-08-01T23:05:06.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3e80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3e80b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49484/0 (socket says 172.21.15.32:49484) 2022-08-01T23:05:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/3156087783 learned_addr learned my addr 172.21.15.32:0/3156087783 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ec9e3700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3e80a92d0 0x7fb3e8150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed9e5700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e8142e30 0x7fb3e80aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3e80a92d0 msgr2=0x7fb3e8150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3e80a92d0 0x7fb3e8150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e8142e30 msgr2=0x7fb3e80aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e8142e30 0x7fb3e80aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 -- 172.21.15.32:0/3156087783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e8150620 con 0x7fb3cc0038e0 2022-08-01T23:05:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ec9e3700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3e80a92d0 0x7fb3e8150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed9e5700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e8142e30 0x7fb3e80aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ed1e4700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3e80b54e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7fb3dc00c000 tx=0x7fb3dc019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3dc01cdc0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc000e90 con 0x7fb3cc0038e0 2022-08-01T23:05:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e8150070 con 0x7fb3cc0038e0 2022-08-01T23:05:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3dc0245e0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.596+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e8063e30 con 0x7fb3cc0038e0 2022-08-01T23:05:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.600+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e81507b0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.600+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb3dc024740 con 0x7fb3cc0038e0 2022-08-01T23:05:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.600+0000 7fb3daffd700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3c4053aa0 0x7fb3c4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.600+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb3dc06d800 con 0x7fb3cc0038e0 2022-08-01T23:05:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.600+0000 7fb3ec9e3700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3c4053aa0 0x7fb3c4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.600+0000 7fb3ec9e3700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3c4053aa0 0x7fb3c4055f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb3e400d000 tx=0x7fb3e4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.631 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.608+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb3dc047ff0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.732+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3e80574e0 con 0x7fb3cc0038e0 2022-08-01T23:05:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.732+0000 7fb3daffd700 1 -- 172.21.15.32:0/3156087783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb3dc00be60 con 0x7fb3cc0038e0 2022-08-01T23:05:06.740 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3c4053aa0 msgr2=0x7fb3c4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3c4053aa0 0x7fb3c4055f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fb3e400d000 tx=0x7fb3e4009040).stop 2022-08-01T23:05:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 msgr2=0x7fb3e80b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3e80b54e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7fb3dc00c000 tx=0x7fb3dc019000).stop 2022-08-01T23:05:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 shutdown_connections 2022-08-01T23:05:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb3c4053aa0 0x7fb3c4055f60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e8142e30 0x7fb3e80aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb3e80a92d0 0x7fb3e8150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 --2- 172.21.15.32:0/3156087783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0038e0 0x7fb3e80b54e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 >> 172.21.15.32:0/3156087783 conn(0x7fb3e8060310 msgr2=0x7fb3e80af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 shutdown_connections 2022-08-01T23:05:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.736+0000 7fb3ee1e6700 1 -- 172.21.15.32:0/3156087783 wait complete. 2022-08-01T23:05:06.758 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:05:06.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4756f4e700 1 Processor -- start 2022-08-01T23:05:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4756f4e700 1 -- start start 2022-08-01T23:05:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4756f4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47500b2710 0x7f475014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4756f4e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500a7250 con 0x7f47500b01a0 2022-08-01T23:05:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4756f4e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500a7480 con 0x7f47500a8c50 2022-08-01T23:05:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4756f4e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500a7720 con 0x7f47500b2710 2022-08-01T23:05:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f475574b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f47500b01a0 0x7f47500a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37034/0 (socket says 172.21.15.32:37034) 2022-08-01T23:05:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f475574b700 1 -- 172.21.15.32:0/1508316600 learned_addr learned my addr 172.21.15.32:0/1508316600 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4755f4c700 1 --2- 172.21.15.32:0/1508316600 >> v2:172.21.15.32:3300/0 conn(0x7f47500b2710 0x7f475014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4754f4a700 1 -- 172.21.15.32:0/1508316600 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986369485 0 0) 0x7f47500a7250 con 0x7f47500b01a0 2022-08-01T23:05:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.856+0000 7f4754f4a700 1 -- 172.21.15.32:0/1508316600 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4738003260 con 0x7f47500b01a0 2022-08-01T23:05:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4755f4c700 1 -- 172.21.15.32:0/1508316600 >> v1:172.21.15.55:6789/0 conn(0x7f47500a8c50 legacy=0x7f4750151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4755f4c700 1 -- 172.21.15.32:0/1508316600 >> v1:172.21.15.32:6789/0 conn(0x7f47500b01a0 legacy=0x7f47500a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4755f4c700 1 -- 172.21.15.32:0/1508316600 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500a79c0 con 0x7f47500b2710 2022-08-01T23:05:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4755f4c700 1 --2- 172.21.15.32:0/1508316600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b2710 0x7f475014afd0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f4740002f70 tx=0x7f4740006df0).ready entity=mon.0 client_cookie=3fab3aea86a60d8a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4754f4a700 1 -- 172.21.15.32:0/1508316600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474000d610 con 0x7f47500b2710 2022-08-01T23:05:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4754f4a700 1 -- 172.21.15.32:0/1508316600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740000e90 con 0x7f47500b2710 2022-08-01T23:05:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4754f4a700 1 -- 172.21.15.32:0/1508316600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474000d960 con 0x7f47500b2710 2022-08-01T23:05:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 -- 172.21.15.32:0/1508316600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b2710 msgr2=0x7f475014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 --2- 172.21.15.32:0/1508316600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b2710 0x7f475014afd0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f4740002f70 tx=0x7f4740006df0).stop 2022-08-01T23:05:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 -- 172.21.15.32:0/1508316600 shutdown_connections 2022-08-01T23:05:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 --2- 172.21.15.32:0/1508316600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b2710 0x7f475014afd0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 -- 172.21.15.32:0/1508316600 >> 172.21.15.32:0/1508316600 conn(0x7f4750060310 msgr2=0x7f47500b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 -- 172.21.15.32:0/1508316600 shutdown_connections 2022-08-01T23:05:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 -- 172.21.15.32:0/1508316600 wait complete. 2022-08-01T23:05:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 Processor -- start 2022-08-01T23:05:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.860+0000 7f4756f4e700 1 -- start start 2022-08-01T23:05:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47500a8c50 0x7f47500ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 0x7f47500b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f47500b2710 0x7f47500b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b2f60 con 0x7f47500b01a0 2022-08-01T23:05:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b3190 con 0x7f47500a8c50 2022-08-01T23:05:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b3430 con 0x7f47500b2710 2022-08-01T23:05:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 0x7f47500b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 0x7f47500b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49500/0 (socket says 172.21.15.32:49500) 2022-08-01T23:05:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 -- 172.21.15.32:0/3278378957 learned_addr learned my addr 172.21.15.32:0/3278378957 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475674d700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f47500b2710 0x7f47500b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4755f4c700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47500a8c50 0x7f47500ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 -- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f47500b2710 msgr2=0x7f47500b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f47500b2710 0x7f47500b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 -- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47500a8c50 msgr2=0x7f47500ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:06.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47500a8c50 0x7f47500ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:06.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 -- 172.21.15.32:0/3278378957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474000b180 con 0x7f47500b01a0 2022-08-01T23:05:06.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f475574b700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 0x7f47500b3a10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f474c001490 tx=0x7f474c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474c005c70 con 0x7f47500b01a0 2022-08-01T23:05:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c005dd0 con 0x7f47500b01a0 2022-08-01T23:05:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474c0072c0 con 0x7f47500b01a0 2022-08-01T23:05:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47500af270 con 0x7f47500b01a0 2022-08-01T23:05:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47500b3f00 con 0x7f47500b01a0 2022-08-01T23:05:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.864+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47500b35f0 con 0x7f47500b01a0 2022-08-01T23:05:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.868+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f474c010070 con 0x7f47500b01a0 2022-08-01T23:05:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.868+0000 7f473effd700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4730053aa0 0x7f4730055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.868+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f474c06ba70 con 0x7f47500b01a0 2022-08-01T23:05:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.868+0000 7f4755f4c700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4730053aa0 0x7f4730055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.868+0000 7f4755f4c700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4730053aa0 0x7f4730055f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f474001a000 tx=0x7f4740014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:06.872+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f474c042790 con 0x7f47500b01a0 2022-08-01T23:05:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.032+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47500af400 con 0x7f47500b01a0 2022-08-01T23:05:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f473effd700 1 -- 172.21.15.32:0/3278378957 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f474c06be60 con 0x7f47500b01a0 2022-08-01T23:05:07.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:07.041 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:02.448548+0000","modified":"2022-08-01T23:05:03.452680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2229490987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2229490987},{"type":"v1","addr":"172.21.15.32:6835","nonce":2229490987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T23:05:07.042 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4730053aa0 msgr2=0x7f4730055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:07.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4730053aa0 0x7f4730055f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f474001a000 tx=0x7f4740014000).stop 2022-08-01T23:05:07.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 msgr2=0x7f47500b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:07.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 0x7f47500b3a10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f474c001490 tx=0x7f474c007e00).stop 2022-08-01T23:05:07.043 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 shutdown_connections 2022-08-01T23:05:07.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4730053aa0 0x7f4730055f60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:07.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f47500b2710 0x7f47500b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:07.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47500b01a0 0x7f47500b3a10 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:07.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 --2- 172.21.15.32:0/3278378957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47500a8c50 0x7f47500ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:07.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 >> 172.21.15.32:0/3278378957 conn(0x7f4750060310 msgr2=0x7f475014aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:07.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 shutdown_connections 2022-08-01T23:05:07.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:07.036+0000 7f4756f4e700 1 -- 172.21.15.32:0/3278378957 wait complete. 2022-08-01T23:05:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2022-08-01T23:05:07.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:05:02.448548+0000', 'modified': '2022-08-01T23:05:03.452680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2229490987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2229490987}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2229490987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:05:07.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:05:07.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:05:07.474 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8427,"want_state":"up:active","state":"up:active","rank_uptime":5.0111732299999998,"mdsmap_epoch":74,"osdmap_epoch":147,"osdmap_epoch_barrier":147,"uptime":9.3371528690000005} 2022-08-01T23:05:07.475 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8427, "mdsmap_epoch": 74, "osdmap_epoch": 147, "osdmap_epoch_barrier": 147, "rank_uptime": 5.01117323, "state": "up:active", "uptime": 9.337152869, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:05:07.475 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:05:07.476 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:05:07.476 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:05:07.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:07.477 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:07.486 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:05:07.545 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:07.545 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:07.559 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:07.560 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:07.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:07.562 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:05:07.568 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:05:07.568 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:07.571 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:05:07.587 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:07.587 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:07.643 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:07.644 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:07.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:07.645 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:05:07.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:05:07.641+0000 7f2e13687080 -1 init, newargv = 0x564707685710 newargc=9ceph-fuse[ 2022-08-01T23:05:07.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:23641]: starting ceph client 2022-08-01T23:05:07.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23641]: starting fuse 2022-08-01T23:05:08.752 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:05:08.771 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:08.771 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:08.827 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:08.829 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:08.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:08.832 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:05:08.839 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:05:08.839 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:05:08.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:08.840 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:05:08.841 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:05:08.841 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:05:08.841 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:05:08.842 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:05:08.842 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:08.843 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:05:08.843 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:05:08.844 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:05:08.844 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:08.844 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:05:08.845 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:05:08.845 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:05:08.846 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:05:08.846 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:08.846 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:05:08.847 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:05:08.847 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:05:08.848 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:05:08.848 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:05:08.849 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:08.849 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:05:08.849 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:05:08.892 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23641.asok 2022-08-01T23:05:08.897 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23641.asok 2022-08-01T23:05:08.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:08.898 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23641.asok status 2022-08-01T23:05:09.031 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:05:09.031 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:05:09.031 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:05:09.032 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:05:09.032 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:05:09.032 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:05:09.032 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:05:09.033 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23641", 2022-08-01T23:05:09.033 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:05:09.033 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:09.033 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:05:09.033 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:05:09.034 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 8595, 2022-08-01T23:05:09.034 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:05:09.034 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:05:09.034 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:05:09.035 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 8595 2022-08-01T23:05:09.035 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:09.035 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:05:09.035 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:05:09.036 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:05:09.036 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 675535042 2022-08-01T23:05:09.036 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:05:09.036 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:09.036 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:05:09.037 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:05:09.037 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:05:09.037 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 675535042 2022-08-01T23:05:09.037 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:09.037 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.8595 172.21.15.130:0/675535042", 2022-08-01T23:05:09.038 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/675535042", 2022-08-01T23:05:09.038 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:05:09.038 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 74, 2022-08-01T23:05:09.038 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 147, 2022-08-01T23:05:09.039 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:05:09.039 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:05:09.039 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:05:09.039 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:05:09.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:09.041 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:09.041 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:09.049 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:05:09.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:05:09.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:09.051 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:09.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333b59e700 1 Processor -- start 2022-08-01T23:05:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333b59e700 1 -- start start 2022-08-01T23:05:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0aee80 con 0x7f333c1405d0 2022-08-01T23:05:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0b70b0 con 0x7f333c0614b0 2022-08-01T23:05:09.222 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0af0b0 con 0x7f333c0ad750 2022-08-01T23:05:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.212+0000 7f333ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f333c1405d0 0x7f333c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37052/0 (socket says 172.21.15.32:37052) 2022-08-01T23:05:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333ad9d700 1 -- 172.21.15.32:0/3246151149 learned_addr learned my addr 172.21.15.32:0/3246151149 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f3339d9b700 1 --2- 172.21.15.32:0/3246151149 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.223 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333959a700 1 -- 172.21.15.32:0/3246151149 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237501223 0 0) 0x7f333c0aee80 con 0x7f333c1405d0 2022-08-01T23:05:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333959a700 1 -- 172.21.15.32:0/3246151149 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320003260 con 0x7f333c1405d0 2022-08-01T23:05:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f3339d9b700 1 -- 172.21.15.32:0/3246151149 >> v1:172.21.15.55:6789/0 conn(0x7f333c0614b0 legacy=0x7f333c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333959a700 1 -- 172.21.15.32:0/3246151149 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312310713 0 0) 0x7f333c0b70b0 con 0x7f333c0614b0 2022-08-01T23:05:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f3339d9b700 1 -- 172.21.15.32:0/3246151149 >> v1:172.21.15.32:6789/0 conn(0x7f333c1405d0 legacy=0x7f333c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f3339d9b700 1 -- 172.21.15.32:0/3246151149 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0af350 con 0x7f333c0ad750 2022-08-01T23:05:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f3339d9b700 1 --2- 172.21.15.32:0/3246151149 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c061890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f332c002f70 tx=0x7f332c006df0).ready entity=mon.2 client_cookie=66cb0d1a9e680ad8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333959a700 1 -- 172.21.15.32:0/3246151149 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f332c00d570 con 0x7f333c0ad750 2022-08-01T23:05:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333959a700 1 -- 172.21.15.32:0/3246151149 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c000e90 con 0x7f333c0ad750 2022-08-01T23:05:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333959a700 1 -- 172.21.15.32:0/3246151149 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f332c00d8c0 con 0x7f333c0ad750 2022-08-01T23:05:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 -- 172.21.15.32:0/3246151149 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 msgr2=0x7f333c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 --2- 172.21.15.32:0/3246151149 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c061890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f332c002f70 tx=0x7f332c006df0).stop 2022-08-01T23:05:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 -- 172.21.15.32:0/3246151149 shutdown_connections 2022-08-01T23:05:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 --2- 172.21.15.32:0/3246151149 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c061890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 -- 172.21.15.32:0/3246151149 >> 172.21.15.32:0/3246151149 conn(0x7f333c066f70 msgr2=0x7f333c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 -- 172.21.15.32:0/3246151149 shutdown_connections 2022-08-01T23:05:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.216+0000 7f333b59e700 1 -- 172.21.15.32:0/3246151149 wait complete. 2022-08-01T23:05:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 Processor -- start 2022-08-01T23:05:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 -- start start 2022-08-01T23:05:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 0x7f333c13e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f333c1405d0 0x7f333c13d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c141990 con 0x7f333c0614b0 2022-08-01T23:05:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c142820 con 0x7f333c1405d0 2022-08-01T23:05:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c144890 con 0x7f333c0ad750 2022-08-01T23:05:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 0x7f333c13e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 0x7f333c13e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49512/0 (socket says 172.21.15.32:49512) 2022-08-01T23:05:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333a59c700 1 -- 172.21.15.32:0/3317185903 learned_addr learned my addr 172.21.15.32:0/3317185903 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f3339d9b700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c13cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.220+0000 7f333ad9d700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f333c1405d0 0x7f333c13d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333a59c700 1 -- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 msgr2=0x7f333c13cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333a59c700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c13cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333a59c700 1 -- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f333c1405d0 msgr2=0x7f333c13d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333a59c700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f333c1405d0 0x7f333c13d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333a59c700 1 -- 172.21.15.32:0/3317185903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c00b180 con 0x7f333c0614b0 2022-08-01T23:05:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333ad9d700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f333c1405d0 0x7f333c13d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333a59c700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 0x7f333c13e900 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f3330000ed0 tx=0x7f3330007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3330012400 con 0x7f333c0614b0 2022-08-01T23:05:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3330012560 con 0x7f333c0614b0 2022-08-01T23:05:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c142110 con 0x7f333c0614b0 2022-08-01T23:05:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33300128c0 con 0x7f333c0614b0 2022-08-01T23:05:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c141250 con 0x7f333c0614b0 2022-08-01T23:05:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c141690 con 0x7f333c0614b0 2022-08-01T23:05:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.224+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3330019030 con 0x7f333c0614b0 2022-08-01T23:05:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.228+0000 7f332b7fe700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3318053a10 0x7f3318055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.228+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f333006b300 con 0x7f333c0614b0 2022-08-01T23:05:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.228+0000 7f3339d9b700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3318053a10 0x7f3318055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.228+0000 7f3339d9b700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3318053a10 0x7f3318055ed0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f332c01a000 tx=0x7f332c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.232+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3330042170 con 0x7f333c0614b0 2022-08-01T23:05:09.363 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.356+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f333c0af5f0 con 0x7f333c0614b0 2022-08-01T23:05:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f332b7fe700 1 -- 172.21.15.32:0/3317185903 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f3330045790 con 0x7f333c0614b0 2022-08-01T23:05:09.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:09.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":147,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:03.451546+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:00.933410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:01.881408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3318053a10 msgr2=0x7f3318055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3318053a10 0x7f3318055ed0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f332c01a000 tx=0x7f332c014000).stop 2022-08-01T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 msgr2=0x7f333c13e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 0x7f333c13e900 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f3330000ed0 tx=0x7f3330007ac0).stop 2022-08-01T23:05:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 shutdown_connections 2022-08-01T23:05:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3318053a10 0x7f3318055ed0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f333c1405d0 0x7f333c13d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f333c0ad750 0x7f333c13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 --2- 172.21.15.32:0/3317185903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f333c0614b0 0x7f333c13e900 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 >> 172.21.15.32:0/3317185903 conn(0x7f333c066f70 msgr2=0x7f333c0b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 shutdown_connections 2022-08-01T23:05:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.360+0000 7f333b59e700 1 -- 172.21.15.32:0/3317185903 wait complete. 2022-08-01T23:05:09.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:05:00.933410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:05:01.881408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:05:09.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:05:09.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbce000e700 1 Processor -- start 2022-08-01T23:05:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbce000e700 1 -- start start 2022-08-01T23:05:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbce000e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbcd80b1b10 0x7fbcd80b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbce000e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd8140220 con 0x7fbcd8060d00 2022-08-01T23:05:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbce000e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd81404c0 con 0x7fbcd8061860 2022-08-01T23:05:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbce000e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd8061130 con 0x7fbcd80b1b10 2022-08-01T23:05:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcde5ab700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbcd8060d00 0x7fbcd805f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37066/0 (socket says 172.21.15.32:37066) 2022-08-01T23:05:09.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcde5ab700 1 -- 172.21.15.32:0/72291448 learned_addr learned my addr 172.21.15.32:0/72291448 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:09.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcdddaa700 1 --2- 172.21.15.32:0/72291448 >> v2:172.21.15.55:3300/0 conn(0x7fbcd80b1b10 0x7fbcd80b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcdcda8700 1 -- 172.21.15.32:0/72291448 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450033096 0 0) 0x7fbcd8140220 con 0x7fbcd8060d00 2022-08-01T23:05:09.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcdcda8700 1 -- 172.21.15.32:0/72291448 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc0003260 con 0x7fbcd8060d00 2022-08-01T23:05:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcdddaa700 1 -- 172.21.15.32:0/72291448 >> v1:172.21.15.55:6789/0 conn(0x7fbcd8061860 legacy=0x7fbcd8139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcdddaa700 1 -- 172.21.15.32:0/72291448 >> v1:172.21.15.32:6789/0 conn(0x7fbcd8060d00 legacy=0x7fbcd805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.488+0000 7fbcdddaa700 1 -- 172.21.15.32:0/72291448 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8140760 con 0x7fbcd80b1b10 2022-08-01T23:05:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbcdddaa700 1 --2- 172.21.15.32:0/72291448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd80b1ef0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fbcd4002f70 tx=0x7fbcd4006df0).ready entity=mon.1 client_cookie=209f511f7490f4b8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbcdcda8700 1 -- 172.21.15.32:0/72291448 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcd400d610 con 0x7fbcd80b1b10 2022-08-01T23:05:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbcdcda8700 1 -- 172.21.15.32:0/72291448 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd4000e90 con 0x7fbcd80b1b10 2022-08-01T23:05:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbcdcda8700 1 -- 172.21.15.32:0/72291448 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcd400d960 con 0x7fbcd80b1b10 2022-08-01T23:05:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 -- 172.21.15.32:0/72291448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 msgr2=0x7fbcd80b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 --2- 172.21.15.32:0/72291448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd80b1ef0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fbcd4002f70 tx=0x7fbcd4006df0).stop 2022-08-01T23:05:09.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 -- 172.21.15.32:0/72291448 shutdown_connections 2022-08-01T23:05:09.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 --2- 172.21.15.32:0/72291448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd80b1ef0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 -- 172.21.15.32:0/72291448 >> 172.21.15.32:0/72291448 conn(0x7fbcd80b2b00 msgr2=0x7fbcd80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:09.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 -- 172.21.15.32:0/72291448 shutdown_connections 2022-08-01T23:05:09.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 -- 172.21.15.32:0/72291448 wait complete. 2022-08-01T23:05:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 Processor -- start 2022-08-01T23:05:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.492+0000 7fbce000e700 1 -- start start 2022-08-01T23:05:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 0x7fbcd813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcd8061860 0x7fbcd8150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd8150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8154310 con 0x7fbcd8060d00 2022-08-01T23:05:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd81547e0 con 0x7fbcd80b1b10 2022-08-01T23:05:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8154540 con 0x7fbcd8061860 2022-08-01T23:05:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 0x7fbcd813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 0x7fbcd813cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49526/0 (socket says 172.21.15.32:49526) 2022-08-01T23:05:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 -- 172.21.15.32:0/2892176965 learned_addr learned my addr 172.21.15.32:0/2892176965 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcde5ab700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd8150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdd5a9700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcd8061860 0x7fbcd8150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 -- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcd8061860 msgr2=0x7fbcd8150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcd8061860 0x7fbcd8150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 -- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 msgr2=0x7fbcd8150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd8150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 -- 172.21.15.32:0/2892176965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd400b180 con 0x7fbcd8060d00 2022-08-01T23:05:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcde5ab700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd8150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcdddaa700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 0x7fbcd813cd00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fbcd4002f70 tx=0x7fbcd4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcd400d290 con 0x7fbcd8060d00 2022-08-01T23:05:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd81504b0 con 0x7fbcd8060d00 2022-08-01T23:05:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd401e400 con 0x7fbcd8060d00 2022-08-01T23:05:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcd401e760 con 0x7fbcd8060d00 2022-08-01T23:05:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.496+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd8142b30 con 0x7fbcd8060d00 2022-08-01T23:05:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.500+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd8140a00 con 0x7fbcd8060d00 2022-08-01T23:05:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.500+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbcd4025030 con 0x7fbcd8060d00 2022-08-01T23:05:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.500+0000 7fbcc6ffd700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcb8053b00 0x7fbcb8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.500+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbcd406d0d0 con 0x7fbcd8060d00 2022-08-01T23:05:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.500+0000 7fbcdd5a9700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcb8053b00 0x7fbcb8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.500+0000 7fbcdd5a9700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcb8053b00 0x7fbcb8055fc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fbcc800b000 tx=0x7fbcc8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.508+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbcd4043d00 con 0x7fbcd8060d00 2022-08-01T23:05:09.633 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.628+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbcd80574e0 con 0x7fbcd8060d00 2022-08-01T23:05:09.634 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.628+0000 7fbcc6ffd700 1 -- 172.21.15.32:0/2892176965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v147) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbcd4023020 con 0x7fbcd8060d00 2022-08-01T23:05:09.635 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:05:09.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.628+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcb8053b00 msgr2=0x7fbcb8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.628+0000 7fbce000e700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcb8053b00 0x7fbcb8055fc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fbcc800b000 tx=0x7fbcc8009040).stop 2022-08-01T23:05:09.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.628+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 msgr2=0x7fbcd813cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.637 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.628+0000 7fbce000e700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 0x7fbcd813cd00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fbcd4002f70 tx=0x7fbcd4014000).stop 2022-08-01T23:05:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 shutdown_connections 2022-08-01T23:05:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbcb8053b00 0x7fbcb8055fc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcd80b1b10 0x7fbcd8150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcd8061860 0x7fbcd8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.638 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 --2- 172.21.15.32:0/2892176965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd8060d00 0x7fbcd813cd00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 >> 172.21.15.32:0/2892176965 conn(0x7fbcd80b2b00 msgr2=0x7fbcd81395c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 shutdown_connections 2022-08-01T23:05:09.639 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.632+0000 7fbce000e700 1 -- 172.21.15.32:0/2892176965 wait complete. 2022-08-01T23:05:09.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:05:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 Processor -- start 2022-08-01T23:05:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 -- start start 2022-08-01T23:05:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01f40614b0 0x7f01f4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40ad750 0x7f01f40b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40aee80 con 0x7f01f41405d0 2022-08-01T23:05:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40b70b0 con 0x7f01f40614b0 2022-08-01T23:05:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f971f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40af0b0 con 0x7f01f40ad750 2022-08-01T23:05:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01f40614b0 0x7f01f4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f8f1e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01f41405d0 0x7f01f41466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37078/0 (socket says 172.21.15.32:37078) 2022-08-01T23:05:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f8f1e700 1 -- 172.21.15.32:0/4134182707 learned_addr learned my addr 172.21.15.32:0/4134182707 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f37fe700 1 --2- 172.21.15.32:0/4134182707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40ad750 0x7f01f40b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f3fff700 1 -- 172.21.15.32:0/4134182707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40ad750 msgr2=0x7f01f40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f3fff700 1 --2- 172.21.15.32:0/4134182707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40ad750 0x7f01f40b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f2ffd700 1 -- 172.21.15.32:0/4134182707 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549624311 0 0) 0x7f01f40aee80 con 0x7f01f41405d0 2022-08-01T23:05:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f3fff700 1 -- 172.21.15.32:0/4134182707 >> v1:172.21.15.32:6789/0 conn(0x7f01f41405d0 legacy=0x7f01f41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.752+0000 7f01f3fff700 1 -- 172.21.15.32:0/4134182707 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40af350 con 0x7f01f40614b0 2022-08-01T23:05:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f3fff700 1 --2- 172.21.15.32:0/4134182707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40614b0 0x7f01f4061890 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f01e4002f70 tx=0x7f01e4006df0).ready entity=mon.0 client_cookie=ccba827c9eb2cffa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f2ffd700 1 -- 172.21.15.32:0/4134182707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e400d570 con 0x7f01f40614b0 2022-08-01T23:05:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f2ffd700 1 -- 172.21.15.32:0/4134182707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4000e90 con 0x7f01f40614b0 2022-08-01T23:05:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f2ffd700 1 -- 172.21.15.32:0/4134182707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e400d8c0 con 0x7f01f40614b0 2022-08-01T23:05:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- 172.21.15.32:0/4134182707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40614b0 msgr2=0x7f01f4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 --2- 172.21.15.32:0/4134182707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40614b0 0x7f01f4061890 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f01e4002f70 tx=0x7f01e4006df0).stop 2022-08-01T23:05:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- 172.21.15.32:0/4134182707 shutdown_connections 2022-08-01T23:05:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 --2- 172.21.15.32:0/4134182707 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40ad750 0x7f01f40b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 --2- 172.21.15.32:0/4134182707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40614b0 0x7f01f4061890 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- 172.21.15.32:0/4134182707 >> 172.21.15.32:0/4134182707 conn(0x7f01f4066f70 msgr2=0x7f01f4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- 172.21.15.32:0/4134182707 shutdown_connections 2022-08-01T23:05:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- 172.21.15.32:0/4134182707 wait complete. 2022-08-01T23:05:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 Processor -- start 2022-08-01T23:05:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- start start 2022-08-01T23:05:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40614b0 0x7f01f4141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 0x7f01f4155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f41405d0 0x7f01f4142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40abe50 con 0x7f01f40ad750 2022-08-01T23:05:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40acee0 con 0x7f01f41405d0 2022-08-01T23:05:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f971f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4156360 con 0x7f01f40614b0 2022-08-01T23:05:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 0x7f01f4155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 0x7f01f4155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49544/0 (socket says 172.21.15.32:49544) 2022-08-01T23:05:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 -- 172.21.15.32:0/2820608878 learned_addr learned my addr 172.21.15.32:0/2820608878 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f8f1e700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f41405d0 0x7f01f4142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f3fff700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40614b0 0x7f01f4141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 -- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40614b0 msgr2=0x7f01f4141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40614b0 0x7f01f4141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 -- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f41405d0 msgr2=0x7f01f4142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f41405d0 0x7f01f4142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f37fe700 1 -- 172.21.15.32:0/2820608878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e400b180 con 0x7f01f40ad750 2022-08-01T23:05:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.756+0000 7f01f3fff700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40614b0 0x7f01f4141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f37fe700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 0x7f01f4155cb0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f01dc00c000 tx=0x7f01dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc010780 con 0x7f01f40ad750 2022-08-01T23:05:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc0108e0 con 0x7f01f40ad750 2022-08-01T23:05:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc010c40 con 0x7f01f40ad750 2022-08-01T23:05:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4156600 con 0x7f01f40ad750 2022-08-01T23:05:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4142e60 con 0x7f01f40ad750 2022-08-01T23:05:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f40af5f0 con 0x7f01f40ad750 2022-08-01T23:05:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f01dc01c460 con 0x7f01f40ad750 2022-08-01T23:05:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f17fa700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01d8053980 0x7f01d8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f01dc06c610 con 0x7f01f40ad750 2022-08-01T23:05:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.760+0000 7f01f3fff700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01d8053980 0x7f01d8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.764+0000 7f01f3fff700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01d8053980 0x7f01d8055e40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f01e4018000 tx=0x7f01e4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.764+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01dc0431e0 con 0x7f01f40ad750 2022-08-01T23:05:09.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f01f40574e0 con 0x7f01f40ad750 2022-08-01T23:05:09.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f17fa700 1 -- 172.21.15.32:0/2820608878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v147) v1 ==== 92+0+10 (secure 0 0 0) 0x7f01dc046800 con 0x7f01f40ad750 2022-08-01T23:05:09.899 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:09.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01d8053980 msgr2=0x7f01d8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01d8053980 0x7f01d8055e40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f01e4018000 tx=0x7f01e4014000).stop 2022-08-01T23:05:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 msgr2=0x7f01f4155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 0x7f01f4155cb0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f01dc00c000 tx=0x7f01dc006df0).stop 2022-08-01T23:05:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 shutdown_connections 2022-08-01T23:05:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f01d8053980 0x7f01d8055e40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f41405d0 0x7f01f4142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f40ad750 0x7f01f4155cb0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 --2- 172.21.15.32:0/2820608878 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f01f40614b0 0x7f01f4141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 >> 172.21.15.32:0/2820608878 conn(0x7f01f4066f70 msgr2=0x7f01f405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 shutdown_connections 2022-08-01T23:05:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:09.892+0000 7f01f971f700 1 -- 172.21.15.32:0/2820608878 wait complete. 2022-08-01T23:05:09.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:05:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.012+0000 7fe1610f3700 1 Processor -- start 2022-08-01T23:05:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.012+0000 7fe1610f3700 1 -- start start 2022-08-01T23:05:10.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.012+0000 7fe1610f3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15c0a9320 0x7fe15c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe15c0b0560 0x7fe15c140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c0b12d0 con 0x7fe15c0b0d10 2022-08-01T23:05:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0a9700 con 0x7fe15c0a9320 2022-08-01T23:05:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0b1500 con 0x7fe15c0b0560 2022-08-01T23:05:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15c0a9320 0x7fe15c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15c0a9320 0x7fe15c1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49548/0 (socket says 172.21.15.32:49548) 2022-08-01T23:05:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 -- 172.21.15.32:0/242381650 learned_addr learned my addr 172.21.15.32:0/242381650 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1608f2700 1 --2- 172.21.15.32:0/242381650 >> v2:172.21.15.55:3300/0 conn(0x7fe15c0b0560 0x7fe15c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 -- 172.21.15.32:0/242381650 >> v2:172.21.15.55:3300/0 conn(0x7fe15c0b0560 msgr2=0x7fe15c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 --2- 172.21.15.32:0/242381650 >> v2:172.21.15.55:3300/0 conn(0x7fe15c0b0560 0x7fe15c140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 -- 172.21.15.32:0/242381650 >> v1:172.21.15.55:6789/0 conn(0x7fe15c0b0d10 legacy=0x7fe15c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 -- 172.21.15.32:0/242381650 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0b17a0 con 0x7fe15c0a9320 2022-08-01T23:05:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15bfff700 1 --2- 172.21.15.32:0/242381650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0a9320 0x7fe15c1466e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fe14c0032f0 tx=0x7fe14c009290).ready entity=mon.0 client_cookie=adcbf9ff9de6c1d1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15affd700 1 -- 172.21.15.32:0/242381650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14c00b870 con 0x7fe15c0a9320 2022-08-01T23:05:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15affd700 1 -- 172.21.15.32:0/242381650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c00b9d0 con 0x7fe15c0a9320 2022-08-01T23:05:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe15affd700 1 -- 172.21.15.32:0/242381650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14c00bb40 con 0x7fe15c0a9320 2022-08-01T23:05:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- 172.21.15.32:0/242381650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0a9320 msgr2=0x7fe15c1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 --2- 172.21.15.32:0/242381650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0a9320 0x7fe15c1466e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fe14c0032f0 tx=0x7fe14c009290).stop 2022-08-01T23:05:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- 172.21.15.32:0/242381650 shutdown_connections 2022-08-01T23:05:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 --2- 172.21.15.32:0/242381650 >> v2:172.21.15.55:3300/0 conn(0x7fe15c0b0560 0x7fe15c140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 --2- 172.21.15.32:0/242381650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0a9320 0x7fe15c1466e0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- 172.21.15.32:0/242381650 >> 172.21.15.32:0/242381650 conn(0x7fe15c0612d0 msgr2=0x7fe15c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- 172.21.15.32:0/242381650 shutdown_connections 2022-08-01T23:05:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.016+0000 7fe1610f3700 1 -- 172.21.15.32:0/242381650 wait complete. 2022-08-01T23:05:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 Processor -- start 2022-08-01T23:05:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 -- start start 2022-08-01T23:05:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe15c0a9320 0x7fe15c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 0x7fe15c145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe15c0b0d10 0x7fe15c13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0b64a0 con 0x7fe15c0b0560 2022-08-01T23:05:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c145760 con 0x7fe15c0b0d10 2022-08-01T23:05:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1610f3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c145500 con 0x7fe15c0a9320 2022-08-01T23:05:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 0x7fe15c145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 0x7fe15c145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49556/0 (socket says 172.21.15.32:49556) 2022-08-01T23:05:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2119155027 learned_addr learned my addr 172.21.15.32:0/2119155027 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1608f2700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe15c0b0d10 0x7fe15c13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15bfff700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe15c0a9320 0x7fe15c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe15c0a9320 msgr2=0x7fe15c0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe15c0a9320 0x7fe15c0af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe15c0b0d10 msgr2=0x7fe15c13ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe15c0b0d10 0x7fe15c13ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2119155027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c009530 con 0x7fe15c0b0560 2022-08-01T23:05:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15bfff700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe15c0a9320 0x7fe15c0af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe15b7fe700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 0x7fe15c145970 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fe15c144900 tx=0x7fe150007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.020+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe150012400 con 0x7fe15c0b0560 2022-08-01T23:05:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe150012560 con 0x7fe15c0b0560 2022-08-01T23:05:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c0b6670 con 0x7fe15c0b0560 2022-08-01T23:05:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1500128c0 con 0x7fe15c0b0560 2022-08-01T23:05:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c1565b0 con 0x7fe15c0b0560 2022-08-01T23:05:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c156770 con 0x7fe15c0b0560 2022-08-01T23:05:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe150019030 con 0x7fe15c0b0560 2022-08-01T23:05:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1597fa700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053b30 0x7fe13c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe15006b470 con 0x7fe15c0b0560 2022-08-01T23:05:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe15bfff700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053b30 0x7fe13c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.024+0000 7fe15bfff700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053b30 0x7fe13c055ff0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fe14c019000 tx=0x7fe14c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.028+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe150042230 con 0x7fe15c0b0560 2022-08-01T23:05:10.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe15c0574e0 con 0x7fe15c0b0560 2022-08-01T23:05:10.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1597fa700 1 -- 172.21.15.32:0/2119155027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe150045850 con 0x7fe15c0b0560 2022-08-01T23:05:10.163 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:10.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053b30 msgr2=0x7fe13c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053b30 0x7fe13c055ff0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fe14c019000 tx=0x7fe14c016000).stop 2022-08-01T23:05:10.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 msgr2=0x7fe15c145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 0x7fe15c145970 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fe15c144900 tx=0x7fe150007c50).stop 2022-08-01T23:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 shutdown_connections 2022-08-01T23:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe13c053b30 0x7fe13c055ff0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe15c0b0d10 0x7fe15c13ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0b0560 0x7fe15c145970 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.156+0000 7fe1610f3700 1 --2- 172.21.15.32:0/2119155027 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe15c0a9320 0x7fe15c0af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.160+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 >> 172.21.15.32:0/2119155027 conn(0x7fe15c0612d0 msgr2=0x7fe15c143580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.160+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 shutdown_connections 2022-08-01T23:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.160+0000 7fe1610f3700 1 -- 172.21.15.32:0/2119155027 wait complete. 2022-08-01T23:05:10.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:05:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 Processor -- start 2022-08-01T23:05:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 -- start start 2022-08-01T23:05:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4f80b4820 0x7fb4f80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f8143cf0 con 0x7fb4f80b4310 2022-08-01T23:05:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f81476a0 con 0x7fb4f80b4820 2022-08-01T23:05:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4fd727700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8143f90 con 0x7fb4f8146ae0 2022-08-01T23:05:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.280+0000 7fb4f77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4f80b4820 0x7fb4f80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4f80b4820 0x7fb4f80b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49566/0 (socket says 172.21.15.32:49566) 2022-08-01T23:05:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 -- 172.21.15.32:0/800581846 learned_addr learned my addr 172.21.15.32:0/800581846 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f6ffd700 1 --2- 172.21.15.32:0/800581846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 -- 172.21.15.32:0/800581846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 msgr2=0x7fb4f81403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 --2- 172.21.15.32:0/800581846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f81403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 -- 172.21.15.32:0/800581846 >> v1:172.21.15.55:6789/0 conn(0x7fb4f80b4310 legacy=0x7fb4f81408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 -- 172.21.15.32:0/800581846 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80b0e20 con 0x7fb4f80b4820 2022-08-01T23:05:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f77fe700 1 --2- 172.21.15.32:0/800581846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4820 0x7fb4f80b5060 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fb4f00032f0 tx=0x7fb4f0009290).ready entity=mon.0 client_cookie=93756f873806f678 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f5ffb700 1 -- 172.21.15.32:0/800581846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4f000b8a0 con 0x7fb4f80b4820 2022-08-01T23:05:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f5ffb700 1 -- 172.21.15.32:0/800581846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f000ba00 con 0x7fb4f80b4820 2022-08-01T23:05:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4f5ffb700 1 -- 172.21.15.32:0/800581846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4f000bb70 con 0x7fb4f80b4820 2022-08-01T23:05:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 -- 172.21.15.32:0/800581846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4820 msgr2=0x7fb4f80b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 --2- 172.21.15.32:0/800581846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4820 0x7fb4f80b5060 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7fb4f00032f0 tx=0x7fb4f0009290).stop 2022-08-01T23:05:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 -- 172.21.15.32:0/800581846 shutdown_connections 2022-08-01T23:05:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 --2- 172.21.15.32:0/800581846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4820 0x7fb4f80b5060 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 --2- 172.21.15.32:0/800581846 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f81403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 -- 172.21.15.32:0/800581846 >> 172.21.15.32:0/800581846 conn(0x7fb4f805c620 msgr2=0x7fb4f805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:10.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 -- 172.21.15.32:0/800581846 shutdown_connections 2022-08-01T23:05:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.284+0000 7fb4fd727700 1 -- 172.21.15.32:0/800581846 wait complete. 2022-08-01T23:05:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 Processor -- start 2022-08-01T23:05:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 -- start start 2022-08-01T23:05:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 0x7fb4f8145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f80b4820 0x7fb4f80adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f80afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8155b50 con 0x7fb4f80b4310 2022-08-01T23:05:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8155df0 con 0x7fb4f80b4820 2022-08-01T23:05:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8156090 con 0x7fb4f8146ae0 2022-08-01T23:05:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 0x7fb4f8145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 0x7fb4f8145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49568/0 (socket says 172.21.15.32:49568) 2022-08-01T23:05:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 -- 172.21.15.32:0/336494682 learned_addr learned my addr 172.21.15.32:0/336494682 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f67fc700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f80b4820 0x7fb4f80adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f77fe700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f80afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 -- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 msgr2=0x7fb4f80afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f80afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 -- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f80b4820 msgr2=0x7fb4f80adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f80b4820 0x7fb4f80adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 -- 172.21.15.32:0/336494682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009530 con 0x7fb4f80b4310 2022-08-01T23:05:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f67fc700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f80b4820 0x7fb4f80adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4f6ffd700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 0x7fb4f8145de0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fb4ec0030b0 tx=0x7fb4ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ec011780 con 0x7fb4f80b4310 2022-08-01T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0118e0 con 0x7fb4f80b4310 2022-08-01T23:05:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ec011c40 con 0x7fb4f80b4310 2022-08-01T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f8156330 con 0x7fb4f80b4310 2022-08-01T23:05:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.288+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f80b8e50 con 0x7fb4f80b4310 2022-08-01T23:05:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.292+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f80ae330 con 0x7fb4f80b4310 2022-08-01T23:05:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.292+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb4ec012420 con 0x7fb4f80b4310 2022-08-01T23:05:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.292+0000 7fb4e7fff700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4d4053a70 0x7fb4d4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.292+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb4ec06bfb0 con 0x7fb4f80b4310 2022-08-01T23:05:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.292+0000 7fb4f67fc700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4d4053a70 0x7fb4d4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.292+0000 7fb4f67fc700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4d4053a70 0x7fb4d4055f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fb4e800b000 tx=0x7fb4e8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.300+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb4ec042c00 con 0x7fb4f80b4310 2022-08-01T23:05:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.424+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4f80574e0 con 0x7fb4f80b4310 2022-08-01T23:05:10.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4e7fff700 1 -- 172.21.15.32:0/336494682 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb4ec046220 con 0x7fb4f80b4310 2022-08-01T23:05:10.433 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4d4053a70 msgr2=0x7fb4d4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4d4053a70 0x7fb4d4055f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fb4e800b000 tx=0x7fb4e8009040).stop 2022-08-01T23:05:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 msgr2=0x7fb4f8145de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 0x7fb4f8145de0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fb4ec0030b0 tx=0x7fb4ec008040).stop 2022-08-01T23:05:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 shutdown_connections 2022-08-01T23:05:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fb4d4053a70 0x7fb4d4055f30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fb4f8146ae0 0x7fb4f80afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f80b4820 0x7fb4f80adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 --2- 172.21.15.32:0/336494682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f80b4310 0x7fb4f8145de0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 >> 172.21.15.32:0/336494682 conn(0x7fb4f805c620 msgr2=0x7fb4f8140190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 shutdown_connections 2022-08-01T23:05:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.428+0000 7fb4fd727700 1 -- 172.21.15.32:0/336494682 wait complete. 2022-08-01T23:05:10.446 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:05:10.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:10.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 Processor -- start 2022-08-01T23:05:10.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 -- start start 2022-08-01T23:05:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40b1410 0x7f64d40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64d40aef50 0x7f64d40b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d41404a0 con 0x7f64d4140e80 2022-08-01T23:05:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d4140740 con 0x7f64d40aef50 2022-08-01T23:05:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64dc42f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d41409e0 con 0x7f64d40b1410 2022-08-01T23:05:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64d99ca700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64d4140e80 0x7f64d40ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37118/0 (socket says 172.21.15.32:37118) 2022-08-01T23:05:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64d99ca700 1 -- 172.21.15.32:0/4201296169 learned_addr learned my addr 172.21.15.32:0/4201296169 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64da9cc700 1 --2- 172.21.15.32:0/4201296169 >> v2:172.21.15.32:3300/0 conn(0x7f64d40aef50 0x7f64d40b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64da1cb700 1 --2- 172.21.15.32:0/4201296169 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40b1410 0x7f64d40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64da9cc700 1 -- 172.21.15.32:0/4201296169 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40b1410 msgr2=0x7f64d40b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64da9cc700 1 --2- 172.21.15.32:0/4201296169 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40b1410 0x7f64d40b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64d91c9700 1 -- 172.21.15.32:0/4201296169 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775438137 0 0) 0x7f64d41404a0 con 0x7f64d4140e80 2022-08-01T23:05:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.544+0000 7f64da9cc700 1 -- 172.21.15.32:0/4201296169 >> v1:172.21.15.32:6789/0 conn(0x7f64d4140e80 legacy=0x7f64d40ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64da9cc700 1 -- 172.21.15.32:0/4201296169 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4140190 con 0x7f64d40aef50 2022-08-01T23:05:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64da9cc700 1 --2- 172.21.15.32:0/4201296169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40aef50 0x7f64d40b22f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f64cc0032f0 tx=0x7f64cc0091d0).ready entity=mon.0 client_cookie=3fbc10dca8febb8f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64d91c9700 1 -- 172.21.15.32:0/4201296169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64cc00b6c0 con 0x7f64d40aef50 2022-08-01T23:05:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64d91c9700 1 -- 172.21.15.32:0/4201296169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00b820 con 0x7f64d40aef50 2022-08-01T23:05:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64d91c9700 1 -- 172.21.15.32:0/4201296169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64cc00b990 con 0x7f64d40aef50 2022-08-01T23:05:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 -- 172.21.15.32:0/4201296169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40aef50 msgr2=0x7f64d40b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 --2- 172.21.15.32:0/4201296169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40aef50 0x7f64d40b22f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f64cc0032f0 tx=0x7f64cc0091d0).stop 2022-08-01T23:05:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 -- 172.21.15.32:0/4201296169 shutdown_connections 2022-08-01T23:05:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 --2- 172.21.15.32:0/4201296169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40aef50 0x7f64d40b22f0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 --2- 172.21.15.32:0/4201296169 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40b1410 0x7f64d40b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 -- 172.21.15.32:0/4201296169 >> 172.21.15.32:0/4201296169 conn(0x7f64d405f840 msgr2=0x7f64d405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 -- 172.21.15.32:0/4201296169 shutdown_connections 2022-08-01T23:05:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 -- 172.21.15.32:0/4201296169 wait complete. 2022-08-01T23:05:10.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 Processor -- start 2022-08-01T23:05:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.548+0000 7f64dc42f700 1 -- start start 2022-08-01T23:05:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40aef50 0x7f64d40ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 0x7f64d412ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64d4140e80 0x7f64d4139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40a86e0 con 0x7f64d40b1410 2022-08-01T23:05:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40a8980 con 0x7f64d4140e80 2022-08-01T23:05:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40aa980 con 0x7f64d40aef50 2022-08-01T23:05:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 0x7f64d412ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 0x7f64d412ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49584/0 (socket says 172.21.15.32:49584) 2022-08-01T23:05:10.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 -- 172.21.15.32:0/840916981 learned_addr learned my addr 172.21.15.32:0/840916981 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:10.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64da9cc700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64d4140e80 0x7f64d4139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64da1cb700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40aef50 0x7f64d40ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 -- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40aef50 msgr2=0x7f64d40ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40aef50 0x7f64d40ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 -- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64d4140e80 msgr2=0x7f64d4139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64d4140e80 0x7f64d4139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 -- 172.21.15.32:0/840916981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc009470 con 0x7f64d40b1410 2022-08-01T23:05:10.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64d99ca700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 0x7f64d412ff40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f64c4000ed0 tx=0x7f64c4007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64c40070d0 con 0x7f64d40b1410 2022-08-01T23:05:10.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4007230 con 0x7f64d40b1410 2022-08-01T23:05:10.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d40aac20 con 0x7f64d40b1410 2022-08-01T23:05:10.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64c4005610 con 0x7f64d40b1410 2022-08-01T23:05:10.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d40a9170 con 0x7f64d40b1410 2022-08-01T23:05:10.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d40aadb0 con 0x7f64d40b1410 2022-08-01T23:05:10.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.552+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f64c4010070 con 0x7f64d40b1410 2022-08-01T23:05:10.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.556+0000 7f64c37fe700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64b4053980 0x7f64b4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:10.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.556+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f64c406b7f0 con 0x7f64d40b1410 2022-08-01T23:05:10.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.556+0000 7f64da1cb700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64b4053980 0x7f64b4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:10.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.556+0000 7f64da1cb700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64b4053980 0x7f64b4055e40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f64d000b000 tx=0x7f64d0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:10.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.560+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f64c40424a0 con 0x7f64d40b1410 2022-08-01T23:05:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.720+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64d40aa520 con 0x7f64d40b1410 2022-08-01T23:05:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.720+0000 7f64c37fe700 1 -- 172.21.15.32:0/840916981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f64c4045ac0 con 0x7f64d40b1410 2022-08-01T23:05:10.729 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:10.729 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:02.448548+0000","modified":"2022-08-01T23:05:03.452680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2229490987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2229490987},{"type":"v1","addr":"172.21.15.32:6835","nonce":2229490987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T23:05:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64b4053980 msgr2=0x7f64b4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64b4053980 0x7f64b4055e40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f64d000b000 tx=0x7f64d0006df0).stop 2022-08-01T23:05:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 msgr2=0x7f64d412ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 0x7f64d412ff40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f64c4000ed0 tx=0x7f64c4007ce0).stop 2022-08-01T23:05:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 shutdown_connections 2022-08-01T23:05:10.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f64b4053980 0x7f64b4055e40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64d4140e80 0x7f64d4139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1410 0x7f64d412ff40 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 --2- 172.21.15.32:0/840916981 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f64d40aef50 0x7f64d40ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:10.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 >> 172.21.15.32:0/840916981 conn(0x7f64d405f840 msgr2=0x7f64d40ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:10.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 shutdown_connections 2022-08-01T23:05:10.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:10.724+0000 7f64dc42f700 1 -- 172.21.15.32:0/840916981 wait complete. 2022-08-01T23:05:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2022-08-01T23:05:11.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:05:02.448548+0000', 'modified': '2022-08-01T23:05:03.452680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/2229490987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2229490987}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 2229490987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:05:11.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:05:11.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T23:05:11.164 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8427,"want_state":"up:active","state":"up:active","rank_uptime":8.7012099109999994,"mdsmap_epoch":74,"osdmap_epoch":147,"osdmap_epoch_barrier":147,"uptime":13.027184639} 2022-08-01T23:05:11.165 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8427, "mdsmap_epoch": 74, "osdmap_epoch": 147, "osdmap_epoch_barrier": 147, "rank_uptime": 8.701209911, "state": "up:active", "uptime": 13.027184639, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:05:11.166 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:05:11.166 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:05:11.167 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:05:11.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:11.168 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:11.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:05:11.192 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:11.193 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:11.252 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:11.253 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:11.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:11.254 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:05:11.259 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:05:11.259 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:11.303 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:05:11.320 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:11.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:11.330 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:11.331 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:11.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:11.332 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:05:11.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:05:11.367+0000 7fd088c87080 -1 init, newargv = 0x561fe09727f0 newargc=9ceph-fuse[23196]: starting ceph client 2022-08-01T23:05:11.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr: 2022-08-01T23:05:11.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23196]: starting fuse 2022-08-01T23:05:12.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:05:12.457 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:12.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:12.516 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:12.517 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:12.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.518 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:05:12.525 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:05:12.526 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:05:12.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.527 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:05:12.527 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:05:12.528 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:05:12.528 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:05:12.528 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:05:12.529 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:12.529 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:05:12.530 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:05:12.530 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:05:12.530 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:12.531 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:05:12.531 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:05:12.532 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:05:12.532 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:05:12.532 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:12.533 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:05:12.533 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:05:12.533 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:05:12.534 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:05:12.535 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:05:12.535 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:12.535 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:05:12.536 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:05:12.580 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.23196.asok 2022-08-01T23:05:12.585 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23196.asok 2022-08-01T23:05:12.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23196.asok status 2022-08-01T23:05:12.719 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:05:12.720 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:05:12.720 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:05:12.720 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:05:12.720 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:05:12.721 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:05:12.721 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:05:12.721 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "23196", 2022-08-01T23:05:12.721 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:05:12.722 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:12.722 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:05:12.722 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:05:12.722 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 8524, 2022-08-01T23:05:12.722 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:05:12.723 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:05:12.723 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:05:12.723 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 8524 2022-08-01T23:05:12.723 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:12.723 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:05:12.724 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:05:12.724 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:05:12.724 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2459012255 2022-08-01T23:05:12.724 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:05:12.724 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:12.725 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:05:12.725 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:05:12.725 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:05:12.725 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2459012255 2022-08-01T23:05:12.726 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:12.726 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.8524 172.21.15.157:0/2459012255", 2022-08-01T23:05:12.726 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/2459012255", 2022-08-01T23:05:12.726 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:05:12.726 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 74, 2022-08-01T23:05:12.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 147, 2022-08-01T23:05:12.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:05:12.727 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:05:12.727 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:05:12.727 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:05:12.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.729 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:12.730 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:12.737 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:05:12.738 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:05:12.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.739 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:12.798 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T23:05:12.798 DEBUG:teuthology.orchestra.run.smithi130:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2022-08-01T23:05:12.833 INFO:teuthology.orchestra.run.smithi130.stdout:1659395112.791299575 2022-08-01T23:05:12.834 INFO:tasks.cephfs.test_misc:rctime = 1659395112.791299575 2022-08-01T23:05:12.834 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:05:12.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:12.855 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:12.877 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:12.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:12.918 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:05:12.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.919 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:12.920 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:12.928 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:05:12.929 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:05:12.929 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi130.front.sepia.ceph.com... 2022-08-01T23:05:12.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:05:12.930 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:12.945 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23641]: fuse finished with error 0 and tester_r 0 2022-08-01T23:05:12.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:12.947 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:12.948 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:12.994 INFO:teuthology.orchestra.run.smithi130.stdout:ext2/ext3 2022-08-01T23:05:12.996 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:05:13.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:05:13.045 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:13.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:13.052 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:13.053 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:13.061 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:05:13.062 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:05:13.063 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:05:13.063 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:05:13.063 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:13.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23196]: fuse finished with error 0 and tester_r 0 2022-08-01T23:05:13.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:13.123 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:13.123 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:13.129 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:05:13.130 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:05:13.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:05:13.175 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:13.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:05:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.284+0000 7f67dd1e9700 1 Processor -- start 2022-08-01T23:05:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.284+0000 7f67dd1e9700 1 -- start start 2022-08-01T23:05:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f67d80a9320 0x7f67d81466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d8140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d80b12d0 con 0x7f67d80b0d10 2022-08-01T23:05:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d80a9700 con 0x7f67d80a9320 2022-08-01T23:05:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80b1500 con 0x7f67d80b0560 2022-08-01T23:05:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67d80b0d10 0x7f67d8146bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37132/0 (socket says 172.21.15.32:37132) 2022-08-01T23:05:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d77fe700 1 -- 172.21.15.32:0/3979829487 learned_addr learned my addr 172.21.15.32:0/3979829487 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dc9e8700 1 --2- 172.21.15.32:0/3979829487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d8140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d7fff700 1 --2- 172.21.15.32:0/3979829487 >> v2:172.21.15.55:3300/0 conn(0x7f67d80a9320 0x7f67d81466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d6ffd700 1 -- 172.21.15.32:0/3979829487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978612078 0 0) 0x7f67d80b12d0 con 0x7f67d80b0d10 2022-08-01T23:05:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d6ffd700 1 -- 172.21.15.32:0/3979829487 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c4003260 con 0x7f67d80b0d10 2022-08-01T23:05:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3979829487 >> v2:172.21.15.55:3300/0 conn(0x7f67d80a9320 msgr2=0x7f67d81466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dc9e8700 1 --2- 172.21.15.32:0/3979829487 >> v2:172.21.15.55:3300/0 conn(0x7f67d80a9320 0x7f67d81466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3979829487 >> v1:172.21.15.32:6789/0 conn(0x7f67d80b0d10 legacy=0x7f67d8146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d6ffd700 1 -- 172.21.15.32:0/3979829487 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632105461 0 0) 0x7f67c4003260 con 0x7f67d80b0d10 2022-08-01T23:05:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3979829487 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80b17a0 con 0x7f67d80b0560 2022-08-01T23:05:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dc9e8700 1 --2- 172.21.15.32:0/3979829487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d8140350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f67c8001820 tx=0x7f67d00091d0).ready entity=mon.2 client_cookie=2117ddde476f616a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d6ffd700 1 -- 172.21.15.32:0/3979829487 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c8001680 con 0x7f67d80b0560 2022-08-01T23:05:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d6ffd700 1 -- 172.21.15.32:0/3979829487 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d000f510 con 0x7f67d80b0560 2022-08-01T23:05:13.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67d6ffd700 1 -- 172.21.15.32:0/3979829487 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d000f680 con 0x7f67d80b0560 2022-08-01T23:05:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3979829487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 msgr2=0x7f67d8140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3979829487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d8140350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f67c8001820 tx=0x7f67d00091d0).stop 2022-08-01T23:05:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3979829487 shutdown_connections 2022-08-01T23:05:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3979829487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d8140350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3979829487 >> v2:172.21.15.55:3300/0 conn(0x7f67d80a9320 0x7f67d81466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3979829487 >> 172.21.15.32:0/3979829487 conn(0x7f67d80612d0 msgr2=0x7f67d8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3979829487 shutdown_connections 2022-08-01T23:05:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.288+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3979829487 wait complete. 2022-08-01T23:05:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 Processor -- start 2022-08-01T23:05:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 -- start start 2022-08-01T23:05:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67d80a9320 0x7f67d80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 0x7f67d8155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8153470 con 0x7f67d80b0d10 2022-08-01T23:05:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d81531d0 con 0x7f67d80a9320 2022-08-01T23:05:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dd1e9700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80ac180 con 0x7f67d80b0560 2022-08-01T23:05:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 0x7f67d8155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 0x7f67d8155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49600/0 (socket says 172.21.15.32:49600) 2022-08-01T23:05:13.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3795191574 learned_addr learned my addr 172.21.15.32:0/3795191574 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67d77fe700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67d7fff700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67d80a9320 0x7f67d80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 msgr2=0x7f67d80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d80b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67d80a9320 msgr2=0x7f67d80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67d80a9320 0x7f67d80b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 -- 172.21.15.32:0/3795191574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0009470 con 0x7f67d80b0d10 2022-08-01T23:05:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.292+0000 7f67dc9e8700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 0x7f67d8155a00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f67c8001820 tx=0x7f67d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d00032d0 con 0x7f67d80b0d10 2022-08-01T23:05:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0011b80 con 0x7f67d80b0d10 2022-08-01T23:05:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d80ac420 con 0x7f67d80b0d10 2022-08-01T23:05:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d0021440 con 0x7f67d80b0d10 2022-08-01T23:05:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d8152df0 con 0x7f67d80b0d10 2022-08-01T23:05:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f67d0027020 con 0x7f67d80b0d10 2022-08-01T23:05:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d8141580 con 0x7f67d80b0d10 2022-08-01T23:05:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d57fa700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f67b8053b00 0x7f67b8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f67d006d040 con 0x7f67d80b0d10 2022-08-01T23:05:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d7fff700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f67b8053b00 0x7f67b8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.296+0000 7f67d7fff700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f67b8053b00 0x7f67b8055fc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f67cc00b000 tx=0x7f67cc006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.300+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67d8141580 con 0x7f67d80b0d10 2022-08-01T23:05:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.428+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f67d8141580 con 0x7f67d80b0d10 2022-08-01T23:05:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.432+0000 7f67d57fa700 1 -- 172.21.15.32:0/3795191574 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v147) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f67d8141580 con 0x7f67d80b0d10 2022-08-01T23:05:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:13.440 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:05:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 147, 2022-08-01T23:05:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:05:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:05:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:05:03.451546+0000", 2022-08-01T23:05:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:05:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:05:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:05:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:05:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:05:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:05:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:05:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:05:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:05:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:05:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:05:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:05:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:05:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:05:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 28, 2022-08-01T23:05:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:05:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:05:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:05:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:05:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:05:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:05:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:05:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:05:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:05:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:05:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:05:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:05:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:05:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:05:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:05:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:05:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:05:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:05:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:05:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:05:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:05:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:05:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:05:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 27, 2022-08-01T23:05:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:05:13.477 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:05:00.933410+0000", 2022-08-01T23:05:13.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:13.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:13.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:13.477 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:13.478 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:13.478 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:13.478 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:13.478 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:13.478 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:13.479 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:13.479 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:13.479 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:13.479 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:13.479 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:13.479 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:13.480 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:13.480 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:13.480 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:13.480 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:13.480 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:13.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.481 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "146", 2022-08-01T23:05:13.481 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:13.481 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:13.482 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:13.482 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:13.482 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:13.482 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:13.482 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:13.483 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:13.483 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:13.483 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:13.483 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:13.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:13.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:13.484 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:13.484 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:13.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:13.485 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:13.485 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:13.485 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:13.485 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:13.485 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:13.486 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:13.486 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:13.486 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:13.486 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:13.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:13.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.487 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:13.487 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:13.487 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:13.488 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:13.488 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:13.488 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:13.488 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:13.489 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:13.489 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:13.489 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:13.489 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:13.489 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:05:13.490 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:05:13.490 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:05:13.490 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:05:13.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.491 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:13.491 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:05:13.491 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:05:13.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.492 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 28, 2022-08-01T23:05:13.492 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:05:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:05:01.881408+0000", 2022-08-01T23:05:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:13.493 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:13.495 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:13.496 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.497 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "147", 2022-08-01T23:05:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:13.498 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:13.499 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:13.500 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:13.501 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:13.502 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:13.504 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:13.505 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:05:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:05:13.506 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:05:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:13.507 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:05:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:05:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:05:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.508 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.509 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.510 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:05:13.511 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:05:13.512 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:05:13.514 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:05:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:05:13.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.518 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:05:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.519 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:05:13.520 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:05:13.521 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:05:13.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:05:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:05:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:05:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.523 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.524 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:05:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:05:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.525 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:13.526 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:05:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:05:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:05:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:05:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:05:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:05:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:05:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:05:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.538 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:05:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:05:13.539 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:05:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:05:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:05:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:05:13.541 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2022-08-01T23:05:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.543 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:05:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:05:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:05:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:05:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:05:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:05:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:05:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:05:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:13.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:05:13.556 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:05:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:05:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:05:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.557 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:05:13.558 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:05:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.559 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2022-08-01T23:05:13.560 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:05:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:05:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:05:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:05:13.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.567 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:05:13.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:05:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.571 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:05:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:05:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:05:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:05:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:05:13.574 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:05:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:05:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:05:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.576 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2022-08-01T23:05:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.578 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:05:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:05:13.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:05:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:05:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:05:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:05:13.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.587 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:05:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:05:13.589 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:05:13.590 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:05:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:05:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:05:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.591 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.592 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:05:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:05:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.593 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.594 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:05:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:05:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:05:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:05:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:05:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:05:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:05:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:05:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:05:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:05:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:05:13.607 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:05:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:05:13.609 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:05:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.610 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:13.611 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:05:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:05:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:05:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:05:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:05:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:05:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.620 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:05:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:05:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:13.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:05:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:05:13.624 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:05:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:05:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.625 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:05:13.626 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:05:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:13.627 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:13.628 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:05:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:05:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:05:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:05:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:05:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:05:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:13.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:05:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:05:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:13.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:05:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:05:13.641 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:05:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:05:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:13.642 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:05:13.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:05:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.644 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:05:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:05:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.646 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:05:13.647 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:05:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.648 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:05:13.649 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:05:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.650 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.651 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:05:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:05:13.652 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.653 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:05:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.654 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:05:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.655 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:05:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:05:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.657 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:05:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:05:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:13.659 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:05:13.660 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:05:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:05:13.661 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:05:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:05:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:05:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:05:13.662 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:05:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:05:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:05:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:05:13.663 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:05:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:05:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:05:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:05:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:05:13.664 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:05:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:05:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:05:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:05:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:05:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:05:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:05:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:05:13.666 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:05:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:05:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:05:13.667 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:05:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:05:13.668 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:05:13.668 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:05:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f67b8053b00 msgr2=0x7f67b8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f67b8053b00 0x7f67b8055fc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f67cc00b000 tx=0x7f67cc006df0).stop 2022-08-01T23:05:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 msgr2=0x7f67d8155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 0x7f67d8155a00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f67c8001820 tx=0x7f67d0016000).stop 2022-08-01T23:05:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 shutdown_connections 2022-08-01T23:05:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f67b8053b00 0x7f67b8055fc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d80b0d10 0x7f67d8155a00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f67d80b0560 0x7f67d80b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 --2- 172.21.15.32:0/3795191574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67d80a9320 0x7f67d80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 >> 172.21.15.32:0/3795191574 conn(0x7f67d80612d0 msgr2=0x7f67d81464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 shutdown_connections 2022-08-01T23:05:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.436+0000 7f67dd1e9700 1 -- 172.21.15.32:0/3795191574 wait complete. 2022-08-01T23:05:13.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 Processor -- start 2022-08-01T23:05:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 -- start start 2022-08-01T23:05:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 0x7f198c0ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0a10 0x7f198c140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0b2c80 con 0x7f198c0b0630 2022-08-01T23:05:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c0b2900 con 0x7f198c0aa2b0 2022-08-01T23:05:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.776+0000 7f1992c30700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0b2eb0 con 0x7f198c0b0a10 2022-08-01T23:05:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 0x7f198c0ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f19911cd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0a10 0x7f198c140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 0x7f198c0ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:59390/0 (socket says 172.21.15.32:59390) 2022-08-01T23:05:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198bfff700 1 -- 172.21.15.32:0/1945412486 learned_addr learned my addr 172.21.15.32:0/1945412486 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f19911cd700 1 -- 172.21.15.32:0/1945412486 >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 msgr2=0x7f198c0ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f19911cd700 1 --2- 172.21.15.32:0/1945412486 >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 0x7f198c0ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f19911cd700 1 -- 172.21.15.32:0/1945412486 >> v1:172.21.15.55:6789/0 conn(0x7f198c0b0630 legacy=0x7f198c0adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f19911cd700 1 -- 172.21.15.32:0/1945412486 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0a9d00 con 0x7f198c0b0a10 2022-08-01T23:05:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198bfff700 1 --2- 172.21.15.32:0/1945412486 >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 0x7f198c0ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f19911cd700 1 --2- 172.21.15.32:0/1945412486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0a10 0x7f198c140730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f19840032f0 tx=0x7f19840091d0).ready entity=mon.2 client_cookie=28dfb0aec8130b2a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198b7fe700 1 -- 172.21.15.32:0/1945412486 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198400b6c0 con 0x7f198c0b0a10 2022-08-01T23:05:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198b7fe700 1 -- 172.21.15.32:0/1945412486 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198400b820 con 0x7f198c0b0a10 2022-08-01T23:05:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f198b7fe700 1 -- 172.21.15.32:0/1945412486 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f198400b990 con 0x7f198c0b0a10 2022-08-01T23:05:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 -- 172.21.15.32:0/1945412486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0a10 msgr2=0x7f198c140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 --2- 172.21.15.32:0/1945412486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0a10 0x7f198c140730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f19840032f0 tx=0x7f19840091d0).stop 2022-08-01T23:05:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 -- 172.21.15.32:0/1945412486 shutdown_connections 2022-08-01T23:05:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 --2- 172.21.15.32:0/1945412486 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0a10 0x7f198c140730 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 --2- 172.21.15.32:0/1945412486 >> v2:172.21.15.55:3300/0 conn(0x7f198c0aa2b0 0x7f198c0ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 -- 172.21.15.32:0/1945412486 >> 172.21.15.32:0/1945412486 conn(0x7f198c060950 msgr2=0x7f198c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 -- 172.21.15.32:0/1945412486 shutdown_connections 2022-08-01T23:05:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.780+0000 7f1992c30700 1 -- 172.21.15.32:0/1945412486 wait complete. 2022-08-01T23:05:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 Processor -- start 2022-08-01T23:05:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 -- start start 2022-08-01T23:05:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 0x7f198c05fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0630 0x7f198c13f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198c0b0a10 0x7f198c13f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c063be0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c063e80 con 0x7f198c0b0a10 2022-08-01T23:05:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c064120 con 0x7f198c0b0630 2022-08-01T23:05:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 0x7f198c05fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f198bfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0630 0x7f198c13f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 0x7f198c05fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49610/0 (socket says 172.21.15.32:49610) 2022-08-01T23:05:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 -- 172.21.15.32:0/4288596392 learned_addr learned my addr 172.21.15.32:0/4288596392 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19911cd700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198c0b0a10 0x7f198c13f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 -- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0630 msgr2=0x7f198c13f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0630 0x7f198c13f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 -- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198c0b0a10 msgr2=0x7f198c13f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198c0b0a10 0x7f198c13f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 -- 172.21.15.32:0/4288596392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197800b1e0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f19909cc700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 0x7f198c05fb80 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f198c0b7900 tx=0x7f1980007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1980012400 con 0x7f198c0aa2b0 2022-08-01T23:05:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980012560 con 0x7f198c0aa2b0 2022-08-01T23:05:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1984009470 con 0x7f198c0aa2b0 2022-08-01T23:05:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19800128c0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.784+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c13feb0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.788+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c0574e0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.788+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1980019030 con 0x7f198c0aa2b0 2022-08-01T23:05:13.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.788+0000 7f1989ffb700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f196c053a70 0x7f196c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:13.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.788+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f198006b430 con 0x7f198c0aa2b0 2022-08-01T23:05:13.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.788+0000 7f198bfff700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f196c053a70 0x7f196c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:13.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.792+0000 7f198bfff700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f196c053a70 0x7f196c055f30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f197800b000 tx=0x7f1978006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:13.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.796+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1980042190 con 0x7f198c0aa2b0 2022-08-01T23:05:13.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.964+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198c0447e0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.964+0000 7f1989ffb700 1 -- 172.21.15.32:0/4288596392 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f19800457b0 con 0x7f198c0aa2b0 2022-08-01T23:05:13.971 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:13.971 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:02.448548+0000","modified":"2022-08-01T23:05:03.452680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/2229490987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2229490987},{"type":"v1","addr":"172.21.15.32:6835","nonce":2229490987}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T23:05:13.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f196c053a70 msgr2=0x7f196c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f196c053a70 0x7f196c055f30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f197800b000 tx=0x7f1978006eb0).stop 2022-08-01T23:05:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 msgr2=0x7f198c05fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 0x7f198c05fb80 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f198c0b7900 tx=0x7f1980007c50).stop 2022-08-01T23:05:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 shutdown_connections 2022-08-01T23:05:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f196c053a70 0x7f196c055f30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f198c0b0a10 0x7f198c13f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f198c0b0630 0x7f198c13f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 --2- 172.21.15.32:0/4288596392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0aa2b0 0x7f198c05fb80 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 >> 172.21.15.32:0/4288596392 conn(0x7f198c060950 msgr2=0x7f198c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 shutdown_connections 2022-08-01T23:05:13.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:13.968+0000 7f1992c30700 1 -- 172.21.15.32:0/4288596392 wait complete. 2022-08-01T23:05:13.977 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2022-08-01T23:05:13.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:05:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb5a83700 1 Processor -- start 2022-08-01T23:05:14.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb5a83700 1 -- start start 2022-08-01T23:05:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb5a83700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb5a83700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb00b5a10 con 0x7f1fb00b07a0 2022-08-01T23:05:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb5a83700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb00ac990 con 0x7f1fb0147730 2022-08-01T23:05:14.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb5a83700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb0145470 con 0x7f1fb00b5630 2022-08-01T23:05:14.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1fb00b07a0 0x7f1fb0067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37160/0 (socket says 172.21.15.32:37160) 2022-08-01T23:05:14.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faffff700 1 -- 172.21.15.32:0/582834354 learned_addr learned my addr 172.21.15.32:0/582834354 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:14.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb4a81700 1 --2- 172.21.15.32:0/582834354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996783225 0 0) 0x7f1fb00b5a10 con 0x7f1fb00b07a0 2022-08-01T23:05:14.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f9c003260 con 0x7f1fb00b07a0 2022-08-01T23:05:14.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233059815 0 0) 0x7f1fb00ac990 con 0x7f1fb0147730 2022-08-01T23:05:14.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb00b5a10 con 0x7f1fb0147730 2022-08-01T23:05:14.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2654930987 0 0) 0x7f1f9c003260 con 0x7f1fb00b07a0 2022-08-01T23:05:14.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb4a81700 1 -- 172.21.15.32:0/582834354 >> v1:172.21.15.55:6789/0 conn(0x7f1fb0147730 legacy=0x7f1fb00acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb4a81700 1 -- 172.21.15.32:0/582834354 >> v1:172.21.15.32:6789/0 conn(0x7f1fb00b07a0 legacy=0x7f1fb0067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb4a81700 1 -- 172.21.15.32:0/582834354 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00b1bc0 con 0x7f1fb00b5630 2022-08-01T23:05:14.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.092+0000 7f1fb4a81700 1 --2- 172.21.15.32:0/582834354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb00af6e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f1fa0001110 tx=0x7f1fa4006df0).ready entity=mon.2 client_cookie=fa442c4f3a655054 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fa0001660 con 0x7f1fb00b5630 2022-08-01T23:05:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa4000e90 con 0x7f1fb00b5630 2022-08-01T23:05:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1faf7fe700 1 -- 172.21.15.32:0/582834354 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fa4012460 con 0x7f1fb00b5630 2022-08-01T23:05:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 -- 172.21.15.32:0/582834354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 msgr2=0x7f1fb00af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/582834354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb00af6e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f1fa0001110 tx=0x7f1fa4006df0).stop 2022-08-01T23:05:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 -- 172.21.15.32:0/582834354 shutdown_connections 2022-08-01T23:05:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/582834354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb00af6e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 -- 172.21.15.32:0/582834354 >> 172.21.15.32:0/582834354 conn(0x7f1fb00626f0 msgr2=0x7f1fb0062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 -- 172.21.15.32:0/582834354 shutdown_connections 2022-08-01T23:05:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 -- 172.21.15.32:0/582834354 wait complete. 2022-08-01T23:05:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 Processor -- start 2022-08-01T23:05:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.096+0000 7f1fb5a83700 1 -- start start 2022-08-01T23:05:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fb00b07a0 0x7f1fb0157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb0149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 0x7f1fb015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb00ae180 con 0x7f1fb0147730 2022-08-01T23:05:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb0064ba0 con 0x7f1fb00b07a0 2022-08-01T23:05:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb015a7d0 con 0x7f1fb00b5630 2022-08-01T23:05:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 0x7f1fb015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 0x7f1fb015af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49628/0 (socket says 172.21.15.32:49628) 2022-08-01T23:05:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1faffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb0149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 -- 172.21.15.32:0/3331320095 learned_addr learned my addr 172.21.15.32:0/3331320095 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb4a81700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fb00b07a0 0x7f1fb0157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 -- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 msgr2=0x7f1fb0149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb0149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 -- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fb00b07a0 msgr2=0x7f1fb0157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fb00b07a0 0x7f1fb0157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 -- 172.21.15.32:0/3331320095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa400b180 con 0x7f1fb0147730 2022-08-01T23:05:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5282700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 0x7f1fb015af10 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f1fa8001490 tx=0x7f1fa8007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fa8002310 con 0x7f1fb0147730 2022-08-01T23:05:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa8002470 con 0x7f1fb0147730 2022-08-01T23:05:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb015aa70 con 0x7f1fb0147730 2022-08-01T23:05:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fa80017a0 con 0x7f1fb0147730 2022-08-01T23:05:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb014a4d0 con 0x7f1fb0147730 2022-08-01T23:05:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.100+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb015ac00 con 0x7f1fb0147730 2022-08-01T23:05:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.104+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1fa8010030 con 0x7f1fb0147730 2022-08-01T23:05:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.104+0000 7f1fadffb700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f90053950 0x7f1f90055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.104+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1fa806be30 con 0x7f1fb0147730 2022-08-01T23:05:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.104+0000 7f1fb4a81700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f90053950 0x7f1f90055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.104+0000 7f1fb4a81700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f90053950 0x7f1f90055e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f1fa401a000 tx=0x7f1fa4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.108+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1fa8042ae0 con 0x7f1fb0147730 2022-08-01T23:05:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.272+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1fb00b1680 con 0x7f1fb0147730 2022-08-01T23:05:14.627 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:05:14.620+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:05:14.645 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.640+0000 7f1fadffb700 1 -- 172.21.15.32:0/3331320095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1fa8046100 con 0x7f1fb0147730 2022-08-01T23:05:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f90053950 msgr2=0x7f1f90055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f90053950 0x7f1f90055e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f1fa401a000 tx=0x7f1fa4014000).stop 2022-08-01T23:05:14.650 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 msgr2=0x7f1fb015af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 0x7f1fb015af10 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f1fa8001490 tx=0x7f1fa8007dd0).stop 2022-08-01T23:05:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 shutdown_connections 2022-08-01T23:05:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1f90053950 0x7f1f90055e10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb0147730 0x7f1fb015af10 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1fb00b5630 0x7f1fb0149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 --2- 172.21.15.32:0/3331320095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fb00b07a0 0x7f1fb0157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 >> 172.21.15.32:0/3331320095 conn(0x7f1fb00626f0 msgr2=0x7f1fb0062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:14.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 shutdown_connections 2022-08-01T23:05:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.644+0000 7f1fb5a83700 1 -- 172.21.15.32:0/3331320095 wait complete. 2022-08-01T23:05:14.653 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:05:14.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:14.676 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:05:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 Processor -- start 2022-08-01T23:05:14.770 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 -- start start 2022-08-01T23:05:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7040b01a0 0x7fc7040a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc7040afd60 0x7fc7040a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7040a7250 con 0x7fc7040b2710 2022-08-01T23:05:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040a7480 con 0x7fc7040b01a0 2022-08-01T23:05:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70b0c0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040a7720 con 0x7fc7040afd60 2022-08-01T23:05:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7040b01a0 0x7fc7040a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7040b01a0 0x7fc7040a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49644/0 (socket says 172.21.15.32:49644) 2022-08-01T23:05:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 -- 172.21.15.32:0/2335749333 learned_addr learned my addr 172.21.15.32:0/2335749333 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc70a8bf700 1 --2- 172.21.15.32:0/2335749333 >> v2:172.21.15.55:3300/0 conn(0x7fc7040afd60 0x7fc7040a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 -- 172.21.15.32:0/2335749333 >> v2:172.21.15.55:3300/0 conn(0x7fc7040afd60 msgr2=0x7fc7040a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 --2- 172.21.15.32:0/2335749333 >> v2:172.21.15.55:3300/0 conn(0x7fc7040afd60 0x7fc7040a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 -- 172.21.15.32:0/2335749333 >> v1:172.21.15.55:6789/0 conn(0x7fc7040b2710 legacy=0x7fc70414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 -- 172.21.15.32:0/2335749333 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040a79c0 con 0x7fc7040b01a0 2022-08-01T23:05:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7098bd700 1 --2- 172.21.15.32:0/2335749333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc7040a8540 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fc70406b950 tx=0x7fc700006eb0).ready entity=mon.0 client_cookie=31fc38b303ecb350 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7090bc700 1 -- 172.21.15.32:0/2335749333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70000d5c0 con 0x7fc7040b01a0 2022-08-01T23:05:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7090bc700 1 -- 172.21.15.32:0/2335749333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc700000e90 con 0x7fc7040b01a0 2022-08-01T23:05:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.764+0000 7fc7090bc700 1 -- 172.21.15.32:0/2335749333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc70000d910 con 0x7fc7040b01a0 2022-08-01T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- 172.21.15.32:0/2335749333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 msgr2=0x7fc7040a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/2335749333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc7040a8540 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fc70406b950 tx=0x7fc700006eb0).stop 2022-08-01T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- 172.21.15.32:0/2335749333 shutdown_connections 2022-08-01T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/2335749333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc7040a8540 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/2335749333 >> v2:172.21.15.55:3300/0 conn(0x7fc7040afd60 0x7fc7040a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.781 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- 172.21.15.32:0/2335749333 >> 172.21.15.32:0/2335749333 conn(0x7fc704060310 msgr2=0x7fc7040b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- 172.21.15.32:0/2335749333 shutdown_connections 2022-08-01T23:05:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- 172.21.15.32:0/2335749333 wait complete. 2022-08-01T23:05:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 Processor -- start 2022-08-01T23:05:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- start start 2022-08-01T23:05:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7040afd60 0x7fc704067070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc70414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7040b2710 0x7fc7040b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040b5e90 con 0x7fc7040b01a0 2022-08-01T23:05:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704063e90 con 0x7fc7040b2710 2022-08-01T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70b0c0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc704066690 con 0x7fc7040afd60 2022-08-01T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc7098bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc70414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc7098bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc70414a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49650/0 (socket says 172.21.15.32:49650) 2022-08-01T23:05:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc7098bd700 1 -- 172.21.15.32:0/3805868744 learned_addr learned my addr 172.21.15.32:0/3805868744 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70a8bf700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7040b2710 0x7fc7040b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.768+0000 7fc70a0be700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7040afd60 0x7fc704067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc7098bd700 1 -- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7040afd60 msgr2=0x7fc704067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc7098bd700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7040afd60 0x7fc704067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc7098bd700 1 -- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7040b2710 msgr2=0x7fc7040b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc7098bd700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7040b2710 0x7fc7040b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc7098bd700 1 -- 172.21.15.32:0/3805868744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70000b1e0 con 0x7fc7040b01a0 2022-08-01T23:05:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc7098bd700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc70414a0c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fc70406b950 tx=0x7fc700015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7000117e0 con 0x7fc7040b01a0 2022-08-01T23:05:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704066930 con 0x7fc7040b01a0 2022-08-01T23:05:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7040b40e0 con 0x7fc7040b01a0 2022-08-01T23:05:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc700011940 con 0x7fc7040b01a0 2022-08-01T23:05:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc700011ca0 con 0x7fc7040b01a0 2022-08-01T23:05:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e0003410 con 0x7fc7040b01a0 2022-08-01T23:05:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc70001a040 con 0x7fc7040b01a0 2022-08-01T23:05:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc6f37fe700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc70006cf70 con 0x7fc7040b01a0 2022-08-01T23:05:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.772+0000 7fc70a0be700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.776+0000 7fc70a0be700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fc6f400b000 tx=0x7fc6f4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.784+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc700043ce0 con 0x7fc7040b01a0 2022-08-01T23:05:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.940+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6e0003c40 con 0x7fc7040b01a0 2022-08-01T23:05:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc6f37fe700 1 -- 172.21.15.32:0/3805868744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fc700023090 con 0x7fc7040b01a0 2022-08-01T23:05:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:14.950 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":75,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:02.448548+0000","modified":"2022-08-01T23:05:14.624826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":148,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T23:05:14.951 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 msgr2=0x7fc6e4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fc6f400b000 tx=0x7fc6f4009040).stop 2022-08-01T23:05:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 msgr2=0x7fc70414a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc70414a0c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fc70406b950 tx=0x7fc700015000).stop 2022-08-01T23:05:14.952 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 shutdown_connections 2022-08-01T23:05:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc6e4053a70 0x7fc6e4055f30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7040b2710 0x7fc7040b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7040b01a0 0x7fc70414a0c0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 --2- 172.21.15.32:0/3805868744 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7040afd60 0x7fc704067070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 >> 172.21.15.32:0/3805868744 conn(0x7fc704060310 msgr2=0x7fc704146750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 shutdown_connections 2022-08-01T23:05:14.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:14.944+0000 7fc70b0c0700 1 -- 172.21.15.32:0/3805868744 wait complete. 2022-08-01T23:05:14.955 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 75 2022-08-01T23:05:14.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:05:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.064+0000 7fad33040700 1 Processor -- start 2022-08-01T23:05:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.064+0000 7fad33040700 1 -- start start 2022-08-01T23:05:15.085 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad2c05c960 0x7fad2c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad2c05d240 0x7fad2c13f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.086 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c13fa10 con 0x7fad2c0acfc0 2022-08-01T23:05:15.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c067f00 con 0x7fad2c05d240 2022-08-01T23:05:15.087 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c05cd40 con 0x7fad2c05c960 2022-08-01T23:05:15.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad2c05d240 0x7fad2c13f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.088 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad2c05d240 0x7fad2c13f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49658/0 (socket says 172.21.15.32:49658) 2022-08-01T23:05:15.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 -- 172.21.15.32:0/4025836788 learned_addr learned my addr 172.21.15.32:0/4025836788 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:15.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3203e700 1 --2- 172.21.15.32:0/4025836788 >> v2:172.21.15.55:3300/0 conn(0x7fad2c05c960 0x7fad2c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.089 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 -- 172.21.15.32:0/4025836788 >> v2:172.21.15.55:3300/0 conn(0x7fad2c05c960 msgr2=0x7fad2c0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:15.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 --2- 172.21.15.32:0/4025836788 >> v2:172.21.15.55:3300/0 conn(0x7fad2c05c960 0x7fad2c0b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.090 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 -- 172.21.15.32:0/4025836788 >> v1:172.21.15.55:6789/0 conn(0x7fad2c0acfc0 legacy=0x7fad2c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 -- 172.21.15.32:0/4025836788 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0b1920 con 0x7fad2c05d240 2022-08-01T23:05:15.091 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3183d700 1 --2- 172.21.15.32:0/4025836788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05d240 0x7fad2c13f520 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fad280032f0 tx=0x7fad28009290).ready entity=mon.0 client_cookie=c7695afd8e0762d0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:15.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3103c700 1 -- 172.21.15.32:0/4025836788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2800b870 con 0x7fad2c05d240 2022-08-01T23:05:15.092 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3103c700 1 -- 172.21.15.32:0/4025836788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2800b9d0 con 0x7fad2c05d240 2022-08-01T23:05:15.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad3103c700 1 -- 172.21.15.32:0/4025836788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2800bb40 con 0x7fad2c05d240 2022-08-01T23:05:15.093 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- 172.21.15.32:0/4025836788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05d240 msgr2=0x7fad2c13f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.094 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 --2- 172.21.15.32:0/4025836788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05d240 0x7fad2c13f520 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fad280032f0 tx=0x7fad28009290).stop 2022-08-01T23:05:15.098 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- 172.21.15.32:0/4025836788 shutdown_connections 2022-08-01T23:05:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 --2- 172.21.15.32:0/4025836788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05d240 0x7fad2c13f520 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 --2- 172.21.15.32:0/4025836788 >> v2:172.21.15.55:3300/0 conn(0x7fad2c05c960 0x7fad2c0b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- 172.21.15.32:0/4025836788 >> 172.21.15.32:0/4025836788 conn(0x7fad2c060310 msgr2=0x7fad2c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:15.100 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- 172.21.15.32:0/4025836788 shutdown_connections 2022-08-01T23:05:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.068+0000 7fad33040700 1 -- 172.21.15.32:0/4025836788 wait complete. 2022-08-01T23:05:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 Processor -- start 2022-08-01T23:05:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- start start 2022-08-01T23:05:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 0x7fad2c143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad2c05d240 0x7fad2c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad2c0acfc0 0x7fad2c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c141f40 con 0x7fad2c05c960 2022-08-01T23:05:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c145a40 con 0x7fad2c0acfc0 2022-08-01T23:05:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c145ce0 con 0x7fad2c05d240 2022-08-01T23:05:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 0x7fad2c143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 0x7fad2c143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49664/0 (socket says 172.21.15.32:49664) 2022-08-01T23:05:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 -- 172.21.15.32:0/2221661412 learned_addr learned my addr 172.21.15.32:0/2221661412 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3283f700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad2c0acfc0 0x7fad2c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3183d700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad2c05d240 0x7fad2c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 -- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad2c05d240 msgr2=0x7fad2c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad2c05d240 0x7fad2c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 -- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad2c0acfc0 msgr2=0x7fad2c152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad2c0acfc0 0x7fad2c152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 -- 172.21.15.32:0/2221661412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009530 con 0x7fad2c05c960 2022-08-01T23:05:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3283f700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad2c0acfc0 0x7fad2c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad3203e700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 0x7fad2c143d10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fad200030b0 tx=0x7fad20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad20011780 con 0x7fad2c05c960 2022-08-01T23:05:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad200118e0 con 0x7fad2c05c960 2022-08-01T23:05:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad20011c40 con 0x7fad2c05c960 2022-08-01T23:05:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c144200 con 0x7fad2c05c960 2022-08-01T23:05:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c1435b0 con 0x7fad2c05c960 2022-08-01T23:05:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c0574e0 con 0x7fad2c05c960 2022-08-01T23:05:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.072+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad2001c460 con 0x7fad2c05c960 2022-08-01T23:05:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.076+0000 7fad1f7fe700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.076+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fad2006c3a0 con 0x7fad2c05c960 2022-08-01T23:05:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.076+0000 7fad3183d700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.076+0000 7fad3183d700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fad28007000 tx=0x7fad28016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.080+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad200430b0 con 0x7fad2c05c960 2022-08-01T23:05:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.244+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fad2c0af660 con 0x7fad2c05c960 2022-08-01T23:05:15.647 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.640+0000 7fad1f7fe700 1 -- 172.21.15.32:0/2221661412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fad200466d0 con 0x7fad2c05c960 2022-08-01T23:05:15.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 msgr2=0x7fad0c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fad28007000 tx=0x7fad28016000).stop 2022-08-01T23:05:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 msgr2=0x7fad2c143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 0x7fad2c143d10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fad200030b0 tx=0x7fad20008040).stop 2022-08-01T23:05:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 shutdown_connections 2022-08-01T23:05:15.652 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fad0c053a70 0x7fad0c055f30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad2c0acfc0 0x7fad2c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fad2c05d240 0x7fad2c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 --2- 172.21.15.32:0/2221661412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad2c05c960 0x7fad2c143d10 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 >> 172.21.15.32:0/2221661412 conn(0x7fad2c060310 msgr2=0x7fad2c0b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 shutdown_connections 2022-08-01T23:05:15.654 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.644+0000 7fad33040700 1 -- 172.21.15.32:0/2221661412 wait complete. 2022-08-01T23:05:15.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:05:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f161790e700 1 Processor -- start 2022-08-01T23:05:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f161790e700 1 -- start start 2022-08-01T23:05:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f161790e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16100b4550 0x7f16100b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f161790e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1610144240 con 0x7f16100b6af0 2022-08-01T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f161790e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1610143d00 con 0x7f16100b6710 2022-08-01T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f161790e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1610143fa0 con 0x7f16100b4550 2022-08-01T23:05:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f1614ea9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16100b4550 0x7f16100b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f1614ea9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16100b4550 0x7f16100b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49674/0 (socket says 172.21.15.32:49674) 2022-08-01T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f1614ea9700 1 -- 172.21.15.32:0/3877472012 learned_addr learned my addr 172.21.15.32:0/3877472012 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f1614ea9700 1 -- 172.21.15.32:0/3877472012 >> v1:172.21.15.55:6789/0 conn(0x7f16100b6710 legacy=0x7f1610061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f1614ea9700 1 -- 172.21.15.32:0/3877472012 >> v1:172.21.15.32:6789/0 conn(0x7f16100b6af0 legacy=0x7f161012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.780+0000 7f1614ea9700 1 -- 172.21.15.32:0/3877472012 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610061780 con 0x7f16100b4550 2022-08-01T23:05:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f1614ea9700 1 --2- 172.21.15.32:0/3877472012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b5d80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f16101432b0 tx=0x7f1600006eb0).ready entity=mon.0 client_cookie=d8d2d56610285061 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f1607fff700 1 -- 172.21.15.32:0/3877472012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f160000d660 con 0x7f16100b4550 2022-08-01T23:05:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f1607fff700 1 -- 172.21.15.32:0/3877472012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600000e90 con 0x7f16100b4550 2022-08-01T23:05:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f1607fff700 1 -- 172.21.15.32:0/3877472012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f160000d9f0 con 0x7f16100b4550 2022-08-01T23:05:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- 172.21.15.32:0/3877472012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 msgr2=0x7f16100b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 --2- 172.21.15.32:0/3877472012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b5d80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f16101432b0 tx=0x7f1600006eb0).stop 2022-08-01T23:05:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- 172.21.15.32:0/3877472012 shutdown_connections 2022-08-01T23:05:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 --2- 172.21.15.32:0/3877472012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b5d80 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- 172.21.15.32:0/3877472012 >> 172.21.15.32:0/3877472012 conn(0x7f161005c620 msgr2=0x7f161005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- 172.21.15.32:0/3877472012 shutdown_connections 2022-08-01T23:05:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- 172.21.15.32:0/3877472012 wait complete. 2022-08-01T23:05:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 Processor -- start 2022-08-01T23:05:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- start start 2022-08-01T23:05:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16100b6710 0x7f1610152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16100b6af0 0x7f16101530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16101525b0 con 0x7f16100b4550 2022-08-01T23:05:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161012a580 con 0x7f16100b6af0 2022-08-01T23:05:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f161790e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16101527a0 con 0x7f16100b6710 2022-08-01T23:05:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.784+0000 7f16156aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49678/0 (socket says 172.21.15.32:49678) 2022-08-01T23:05:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 -- 172.21.15.32:0/3045718290 learned_addr learned my addr 172.21.15.32:0/3045718290 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f1614ea9700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16100b6710 0x7f1610152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f1615eab700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16100b6af0 0x7f16101530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 -- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16100b6710 msgr2=0x7f1610152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16100b6710 0x7f1610152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 -- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16100b6af0 msgr2=0x7f16101530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16100b6af0 0x7f16101530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 -- 172.21.15.32:0/3045718290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160000b1e0 con 0x7f16100b4550 2022-08-01T23:05:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16156aa700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b78e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f1610126f50 tx=0x7f160c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f160c007030 con 0x7f16100b4550 2022-08-01T23:05:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c007190 con 0x7f16100b4550 2022-08-01T23:05:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161012a2e0 con 0x7f16100b4550 2022-08-01T23:05:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f160c006510 con 0x7f16100b4550 2022-08-01T23:05:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16100b2b70 con 0x7f16100b4550 2022-08-01T23:05:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161012aa70 con 0x7f16100b4550 2022-08-01T23:05:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f160c010030 con 0x7f16100b4550 2022-08-01T23:05:15.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16067fc700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15f80539e0 0x7f15f8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:15.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f160c06bbf0 con 0x7f16100b4550 2022-08-01T23:05:15.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.788+0000 7f1614ea9700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15f80539e0 0x7f15f8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:15.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.792+0000 7f1614ea9700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15f80539e0 0x7f15f8055ea0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f160001b000 tx=0x7f1600015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:15.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.800+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f160c042960 con 0x7f16100b4550 2022-08-01T23:05:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:15.920+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1610061a20 con 0x7f16100b4550 2022-08-01T23:05:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.643+0000 7f16067fc700 1 -- 172.21.15.32:0/3045718290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v149) v1 ==== 185+0+0 (secure 0 0 0) 0x7f160c045f80 con 0x7f16100b4550 2022-08-01T23:05:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15f80539e0 msgr2=0x7f15f8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15f80539e0 0x7f15f8055ea0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f160001b000 tx=0x7f1600015000).stop 2022-08-01T23:05:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 msgr2=0x7f16100b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b78e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f1610126f50 tx=0x7f160c007c10).stop 2022-08-01T23:05:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 shutdown_connections 2022-08-01T23:05:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f15f80539e0 0x7f15f8055ea0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16100b6af0 0x7f16101530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f16100b6710 0x7f1610152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 --2- 172.21.15.32:0/3045718290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16100b4550 0x7f16100b78e0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 >> 172.21.15.32:0/3045718290 conn(0x7f161005c620 msgr2=0x7f1610127270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 shutdown_connections 2022-08-01T23:05:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.647+0000 7f161790e700 1 -- 172.21.15.32:0/3045718290 wait complete. 2022-08-01T23:05:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:05:16.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:05:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.775+0000 7f1d389b4700 1 Processor -- start 2022-08-01T23:05:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.775+0000 7f1d389b4700 1 -- start start 2022-08-01T23:05:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d389b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d34060330 0x7f1d340b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d389b4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d340aa630 con 0x7f1d3405c6f0 2022-08-01T23:05:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d389b4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3405fd10 con 0x7f1d340af550 2022-08-01T23:05:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d389b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340b15e0 con 0x7f1d34060330 2022-08-01T23:05:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d33fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d3405c6f0 0x7f1d340a8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37232/0 (socket says 172.21.15.32:37232) 2022-08-01T23:05:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d33fff700 1 -- 172.21.15.32:0/694602351 learned_addr learned my addr 172.21.15.32:0/694602351 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d2bfff700 1 --2- 172.21.15.32:0/694602351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d34060330 0x7f1d340b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:16.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831135230 0 0) 0x7f1d340aa630 con 0x7f1d3405c6f0 2022-08-01T23:05:16.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d18003260 con 0x7f1d3405c6f0 2022-08-01T23:05:16.792 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051704333 0 0) 0x7f1d3405fd10 con 0x7f1d340af550 2022-08-01T23:05:16.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d340aa630 con 0x7f1d340af550 2022-08-01T23:05:16.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d2bfff700 1 -- 172.21.15.32:0/694602351 >> v1:172.21.15.55:6789/0 conn(0x7f1d340af550 legacy=0x7f1d341445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d2bfff700 1 -- 172.21.15.32:0/694602351 >> v1:172.21.15.32:6789/0 conn(0x7f1d3405c6f0 legacy=0x7f1d340a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d2bfff700 1 -- 172.21.15.32:0/694602351 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3405f840 con 0x7f1d34060330 2022-08-01T23:05:16.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d2bfff700 1 --2- 172.21.15.32:0/694602351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d34060330 0x7f1d340b0e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c006df0).ready entity=mon.2 client_cookie=129d35dc5401bbfc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:16.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c00d5c0 con 0x7f1d34060330 2022-08-01T23:05:16.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c000e90 con 0x7f1d34060330 2022-08-01T23:05:16.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d32ffd700 1 -- 172.21.15.32:0/694602351 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c00d910 con 0x7f1d34060330 2022-08-01T23:05:16.794 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d389b4700 1 -- 172.21.15.32:0/694602351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d34060330 msgr2=0x7f1d340b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.779+0000 7f1d389b4700 1 --2- 172.21.15.32:0/694602351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d34060330 0x7f1d340b0e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c006df0).stop 2022-08-01T23:05:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- 172.21.15.32:0/694602351 shutdown_connections 2022-08-01T23:05:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 --2- 172.21.15.32:0/694602351 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d34060330 0x7f1d340b0e40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- 172.21.15.32:0/694602351 >> 172.21.15.32:0/694602351 conn(0x7f1d340612d0 msgr2=0x7f1d34062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- 172.21.15.32:0/694602351 shutdown_connections 2022-08-01T23:05:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- 172.21.15.32:0/694602351 wait complete. 2022-08-01T23:05:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 Processor -- start 2022-08-01T23:05:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- start start 2022-08-01T23:05:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3405c6f0 0x7f1d341432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 0x7f1d341535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d340af550 0x7f1d34153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34144880 con 0x7f1d34060330 2022-08-01T23:05:16.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34144ad0 con 0x7f1d340af550 2022-08-01T23:05:16.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34144d70 con 0x7f1d3405c6f0 2022-08-01T23:05:16.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 0x7f1d341535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:16.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 0x7f1d341535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49694/0 (socket says 172.21.15.32:49694) 2022-08-01T23:05:16.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 -- 172.21.15.32:0/3010586047 learned_addr learned my addr 172.21.15.32:0/3010586047 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d337fe700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3405c6f0 0x7f1d341432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 -- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3405c6f0 msgr2=0x7f1d341432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d33fff700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d340af550 0x7f1d34153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:16.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3405c6f0 0x7f1d341432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 -- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d340af550 msgr2=0x7f1d34153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d340af550 0x7f1d34153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 -- 172.21.15.32:0/3010586047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c00b180 con 0x7f1d34060330 2022-08-01T23:05:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d337fe700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3405c6f0 0x7f1d341432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:16.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d33fff700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d340af550 0x7f1d34153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d2bfff700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 0x7f1d341535f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c01e400 con 0x7f1d34060330 2022-08-01T23:05:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c01e560 con 0x7f1d34060330 2022-08-01T23:05:16.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d34143ab0 con 0x7f1d34060330 2022-08-01T23:05:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c01e8c0 con 0x7f1d34060330 2022-08-01T23:05:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.783+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d34143f50 con 0x7f1d34060330 2022-08-01T23:05:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.787+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34143c40 con 0x7f1d34060330 2022-08-01T23:05:16.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.787+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d1c01c070 con 0x7f1d34060330 2022-08-01T23:05:16.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.787+0000 7f1d317fa700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d0c053b30 0x7f1d0c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:16.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.787+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f1d1c06c9a0 con 0x7f1d34060330 2022-08-01T23:05:16.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.787+0000 7f1d337fe700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d0c053b30 0x7f1d0c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.787+0000 7f1d337fe700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d0c053b30 0x7f1d0c055ff0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1d24008000 tx=0x7f1d24009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:16.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.795+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d1c043820 con 0x7f1d34060330 2022-08-01T23:05:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:16.919+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d340574e0 con 0x7f1d34060330 2022-08-01T23:05:17.667 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.659+0000 7f1d317fa700 1 -- 172.21.15.32:0/3010586047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v150) v1 ==== 173+0+0 (secure 0 0 0) 0x7f1d1c023070 con 0x7f1d34060330 2022-08-01T23:05:17.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d0c053b30 msgr2=0x7f1d0c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d0c053b30 0x7f1d0c055ff0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1d24008000 tx=0x7f1d24009040).stop 2022-08-01T23:05:17.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 msgr2=0x7f1d341535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 0x7f1d341535f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f1d1c002f70 tx=0x7f1d1c014000).stop 2022-08-01T23:05:17.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 shutdown_connections 2022-08-01T23:05:17.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d0c053b30 0x7f1d0c055ff0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d340af550 0x7f1d34153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d34060330 0x7f1d341535f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 --2- 172.21.15.32:0/3010586047 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d3405c6f0 0x7f1d341432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 >> 172.21.15.32:0/3010586047 conn(0x7f1d340612d0 msgr2=0x7f1d340a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 shutdown_connections 2022-08-01T23:05:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.663+0000 7f1d389b4700 1 -- 172.21.15.32:0/3010586047 wait complete. 2022-08-01T23:05:17.675 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:05:17.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T23:05:17.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 Processor -- start 2022-08-01T23:05:17.793 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- start start 2022-08-01T23:05:17.796 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8a0061860 0x7fe8a00b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:17.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a0140220 con 0x7fe8a00b1b10 2022-08-01T23:05:17.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a01404c0 con 0x7fe8a0060d00 2022-08-01T23:05:17.797 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a0061130 con 0x7fe8a0061860 2022-08-01T23:05:17.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a5d4e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8a00b1b10 0x7fe8a0139e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37242/0 (socket says 172.21.15.32:37242) 2022-08-01T23:05:17.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a5d4e700 1 -- 172.21.15.32:0/1076282455 learned_addr learned my addr 172.21.15.32:0/1076282455 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:17.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a554d700 1 --2- 172.21.15.32:0/1076282455 >> v2:172.21.15.55:3300/0 conn(0x7fe8a0061860 0x7fe8a00b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:17.798 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a4d4c700 1 -- 172.21.15.32:0/1076282455 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746250815 0 0) 0x7fe8a0140220 con 0x7fe8a00b1b10 2022-08-01T23:05:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a4d4c700 1 -- 172.21.15.32:0/1076282455 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe888003260 con 0x7fe8a00b1b10 2022-08-01T23:05:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a554d700 1 -- 172.21.15.32:0/1076282455 >> v1:172.21.15.55:6789/0 conn(0x7fe8a0060d00 legacy=0x7fe8a005f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a554d700 1 -- 172.21.15.32:0/1076282455 >> v1:172.21.15.32:6789/0 conn(0x7fe8a00b1b10 legacy=0x7fe8a0139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.799 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a554d700 1 -- 172.21.15.32:0/1076282455 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0140760 con 0x7fe8a0061860 2022-08-01T23:05:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a554d700 1 --2- 172.21.15.32:0/1076282455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a00b1ef0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fe890002f70 tx=0x7fe890006df0).ready entity=mon.1 client_cookie=92d7ab444efdf178 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a4d4c700 1 -- 172.21.15.32:0/1076282455 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89000d610 con 0x7fe8a0061860 2022-08-01T23:05:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a4d4c700 1 -- 172.21.15.32:0/1076282455 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe890000e90 con 0x7fe8a0061860 2022-08-01T23:05:17.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a4d4c700 1 -- 172.21.15.32:0/1076282455 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89000d960 con 0x7fe8a0061860 2022-08-01T23:05:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1076282455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 msgr2=0x7fe8a00b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1076282455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a00b1ef0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fe890002f70 tx=0x7fe890006df0).stop 2022-08-01T23:05:17.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1076282455 shutdown_connections 2022-08-01T23:05:17.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1076282455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a00b1ef0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1076282455 >> 172.21.15.32:0/1076282455 conn(0x7fe8a00b2b00 msgr2=0x7fe8a00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:17.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1076282455 shutdown_connections 2022-08-01T23:05:17.803 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.787+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1076282455 wait complete. 2022-08-01T23:05:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 Processor -- start 2022-08-01T23:05:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 -- start start 2022-08-01T23:05:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 0x7fe8a013bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a013c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8a00b1b10 0x7fe8a00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a0150190 con 0x7fe8a0060d00 2022-08-01T23:05:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a013b460 con 0x7fe8a0061860 2022-08-01T23:05:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a013b700 con 0x7fe8a00b1b10 2022-08-01T23:05:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 0x7fe8a013bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 0x7fe8a013bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49708/0 (socket says 172.21.15.32:49708) 2022-08-01T23:05:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 -- 172.21.15.32:0/1442949468 learned_addr learned my addr 172.21.15.32:0/1442949468 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a554d700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a013c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a654f700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8a00b1b10 0x7fe8a00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 -- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8a00b1b10 msgr2=0x7fe8a00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8a00b1b10 0x7fe8a00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 -- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 msgr2=0x7fe8a013c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a013c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 -- 172.21.15.32:0/1442949468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89000b180 con 0x7fe8a0060d00 2022-08-01T23:05:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a5d4e700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 0x7fe8a013bba0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fe89c001110 tx=0x7fe89c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:17.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c0070d0 con 0x7fe8a0060d00 2022-08-01T23:05:17.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c007230 con 0x7fe8a0060d00 2022-08-01T23:05:17.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a013a910 con 0x7fe8a0060d00 2022-08-01T23:05:17.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89c005610 con 0x7fe8a0060d00 2022-08-01T23:05:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.791+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a013af00 con 0x7fe8a0060d00 2022-08-01T23:05:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.795+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a013b2c0 con 0x7fe8a0060d00 2022-08-01T23:05:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.795+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe89c010070 con 0x7fe8a0060d00 2022-08-01T23:05:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.795+0000 7fe88effd700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe8800539e0 0x7fe880055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.795+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe89c06b3e0 con 0x7fe8a0060d00 2022-08-01T23:05:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.795+0000 7fe8a554d700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe8800539e0 0x7fe880055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.795+0000 7fe8a554d700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe8800539e0 0x7fe880055ea0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fe89001a000 tx=0x7fe890014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:17.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.799+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe8a013b2c0 con 0x7fe8a0060d00 2022-08-01T23:05:17.933 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.927+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe8a013b2c0 con 0x7fe8a0060d00 2022-08-01T23:05:17.941 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe88effd700 1 -- 172.21.15.32:0/1442949468 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe8a013b2c0 con 0x7fe8a0060d00 2022-08-01T23:05:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe8800539e0 msgr2=0x7fe880055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe8800539e0 0x7fe880055ea0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fe89001a000 tx=0x7fe890014000).stop 2022-08-01T23:05:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 msgr2=0x7fe8a013bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 0x7fe8a013bba0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fe89c001110 tx=0x7fe89c007ad0).stop 2022-08-01T23:05:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 shutdown_connections 2022-08-01T23:05:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fe8800539e0 0x7fe880055ea0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fe8a00b1b10 0x7fe8a00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8a0061860 0x7fe8a013c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 --2- 172.21.15.32:0/1442949468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a0060d00 0x7fe8a013bba0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 >> 172.21.15.32:0/1442949468 conn(0x7fe8a00b2b00 msgr2=0x7fe8a00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 shutdown_connections 2022-08-01T23:05:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:17.935+0000 7fe8a7fb2700 1 -- 172.21.15.32:0/1442949468 wait complete. 2022-08-01T23:05:17.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-08-01T23:05:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 Processor -- start 2022-08-01T23:05:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- start start 2022-08-01T23:05:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84140a7840 0x7f8414066c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.064 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140b1b00 0x7f84140670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84140675e0 con 0x7f84140b01f0 2022-08-01T23:05:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140a7c20 con 0x7f84140a7840 2022-08-01T23:05:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84141477b0 con 0x7f84140b1b00 2022-08-01T23:05:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84140a7840 0x7f8414066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84140a7840 0x7f8414066c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49714/0 (socket says 172.21.15.32:49714) 2022-08-01T23:05:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 -- 172.21.15.32:0/3584197694 learned_addr learned my addr 172.21.15.32:0/3584197694 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8419317700 1 --1- 172.21.15.32:0/3584197694 >> v1:172.21.15.32:6789/0 conn(0x7f84140b01f0 0x7f841405f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37260/0 (socket says 172.21.15.32:37260) 2022-08-01T23:05:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8413fff700 1 --2- 172.21.15.32:0/3584197694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140b1b00 0x7f84140670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 -- 172.21.15.32:0/3584197694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140b1b00 msgr2=0x7f84140670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 --2- 172.21.15.32:0/3584197694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140b1b00 0x7f84140670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 -- 172.21.15.32:0/3584197694 >> v1:172.21.15.32:6789/0 conn(0x7f84140b01f0 legacy=0x7f841405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 -- 172.21.15.32:0/3584197694 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8414147a50 con 0x7f84140a7840 2022-08-01T23:05:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f8418b16700 1 --2- 172.21.15.32:0/3584197694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140a7840 0x7f8414066c00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f8408002f70 tx=0x7f8408006eb0).ready entity=mon.0 client_cookie=41103ac31cc40eb4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f84137fe700 1 -- 172.21.15.32:0/3584197694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840800d5c0 con 0x7f84140a7840 2022-08-01T23:05:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f84137fe700 1 -- 172.21.15.32:0/3584197694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408000e90 con 0x7f84140a7840 2022-08-01T23:05:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f84137fe700 1 -- 172.21.15.32:0/3584197694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840800d950 con 0x7f84140a7840 2022-08-01T23:05:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- 172.21.15.32:0/3584197694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140a7840 msgr2=0x7f8414066c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 --2- 172.21.15.32:0/3584197694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140a7840 0x7f8414066c00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f8408002f70 tx=0x7f8408006eb0).stop 2022-08-01T23:05:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- 172.21.15.32:0/3584197694 shutdown_connections 2022-08-01T23:05:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 --2- 172.21.15.32:0/3584197694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140b1b00 0x7f84140670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 --2- 172.21.15.32:0/3584197694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140a7840 0x7f8414066c00 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- 172.21.15.32:0/3584197694 >> 172.21.15.32:0/3584197694 conn(0x7f84140b5570 msgr2=0x7f84140b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:18.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- 172.21.15.32:0/3584197694 shutdown_connections 2022-08-01T23:05:18.070 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.055+0000 7f841ad7a700 1 -- 172.21.15.32:0/3584197694 wait complete. 2022-08-01T23:05:18.073 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 Processor -- start 2022-08-01T23:05:18.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 -- start start 2022-08-01T23:05:18.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140a7840 0x7f8414149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 0x7f84140adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.074 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84140b1b00 0x7f84140ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414156ad0 con 0x7f84140b01f0 2022-08-01T23:05:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84140ae640 con 0x7f84140b1b00 2022-08-01T23:05:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8414157a90 con 0x7f84140a7840 2022-08-01T23:05:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 0x7f84140adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 0x7f84140adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49724/0 (socket says 172.21.15.32:49724) 2022-08-01T23:05:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 -- 172.21.15.32:0/255011860 learned_addr learned my addr 172.21.15.32:0/255011860 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8418b16700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140a7840 0x7f8414149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8419317700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84140b1b00 0x7f84140ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 -- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140a7840 msgr2=0x7f8414149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140a7840 0x7f8414149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 -- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84140b1b00 msgr2=0x7f84140ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84140b1b00 0x7f84140ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 -- 172.21.15.32:0/255011860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840800b1e0 con 0x7f84140b01f0 2022-08-01T23:05:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8419317700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84140b1b00 0x7f84140ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8413fff700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 0x7f84140adc60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f84140ae780 tx=0x7f8404008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8404011780 con 0x7f84140b01f0 2022-08-01T23:05:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84040118e0 con 0x7f84140b01f0 2022-08-01T23:05:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8414156cc0 con 0x7f84140b01f0 2022-08-01T23:05:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8404011c40 con 0x7f84140b01f0 2022-08-01T23:05:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.059+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84140ac160 con 0x7f84140b01f0 2022-08-01T23:05:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.063+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140574e0 con 0x7f84140b01f0 2022-08-01T23:05:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.063+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f840401c460 con 0x7f84140b01f0 2022-08-01T23:05:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.063+0000 7f8411ffb700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f83f40539e0 0x7f83f4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.063+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f840406bfe0 con 0x7f84140b01f0 2022-08-01T23:05:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.063+0000 7f8418b16700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f83f40539e0 0x7f83f4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.063+0000 7f8418b16700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f83f40539e0 0x7f83f4055ea0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f840801c000 tx=0x7f8408015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:18.081 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.071+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8404042f50 con 0x7f84140b01f0 2022-08-01T23:05:18.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.235+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f84140673c0 con 0x7f84140b01f0 2022-08-01T23:05:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.663+0000 7f8411ffb700 1 -- 172.21.15.32:0/255011860 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v440) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8404046570 con 0x7f84140b01f0 2022-08-01T23:05:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f83f40539e0 msgr2=0x7f83f4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f83f40539e0 0x7f83f4055ea0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f840801c000 tx=0x7f8408015000).stop 2022-08-01T23:05:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 msgr2=0x7f84140adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 0x7f84140adc60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f84140ae780 tx=0x7f8404008040).stop 2022-08-01T23:05:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 shutdown_connections 2022-08-01T23:05:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f83f40539e0 0x7f83f4055ea0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84140b1b00 0x7f84140ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84140b01f0 0x7f84140adc60 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 --2- 172.21.15.32:0/255011860 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f84140a7840 0x7f8414149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 >> 172.21.15.32:0/255011860 conn(0x7f84140b5570 msgr2=0x7f84140600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 shutdown_connections 2022-08-01T23:05:18.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.667+0000 7f841ad7a700 1 -- 172.21.15.32:0/255011860 wait complete. 2022-08-01T23:05:18.691 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T23:05:18.691 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:05:18.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2022-08-01T23:05:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.791+0000 7f4cd359e700 1 Processor -- start 2022-08-01T23:05:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.791+0000 7f4cd359e700 1 -- start start 2022-08-01T23:05:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4cd40ad750 0x7f4cd4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd41405d0 0x7f4cd4140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd40aee80 con 0x7f4cd40614b0 2022-08-01T23:05:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd40b70b0 con 0x7f4cd40ad750 2022-08-01T23:05:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40af0b0 con 0x7f4cd41405d0 2022-08-01T23:05:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4cd40614b0 0x7f4cd40b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37270/0 (socket says 172.21.15.32:37270) 2022-08-01T23:05:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd259c700 1 -- 172.21.15.32:0/2110659487 learned_addr learned my addr 172.21.15.32:0/2110659487 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd2d9d700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd41405d0 0x7f4cd4140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4ccbfff700 1 --2- 172.21.15.32:0/2110659487 >> v2:172.21.15.55:3300/0 conn(0x7f4cd40ad750 0x7f4cd4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd1d9b700 1 -- 172.21.15.32:0/2110659487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567251163 0 0) 0x7f4cd40aee80 con 0x7f4cd40614b0 2022-08-01T23:05:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd1d9b700 1 -- 172.21.15.32:0/2110659487 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb4003260 con 0x7f4cd40614b0 2022-08-01T23:05:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4ccbfff700 1 -- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd41405d0 msgr2=0x7f4cd4140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4ccbfff700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd41405d0 0x7f4cd4140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4ccbfff700 1 -- 172.21.15.32:0/2110659487 >> v1:172.21.15.32:6789/0 conn(0x7f4cd40614b0 legacy=0x7f4cd40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd1d9b700 1 -- 172.21.15.32:0/2110659487 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 231623532 0 0) 0x7f4cb4003260 con 0x7f4cd40614b0 2022-08-01T23:05:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4ccbfff700 1 -- 172.21.15.32:0/2110659487 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40af350 con 0x7f4cd40ad750 2022-08-01T23:05:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd2d9d700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd41405d0 0x7f4cd4140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4ccbfff700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4061890 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f4cbc002f70 tx=0x7f4cbc006df0).ready entity=mon.1 client_cookie=952e73c164abdf00 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd1d9b700 1 -- 172.21.15.32:0/2110659487 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cbc00d4a0 con 0x7f4cd40ad750 2022-08-01T23:05:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd1d9b700 1 -- 172.21.15.32:0/2110659487 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc000e90 con 0x7f4cd40ad750 2022-08-01T23:05:18.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.795+0000 7f4cd1d9b700 1 -- 172.21.15.32:0/2110659487 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cbc00d7f0 con 0x7f4cd40ad750 2022-08-01T23:05:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 msgr2=0x7f4cd4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4061890 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f4cbc002f70 tx=0x7f4cbc006df0).stop 2022-08-01T23:05:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- 172.21.15.32:0/2110659487 shutdown_connections 2022-08-01T23:05:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd41405d0 0x7f4cd4140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 --2- 172.21.15.32:0/2110659487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4061890 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- 172.21.15.32:0/2110659487 >> 172.21.15.32:0/2110659487 conn(0x7f4cd4066f70 msgr2=0x7f4cd4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- 172.21.15.32:0/2110659487 shutdown_connections 2022-08-01T23:05:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- 172.21.15.32:0/2110659487 wait complete. 2022-08-01T23:05:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 Processor -- start 2022-08-01T23:05:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- start start 2022-08-01T23:05:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd40614b0 0x7f4cd4148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 0x7f4cd4152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd4152450 con 0x7f4cd41405d0 2022-08-01T23:05:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd414a880 con 0x7f4cd40ad750 2022-08-01T23:05:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd414a5e0 con 0x7f4cd40614b0 2022-08-01T23:05:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 0x7f4cd4152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 0x7f4cd4152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49738/0 (socket says 172.21.15.32:49738) 2022-08-01T23:05:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd2d9d700 1 -- 172.21.15.32:0/300999975 learned_addr learned my addr 172.21.15.32:0/300999975 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4ccbfff700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.799+0000 7f4cd259c700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd40614b0 0x7f4cd4148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd2d9d700 1 -- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd40614b0 msgr2=0x7f4cd4148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd2d9d700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd40614b0 0x7f4cd4148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd2d9d700 1 -- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 msgr2=0x7f4cd4149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd2d9d700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd2d9d700 1 -- 172.21.15.32:0/300999975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc009470 con 0x7f4cd41405d0 2022-08-01T23:05:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4ccbfff700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd2d9d700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 0x7f4cd4152b50 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f4ccc00c000 tx=0x7f4ccc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ccc00ee60 con 0x7f4cd41405d0 2022-08-01T23:05:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc00c850 con 0x7f4cd41405d0 2022-08-01T23:05:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc00b180 con 0x7f4cd41405d0 2022-08-01T23:05:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ccc00cbb0 con 0x7f4cd41405d0 2022-08-01T23:05:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd40b1770 con 0x7f4cd41405d0 2022-08-01T23:05:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd40a9f20 con 0x7f4cd41405d0 2022-08-01T23:05:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.803+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ccc022070 con 0x7f4cd41405d0 2022-08-01T23:05:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.807+0000 7f4ccb7fe700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4cb80539e0 0x7f4cb8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.807+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4ccc06c560 con 0x7f4cd41405d0 2022-08-01T23:05:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.807+0000 7f4cd259c700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4cb80539e0 0x7f4cb8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.807+0000 7f4cd259c700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4cb80539e0 0x7f4cb8055ea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f4cc400d000 tx=0x7f4cc4001740).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.811+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ccc0434f0 con 0x7f4cd41405d0 2022-08-01T23:05:18.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:18.979+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7f4cd40af5f0 con 0x7f4cd41405d0 2022-08-01T23:05:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.667+0000 7f4ccb7fe700 1 -- 172.21.15.32:0/300999975 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v441) v1 ==== 139+0+0 (secure 0 0 0) 0x7f4ccc046b10 con 0x7f4cd41405d0 2022-08-01T23:05:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4cb80539e0 msgr2=0x7f4cb8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4cb80539e0 0x7f4cb8055ea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f4cc400d000 tx=0x7f4cc4001740).stop 2022-08-01T23:05:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 msgr2=0x7f4cd4152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 0x7f4cd4152b50 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f4ccc00c000 tx=0x7f4ccc0091d0).stop 2022-08-01T23:05:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 shutdown_connections 2022-08-01T23:05:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4cb80539e0 0x7f4cb8055ea0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cd41405d0 0x7f4cd4152b50 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cd40ad750 0x7f4cd4149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 --2- 172.21.15.32:0/300999975 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4cd40614b0 0x7f4cd4148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.671+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 >> 172.21.15.32:0/300999975 conn(0x7f4cd4066f70 msgr2=0x7f4cd40b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.675+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 shutdown_connections 2022-08-01T23:05:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.675+0000 7f4cd359e700 1 -- 172.21.15.32:0/300999975 wait complete. 2022-08-01T23:05:19.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 Processor -- start 2022-08-01T23:05:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 -- start start 2022-08-01T23:05:19.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d30146ae0 0x7f1d301403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:19.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d300b4310 0x7f1d301408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:19.812 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d30143cf0 con 0x7f1d300b4820 2022-08-01T23:05:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d301476a0 con 0x7f1d300b4310 2022-08-01T23:05:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d36d9e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30143f90 con 0x7f1d30146ae0 2022-08-01T23:05:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d300b4310 0x7f1d301408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d34b3a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d30146ae0 0x7f1d301403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d300b4310 0x7f1d301408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:59530/0 (socket says 172.21.15.32:59530) 2022-08-01T23:05:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d34b3a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d30146ae0 0x7f1d301403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:33250/0 (socket says 172.21.15.32:33250) 2022-08-01T23:05:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 -- 172.21.15.32:0/4147271255 learned_addr learned my addr 172.21.15.32:0/4147271255 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d3533b700 1 --1- 172.21.15.32:0/4147271255 >> v1:172.21.15.55:6789/0 conn(0x7f1d300b4820 0x7f1d300b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:58196/0 (socket says 172.21.15.32:58196) 2022-08-01T23:05:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 -- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d30146ae0 msgr2=0x7f1d301403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 --2- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d30146ae0 0x7f1d301403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 -- 172.21.15.32:0/4147271255 >> v1:172.21.15.55:6789/0 conn(0x7f1d300b4820 legacy=0x7f1d300b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2f7fe700 1 -- 172.21.15.32:0/4147271255 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126995897 0 0) 0x7f1d30143cf0 con 0x7f1d300b4820 2022-08-01T23:05:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 -- 172.21.15.32:0/4147271255 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300b0e20 con 0x7f1d300b4310 2022-08-01T23:05:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2ffff700 1 --2- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d300b4310 0x7f1d301408b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f1d24002f70 tx=0x7f1d24006eb0).ready entity=mon.1 client_cookie=1075a6c0f6574953 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2f7fe700 1 -- 172.21.15.32:0/4147271255 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2400d570 con 0x7f1d300b4310 2022-08-01T23:05:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2f7fe700 1 -- 172.21.15.32:0/4147271255 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d24000e90 con 0x7f1d300b4310 2022-08-01T23:05:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.803+0000 7f1d2f7fe700 1 -- 172.21.15.32:0/4147271255 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2400d8c0 con 0x7f1d300b4310 2022-08-01T23:05:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 -- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d300b4310 msgr2=0x7f1d301408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d300b4310 0x7f1d301408b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f1d24002f70 tx=0x7f1d24006eb0).stop 2022-08-01T23:05:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 -- 172.21.15.32:0/4147271255 shutdown_connections 2022-08-01T23:05:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d300b4310 0x7f1d301408b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/4147271255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d30146ae0 0x7f1d301403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 -- 172.21.15.32:0/4147271255 >> 172.21.15.32:0/4147271255 conn(0x7f1d3005c620 msgr2=0x7f1d3005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 -- 172.21.15.32:0/4147271255 shutdown_connections 2022-08-01T23:05:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 -- 172.21.15.32:0/4147271255 wait complete. 2022-08-01T23:05:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 Processor -- start 2022-08-01T23:05:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.807+0000 7f1d36d9e700 1 -- start start 2022-08-01T23:05:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 0x7f1d300adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d300b4820 0x7f1d30152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d30146ae0 0x7f1d30152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30142500 con 0x7f1d300b4310 2022-08-01T23:05:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300b9690 con 0x7f1d30146ae0 2022-08-01T23:05:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300ae440 con 0x7f1d300b4820 2022-08-01T23:05:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 0x7f1d300adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d2ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d300b4820 0x7f1d30152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 0x7f1d300adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49750/0 (socket says 172.21.15.32:49750) 2022-08-01T23:05:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 -- 172.21.15.32:0/1711997469 learned_addr learned my addr 172.21.15.32:0/1711997469 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d3533b700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d30146ae0 0x7f1d30152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 -- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d300b4820 msgr2=0x7f1d30152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d300b4820 0x7f1d30152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 -- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d30146ae0 msgr2=0x7f1d30152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d30146ae0 0x7f1d30152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 -- 172.21.15.32:0/1711997469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2400b1e0 con 0x7f1d300b4310 2022-08-01T23:05:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d34b3a700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 0x7f1d300adac0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f1d2000c000 tx=0x7f1d20006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d20011750 con 0x7f1d300b4310 2022-08-01T23:05:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d30142730 con 0x7f1d300b4310 2022-08-01T23:05:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d200118b0 con 0x7f1d300b4310 2022-08-01T23:05:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d30142a10 con 0x7f1d300b4310 2022-08-01T23:05:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d20011c10 con 0x7f1d300b4310 2022-08-01T23:05:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.811+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d30141be0 con 0x7f1d300b4310 2022-08-01T23:05:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.815+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d20011d70 con 0x7f1d300b4310 2022-08-01T23:05:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.815+0000 7f1d2dffb700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d100539e0 0x7f1d10055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.815+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f1d2006c2f0 con 0x7f1d300b4310 2022-08-01T23:05:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.815+0000 7f1d2ffff700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d100539e0 0x7f1d10055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.815+0000 7f1d2ffff700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d100539e0 0x7f1d10055ea0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f1d2401c000 tx=0x7f1d24015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.823+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d30141be0 con 0x7f1d300b4310 2022-08-01T23:05:19.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.951+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1d30141be0 con 0x7f1d300b4310 2022-08-01T23:05:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d2dffb700 1 -- 172.21.15.32:0/1711997469 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1d30141be0 con 0x7f1d300b4310 2022-08-01T23:05:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d100539e0 msgr2=0x7f1d10055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d100539e0 0x7f1d10055ea0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f1d2401c000 tx=0x7f1d24015000).stop 2022-08-01T23:05:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 msgr2=0x7f1d300adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 0x7f1d300adac0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f1d2000c000 tx=0x7f1d20006df0).stop 2022-08-01T23:05:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 shutdown_connections 2022-08-01T23:05:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1d100539e0 0x7f1d10055ea0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d30146ae0 0x7f1d30152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1d300b4820 0x7f1d30152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 --2- 172.21.15.32:0/1711997469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d300b4310 0x7f1d300adac0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:19.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 >> 172.21.15.32:0/1711997469 conn(0x7f1d3005c620 msgr2=0x7f1d301434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:19.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 shutdown_connections 2022-08-01T23:05:19.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:19.967+0000 7f1d36d9e700 1 -- 172.21.15.32:0/1711997469 wait complete. 2022-08-01T23:05:19.988 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:05:19.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:20.006 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:20.026 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:20.047 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:05:20.067 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:05:20.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:20.069 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:20.069 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:20.076 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T23:05:20.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:05:20.077 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T23:05:20.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:20.078 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:20.079 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:20.115 INFO:teuthology.orchestra.run.smithi157.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T23:05:20.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:05:20.117 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T23:05:20.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:05:20.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 Processor -- start 2022-08-01T23:05:20.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 -- start start 2022-08-01T23:05:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcdf00614b0 0x7fcdf0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00aee80 con 0x7fcdf00ad750 2022-08-01T23:05:20.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00b70b0 con 0x7fcdf00614b0 2022-08-01T23:05:20.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf6257700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00af0b0 con 0x7fcdf01405d0 2022-08-01T23:05:20.234 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf4a54700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcdf00ad750 0x7fcdf00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37300/0 (socket says 172.21.15.32:37300) 2022-08-01T23:05:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf4a54700 1 -- 172.21.15.32:0/15107890 learned_addr learned my addr 172.21.15.32:0/15107890 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf5255700 1 --2- 172.21.15.32:0/15107890 >> v2:172.21.15.55:3300/0 conn(0x7fcdf00614b0 0x7fcdf0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.223+0000 7fcdf5a56700 1 --2- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcde7fff700 1 -- 172.21.15.32:0/15107890 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816327917 0 0) 0x7fcdf00aee80 con 0x7fcdf00ad750 2022-08-01T23:05:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcde7fff700 1 -- 172.21.15.32:0/15107890 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd4003260 con 0x7fcdf00ad750 2022-08-01T23:05:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf5255700 1 -- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 msgr2=0x7fcdf0140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf5255700 1 --2- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf5255700 1 -- 172.21.15.32:0/15107890 >> v1:172.21.15.32:6789/0 conn(0x7fcdf00ad750 legacy=0x7fcdf00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf5255700 1 -- 172.21.15.32:0/15107890 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00af350 con 0x7fcdf00614b0 2022-08-01T23:05:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf5255700 1 --2- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00614b0 0x7fcdf0061890 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fcdf00aed30 tx=0x7fcde0006df0).ready entity=mon.1 client_cookie=5e2db9b71f41fcbf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcde7fff700 1 -- 172.21.15.32:0/15107890 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcde000d570 con 0x7fcdf00614b0 2022-08-01T23:05:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcde7fff700 1 -- 172.21.15.32:0/15107890 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0000e90 con 0x7fcdf00614b0 2022-08-01T23:05:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcde7fff700 1 -- 172.21.15.32:0/15107890 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcde000d8c0 con 0x7fcdf00614b0 2022-08-01T23:05:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 -- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00614b0 msgr2=0x7fcdf0061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 --2- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00614b0 0x7fcdf0061890 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fcdf00aed30 tx=0x7fcde0006df0).stop 2022-08-01T23:05:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 -- 172.21.15.32:0/15107890 shutdown_connections 2022-08-01T23:05:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 --2- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 --2- 172.21.15.32:0/15107890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00614b0 0x7fcdf0061890 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 -- 172.21.15.32:0/15107890 >> 172.21.15.32:0/15107890 conn(0x7fcdf0066f70 msgr2=0x7fcdf0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 -- 172.21.15.32:0/15107890 shutdown_connections 2022-08-01T23:05:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 -- 172.21.15.32:0/15107890 wait complete. 2022-08-01T23:05:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 Processor -- start 2022-08-01T23:05:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.227+0000 7fcdf6257700 1 -- start start 2022-08-01T23:05:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 0x7fcdf00aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00ad750 0x7fcdf00b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b0d00 con 0x7fcdf00614b0 2022-08-01T23:05:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b0fa0 con 0x7fcdf00ad750 2022-08-01T23:05:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0155d80 con 0x7fcdf01405d0 2022-08-01T23:05:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 0x7fcdf00aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 0x7fcdf00aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49764/0 (socket says 172.21.15.32:49764) 2022-08-01T23:05:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 -- 172.21.15.32:0/3248418225 learned_addr learned my addr 172.21.15.32:0/3248418225 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf4a54700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00ad750 0x7fcdf00b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5a56700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 -- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 msgr2=0x7fcdf0156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 -- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00ad750 msgr2=0x7fcdf00b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00ad750 0x7fcdf00b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 -- 172.21.15.32:0/3248418225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde000b180 con 0x7fcdf00614b0 2022-08-01T23:05:20.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf4a54700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00ad750 0x7fcdf00b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:20.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf5255700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 0x7fcdf00aa8b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fcdf00aed30 tx=0x7fcde0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:20.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcde0016070 con 0x7fcdf00614b0 2022-08-01T23:05:20.247 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00abbb0 con 0x7fcdf00614b0 2022-08-01T23:05:20.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0013a30 con 0x7fcdf00614b0 2022-08-01T23:05:20.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcde0013d90 con 0x7fcdf00614b0 2022-08-01T23:05:20.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.231+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00ab2b0 con 0x7fcdf00614b0 2022-08-01T23:05:20.248 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.235+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf0155f40 con 0x7fcdf00614b0 2022-08-01T23:05:20.249 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.235+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcde000f030 con 0x7fcdf00614b0 2022-08-01T23:05:20.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.235+0000 7fcde67fc700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.235+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcde006cd50 con 0x7fcdf00614b0 2022-08-01T23:05:20.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.235+0000 7fcdf4a54700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.250 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.235+0000 7fcdf4a54700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fcdec00b000 tx=0x7fcdec009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.243+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcde0043c60 con 0x7fcdf00614b0 2022-08-01T23:05:20.373 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.367+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcdf00af5f0 con 0x7fcdf00614b0 2022-08-01T23:05:20.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.367+0000 7fcde67fc700 1 -- 172.21.15.32:0/3248418225 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v150) v1 ==== 81+0+29229 (secure 0 0 0) 0x7fcde0023090 con 0x7fcdf00614b0 2022-08-01T23:05:20.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:20.377 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:05:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 150, 2022-08-01T23:05:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:05:20.377 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:05:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:05:17.644900+0000", 2022-08-01T23:05:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:05:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:05:20.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:05:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:05:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:05:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:05:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:05:20.379 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:05:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:05:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:05:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:05:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:05:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:05:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:05:20.381 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 28, 2022-08-01T23:05:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:05:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:05:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:05:20.382 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:05:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:05:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:05:20.383 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:05:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:05:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:20.384 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:20.385 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:05:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:05:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:05:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:05:20.386 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:05:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:05:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:20.389 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:20.390 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:20.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:20.392 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:20.393 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:05:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:05:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:05:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:05:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:05:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:05:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:05:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:05:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:05:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:05:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:05:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:05:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:05:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:05:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:05:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:05:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:05:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:05:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:05:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:05:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:05:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:05:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:05:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:05:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:05:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:05:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:05:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:05:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:05:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:05:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:05:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:05:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:05:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:05:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:05:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:05:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:05:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:05:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:05:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:05:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2022-08-01T23:05:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:05:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:05:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:05:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:05:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:05:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:05:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:05:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:05:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:05:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:05:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:05:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:05:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:05:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:05:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 134, 2022-08-01T23:05:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:05:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:05:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:05:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:05:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:05:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:05:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:05:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:05:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:05:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:05:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 143, 2022-08-01T23:05:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:05:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:05:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:05:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:05:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:05:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:05:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:05:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:05:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:05:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:05:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:05:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:05:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:05:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:05:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:05:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:05:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:05:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:05:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:05:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:05:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:05:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:05:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:05:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:05:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:05:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:05:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:05:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:05:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:05:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:05:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:05:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:05:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:05:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:05:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:05:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:05:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:05:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:05:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:05:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:05:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:05:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:05:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 144, 2022-08-01T23:05:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:05:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:05:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:05:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:05:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:05:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:05:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:05:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:05:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:05:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:05:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:05:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:05:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:05:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:05:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:05:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:05:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:05:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:05:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:05:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:05:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:05:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:05:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:05:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:05:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:05:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:05:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:05:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:05:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:05:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:05:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:05:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:05:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:05:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": { 2022-08-01T23:05:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6835/2229490987": "2022-08-02T23:05:14.278080+0000", 2022-08-01T23:05:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6834/2229490987": "2022-08-02T23:05:14.278080+0000" 2022-08-01T23:05:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:05:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:05:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:05:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:05:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:05:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:05:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:05:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:05:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:05:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:05:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:05:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:05:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:05:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:05:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:05:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:05:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:05:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:05:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:05:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:05:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:05:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:05:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:05:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:05:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:05:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 msgr2=0x7fcdd8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fcdec00b000 tx=0x7fcdec009040).stop 2022-08-01T23:05:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 msgr2=0x7fcdf00aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 0x7fcdf00aa8b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fcdf00aed30 tx=0x7fcde0014000).stop 2022-08-01T23:05:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 shutdown_connections 2022-08-01T23:05:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcdd8053950 0x7fcdd8055e10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcdf01405d0 0x7fcdf0156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf00ad750 0x7fcdf00b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 --2- 172.21.15.32:0/3248418225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00614b0 0x7fcdf00aa8b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 >> 172.21.15.32:0/3248418225 conn(0x7fcdf0066f70 msgr2=0x7fcdf00b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 shutdown_connections 2022-08-01T23:05:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.371+0000 7fcdf6257700 1 -- 172.21.15.32:0/3248418225 wait complete. 2022-08-01T23:05:20.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 Processor -- start 2022-08-01T23:05:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 -- start start 2022-08-01T23:05:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb3c0614b0 0x7fcb3c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb3c0ad750 0x7fcb3c0b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0aee80 con 0x7fcb3c1405d0 2022-08-01T23:05:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0b70b0 con 0x7fcb3c0ad750 2022-08-01T23:05:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb40871700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0af0b0 con 0x7fcb3c0614b0 2022-08-01T23:05:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb3c0ad750 0x7fcb3c0b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb3c0ad750 0x7fcb3c0b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49774/0 (socket says 172.21.15.32:49774) 2022-08-01T23:05:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 -- 172.21.15.32:0/3342181023 learned_addr learned my addr 172.21.15.32:0/3342181023 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3ad9d700 1 --2- 172.21.15.32:0/3342181023 >> v2:172.21.15.55:3300/0 conn(0x7fcb3c0614b0 0x7fcb3c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 -- 172.21.15.32:0/3342181023 >> v2:172.21.15.55:3300/0 conn(0x7fcb3c0614b0 msgr2=0x7fcb3c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 --2- 172.21.15.32:0/3342181023 >> v2:172.21.15.55:3300/0 conn(0x7fcb3c0614b0 0x7fcb3c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 -- 172.21.15.32:0/3342181023 >> v1:172.21.15.32:6789/0 conn(0x7fcb3c1405d0 legacy=0x7fcb3c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb39d9b700 1 -- 172.21.15.32:0/3342181023 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101247588 0 0) 0x7fcb3c0aee80 con 0x7fcb3c1405d0 2022-08-01T23:05:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 -- 172.21.15.32:0/3342181023 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0af350 con 0x7fcb3c0ad750 2022-08-01T23:05:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb3a59c700 1 --2- 172.21.15.32:0/3342181023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b9ff0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7fcb3c0aed30 tx=0x7fcb2c009290).ready entity=mon.0 client_cookie=f318e43101fad4a8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb39d9b700 1 -- 172.21.15.32:0/3342181023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb2c00b860 con 0x7fcb3c0ad750 2022-08-01T23:05:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb39d9b700 1 -- 172.21.15.32:0/3342181023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c00b9c0 con 0x7fcb3c0ad750 2022-08-01T23:05:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.695+0000 7fcb39d9b700 1 -- 172.21.15.32:0/3342181023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb2c00bb30 con 0x7fcb3c0ad750 2022-08-01T23:05:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 -- 172.21.15.32:0/3342181023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0ad750 msgr2=0x7fcb3c0b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 --2- 172.21.15.32:0/3342181023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b9ff0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7fcb3c0aed30 tx=0x7fcb2c009290).stop 2022-08-01T23:05:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 -- 172.21.15.32:0/3342181023 shutdown_connections 2022-08-01T23:05:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 --2- 172.21.15.32:0/3342181023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b9ff0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 --2- 172.21.15.32:0/3342181023 >> v2:172.21.15.55:3300/0 conn(0x7fcb3c0614b0 0x7fcb3c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 -- 172.21.15.32:0/3342181023 >> 172.21.15.32:0/3342181023 conn(0x7fcb3c066f70 msgr2=0x7fcb3c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 -- 172.21.15.32:0/3342181023 shutdown_connections 2022-08-01T23:05:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 -- 172.21.15.32:0/3342181023 wait complete. 2022-08-01T23:05:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 Processor -- start 2022-08-01T23:05:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.699+0000 7fcb40871700 1 -- start start 2022-08-01T23:05:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 0x7fcb3c141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb3c1405d0 0x7fcb3c1426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0afe40 con 0x7fcb3c0614b0 2022-08-01T23:05:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c09eb50 con 0x7fcb3c0ad750 2022-08-01T23:05:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c09edf0 con 0x7fcb3c1405d0 2022-08-01T23:05:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 0x7fcb3c141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 0x7fcb3c141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49778/0 (socket says 172.21.15.32:49778) 2022-08-01T23:05:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3ad9d700 1 -- 172.21.15.32:0/2543498590 learned_addr learned my addr 172.21.15.32:0/2543498590 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3b59e700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb3c1405d0 0x7fcb3c1426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 -- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb3c1405d0 msgr2=0x7fcb3c1426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb3c1405d0 0x7fcb3c1426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 -- 172.21.15.32:0/2543498590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 msgr2=0x7fcb3c141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 0x7fcb3c141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 -- 172.21.15.32:0/2543498590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009530 con 0x7fcb3c0ad750 2022-08-01T23:05:20.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3ad9d700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 0x7fcb3c141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:20.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb3a59c700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b3470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fcb3c0b33c0 tx=0x7fcb2c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:20.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb2c021630 con 0x7fcb3c0ad750 2022-08-01T23:05:20.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c021790 con 0x7fcb3c0ad750 2022-08-01T23:05:20.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c0b0290 con 0x7fcb3c0ad750 2022-08-01T23:05:20.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb2c021af0 con 0x7fcb3c0ad750 2022-08-01T23:05:20.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.703+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c0afac0 con 0x7fcb3c0ad750 2022-08-01T23:05:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.707+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c0b3b00 con 0x7fcb3c0ad750 2022-08-01T23:05:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.707+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb2c027020 con 0x7fcb3c0ad750 2022-08-01T23:05:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.707+0000 7fcb33fff700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb200539e0 0x7fcb20055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.707+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcb2c06d4e0 con 0x7fcb3c0ad750 2022-08-01T23:05:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.707+0000 7fcb3ad9d700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb200539e0 0x7fcb20055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.707+0000 7fcb3ad9d700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb200539e0 0x7fcb20055ea0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fcb2400b000 tx=0x7fcb24006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:20.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.719+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb2c044430 con 0x7fcb3c0ad750 2022-08-01T23:05:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.879+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb3c0b0420 con 0x7fcb3c0ad750 2022-08-01T23:05:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.883+0000 7fcb33fff700 1 -- 172.21.15.32:0/2543498590 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fcb2c015070 con 0x7fcb3c0ad750 2022-08-01T23:05:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:20.890 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/2132302387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2132302387},{"type":"v1","addr":"172.21.15.32:6835","nonce":2132302387}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[]} 2022-08-01T23:05:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb200539e0 msgr2=0x7fcb20055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb200539e0 0x7fcb20055ea0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fcb2400b000 tx=0x7fcb24006df0).stop 2022-08-01T23:05:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0ad750 msgr2=0x7fcb3c0b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b3470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fcb3c0b33c0 tx=0x7fcb2c016000).stop 2022-08-01T23:05:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 shutdown_connections 2022-08-01T23:05:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcb200539e0 0x7fcb20055ea0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcb3c1405d0 0x7fcb3c1426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0ad750 0x7fcb3c0b3470 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 --2- 172.21.15.32:0/2543498590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0614b0 0x7fcb3c141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 >> 172.21.15.32:0/2543498590 conn(0x7fcb3c066f70 msgr2=0x7fcb3c0b8910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 shutdown_connections 2022-08-01T23:05:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:20.887+0000 7fcb40871700 1 -- 172.21.15.32:0/2543498590 wait complete. 2022-08-01T23:05:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 77 2022-08-01T23:05:20.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 Processor -- start 2022-08-01T23:05:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 -- start start 2022-08-01T23:05:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35080b2710 0x7f350814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35080b01a0 0x7f350814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35080a7250 con 0x7f35080afd60 2022-08-01T23:05:21.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35080a7480 con 0x7f35080b2710 2022-08-01T23:05:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f350cd95700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35080a7720 con 0x7f35080b01a0 2022-08-01T23:05:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35080b2710 0x7f350814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35080b2710 0x7f350814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49788/0 (socket says 172.21.15.32:49788) 2022-08-01T23:05:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f3506ffd700 1 --2- 172.21.15.32:0/1629881706 >> v2:172.21.15.55:3300/0 conn(0x7f35080b01a0 0x7f350814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 -- 172.21.15.32:0/1629881706 learned_addr learned my addr 172.21.15.32:0/1629881706 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 -- 172.21.15.32:0/1629881706 >> v2:172.21.15.55:3300/0 conn(0x7f35080b01a0 msgr2=0x7f350814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 --2- 172.21.15.32:0/1629881706 >> v2:172.21.15.55:3300/0 conn(0x7f35080b01a0 0x7f350814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 -- 172.21.15.32:0/1629881706 >> v1:172.21.15.55:6789/0 conn(0x7f35080afd60 legacy=0x7f35080a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 -- 172.21.15.32:0/1629881706 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080a79c0 con 0x7f35080b2710 2022-08-01T23:05:21.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35077fe700 1 --2- 172.21.15.32:0/1629881706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080b2710 0x7f350814afd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f34fc002f70 tx=0x7f34fc006eb0).ready entity=mon.0 client_cookie=7acd7402a1c4206 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35067fc700 1 -- 172.21.15.32:0/1629881706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34fc00d5c0 con 0x7f35080b2710 2022-08-01T23:05:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35067fc700 1 -- 172.21.15.32:0/1629881706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc000e90 con 0x7f35080b2710 2022-08-01T23:05:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.003+0000 7f35067fc700 1 -- 172.21.15.32:0/1629881706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34fc00d950 con 0x7f35080b2710 2022-08-01T23:05:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 -- 172.21.15.32:0/1629881706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080b2710 msgr2=0x7f350814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 --2- 172.21.15.32:0/1629881706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080b2710 0x7f350814afd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f34fc002f70 tx=0x7f34fc006eb0).stop 2022-08-01T23:05:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 -- 172.21.15.32:0/1629881706 shutdown_connections 2022-08-01T23:05:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 --2- 172.21.15.32:0/1629881706 >> v2:172.21.15.55:3300/0 conn(0x7f35080b01a0 0x7f350814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 --2- 172.21.15.32:0/1629881706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080b2710 0x7f350814afd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 -- 172.21.15.32:0/1629881706 >> 172.21.15.32:0/1629881706 conn(0x7f3508060310 msgr2=0x7f35080b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 -- 172.21.15.32:0/1629881706 shutdown_connections 2022-08-01T23:05:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 -- 172.21.15.32:0/1629881706 wait complete. 2022-08-01T23:05:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 Processor -- start 2022-08-01T23:05:21.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.007+0000 7f350cd95700 1 -- start start 2022-08-01T23:05:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35080afd60 0x7f35080b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35080b01a0 0x7f3508063980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 0x7f35081539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508063190 con 0x7f3508063e70 2022-08-01T23:05:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508157750 con 0x7f35080afd60 2022-08-01T23:05:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3508157250 con 0x7f35080b01a0 2022-08-01T23:05:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 0x7f35081539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 0x7f35081539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49796/0 (socket says 172.21.15.32:49796) 2022-08-01T23:05:21.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 -- 172.21.15.32:0/719631343 learned_addr learned my addr 172.21.15.32:0/719631343 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3506ffd700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35080b01a0 0x7f3508063980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f35077fe700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35080afd60 0x7f35080b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 -- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35080b01a0 msgr2=0x7f3508063980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35080b01a0 0x7f3508063980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 -- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35080afd60 msgr2=0x7f35080b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35080afd60 0x7f35080b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 -- 172.21.15.32:0/719631343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc00b1e0 con 0x7f3508063e70 2022-08-01T23:05:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f35077fe700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35080afd60 0x7f35080b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3507fff700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 0x7f35081539f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f350806b950 tx=0x7f3500007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350000f470 con 0x7f3508063e70 2022-08-01T23:05:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350000f5d0 con 0x7f3508063e70 2022-08-01T23:05:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35080633c0 con 0x7f3508063e70 2022-08-01T23:05:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f350000f930 con 0x7f3508063e70 2022-08-01T23:05:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35080ac240 con 0x7f3508063e70 2022-08-01T23:05:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35080b5380 con 0x7f3508063e70 2022-08-01T23:05:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.011+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3500019030 con 0x7f3508063e70 2022-08-01T23:05:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.015+0000 7f3504ff9700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34e8053aa0 0x7f34e8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.015+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f350006b5e0 con 0x7f3508063e70 2022-08-01T23:05:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.015+0000 7f35077fe700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34e8053aa0 0x7f34e8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.015+0000 7f35077fe700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34e8053aa0 0x7f34e8055f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f34fc01a000 tx=0x7f34fc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.019+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35080b5380 con 0x7f3508063e70 2022-08-01T23:05:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35080b5380 con 0x7f3508063e70 2022-08-01T23:05:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f3504ff9700 1 -- 172.21.15.32:0/719631343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f35080b5380 con 0x7f3508063e70 2022-08-01T23:05:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:21.186 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.55:6835/1226575076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1226575076},{"type":"v1","addr":"172.21.15.55:6835","nonce":1226575076}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/1127680282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1127680282},{"type":"v1","addr":"172.21.15.32:6833","nonce":1127680282}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.32:6835/2132302387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2132302387},{"type":"v1","addr":"172.21.15.32:6835","nonce":2132302387}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[]} 2022-08-01T23:05:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34e8053aa0 msgr2=0x7f34e8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34e8053aa0 0x7f34e8055f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f34fc01a000 tx=0x7f34fc015000).stop 2022-08-01T23:05:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 msgr2=0x7f35081539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 0x7f35081539f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f350806b950 tx=0x7f3500007f10).stop 2022-08-01T23:05:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 shutdown_connections 2022-08-01T23:05:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f34e8053aa0 0x7f34e8055f60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3508063e70 0x7f35081539f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f35080b01a0 0x7f3508063980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 --2- 172.21.15.32:0/719631343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35080afd60 0x7f35080b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 >> 172.21.15.32:0/719631343 conn(0x7f3508060310 msgr2=0x7f35080678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 shutdown_connections 2022-08-01T23:05:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.179+0000 7f350cd95700 1 -- 172.21.15.32:0/719631343 wait complete. 2022-08-01T23:05:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 77 2022-08-01T23:05:21.200 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T23:05:21.201 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T23:05:21.201 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:05:21.202 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:05:21.202 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T23:05:21.203 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T23:05:21.203 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:05:21.204 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:05:21.204 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T23:05:21.205 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T23:05:21.205 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:05:21.206 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:05:21.214 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:05:21.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T23:05:21.216 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:05:21.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T23:05:21.217 INFO:tasks.ceph.mds.a:Started 2022-08-01T23:05:21.218 DEBUG:teuthology.parallel:result is None 2022-08-01T23:05:21.219 INFO:tasks.ceph.mds.c:Started 2022-08-01T23:05:21.219 DEBUG:teuthology.parallel:result is None 2022-08-01T23:05:21.222 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:05:21.222 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T23:05:21.266 INFO:tasks.ceph.mds.b:Started 2022-08-01T23:05:21.267 DEBUG:teuthology.parallel:result is None 2022-08-01T23:05:21.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T23:05:21.290 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2022-08-01T23:05:21.292 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:05:21.339 INFO:tasks.ceph.mds.b.smithi055.stdout:starting mds.b at 2022-08-01T23:05:21.364 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.355+0000 7f1301076700 1 Processor -- start 2022-08-01T23:05:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- start start 2022-08-01T23:05:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b5630 0x7f12fc0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12fc0b07a0 0x7f12fc147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0b5a10 con 0x7f12fc067730 2022-08-01T23:05:21.367 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc145470 con 0x7f12fc0b07a0 2022-08-01T23:05:21.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0ae180 con 0x7f12fc0b5630 2022-08-01T23:05:21.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12fc0b07a0 0x7f12fc147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12fc0b07a0 0x7f12fc147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49826/0 (socket says 172.21.15.32:49826) 2022-08-01T23:05:21.368 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 -- 172.21.15.32:0/3259206651 learned_addr learned my addr 172.21.15.32:0/3259206651 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:21.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fbfff700 1 --2- 172.21.15.32:0/3259206651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b5630 0x7f12fc0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 -- 172.21.15.32:0/3259206651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b5630 msgr2=0x7f12fc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:21.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 --2- 172.21.15.32:0/3259206651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b5630 0x7f12fc0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 -- 172.21.15.32:0/3259206651 >> v1:172.21.15.55:6789/0 conn(0x7f12fc067730 legacy=0x7f12fc0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.369 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12faffd700 1 -- 172.21.15.32:0/3259206651 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993357718 0 0) 0x7f12fc0b5a10 con 0x7f12fc067730 2022-08-01T23:05:21.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 -- 172.21.15.32:0/3259206651 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0ac990 con 0x7f12fc0b07a0 2022-08-01T23:05:21.370 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12fb7fe700 1 --2- 172.21.15.32:0/3259206651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc0b07a0 0x7f12fc147730 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f12f0002f70 tx=0x7f12f0006eb0).ready entity=mon.0 client_cookie=d8ab56504eb59382 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12faffd700 1 -- 172.21.15.32:0/3259206651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f000d5c0 con 0x7f12fc0b07a0 2022-08-01T23:05:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12faffd700 1 -- 172.21.15.32:0/3259206651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0000e90 con 0x7f12fc0b07a0 2022-08-01T23:05:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f12faffd700 1 -- 172.21.15.32:0/3259206651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f000d910 con 0x7f12fc0b07a0 2022-08-01T23:05:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- 172.21.15.32:0/3259206651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc0b07a0 msgr2=0x7f12fc147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 --2- 172.21.15.32:0/3259206651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc0b07a0 0x7f12fc147730 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f12f0002f70 tx=0x7f12f0006eb0).stop 2022-08-01T23:05:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- 172.21.15.32:0/3259206651 shutdown_connections 2022-08-01T23:05:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 --2- 172.21.15.32:0/3259206651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc0b07a0 0x7f12fc147730 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 --2- 172.21.15.32:0/3259206651 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b5630 0x7f12fc0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.359+0000 7f1301076700 1 -- 172.21.15.32:0/3259206651 >> 172.21.15.32:0/3259206651 conn(0x7f12fc0626f0 msgr2=0x7f12fc062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.363+0000 7f1301076700 1 -- 172.21.15.32:0/3259206651 shutdown_connections 2022-08-01T23:05:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.363+0000 7f1301076700 1 -- 172.21.15.32:0/3259206651 wait complete. 2022-08-01T23:05:21.379 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.363+0000 7f1301076700 1 Processor -- start 2022-08-01T23:05:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.363+0000 7f1301076700 1 -- start start 2022-08-01T23:05:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 0x7f12fc0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b07a0 0x7f12fc156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0b5630 0x7f12fc1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc156fe0 con 0x7f12fc067730 2022-08-01T23:05:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0668f0 con 0x7f12fc0b5630 2022-08-01T23:05:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc066cb0 con 0x7f12fc0b07a0 2022-08-01T23:05:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 0x7f12fc0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 0x7f12fc0b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49830/0 (socket says 172.21.15.32:49830) 2022-08-01T23:05:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 -- 172.21.15.32:0/2780927730 learned_addr learned my addr 172.21.15.32:0/2780927730 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1300875700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0b5630 0x7f12fc1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 -- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b07a0 msgr2=0x7f12fc156af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-08-01T23:05:21.382 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b07a0 0x7f12fc156af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T23:05:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 -- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0b5630 msgr2=0x7f12fc1473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0b5630 0x7f12fc1473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 -- 172.21.15.32:0/2780927730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f000b1e0 con 0x7f12fc067730 2022-08-01T23:05:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1300875700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0b5630 0x7f12fc1473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:21.383 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12fbfff700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 0x7f12fc0b44a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f12ec0030b0 tx=0x7f12ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12ec011780 con 0x7f12fc067730 2022-08-01T23:05:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec0118e0 con 0x7f12fc067730 2022-08-01T23:05:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc066f50 con 0x7f12fc067730 2022-08-01T23:05:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc146cf0 con 0x7f12fc067730 2022-08-01T23:05:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.367+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12ec011c40 con 0x7f12fc067730 2022-08-01T23:05:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.371+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12ec012420 con 0x7f12fc067730 2022-08-01T23:05:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.371+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d4003410 con 0x7f12fc067730 2022-08-01T23:05:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.371+0000 7f12f97fa700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12dc053a70 0x7f12dc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.371+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f12ec06c2b0 con 0x7f12fc067730 2022-08-01T23:05:21.386 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.371+0000 7f12fb7fe700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12dc053a70 0x7f12dc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:21.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.371+0000 7f12fb7fe700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12dc053a70 0x7f12dc055f30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f12f0002f70 tx=0x7f12f0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:21.387 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.375+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12ec0431e0 con 0x7f12fc067730 2022-08-01T23:05:21.518 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:21.511+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f12d4003c40 con 0x7f12fc067730 2022-08-01T23:05:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.379+0000 7f12f97fa700 1 -- 172.21.15.32:0/2780927730 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v154) v1 ==== 97+0+0 (secure 0 0 0) 0x7f12ec046800 con 0x7f12fc067730 2022-08-01T23:05:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12dc053a70 msgr2=0x7f12dc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12dc053a70 0x7f12dc055f30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f12f0002f70 tx=0x7f12f0015000).stop 2022-08-01T23:05:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 msgr2=0x7f12fc0b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 0x7f12fc0b44a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f12ec0030b0 tx=0x7f12ec008040).stop 2022-08-01T23:05:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 shutdown_connections 2022-08-01T23:05:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f12dc053a70 0x7f12dc055f30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0b5630 0x7f12fc1473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f12fc0b07a0 0x7f12fc156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T23:05:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 --2- 172.21.15.32:0/2780927730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12fc067730 0x7f12fc0b44a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 >> 172.21.15.32:0/2780927730 conn(0x7f12fc0626f0 msgr2=0x7f12fc0acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 shutdown_connections 2022-08-01T23:05:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.383+0000 7f1301076700 1 -- 172.21.15.32:0/2780927730 wait complete. 2022-08-01T23:05:22.392 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blacklist entries 2022-08-01T23:05:22.405 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T23:05:22.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T23:05:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 Processor -- start 2022-08-01T23:05:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- start start 2022-08-01T23:05:22.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f900adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4f900aa2b0 0x7f4f900ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f900b2c80 con 0x7f4f900b0a10 2022-08-01T23:05:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f900b2900 con 0x7f4f900aa2b0 2022-08-01T23:05:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900b2eb0 con 0x7f4f900b0630 2022-08-01T23:05:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f900b0a10 0x7f4f901438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37386/0 (socket says 172.21.15.32:37386) 2022-08-01T23:05:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8ed9d700 1 -- 172.21.15.32:0/930584222 learned_addr learned my addr 172.21.15.32:0/930584222 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8dd9b700 1 --2- 172.21.15.32:0/930584222 >> v2:172.21.15.55:3300/0 conn(0x7f4f900aa2b0 0x7f4f900ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8e59c700 1 --2- 172.21.15.32:0/930584222 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f900adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8d59a700 1 -- 172.21.15.32:0/930584222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483883742 0 0) 0x7f4f900b2c80 con 0x7f4f900b0a10 2022-08-01T23:05:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8d59a700 1 -- 172.21.15.32:0/930584222 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f74003260 con 0x7f4f900b0a10 2022-08-01T23:05:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8e59c700 1 -- 172.21.15.32:0/930584222 >> v2:172.21.15.55:3300/0 conn(0x7f4f900aa2b0 msgr2=0x7f4f900ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8e59c700 1 --2- 172.21.15.32:0/930584222 >> v2:172.21.15.55:3300/0 conn(0x7f4f900aa2b0 0x7f4f900ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8e59c700 1 -- 172.21.15.32:0/930584222 >> v1:172.21.15.32:6789/0 conn(0x7f4f900b0a10 legacy=0x7f4f901438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8e59c700 1 -- 172.21.15.32:0/930584222 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900a9d00 con 0x7f4f900b0630 2022-08-01T23:05:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8dd9b700 1 --2- 172.21.15.32:0/930584222 >> v2:172.21.15.55:3300/0 conn(0x7f4f900aa2b0 0x7f4f900ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8e59c700 1 --2- 172.21.15.32:0/930584222 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f900adec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f4f84002f70 tx=0x7f4f84006df0).ready entity=mon.2 client_cookie=32638a4400877aa4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8d59a700 1 -- 172.21.15.32:0/930584222 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f8400d4a0 con 0x7f4f900b0630 2022-08-01T23:05:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8d59a700 1 -- 172.21.15.32:0/930584222 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84000e90 con 0x7f4f900b0630 2022-08-01T23:05:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f8d59a700 1 -- 172.21.15.32:0/930584222 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f8400d7f0 con 0x7f4f900b0630 2022-08-01T23:05:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- 172.21.15.32:0/930584222 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 msgr2=0x7f4f900adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 --2- 172.21.15.32:0/930584222 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f900adec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f4f84002f70 tx=0x7f4f84006df0).stop 2022-08-01T23:05:22.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- 172.21.15.32:0/930584222 shutdown_connections 2022-08-01T23:05:22.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 --2- 172.21.15.32:0/930584222 >> v2:172.21.15.55:3300/0 conn(0x7f4f900aa2b0 0x7f4f900ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 --2- 172.21.15.32:0/930584222 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f900adec0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- 172.21.15.32:0/930584222 >> 172.21.15.32:0/930584222 conn(0x7f4f90060950 msgr2=0x7f4f90060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:22.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.555+0000 7f4f94c24700 1 -- 172.21.15.32:0/930584222 shutdown_connections 2022-08-01T23:05:22.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 -- 172.21.15.32:0/930584222 wait complete. 2022-08-01T23:05:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 Processor -- start 2022-08-01T23:05:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 -- start start 2022-08-01T23:05:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 0x7f4f90063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f90066520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f900b0a10 0x7f4f90066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901375b0 con 0x7f4f900aa2b0 2022-08-01T23:05:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900ab220 con 0x7f4f900b0a10 2022-08-01T23:05:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900aace0 con 0x7f4f900b0630 2022-08-01T23:05:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 0x7f4f90063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 0x7f4f90063da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49846/0 (socket says 172.21.15.32:49846) 2022-08-01T23:05:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 -- 172.21.15.32:0/2300073407 learned_addr learned my addr 172.21.15.32:0/2300073407 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:22.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8ed9d700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f900b0a10 0x7f4f90066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8dd9b700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f90066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 -- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 msgr2=0x7f4f90066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f90066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 -- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f900b0a10 msgr2=0x7f4f90066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f900b0a10 0x7f4f90066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 -- 172.21.15.32:0/2300073407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8000b180 con 0x7f4f900aa2b0 2022-08-01T23:05:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8dd9b700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f90066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f8e59c700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 0x7f4f90063da0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f4f84002f70 tx=0x7f4f84014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:22.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f84016070 con 0x7f4f900aa2b0 2022-08-01T23:05:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84013a40 con 0x7f4f900aa2b0 2022-08-01T23:05:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8400b180 con 0x7f4f900aa2b0 2022-08-01T23:05:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.559+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f84013da0 con 0x7f4f900aa2b0 2022-08-01T23:05:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.563+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f900ab470 con 0x7f4f900aa2b0 2022-08-01T23:05:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.563+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f900662c0 con 0x7f4f900aa2b0 2022-08-01T23:05:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.563+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4f84025030 con 0x7f4f900aa2b0 2022-08-01T23:05:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.563+0000 7f4f7f7fe700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4f6c053b00 0x7f4f6c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.563+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4f8406cb80 con 0x7f4f900aa2b0 2022-08-01T23:05:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.563+0000 7f4f8dd9b700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4f6c053b00 0x7f4f6c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.567+0000 7f4f8dd9b700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4f6c053b00 0x7f4f6c055fc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f4f8000b000 tx=0x7f4f80008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.571+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f84043c80 con 0x7f4f900aa2b0 2022-08-01T23:05:22.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.739+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f900574e0 con 0x7f4f900aa2b0 2022-08-01T23:05:22.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.739+0000 7f4f7f7fe700 1 -- 172.21.15.32:0/2300073407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f4f84023090 con 0x7f4f900aa2b0 2022-08-01T23:05:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:22.746 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:05:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2022-08-01T23:05:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2022-08-01T23:05:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiiA1AHxAA6/BgaAmkDq6AIKY4UL5jcw==", 2022-08-01T23:05:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:05:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2022-08-01T23:05:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehixByHKRAADL1G28zWOMQb8lWbSrsDAA==", 2022-08-01T23:05:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:05:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2022-08-01T23:05:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6F+JJRAAsY/KTXSnFVIcxqv06CQ/Gw==", 2022-08-01T23:05:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:05:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2022-08-01T23:05:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC/Wehiu03NIhAATFdB0tZo4bs6FbnU+OvxXw==", 2022-08-01T23:05:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2022-08-01T23:05:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDCWehiBf6xBxAADTz8oVfnlTmFjYP9jc7Gkw==", 2022-08-01T23:05:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2022-08-01T23:05:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDEWehi9kIRKRAAOReUHcx1glUSQZJpyNCu/A==", 2022-08-01T23:05:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2022-08-01T23:05:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDHWehi2+j0DxAALXmLb/wyDkA3PCyimddzKQ==", 2022-08-01T23:05:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2022-08-01T23:05:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPWehiyRmUJxAAZIHcLPjujbvEsir/DIBJlA==", 2022-08-01T23:05:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2022-08-01T23:05:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDSWehi8Iv6DhAATyX6mM5zrVK44zXbdtQJBw==", 2022-08-01T23:05:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2022-08-01T23:05:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUWehiXnQ6LxAAA/h+isPaG4W9VASbOBFc5Q==", 2022-08-01T23:05:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2022-08-01T23:05:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXWehi+csKFhAAwDYroG9Sh6ypdE5Zjx2aHw==", 2022-08-01T23:05:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2022-08-01T23:05:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi042VLRAAamOzYktlPZOXMRSN0KGM1Q==", 2022-08-01T23:05:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:05:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:05:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:05:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2022-08-01T23:05:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehi6GhcMRAAwYPo0Y3Hgt8/iGQPSS0t4g==", 2022-08-01T23:05:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2022-08-01T23:05:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2022-08-01T23:05:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T23:05:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2022-08-01T23:05:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehiu/LmAhAA6c6E46OTC4A0c8uT6TEQGA==", 2022-08-01T23:05:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:05:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2022-08-01T23:05:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2022-08-01T23:05:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2022-08-01T23:05:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+Wehij6lqHBAAfxkiNhyn77F1sb15neU8aQ==", 2022-08-01T23:05:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:05:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:05:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2022-08-01T23:05:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQC+WehiH+fKGBAAfvEUnjGda/1kumbSSiziYQ==", 2022-08-01T23:05:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2022-08-01T23:05:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2022-08-01T23:05:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2022-08-01T23:05:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2022-08-01T23:05:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:22.781 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:05:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4f6c053b00 msgr2=0x7f4f6c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4f6c053b00 0x7f4f6c055fc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f4f8000b000 tx=0x7f4f80008040).stop 2022-08-01T23:05:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 msgr2=0x7f4f90063da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 0x7f4f90063da0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f4f84002f70 tx=0x7f4f84014000).stop 2022-08-01T23:05:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 shutdown_connections 2022-08-01T23:05:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4f6c053b00 0x7f4f6c055fc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f900b0a10 0x7f4f90066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4f900b0630 0x7f4f90066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 --2- 172.21.15.32:0/2300073407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f900aa2b0 0x7f4f90063da0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 >> 172.21.15.32:0/2300073407 conn(0x7f4f90060950 msgr2=0x7f4f900b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 shutdown_connections 2022-08-01T23:05:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.743+0000 7f4f94c24700 1 -- 172.21.15.32:0/2300073407 wait complete. 2022-08-01T23:05:22.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 Processor -- start 2022-08-01T23:05:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 -- start start 2022-08-01T23:05:22.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdfc0a9320 0x7efdfc1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0b12d0 con 0x7efdfc0b0560 2022-08-01T23:05:22.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0a9700 con 0x7efdfc0a9320 2022-08-01T23:05:22.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efe008f4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0b1500 con 0x7efdfc0b0d10 2022-08-01T23:05:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.883+0000 7efdfb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdfc0a9320 0x7efdfc1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdfc0a9320 0x7efdfc1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:59640/0 (socket says 172.21.15.32:59640) 2022-08-01T23:05:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 -- 172.21.15.32:0/4220664732 learned_addr learned my addr 172.21.15.32:0/4220664732 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfbfff700 1 --1- 172.21.15.32:0/4220664732 >> v1:172.21.15.55:6789/0 conn(0x7efdfc0b0560 0x7efdfc140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:58304/0 (socket says 172.21.15.32:58304) 2022-08-01T23:05:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfaffd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 -- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 msgr2=0x7efdfc146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 --2- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 -- 172.21.15.32:0/4220664732 >> v1:172.21.15.55:6789/0 conn(0x7efdfc0b0560 legacy=0x7efdfc140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 -- 172.21.15.32:0/4220664732 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0b17a0 con 0x7efdfc0a9320 2022-08-01T23:05:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfb7fe700 1 --2- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0a9320 0x7efdfc1466e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec009290).ready entity=mon.1 client_cookie=1c4ad96e7ae6c41d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfa7fc700 1 -- 172.21.15.32:0/4220664732 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00b8a0 con 0x7efdfc0a9320 2022-08-01T23:05:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfa7fc700 1 -- 172.21.15.32:0/4220664732 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00ba00 con 0x7efdfc0a9320 2022-08-01T23:05:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efdfa7fc700 1 -- 172.21.15.32:0/4220664732 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00bb70 con 0x7efdfc0a9320 2022-08-01T23:05:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 -- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0a9320 msgr2=0x7efdfc1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 --2- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0a9320 0x7efdfc1466e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec009290).stop 2022-08-01T23:05:22.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 -- 172.21.15.32:0/4220664732 shutdown_connections 2022-08-01T23:05:22.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 --2- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.899 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 --2- 172.21.15.32:0/4220664732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0a9320 0x7efdfc1466e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 -- 172.21.15.32:0/4220664732 >> 172.21.15.32:0/4220664732 conn(0x7efdfc0612d0 msgr2=0x7efdfc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:22.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 -- 172.21.15.32:0/4220664732 shutdown_connections 2022-08-01T23:05:22.900 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 -- 172.21.15.32:0/4220664732 wait complete. 2022-08-01T23:05:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 Processor -- start 2022-08-01T23:05:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.887+0000 7efe008f4700 1 -- start start 2022-08-01T23:05:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 0x7efdfc063e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0b0560 0x7efdfc0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc132a90 con 0x7efdfc0a9320 2022-08-01T23:05:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc067a80 con 0x7efdfc0b0560 2022-08-01T23:05:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1534f0 con 0x7efdfc0b0d10 2022-08-01T23:05:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 0x7efdfc063e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 0x7efdfc063e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49860/0 (socket says 172.21.15.32:49860) 2022-08-01T23:05:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 -- 172.21.15.32:0/1310799141 learned_addr learned my addr 172.21.15.32:0/1310799141 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfbfff700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfaffd700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0b0560 0x7efdfc0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 -- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 msgr2=0x7efdfc0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc0b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 -- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0b0560 msgr2=0x7efdfc0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0b0560 0x7efdfc0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 -- 172.21.15.32:0/1310799141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009530 con 0x7efdfc0a9320 2022-08-01T23:05:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfaffd700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0b0560 0x7efdfc0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:22.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdfb7fe700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 0x7efdfc063e30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:22.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00b8a0 con 0x7efdfc0a9320 2022-08-01T23:05:22.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec021630 con 0x7efdfc0a9320 2022-08-01T23:05:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc153790 con 0x7efdfc0a9320 2022-08-01T23:05:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec021990 con 0x7efdfc0a9320 2022-08-01T23:05:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.891+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc066120 con 0x7efdfc0a9320 2022-08-01T23:05:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.895+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0574e0 con 0x7efdfc0a9320 2022-08-01T23:05:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.895+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efdec00f070 con 0x7efdfc0a9320 2022-08-01T23:05:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.895+0000 7efdf8ff9700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efde00539e0 0x7efde0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.895+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7efdec06ccd0 con 0x7efdfc0a9320 2022-08-01T23:05:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.895+0000 7efdfaffd700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efde00539e0 0x7efde0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.895+0000 7efdfaffd700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efde00539e0 0x7efde0055ea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7efde400b000 tx=0x7efde4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:22.899+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efdec043dd0 con 0x7efdfc0a9320 2022-08-01T23:05:23.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.023+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdfc0afd50 con 0x7efdfc0a9320 2022-08-01T23:05:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efdf8ff9700 1 -- 172.21.15.32:0/1310799141 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13424 (secure 0 0 0) 0x7efdec015030 con 0x7efdfc0a9320 2022-08-01T23:05:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:23.034 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":154,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:22.375257+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efde00539e0 msgr2=0x7efde0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efde00539e0 0x7efde0055ea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7efde400b000 tx=0x7efde4008040).stop 2022-08-01T23:05:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 msgr2=0x7efdfc063e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 0x7efdfc063e30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec016000).stop 2022-08-01T23:05:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 shutdown_connections 2022-08-01T23:05:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7efde00539e0 0x7efde0055ea0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7efdfc0b0d10 0x7efdfc0b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdfc0b0560 0x7efdfc0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 --2- 172.21.15.32:0/1310799141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdfc0a9320 0x7efdfc063e30 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 >> 172.21.15.32:0/1310799141 conn(0x7efdfc0612d0 msgr2=0x7efdfc05be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 shutdown_connections 2022-08-01T23:05:23.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.027+0000 7efe008f4700 1 -- 172.21.15.32:0/1310799141 wait complete. 2022-08-01T23:05:23.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T23:05:23.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:05:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 Processor -- start 2022-08-01T23:05:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 -- start start 2022-08-01T23:05:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e8c0accd0 0x7f2e8c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0b1100 0x7f2e8c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c0b06c0 con 0x7f2e8c0b2580 2022-08-01T23:05:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0b2960 con 0x7f2e8c0accd0 2022-08-01T23:05:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e91eaa700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0ae360 con 0x7f2e8c0b1100 2022-08-01T23:05:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e90ea8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e8c0b2580 0x7f2e8c0ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37410/0 (socket says 172.21.15.32:37410) 2022-08-01T23:05:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e90ea8700 1 -- 172.21.15.32:0/2795144645 learned_addr learned my addr 172.21.15.32:0/2795144645 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e916a9700 1 --2- 172.21.15.32:0/2795144645 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0b1100 0x7f2e8c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e8bfff700 1 --2- 172.21.15.32:0/2795144645 >> v2:172.21.15.55:3300/0 conn(0x7f2e8c0accd0 0x7f2e8c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e8b7fe700 1 -- 172.21.15.32:0/2795144645 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440531413 0 0) 0x7f2e8c0b06c0 con 0x7f2e8c0b2580 2022-08-01T23:05:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e8b7fe700 1 -- 172.21.15.32:0/2795144645 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e6c003260 con 0x7f2e8c0b2580 2022-08-01T23:05:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e916a9700 1 -- 172.21.15.32:0/2795144645 >> v2:172.21.15.55:3300/0 conn(0x7f2e8c0accd0 msgr2=0x7f2e8c0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e916a9700 1 --2- 172.21.15.32:0/2795144645 >> v2:172.21.15.55:3300/0 conn(0x7f2e8c0accd0 0x7f2e8c0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e8b7fe700 1 -- 172.21.15.32:0/2795144645 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2233658934 0 0) 0x7f2e6c003260 con 0x7f2e8c0b2580 2022-08-01T23:05:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e916a9700 1 -- 172.21.15.32:0/2795144645 >> v1:172.21.15.32:6789/0 conn(0x7f2e8c0b2580 legacy=0x7f2e8c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.151+0000 7f2e916a9700 1 -- 172.21.15.32:0/2795144645 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0b0330 con 0x7f2e8c0b1100 2022-08-01T23:05:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e916a9700 1 --2- 172.21.15.32:0/2795144645 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0b1100 0x7f2e8c1466f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f2e84001b80 tx=0x7f2e7c006df0).ready entity=mon.2 client_cookie=9b2f33f1833c0308 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e8b7fe700 1 -- 172.21.15.32:0/2795144645 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e840019e0 con 0x7f2e8c0b1100 2022-08-01T23:05:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e8b7fe700 1 -- 172.21.15.32:0/2795144645 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c000e90 con 0x7f2e8c0b1100 2022-08-01T23:05:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e8b7fe700 1 -- 172.21.15.32:0/2795144645 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e7c012460 con 0x7f2e8c0b1100 2022-08-01T23:05:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 -- 172.21.15.32:0/2795144645 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0b1100 msgr2=0x7f2e8c1466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/2795144645 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0b1100 0x7f2e8c1466f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f2e84001b80 tx=0x7f2e7c006df0).stop 2022-08-01T23:05:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 -- 172.21.15.32:0/2795144645 shutdown_connections 2022-08-01T23:05:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/2795144645 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0b1100 0x7f2e8c1466f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/2795144645 >> v2:172.21.15.55:3300/0 conn(0x7f2e8c0accd0 0x7f2e8c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 -- 172.21.15.32:0/2795144645 >> 172.21.15.32:0/2795144645 conn(0x7f2e8c05c620 msgr2=0x7f2e8c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 -- 172.21.15.32:0/2795144645 shutdown_connections 2022-08-01T23:05:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.155+0000 7f2e91eaa700 1 -- 172.21.15.32:0/2795144645 wait complete. 2022-08-01T23:05:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 Processor -- start 2022-08-01T23:05:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 -- start start 2022-08-01T23:05:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0accd0 0x7f2e8c144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 0x7f2e8c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8c0b2580 0x7f2e8c09f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c141f40 con 0x7f2e8c0b1100 2022-08-01T23:05:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c1448d0 con 0x7f2e8c0b2580 2022-08-01T23:05:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c142170 con 0x7f2e8c0accd0 2022-08-01T23:05:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 0x7f2e8c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 0x7f2e8c152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49876/0 (socket says 172.21.15.32:49876) 2022-08-01T23:05:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 -- 172.21.15.32:0/184182678 learned_addr learned my addr 172.21.15.32:0/184182678 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e90ea8700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0accd0 0x7f2e8c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e916a9700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8c0b2580 0x7f2e8c09f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 -- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0accd0 msgr2=0x7f2e8c144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0accd0 0x7f2e8c144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 -- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8c0b2580 msgr2=0x7f2e8c09f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8c0b2580 0x7f2e8c09f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 -- 172.21.15.32:0/184182678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c00b180 con 0x7f2e8c0b1100 2022-08-01T23:05:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e8bfff700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 0x7f2e8c152100 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f2e7400c000 tx=0x7f2e74006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e74010780 con 0x7f2e8c0b1100 2022-08-01T23:05:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e740108e0 con 0x7f2e8c0b1100 2022-08-01T23:05:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c156060 con 0x7f2e8c0b1100 2022-08-01T23:05:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e74010c40 con 0x7f2e8c0b1100 2022-08-01T23:05:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.159+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c1424c0 con 0x7f2e8c0b1100 2022-08-01T23:05:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.163+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c0aa6c0 con 0x7f2e8c0b1100 2022-08-01T23:05:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.163+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2e7401c460 con 0x7f2e8c0b1100 2022-08-01T23:05:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.163+0000 7f2e89ffb700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2e70053980 0x7f2e70055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.163+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2e7406c210 con 0x7f2e8c0b1100 2022-08-01T23:05:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.163+0000 7f2e90ea8700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2e70053980 0x7f2e70055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.163+0000 7f2e90ea8700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2e70053980 0x7f2e70055e40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f2e8400b000 tx=0x7f2e84001f40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.171+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e8c0aa6c0 con 0x7f2e8c0b1100 2022-08-01T23:05:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2e8c156500 con 0x7f2e8c0b1100 2022-08-01T23:05:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e89ffb700 1 -- 172.21.15.32:0/184182678 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v154) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2e740432b0 con 0x7f2e8c0b1100 2022-08-01T23:05:23.307 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:05:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2e70053980 msgr2=0x7f2e70055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2e70053980 0x7f2e70055e40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f2e8400b000 tx=0x7f2e84001f40).stop 2022-08-01T23:05:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 msgr2=0x7f2e8c152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 0x7f2e8c152100 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f2e7400c000 tx=0x7f2e74006df0).stop 2022-08-01T23:05:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 shutdown_connections 2022-08-01T23:05:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f2e70053980 0x7f2e70055e40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8c0b2580 0x7f2e8c09f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8c0b1100 0x7f2e8c152100 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 --2- 172.21.15.32:0/184182678 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f2e8c0accd0 0x7f2e8c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 >> 172.21.15.32:0/184182678 conn(0x7f2e8c05c620 msgr2=0x7f2e8c0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 shutdown_connections 2022-08-01T23:05:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.299+0000 7f2e91eaa700 1 -- 172.21.15.32:0/184182678 wait complete. 2022-08-01T23:05:23.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:05:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.419+0000 7f58c9dbb700 1 Processor -- start 2022-08-01T23:05:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.419+0000 7f58c9dbb700 1 -- start start 2022-08-01T23:05:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c9dbb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58c40af550 0x7f58c40b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c9dbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c41445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c9dbb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40aa630 con 0x7f58c4144ad0 2022-08-01T23:05:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c9dbb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40b15e0 con 0x7f58c40af550 2022-08-01T23:05:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c9dbb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40af930 con 0x7f58c4060330 2022-08-01T23:05:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c41445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c41445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3301/0 says I am v2:172.21.15.32:33388/0 (socket says 172.21.15.32:33388) 2022-08-01T23:05:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 -- 172.21.15.32:0/2444934 learned_addr learned my addr 172.21.15.32:0/2444934 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c8db9700 1 --2- 172.21.15.32:0/2444934 >> v2:172.21.15.55:3300/0 conn(0x7f58c40af550 0x7f58c40b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 -- 172.21.15.32:0/2444934 >> v2:172.21.15.55:3300/0 conn(0x7f58c40af550 msgr2=0x7f58c40b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 --2- 172.21.15.32:0/2444934 >> v2:172.21.15.55:3300/0 conn(0x7f58c40af550 0x7f58c40b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 -- 172.21.15.32:0/2444934 >> v1:172.21.15.55:6789/0 conn(0x7f58c4144ad0 legacy=0x7f58c40a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 -- 172.21.15.32:0/2444934 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c405fd10 con 0x7f58c4060330 2022-08-01T23:05:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c3fff700 1 --2- 172.21.15.32:0/2444934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c41445e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f58b0002f70 tx=0x7f58b0006eb0).ready entity=mon.2 client_cookie=d62daacdc79e35bb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c37fe700 1 -- 172.21.15.32:0/2444934 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58b000d570 con 0x7f58c4060330 2022-08-01T23:05:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c37fe700 1 -- 172.21.15.32:0/2444934 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0000e90 con 0x7f58c4060330 2022-08-01T23:05:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.423+0000 7f58c37fe700 1 -- 172.21.15.32:0/2444934 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58b000d900 con 0x7f58c4060330 2022-08-01T23:05:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2444934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 msgr2=0x7f58c41445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2444934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c41445e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f58b0002f70 tx=0x7f58b0006eb0).stop 2022-08-01T23:05:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2444934 shutdown_connections 2022-08-01T23:05:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2444934 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c41445e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2444934 >> v2:172.21.15.55:3300/0 conn(0x7f58c40af550 0x7f58c40b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2444934 >> 172.21.15.32:0/2444934 conn(0x7f58c40612d0 msgr2=0x7f58c4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2444934 shutdown_connections 2022-08-01T23:05:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2444934 wait complete. 2022-08-01T23:05:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 Processor -- start 2022-08-01T23:05:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.427+0000 7f58c9dbb700 1 -- start start 2022-08-01T23:05:23.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c4141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58c40af550 0x7f58c4153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 0x7f58c41541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c413f350 con 0x7f58c4144ad0 2022-08-01T23:05:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41546a0 con 0x7f58c40af550 2022-08-01T23:05:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4141ac0 con 0x7f58c4060330 2022-08-01T23:05:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 0x7f58c41541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 0x7f58c41541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49892/0 (socket says 172.21.15.32:49892) 2022-08-01T23:05:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 -- 172.21.15.32:0/2979011383 learned_addr learned my addr 172.21.15.32:0/2979011383 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c3fff700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58c40af550 0x7f58c4153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c8db9700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c4141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 -- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 msgr2=0x7f58c4141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c4141000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 -- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58c40af550 msgr2=0x7f58c4153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58c40af550 0x7f58c4153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 -- 172.21.15.32:0/2979011383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b000b1e0 con 0x7f58c4144ad0 2022-08-01T23:05:23.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c95ba700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 0x7f58c41541b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f58c4154830 tx=0x7f58bc007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58bc006aa0 con 0x7f58c4144ad0 2022-08-01T23:05:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc006c00 con 0x7f58c4144ad0 2022-08-01T23:05:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c405f840 con 0x7f58c4144ad0 2022-08-01T23:05:23.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58bc007030 con 0x7f58c4144ad0 2022-08-01T23:05:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.431+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c40640b0 con 0x7f58c4144ad0 2022-08-01T23:05:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.435+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4063f20 con 0x7f58c4144ad0 2022-08-01T23:05:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.435+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58bc010030 con 0x7f58c4144ad0 2022-08-01T23:05:23.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.435+0000 7f58c1ffb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58a4053950 0x7f58a4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.435+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f58bc06bc50 con 0x7f58c4144ad0 2022-08-01T23:05:23.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.435+0000 7f58c8db9700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58a4053950 0x7f58a4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.435+0000 7f58c8db9700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58a4053950 0x7f58a4055e10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f58b800b000 tx=0x7f58b8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.443+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58bc042c40 con 0x7f58c4144ad0 2022-08-01T23:05:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58c4064240 con 0x7f58c4144ad0 2022-08-01T23:05:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c1ffb700 1 -- 172.21.15.32:0/2979011383 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v154) v1 ==== 92+0+10 (secure 0 0 0) 0x7f58bc046260 con 0x7f58c4144ad0 2022-08-01T23:05:23.574 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58a4053950 msgr2=0x7f58a4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58a4053950 0x7f58a4055e10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f58b800b000 tx=0x7f58b8006df0).stop 2022-08-01T23:05:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 msgr2=0x7f58c41541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 0x7f58c41541b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f58c4154830 tx=0x7f58bc007f70).stop 2022-08-01T23:05:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 shutdown_connections 2022-08-01T23:05:23.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f58a4053950 0x7f58a4055e10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4144ad0 0x7f58c41541b0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58c40af550 0x7f58c4153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 --2- 172.21.15.32:0/2979011383 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f58c4060330 0x7f58c4141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.567+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 >> 172.21.15.32:0/2979011383 conn(0x7f58c40612d0 msgr2=0x7f58c40b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.571+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 shutdown_connections 2022-08-01T23:05:23.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.571+0000 7f58c9dbb700 1 -- 172.21.15.32:0/2979011383 wait complete. 2022-08-01T23:05:23.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T23:05:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.683+0000 7f21c359e700 1 Processor -- start 2022-08-01T23:05:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.683+0000 7f21c359e700 1 -- start start 2022-08-01T23:05:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc0425e0 0x7f21bc040c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc03de50 con 0x7f21bc0d3f40 2022-08-01T23:05:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc041bd0 con 0x7f21bc03da70 2022-08-01T23:05:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc03e680 con 0x7f21bc0425e0 2022-08-01T23:05:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21bc0d3f40 0x7f21bc03c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37440/0 (socket says 172.21.15.32:37440) 2022-08-01T23:05:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c2d9d700 1 -- 172.21.15.32:0/913833392 learned_addr learned my addr 172.21.15.32:0/913833392 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:23.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c1d9b700 1 --2- 172.21.15.32:0/913833392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc0425e0 0x7f21bc040c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901459218 0 0) 0x7f21bc03de50 con 0x7f21bc0d3f40 2022-08-01T23:05:23.697 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21a8003260 con 0x7f21bc0d3f40 2022-08-01T23:05:23.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147156757 0 0) 0x7f21bc041bd0 con 0x7f21bc03da70 2022-08-01T23:05:23.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21bc03de50 con 0x7f21bc03da70 2022-08-01T23:05:23.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c1d9b700 1 -- 172.21.15.32:0/913833392 >> v1:172.21.15.55:6789/0 conn(0x7f21bc03da70 legacy=0x7f21bc03f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c1d9b700 1 -- 172.21.15.32:0/913833392 >> v1:172.21.15.32:6789/0 conn(0x7f21bc0d3f40 legacy=0x7f21bc03c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.698 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c1d9b700 1 -- 172.21.15.32:0/913833392 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc041e70 con 0x7f21bc0425e0 2022-08-01T23:05:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c1d9b700 1 --2- 172.21.15.32:0/913833392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc0425e0 0x7f21bc040c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f21b8002f70 tx=0x7f21b8006df0).ready entity=mon.2 client_cookie=d48fedcc984dcd23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b800d5c0 con 0x7f21bc0425e0 2022-08-01T23:05:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8000e90 con 0x7f21bc0425e0 2022-08-01T23:05:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c159a700 1 -- 172.21.15.32:0/913833392 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b800d910 con 0x7f21bc0425e0 2022-08-01T23:05:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- 172.21.15.32:0/913833392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc0425e0 msgr2=0x7f21bc040c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 --2- 172.21.15.32:0/913833392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc0425e0 0x7f21bc040c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f21b8002f70 tx=0x7f21b8006df0).stop 2022-08-01T23:05:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- 172.21.15.32:0/913833392 shutdown_connections 2022-08-01T23:05:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 --2- 172.21.15.32:0/913833392 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc0425e0 0x7f21bc040c90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- 172.21.15.32:0/913833392 >> 172.21.15.32:0/913833392 conn(0x7f21bc0462f0 msgr2=0x7f21bc046700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- 172.21.15.32:0/913833392 shutdown_connections 2022-08-01T23:05:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.687+0000 7f21c359e700 1 -- 172.21.15.32:0/913833392 wait complete. 2022-08-01T23:05:23.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 Processor -- start 2022-08-01T23:05:23.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- start start 2022-08-01T23:05:23.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc03da70 0x7f21bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 0x7f21bc0dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21bc0d3f40 0x7f21bc0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0d5350 con 0x7f21bc0425e0 2022-08-01T23:05:23.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0d7e90 con 0x7f21bc0d3f40 2022-08-01T23:05:23.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0a3c70 con 0x7f21bc03da70 2022-08-01T23:05:23.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 0x7f21bc0dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 0x7f21bc0dadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49904/0 (socket says 172.21.15.32:49904) 2022-08-01T23:05:23.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 -- 172.21.15.32:0/2529874606 learned_addr learned my addr 172.21.15.32:0/2529874606 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:23.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c259c700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc03da70 0x7f21bc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c2d9d700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21bc0d3f40 0x7f21bc0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 -- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc03da70 msgr2=0x7f21bc03b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc03da70 0x7f21bc03b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 -- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21bc0d3f40 msgr2=0x7f21bc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21bc0d3f40 0x7f21bc0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 -- 172.21.15.32:0/2529874606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b800b180 con 0x7f21bc0425e0 2022-08-01T23:05:23.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c259c700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc03da70 0x7f21bc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:23.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c1d9b700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 0x7f21bc0dadc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f21b8002f70 tx=0x7f21b8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b801e400 con 0x7f21bc0425e0 2022-08-01T23:05:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b801e560 con 0x7f21bc0425e0 2022-08-01T23:05:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc0e76b0 con 0x7f21bc0425e0 2022-08-01T23:05:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21b801e8c0 con 0x7f21bc0425e0 2022-08-01T23:05:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc03bbb0 con 0x7f21bc0425e0 2022-08-01T23:05:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.691+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc004d30 con 0x7f21bc0425e0 2022-08-01T23:05:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.695+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f21b8025030 con 0x7f21bc0425e0 2022-08-01T23:05:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.695+0000 7f21b37fe700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21a0053a70 0x7f21a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.695+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f21b806cb20 con 0x7f21bc0425e0 2022-08-01T23:05:23.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.695+0000 7f21c259c700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21a0053a70 0x7f21a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.695+0000 7f21c259c700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21a0053a70 0x7f21a0055f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f21c4062000 tx=0x7f21c406b4a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.703+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f21b8043b10 con 0x7f21bc0425e0 2022-08-01T23:05:23.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.827+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f21bc03cc60 con 0x7f21bc0425e0 2022-08-01T23:05:23.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.827+0000 7f21b37fe700 1 -- 172.21.15.32:0/2529874606 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v154) v1 ==== 84+0+143 (secure 0 0 0) 0x7f21b8023070 con 0x7f21bc0425e0 2022-08-01T23:05:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:23.836 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2022-08-01T23:05:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2022-08-01T23:05:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2022-08-01T23:05:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2022-08-01T23:05:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2022-08-01T23:05:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:23.838 INFO:teuthology.orchestra.run.smithi032.stdout:] 2022-08-01T23:05:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21a0053a70 msgr2=0x7f21a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21a0053a70 0x7f21a0055f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f21c4062000 tx=0x7f21c406b4a0).stop 2022-08-01T23:05:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 msgr2=0x7f21bc0dadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 0x7f21bc0dadc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f21b8002f70 tx=0x7f21b8014000).stop 2022-08-01T23:05:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 shutdown_connections 2022-08-01T23:05:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f21a0053a70 0x7f21a0055f30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21bc0d3f40 0x7f21bc0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21bc0425e0 0x7f21bc0dadc0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 --2- 172.21.15.32:0/2529874606 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f21bc03da70 0x7f21bc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 >> 172.21.15.32:0/2529874606 conn(0x7f21bc0462f0 msgr2=0x7f21bc0d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 shutdown_connections 2022-08-01T23:05:23.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.831+0000 7f21c359e700 1 -- 172.21.15.32:0/2529874606 wait complete. 2022-08-01T23:05:23.848 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T23:05:23.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T23:05:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 Processor -- start 2022-08-01T23:05:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- start start 2022-08-01T23:05:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6f9c0b2050 0x7f6f9c0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2430 0x7f6f9c1413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c0b05b0 con 0x7f6f9c0b0140 2022-08-01T23:05:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a7960 con 0x7f6f9c0b2050 2022-08-01T23:05:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0b07e0 con 0x7f6f9c0b2430 2022-08-01T23:05:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f9c0b0140 0x7f6f9c0a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37452/0 (socket says 172.21.15.32:37452) 2022-08-01T23:05:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9b7fe700 1 -- 172.21.15.32:0/1498313667 learned_addr learned my addr 172.21.15.32:0/1498313667 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9bfff700 1 --2- 172.21.15.32:0/1498313667 >> v2:172.21.15.55:3300/0 conn(0x7f6f9c0b2050 0x7f6f9c0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa0df9700 1 --2- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2430 0x7f6f9c1413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9affd700 1 -- 172.21.15.32:0/1498313667 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126069298 0 0) 0x7f6f9c0b05b0 con 0x7f6f9c0b0140 2022-08-01T23:05:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9affd700 1 -- 172.21.15.32:0/1498313667 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f88003260 con 0x7f6f9c0b0140 2022-08-01T23:05:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9bfff700 1 -- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2430 msgr2=0x7f6f9c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9bfff700 1 --2- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2430 0x7f6f9c1413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9bfff700 1 -- 172.21.15.32:0/1498313667 >> v1:172.21.15.32:6789/0 conn(0x7f6f9c0b0140 legacy=0x7f6f9c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9bfff700 1 -- 172.21.15.32:0/1498313667 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0b0a80 con 0x7f6f9c0b2050 2022-08-01T23:05:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9bfff700 1 --2- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b2050 0x7f6f9c0b1460 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f6f8c002f70 tx=0x7f6f8c006df0).ready entity=mon.1 client_cookie=1a9a92df49d7300c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9affd700 1 -- 172.21.15.32:0/1498313667 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8c00d4a0 con 0x7f6f9c0b2050 2022-08-01T23:05:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9affd700 1 -- 172.21.15.32:0/1498313667 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c000e90 con 0x7f6f9c0b2050 2022-08-01T23:05:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6f9affd700 1 -- 172.21.15.32:0/1498313667 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8c00d7f0 con 0x7f6f9c0b2050 2022-08-01T23:05:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b2050 msgr2=0x7f6f9c0b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 --2- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b2050 0x7f6f9c0b1460 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f6f8c002f70 tx=0x7f6f8c006df0).stop 2022-08-01T23:05:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- 172.21.15.32:0/1498313667 shutdown_connections 2022-08-01T23:05:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 --2- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2430 0x7f6f9c1413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 --2- 172.21.15.32:0/1498313667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b2050 0x7f6f9c0b1460 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- 172.21.15.32:0/1498313667 >> 172.21.15.32:0/1498313667 conn(0x7f6f9c05f840 msgr2=0x7f6f9c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- 172.21.15.32:0/1498313667 shutdown_connections 2022-08-01T23:05:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.947+0000 7f6fa285c700 1 -- 172.21.15.32:0/1498313667 wait complete. 2022-08-01T23:05:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 Processor -- start 2022-08-01T23:05:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 -- start start 2022-08-01T23:05:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b0140 0x7f6f9c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2050 0x7f6f9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 0x7f6f9c0aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0aa200 con 0x7f6f9c0b2430 2022-08-01T23:05:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a9cc0 con 0x7f6f9c0b0140 2022-08-01T23:05:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a9f60 con 0x7f6f9c0b2050 2022-08-01T23:05:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 0x7f6f9c0aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 0x7f6f9c0aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49920/0 (socket says 172.21.15.32:49920) 2022-08-01T23:05:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 -- 172.21.15.32:0/3321520493 learned_addr learned my addr 172.21.15.32:0/3321520493 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6f9bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b0140 0x7f6f9c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6f9b7fe700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2050 0x7f6f9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 -- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2050 msgr2=0x7f6f9c0b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2050 0x7f6f9c0b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 -- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b0140 msgr2=0x7f6f9c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b0140 0x7f6f9c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 -- 172.21.15.32:0/3321520493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c00b180 con 0x7f6f9c0b2430 2022-08-01T23:05:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa0df9700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 0x7f6f9c0aba40 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f6f94006e00 tx=0x7f6f940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f94012600 con 0x7f6f9c0b2430 2022-08-01T23:05:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94012760 con 0x7f6f9c0b2430 2022-08-01T23:05:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.951+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c13d9c0 con 0x7f6f9c0b2430 2022-08-01T23:05:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f94012ac0 con 0x7f6f9c0b2430 2022-08-01T23:05:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c1576f0 con 0x7f6f9c0b2430 2022-08-01T23:05:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c0b52d0 con 0x7f6f9c0b2430 2022-08-01T23:05:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6f94022070 con 0x7f6f9c0b2430 2022-08-01T23:05:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6f997fa700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6f7c0539e0 0x7f6f7c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6f9406c360 con 0x7f6f9c0b2430 2022-08-01T23:05:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.955+0000 7f6f9bfff700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6f7c0539e0 0x7f6f7c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.959+0000 7f6f9bfff700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6f7c0539e0 0x7f6f7c055ea0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f6f8c01a000 tx=0x7f6f8c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:23.963+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f94043380 con 0x7f6f9c0b2430 2022-08-01T23:05:24.096 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.087+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6f9c0b0d20 con 0x7f6f9c0b2430 2022-08-01T23:05:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.383+0000 7f6f997fa700 1 -- 172.21.15.32:0/3321520493 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v155) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6f940160a0 con 0x7f6f9c0b2430 2022-08-01T23:05:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.387+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6f7c0539e0 msgr2=0x7f6f7c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.387+0000 7f6fa285c700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6f7c0539e0 0x7f6f7c055ea0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f6f8c01a000 tx=0x7f6f8c014000).stop 2022-08-01T23:05:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.387+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 msgr2=0x7f6f9c0aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.387+0000 7f6fa285c700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 0x7f6f9c0aba40 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f6f94006e00 tx=0x7f6f940091d0).stop 2022-08-01T23:05:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 shutdown_connections 2022-08-01T23:05:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6f7c0539e0 0x7f6f7c055ea0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f9c0b2430 0x7f6f9c0aba40 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6f9c0b2050 0x7f6f9c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 --2- 172.21.15.32:0/3321520493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f9c0b0140 0x7f6f9c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 >> 172.21.15.32:0/3321520493 conn(0x7f6f9c05f840 msgr2=0x7f6f9c145760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:24.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 shutdown_connections 2022-08-01T23:05:24.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.391+0000 7f6fa285c700 1 -- 172.21.15.32:0/3321520493 wait complete. 2022-08-01T23:05:24.400 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2022-08-01T23:05:24.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T23:05:24.529 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 Processor -- start 2022-08-01T23:05:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- start start 2022-08-01T23:05:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9a00b3190 0x7ff9a00b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9a00a9da0 0x7ff9a0147e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a00b3570 con 0x7ff9a00b1a80 2022-08-01T23:05:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00b2980 con 0x7ff9a00a9da0 2022-08-01T23:05:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00b0e00 con 0x7ff9a00b3190 2022-08-01T23:05:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a6cd2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9a00a9da0 0x7ff9a0147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a5cd0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9a00b1a80 0x7ff9a00656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37466/0 (socket says 172.21.15.32:37466) 2022-08-01T23:05:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a5cd0700 1 -- 172.21.15.32:0/3071682224 learned_addr learned my addr 172.21.15.32:0/3071682224 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a64d1700 1 --2- 172.21.15.32:0/3071682224 >> v2:172.21.15.55:3300/0 conn(0x7ff9a00b3190 0x7ff9a00b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a6cd2700 1 -- 172.21.15.32:0/3071682224 >> v2:172.21.15.55:3300/0 conn(0x7ff9a00b3190 msgr2=0x7ff9a00b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a6cd2700 1 --2- 172.21.15.32:0/3071682224 >> v2:172.21.15.55:3300/0 conn(0x7ff9a00b3190 0x7ff9a00b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a6cd2700 1 -- 172.21.15.32:0/3071682224 >> v1:172.21.15.32:6789/0 conn(0x7ff9a00b1a80 legacy=0x7ff9a00656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:24.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a6cd2700 1 -- 172.21.15.32:0/3071682224 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00b10a0 con 0x7ff9a00a9da0 2022-08-01T23:05:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a64d1700 1 --2- 172.21.15.32:0/3071682224 >> v2:172.21.15.55:3300/0 conn(0x7ff9a00b3190 0x7ff9a00b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a6cd2700 1 --2- 172.21.15.32:0/3071682224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00a9da0 0x7ff9a0147e30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7ff9980032f0 tx=0x7ff9980091d0).ready entity=mon.0 client_cookie=f88a224f078028e9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a54cf700 1 -- 172.21.15.32:0/3071682224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff998009ea0 con 0x7ff9a00a9da0 2022-08-01T23:05:24.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a54cf700 1 -- 172.21.15.32:0/3071682224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800b870 con 0x7ff9a00a9da0 2022-08-01T23:05:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a54cf700 1 -- 172.21.15.32:0/3071682224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99800b9e0 con 0x7ff9a00a9da0 2022-08-01T23:05:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- 172.21.15.32:0/3071682224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00a9da0 msgr2=0x7ff9a0147e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/3071682224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00a9da0 0x7ff9a0147e30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7ff9980032f0 tx=0x7ff9980091d0).stop 2022-08-01T23:05:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- 172.21.15.32:0/3071682224 shutdown_connections 2022-08-01T23:05:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/3071682224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00a9da0 0x7ff9a0147e30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/3071682224 >> v2:172.21.15.55:3300/0 conn(0x7ff9a00b3190 0x7ff9a00b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- 172.21.15.32:0/3071682224 >> 172.21.15.32:0/3071682224 conn(0x7ff9a0063870 msgr2=0x7ff9a0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- 172.21.15.32:0/3071682224 shutdown_connections 2022-08-01T23:05:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.523+0000 7ff9a74d3700 1 -- 172.21.15.32:0/3071682224 wait complete. 2022-08-01T23:05:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 Processor -- start 2022-08-01T23:05:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 -- start start 2022-08-01T23:05:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9a00a9da0 0x7ff9a014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 0x7ff9a01435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a00b3190 0x7ff9a014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0149250 con 0x7ff9a00b1a80 2022-08-01T23:05:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00b43a0 con 0x7ff9a00b3190 2022-08-01T23:05:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a74d3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a014d930 con 0x7ff9a00a9da0 2022-08-01T23:05:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 0x7ff9a01435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 0x7ff9a01435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49930/0 (socket says 172.21.15.32:49930) 2022-08-01T23:05:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 -- 172.21.15.32:0/2699470634 learned_addr learned my addr 172.21.15.32:0/2699470634 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a6cd2700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a00b3190 0x7ff9a014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a64d1700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9a00a9da0 0x7ff9a014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 -- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9a00a9da0 msgr2=0x7ff9a014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9a00a9da0 0x7ff9a014a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 -- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a00b3190 msgr2=0x7ff9a014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a00b3190 0x7ff9a014bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 -- 172.21.15.32:0/2699470634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998009470 con 0x7ff9a00b1a80 2022-08-01T23:05:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a6cd2700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a00b3190 0x7ff9a014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.527+0000 7ff9a5cd0700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 0x7ff9a01435d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7ff9a014dac0 tx=0x7ff99c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c0070d0 con 0x7ff9a00b1a80 2022-08-01T23:05:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c007230 con 0x7ff9a00b1a80 2022-08-01T23:05:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a014d690 con 0x7ff9a00b1a80 2022-08-01T23:05:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c005610 con 0x7ff9a00b1a80 2022-08-01T23:05:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a0148580 con 0x7ff9a00b1a80 2022-08-01T23:05:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a00b6270 con 0x7ff9a00b1a80 2022-08-01T23:05:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff99c010070 con 0x7ff9a00b1a80 2022-08-01T23:05:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff98f7fe700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9800539e0 0x7ff980055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5954+0+0 (secure 0 0 0) 0x7ff99c06b840 con 0x7ff9a00b1a80 2022-08-01T23:05:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.531+0000 7ff9a64d1700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9800539e0 0x7ff980055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.535+0000 7ff9a64d1700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9800539e0 0x7ff980055ea0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7ff99000c000 tx=0x7ff990008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.539+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff99c0427d0 con 0x7ff9a00b1a80 2022-08-01T23:05:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:24.667+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff9a00574e0 con 0x7ff9a00b1a80 2022-08-01T23:05:24.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:24.806 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:24.809 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:24.812 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.399+0000 7ff98f7fe700 1 -- 172.21.15.32:0/2699470634 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v156) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff99c045df0 con 0x7ff9a00b1a80 2022-08-01T23:05:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9800539e0 msgr2=0x7ff980055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9800539e0 0x7ff980055ea0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7ff99000c000 tx=0x7ff990008040).stop 2022-08-01T23:05:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 msgr2=0x7ff9a01435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 0x7ff9a01435d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7ff9a014dac0 tx=0x7ff99c007c00).stop 2022-08-01T23:05:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 shutdown_connections 2022-08-01T23:05:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9800539e0 0x7ff980055ea0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9a00b3190 0x7ff9a014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a00b1a80 0x7ff9a01435d0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 --2- 172.21.15.32:0/2699470634 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9a00a9da0 0x7ff9a014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 >> 172.21.15.32:0/2699470634 conn(0x7ff9a0063870 msgr2=0x7ff9a005fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 shutdown_connections 2022-08-01T23:05:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.403+0000 7ff9a74d3700 1 -- 172.21.15.32:0/2699470634 wait complete. 2022-08-01T23:05:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2022-08-01T23:05:25.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T23:05:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 Processor -- start 2022-08-01T23:05:25.535 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 -- start start 2022-08-01T23:05:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f678c0614b0 0x7f678c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c0ad750 0x7f678c0b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0aee80 con 0x7f678c1405d0 2022-08-01T23:05:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0b70b0 con 0x7f678c0614b0 2022-08-01T23:05:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6792563700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0af0b0 con 0x7f678c0ad750 2022-08-01T23:05:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.527+0000 7f6791561700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f678c0614b0 0x7f678c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791d62700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f678c1405d0 0x7f678c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37484/0 (socket says 172.21.15.32:37484) 2022-08-01T23:05:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791d62700 1 -- 172.21.15.32:0/1710736453 learned_addr learned my addr 172.21.15.32:0/1710736453 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6790d60700 1 --2- 172.21.15.32:0/1710736453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c0ad750 0x7f678c0b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791561700 1 -- 172.21.15.32:0/1710736453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c0ad750 msgr2=0x7f678c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791561700 1 --2- 172.21.15.32:0/1710736453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c0ad750 0x7f678c0b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791561700 1 -- 172.21.15.32:0/1710736453 >> v1:172.21.15.32:6789/0 conn(0x7f678c1405d0 legacy=0x7f678c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f677bfff700 1 -- 172.21.15.32:0/1710736453 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866441012 0 0) 0x7f678c0aee80 con 0x7f678c1405d0 2022-08-01T23:05:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791561700 1 -- 172.21.15.32:0/1710736453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0af350 con 0x7f678c0614b0 2022-08-01T23:05:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6791561700 1 --2- 172.21.15.32:0/1710736453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0614b0 0x7f678c061890 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f677c002f70 tx=0x7f677c006df0).ready entity=mon.0 client_cookie=1154ca0be5e6dee6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f677bfff700 1 -- 172.21.15.32:0/1710736453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677c00d570 con 0x7f678c0614b0 2022-08-01T23:05:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f677bfff700 1 -- 172.21.15.32:0/1710736453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c000e90 con 0x7f678c0614b0 2022-08-01T23:05:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f677bfff700 1 -- 172.21.15.32:0/1710736453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677c00d8c0 con 0x7f678c0614b0 2022-08-01T23:05:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 -- 172.21.15.32:0/1710736453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0614b0 msgr2=0x7f678c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 --2- 172.21.15.32:0/1710736453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0614b0 0x7f678c061890 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f677c002f70 tx=0x7f677c006df0).stop 2022-08-01T23:05:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 -- 172.21.15.32:0/1710736453 shutdown_connections 2022-08-01T23:05:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 --2- 172.21.15.32:0/1710736453 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c0ad750 0x7f678c0b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 --2- 172.21.15.32:0/1710736453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0614b0 0x7f678c061890 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 -- 172.21.15.32:0/1710736453 >> 172.21.15.32:0/1710736453 conn(0x7f678c066f70 msgr2=0x7f678c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 -- 172.21.15.32:0/1710736453 shutdown_connections 2022-08-01T23:05:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 -- 172.21.15.32:0/1710736453 wait complete. 2022-08-01T23:05:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 Processor -- start 2022-08-01T23:05:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.531+0000 7f6792563700 1 -- start start 2022-08-01T23:05:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c0614b0 0x7f678c0b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 0x7f678c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c1405d0 0x7f678c13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b1670 con 0x7f678c0ad750 2022-08-01T23:05:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b1410 con 0x7f678c0614b0 2022-08-01T23:05:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b3e60 con 0x7f678c1405d0 2022-08-01T23:05:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6790d60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 0x7f678c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c1405d0 0x7f678c13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791561700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c0614b0 0x7f678c0b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6790d60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 0x7f678c13f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49948/0 (socket says 172.21.15.32:49948) 2022-08-01T23:05:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6790d60700 1 -- 172.21.15.32:0/485806730 learned_addr learned my addr 172.21.15.32:0/485806730 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 -- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c0614b0 msgr2=0x7f678c0b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c0614b0 0x7f678c0b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 -- 172.21.15.32:0/485806730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 msgr2=0x7f678c13f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 0x7f678c13f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 -- 172.21.15.32:0/485806730 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c00b180 con 0x7f678c1405d0 2022-08-01T23:05:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791561700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c0614b0 0x7f678c0b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6790d60700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 0x7f678c13f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6791d62700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c1405d0 0x7f678c13fe10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6784001250 tx=0x7f6784007f20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:25.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678400ce80 con 0x7f678c1405d0 2022-08-01T23:05:25.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6784005ba0 con 0x7f678c1405d0 2022-08-01T23:05:25.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67840048a0 con 0x7f678c1405d0 2022-08-01T23:05:25.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c140300 con 0x7f678c1405d0 2022-08-01T23:05:25.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c1431b0 con 0x7f678c1405d0 2022-08-01T23:05:25.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.535+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c0b4060 con 0x7f678c1405d0 2022-08-01T23:05:25.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.539+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6784004a00 con 0x7f678c1405d0 2022-08-01T23:05:25.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.539+0000 7f677a7fc700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f676c053980 0x7f676c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.539+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f678406b9c0 con 0x7f678c1405d0 2022-08-01T23:05:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.539+0000 7f6791561700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f676c053980 0x7f676c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.539+0000 7f6791561700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f676c053980 0x7f676c055e40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f677c01a000 tx=0x7f677c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:25.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.547+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6784042800 con 0x7f678c1405d0 2022-08-01T23:05:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.711+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f678c0af5f0 con 0x7f678c1405d0 2022-08-01T23:05:25.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.727+0000 7f677a7fc700 1 -- 172.21.15.32:0/485806730 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v79) v1 ==== 189+0+0 (secure 0 0 0) 0x7f6784045e20 con 0x7f678c1405d0 2022-08-01T23:05:25.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f676c053980 msgr2=0x7f676c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f676c053980 0x7f676c055e40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f677c01a000 tx=0x7f677c014000).stop 2022-08-01T23:05:25.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c1405d0 msgr2=0x7f678c13fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c1405d0 0x7f678c13fe10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6784001250 tx=0x7f6784007f20).stop 2022-08-01T23:05:25.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 shutdown_connections 2022-08-01T23:05:25.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f676c053980 0x7f676c055e40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f678c1405d0 0x7f678c13fe10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678c0ad750 0x7f678c13f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 --2- 172.21.15.32:0/485806730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f678c0614b0 0x7f678c0b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 >> 172.21.15.32:0/485806730 conn(0x7f678c066f70 msgr2=0x7f678c05ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 shutdown_connections 2022-08-01T23:05:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.731+0000 7f6792563700 1 -- 172.21.15.32:0/485806730 wait complete. 2022-08-01T23:05:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 29 and data pool 30 2022-08-01T23:05:25.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:25.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 Processor -- start 2022-08-01T23:05:25.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 -- start start 2022-08-01T23:05:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bc00b01a0 0x7f8bc00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8bc00afd60 0x7f8bc00a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc00a7250 con 0x7f8bc00b2710 2022-08-01T23:05:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc00a7480 con 0x7f8bc00b01a0 2022-08-01T23:05:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.839+0000 7f8bc5b4d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc00a7720 con 0x7f8bc00afd60 2022-08-01T23:05:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bc00b01a0 0x7f8bc00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bc00b01a0 0x7f8bc00a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49960/0 (socket says 172.21.15.32:49960) 2022-08-01T23:05:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 -- 172.21.15.32:0/664709006 learned_addr learned my addr 172.21.15.32:0/664709006 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc534c700 1 --2- 172.21.15.32:0/664709006 >> v2:172.21.15.55:3300/0 conn(0x7f8bc00afd60 0x7f8bc00a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 -- 172.21.15.32:0/664709006 >> v2:172.21.15.55:3300/0 conn(0x7f8bc00afd60 msgr2=0x7f8bc00a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 --2- 172.21.15.32:0/664709006 >> v2:172.21.15.55:3300/0 conn(0x7f8bc00afd60 0x7f8bc00a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 -- 172.21.15.32:0/664709006 >> v1:172.21.15.55:6789/0 conn(0x7f8bc00b2710 legacy=0x7f8bc014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 -- 172.21.15.32:0/664709006 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00a79c0 con 0x7f8bc00b01a0 2022-08-01T23:05:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbffff700 1 --2- 172.21.15.32:0/664709006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b01a0 0x7f8bc00a8540 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f8ba8002f70 tx=0x7f8ba8006eb0).ready entity=mon.0 client_cookie=6304c71011433aa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:25.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbf7fe700 1 -- 172.21.15.32:0/664709006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba800d5c0 con 0x7f8bc00b01a0 2022-08-01T23:05:25.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbf7fe700 1 -- 172.21.15.32:0/664709006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8000e90 con 0x7f8bc00b01a0 2022-08-01T23:05:25.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bbf7fe700 1 -- 172.21.15.32:0/664709006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba800d950 con 0x7f8bc00b01a0 2022-08-01T23:05:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/664709006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b01a0 msgr2=0x7f8bc00a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/664709006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b01a0 0x7f8bc00a8540 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f8ba8002f70 tx=0x7f8ba8006eb0).stop 2022-08-01T23:05:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/664709006 shutdown_connections 2022-08-01T23:05:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/664709006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b01a0 0x7f8bc00a8540 secure :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0x7f8ba8002f70 tx=0x7f8ba8006eb0).stop 2022-08-01T23:05:25.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/664709006 >> v2:172.21.15.55:3300/0 conn(0x7f8bc00afd60 0x7f8bc00a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/664709006 >> 172.21.15.32:0/664709006 conn(0x7f8bc0060310 msgr2=0x7f8bc00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:25.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/664709006 shutdown_connections 2022-08-01T23:05:25.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.843+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/664709006 wait complete. 2022-08-01T23:05:25.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 Processor -- start 2022-08-01T23:05:25.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 -- start start 2022-08-01T23:05:25.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8bc00afd60 0x7f8bc00646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 0x7f8bc0156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc0066380 0x7f8bc0066810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00636f0 con 0x7f8bc00b2710 2022-08-01T23:05:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00abe90 con 0x7f8bc0066380 2022-08-01T23:05:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc5b4d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0064ba0 con 0x7f8bc00afd60 2022-08-01T23:05:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 0x7f8bc0156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 0x7f8bc0156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49966/0 (socket says 172.21.15.32:49966) 2022-08-01T23:05:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 -- 172.21.15.32:0/714922503 learned_addr learned my addr 172.21.15.32:0/714922503 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc4b4b700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8bc00afd60 0x7f8bc00646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bc534c700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc0066380 0x7f8bc0066810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 -- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8bc00afd60 msgr2=0x7f8bc00646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8bc00afd60 0x7f8bc00646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 -- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc0066380 msgr2=0x7f8bc0066810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc0066380 0x7f8bc0066810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 -- 172.21.15.32:0/714922503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba800b1e0 con 0x7f8bc00b2710 2022-08-01T23:05:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbffff700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 0x7f8bc0156c00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f8ba8000b60 tx=0x7f8ba8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.847+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba8011840 con 0x7f8bc00b2710 2022-08-01T23:05:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba80119a0 con 0x7f8bc00b2710 2022-08-01T23:05:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc0066ec0 con 0x7f8bc00b2710 2022-08-01T23:05:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba8011d00 con 0x7f8bc00b2710 2022-08-01T23:05:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc00abb00 con 0x7f8bc00b2710 2022-08-01T23:05:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc00ac000 con 0x7f8bc00b2710 2022-08-01T23:05:25.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ba801a040 con 0x7f8bc00b2710 2022-08-01T23:05:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bbdffb700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ba4053aa0 0x7f8ba4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8ba806cfe0 con 0x7f8bc00b2710 2022-08-01T23:05:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.851+0000 7f8bc4b4b700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ba4053aa0 0x7f8ba4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.855+0000 7f8bc4b4b700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ba4053aa0 0x7f8ba4055f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f8bb800d000 tx=0x7f8bb8007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.859+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8bc00ac000 con 0x7f8bc00b2710 2022-08-01T23:05:25.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.987+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bc00ac000 con 0x7f8bc00b2710 2022-08-01T23:05:25.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bbdffb700 1 -- 172.21.15.32:0/714922503 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f8bc00ac000 con 0x7f8bc00b2710 2022-08-01T23:05:25.997 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:25.998 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":157,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:25.720295+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:24.145656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:24.725583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:26.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ba4053aa0 msgr2=0x7f8ba4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ba4053aa0 0x7f8ba4055f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f8bb800d000 tx=0x7f8bb8007f10).stop 2022-08-01T23:05:26.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 msgr2=0x7f8bc0156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 0x7f8bc0156c00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f8ba8000b60 tx=0x7f8ba8015000).stop 2022-08-01T23:05:26.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 shutdown_connections 2022-08-01T23:05:26.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8ba4053aa0 0x7f8ba4055f60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc0066380 0x7f8bc0066810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bc00b2710 0x7f8bc0156c00 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 --2- 172.21.15.32:0/714922503 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8bc00afd60 0x7f8bc00646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 >> 172.21.15.32:0/714922503 conn(0x7f8bc0060310 msgr2=0x7f8bc0148ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 shutdown_connections 2022-08-01T23:05:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:25.991+0000 7f8bc5b4d700 1 -- 172.21.15.32:0/714922503 wait complete. 2022-08-01T23:05:26.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 Processor -- start 2022-08-01T23:05:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 -- start start 2022-08-01T23:05:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96dc0b7840 0x7f96dc065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc061120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0b2c20 con 0x7f96dc14b040 2022-08-01T23:05:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc065c10 con 0x7f96dc0b7840 2022-08-01T23:05:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e302c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0b7c20 con 0x7f96dc0b3070 2022-08-01T23:05:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e282b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96dc14b040 0x7f96dc14b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37518/0 (socket says 172.21.15.32:37518) 2022-08-01T23:05:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e282b700 1 -- 172.21.15.32:0/3082138903 learned_addr learned my addr 172.21.15.32:0/3082138903 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e202a700 1 --2- 172.21.15.32:0/3082138903 >> v2:172.21.15.32:3300/0 conn(0x7f96dc0b7840 0x7f96dc065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.111+0000 7f96e1829700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e1028700 1 -- 172.21.15.32:0/3082138903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152767853 0 0) 0x7f96dc0b2c20 con 0x7f96dc14b040 2022-08-01T23:05:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e1028700 1 -- 172.21.15.32:0/3082138903 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c4003260 con 0x7f96dc14b040 2022-08-01T23:05:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e202a700 1 -- 172.21.15.32:0/3082138903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 msgr2=0x7f96dc061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e202a700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc061120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e202a700 1 -- 172.21.15.32:0/3082138903 >> v1:172.21.15.32:6789/0 conn(0x7f96dc14b040 legacy=0x7f96dc14b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e202a700 1 -- 172.21.15.32:0/3082138903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc0b2980 con 0x7f96dc0b7840 2022-08-01T23:05:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e1829700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc061120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e202a700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc065720 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f96d8002f70 tx=0x7f96d8006df0).ready entity=mon.0 client_cookie=c04c3354144e2805 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e1028700 1 -- 172.21.15.32:0/3082138903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800e470 con 0x7f96dc0b7840 2022-08-01T23:05:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e1028700 1 -- 172.21.15.32:0/3082138903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d8000e90 con 0x7f96dc0b7840 2022-08-01T23:05:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e1028700 1 -- 172.21.15.32:0/3082138903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800e7c0 con 0x7f96dc0b7840 2022-08-01T23:05:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 -- 172.21.15.32:0/3082138903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 msgr2=0x7f96dc065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc065720 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f96d8002f70 tx=0x7f96d8006df0).stop 2022-08-01T23:05:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 -- 172.21.15.32:0/3082138903 shutdown_connections 2022-08-01T23:05:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc061120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 --2- 172.21.15.32:0/3082138903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc065720 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 -- 172.21.15.32:0/3082138903 >> 172.21.15.32:0/3082138903 conn(0x7f96dc05f840 msgr2=0x7f96dc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 -- 172.21.15.32:0/3082138903 shutdown_connections 2022-08-01T23:05:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.115+0000 7f96e302c700 1 -- 172.21.15.32:0/3082138903 wait complete. 2022-08-01T23:05:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 Processor -- start 2022-08-01T23:05:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 -- start start 2022-08-01T23:05:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96dc14b040 0x7f96dc0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc15abf0 con 0x7f96dc0b7840 2022-08-01T23:05:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0a1c50 con 0x7f96dc14b040 2022-08-01T23:05:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0a1ef0 con 0x7f96dc0b3070 2022-08-01T23:05:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc0b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49980/0 (socket says 172.21.15.32:49980) 2022-08-01T23:05:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 -- 172.21.15.32:0/226995559 learned_addr learned my addr 172.21.15.32:0/226995559 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e202a700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e282b700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96dc14b040 0x7f96dc0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 -- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 msgr2=0x7f96dc15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 -- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96dc14b040 msgr2=0x7f96dc0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96dc14b040 0x7f96dc0aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 -- 172.21.15.32:0/226995559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc00b180 con 0x7f96dc0b7840 2022-08-01T23:05:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e1829700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc0b0c60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f96cc00c000 tx=0x7f96cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96cc010780 con 0x7f96dc0b7840 2022-08-01T23:05:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc0108e0 con 0x7f96dc0b7840 2022-08-01T23:05:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d800b120 con 0x7f96dc0b7840 2022-08-01T23:05:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96cc010c40 con 0x7f96dc0b7840 2022-08-01T23:05:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.119+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc0b1150 con 0x7f96dc0b7840 2022-08-01T23:05:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.123+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0a2390 con 0x7f96dc0b7840 2022-08-01T23:05:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.123+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96cc01c460 con 0x7f96dc0b7840 2022-08-01T23:05:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.123+0000 7f96cb7fe700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96bc053b00 0x7f96bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.123+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96cc06c650 con 0x7f96dc0b7840 2022-08-01T23:05:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.123+0000 7f96e202a700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96bc053b00 0x7f96bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.123+0000 7f96e202a700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96bc053b00 0x7f96bc055fc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f96d801a000 tx=0x7f96d8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.127+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96cc0432a0 con 0x7f96dc0b7840 2022-08-01T23:05:26.262 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.255+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96dc0609e0 con 0x7f96dc0b7840 2022-08-01T23:05:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.255+0000 7f96cb7fe700 1 -- 172.21.15.32:0/226995559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f96cc0468c0 con 0x7f96dc0b7840 2022-08-01T23:05:26.264 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:26.265 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":157,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:25.720295+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:24.145656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:24.725583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:26.267 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96bc053b00 msgr2=0x7f96bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96bc053b00 0x7f96bc055fc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f96d801a000 tx=0x7f96d8014000).stop 2022-08-01T23:05:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 msgr2=0x7f96dc0b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc0b0c60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f96cc00c000 tx=0x7f96cc006df0).stop 2022-08-01T23:05:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 shutdown_connections 2022-08-01T23:05:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f96bc053b00 0x7f96bc055fc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96dc14b040 0x7f96dc0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0b7840 0x7f96dc0b0c60 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 --2- 172.21.15.32:0/226995559 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f96dc0b3070 0x7f96dc15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 >> 172.21.15.32:0/226995559 conn(0x7f96dc05f840 msgr2=0x7f96dc05cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 shutdown_connections 2022-08-01T23:05:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.259+0000 7f96e302c700 1 -- 172.21.15.32:0/226995559 wait complete. 2022-08-01T23:05:26.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T23:05:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e52e0700 1 Processor -- start 2022-08-01T23:05:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e52e0700 1 -- start start 2022-08-01T23:05:26.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e52e0700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79e00614b0 0x7f79e0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.395 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e52e0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e00aee80 con 0x7f79e01405d0 2022-08-01T23:05:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e52e0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e00b70b0 con 0x7f79e00ad750 2022-08-01T23:05:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e52e0700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e00af0b0 con 0x7f79e00614b0 2022-08-01T23:05:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e4adf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79e01405d0 0x7f79e01466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37532/0 (socket says 172.21.15.32:37532) 2022-08-01T23:05:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79e4adf700 1 -- 172.21.15.32:0/1244359173 learned_addr learned my addr 172.21.15.32:0/1244359173 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79dffff700 1 --2- 172.21.15.32:0/1244359173 >> v2:172.21.15.55:3300/0 conn(0x7f79e00614b0 0x7f79e0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79deffd700 1 -- 172.21.15.32:0/1244359173 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070420847 0 0) 0x7f79e00aee80 con 0x7f79e01405d0 2022-08-01T23:05:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79deffd700 1 -- 172.21.15.32:0/1244359173 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c0003260 con 0x7f79e01405d0 2022-08-01T23:05:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79dffff700 1 -- 172.21.15.32:0/1244359173 >> v1:172.21.15.55:6789/0 conn(0x7f79e00ad750 legacy=0x7f79e00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79dffff700 1 -- 172.21.15.32:0/1244359173 >> v1:172.21.15.32:6789/0 conn(0x7f79e01405d0 legacy=0x7f79e01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79dffff700 1 -- 172.21.15.32:0/1244359173 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00af350 con 0x7f79e00614b0 2022-08-01T23:05:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.387+0000 7f79dffff700 1 --2- 172.21.15.32:0/1244359173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0061890 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f79e00aed30 tx=0x7f79d00091d0).ready entity=mon.1 client_cookie=62622254a7f0d716 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79deffd700 1 -- 172.21.15.32:0/1244359173 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d000b680 con 0x7f79e00614b0 2022-08-01T23:05:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79deffd700 1 -- 172.21.15.32:0/1244359173 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d000b7e0 con 0x7f79e00614b0 2022-08-01T23:05:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79deffd700 1 -- 172.21.15.32:0/1244359173 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d000b950 con 0x7f79e00614b0 2022-08-01T23:05:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 -- 172.21.15.32:0/1244359173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 msgr2=0x7f79e0061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 --2- 172.21.15.32:0/1244359173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0061890 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f79e00aed30 tx=0x7f79d00091d0).stop 2022-08-01T23:05:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 -- 172.21.15.32:0/1244359173 shutdown_connections 2022-08-01T23:05:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 --2- 172.21.15.32:0/1244359173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0061890 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 -- 172.21.15.32:0/1244359173 >> 172.21.15.32:0/1244359173 conn(0x7f79e0066f70 msgr2=0x7f79e0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 -- 172.21.15.32:0/1244359173 shutdown_connections 2022-08-01T23:05:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 -- 172.21.15.32:0/1244359173 wait complete. 2022-08-01T23:05:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 Processor -- start 2022-08-01T23:05:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.391+0000 7f79e52e0700 1 -- start start 2022-08-01T23:05:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e52e0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e52e0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79e00ad750 0x7f79e0152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e52e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 0x7f79e01530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e52e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0152800 con 0x7f79e01405d0 2022-08-01T23:05:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e52e0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00aaf00 con 0x7f79e00614b0 2022-08-01T23:05:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e52e0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0143b50 con 0x7f79e00ad750 2022-08-01T23:05:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 0x7f79e01530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79dffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 0x7f79e01530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49996/0 (socket says 172.21.15.32:49996) 2022-08-01T23:05:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79dffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:59778/0 (socket says 172.21.15.32:59778) 2022-08-01T23:05:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 -- 172.21.15.32:0/2330888999 learned_addr learned my addr 172.21.15.32:0/2330888999 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79df7fe700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79e00ad750 0x7f79e0152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 -- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79e00ad750 msgr2=0x7f79e0152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79e00ad750 0x7f79e0152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 -- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 msgr2=0x7f79e0147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 -- 172.21.15.32:0/2330888999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009470 con 0x7f79e01405d0 2022-08-01T23:05:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.395+0000 7f79e4adf700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 0x7f79e01530a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f79e00acb30 tx=0x7f79d8007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d800cd90 con 0x7f79e01405d0 2022-08-01T23:05:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d80070d0 con 0x7f79e01405d0 2022-08-01T23:05:26.410 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e00aa720 con 0x7f79e01405d0 2022-08-01T23:05:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d8007430 con 0x7f79e01405d0 2022-08-01T23:05:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e00aac60 con 0x7f79e01405d0 2022-08-01T23:05:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e0152950 con 0x7f79e01405d0 2022-08-01T23:05:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f79d801a060 con 0x7f79e01405d0 2022-08-01T23:05:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dd7fa700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79c4053950 0x7f79c4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f79d806b630 con 0x7f79e01405d0 2022-08-01T23:05:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.399+0000 7f79dffff700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79c4053950 0x7f79c4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.403+0000 7f79dffff700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79c4053950 0x7f79c4055e10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f79d001b000 tx=0x7f79d0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:26.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.407+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79d80423f0 con 0x7f79e01405d0 2022-08-01T23:05:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:26.563+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f79e00af5f0 con 0x7f79e01405d0 2022-08-01T23:05:27.409 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.403+0000 7f79dd7fa700 1 -- 172.21.15.32:0/2330888999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7f79d8018070 con 0x7f79e01405d0 2022-08-01T23:05:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79c4053950 msgr2=0x7f79c4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79c4053950 0x7f79c4055e10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f79d001b000 tx=0x7f79d0016000).stop 2022-08-01T23:05:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 msgr2=0x7f79e01530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 0x7f79e01530a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f79e00acb30 tx=0x7f79d8007a80).stop 2022-08-01T23:05:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 shutdown_connections 2022-08-01T23:05:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f79c4053950 0x7f79c4055e10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79e01405d0 0x7f79e01530a0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f79e00ad750 0x7f79e0152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 --2- 172.21.15.32:0/2330888999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e00614b0 0x7f79e0147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 >> 172.21.15.32:0/2330888999 conn(0x7f79e0066f70 msgr2=0x7f79e01464c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 shutdown_connections 2022-08-01T23:05:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.407+0000 7f79e52e0700 1 -- 172.21.15.32:0/2330888999 wait complete. 2022-08-01T23:05:27.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:27.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 Processor -- start 2022-08-01T23:05:27.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 -- start start 2022-08-01T23:05:27.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc280b3630 0x7fcc28143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280aa0d0 0x7fcc280af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc280af6f0 0x7fcc2805f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc281443f0 con 0x7fcc280b3630 2022-08-01T23:05:27.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280b3a10 con 0x7fcc280af6f0 2022-08-01T23:05:27.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc2d58a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280afb80 con 0x7fcc280aa0d0 2022-08-01T23:05:27.546 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc280b3630 0x7fcc28143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc280b3630 0x7fcc28143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50000/0 (socket says 172.21.15.32:50000) 2022-08-01T23:05:27.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 -- 172.21.15.32:0/8941458 learned_addr learned my addr 172.21.15.32:0/8941458 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:27.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc267fc700 1 --2- 172.21.15.32:0/8941458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280aa0d0 0x7fcc280af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.547 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc277fe700 1 --2- 172.21.15.32:0/8941458 >> v2:172.21.15.55:3300/0 conn(0x7fcc280af6f0 0x7fcc2805f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 -- 172.21.15.32:0/8941458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280aa0d0 msgr2=0x7fcc280af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:27.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 --2- 172.21.15.32:0/8941458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280aa0d0 0x7fcc280af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 -- 172.21.15.32:0/8941458 >> v2:172.21.15.55:3300/0 conn(0x7fcc280af6f0 msgr2=0x7fcc2805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:27.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 --2- 172.21.15.32:0/8941458 >> v2:172.21.15.55:3300/0 conn(0x7fcc280af6f0 0x7fcc2805f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 -- 172.21.15.32:0/8941458 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28146b40 con 0x7fcc280b3630 2022-08-01T23:05:27.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc26ffd700 1 --2- 172.21.15.32:0/8941458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280b3630 0x7fcc28143f00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fcc180032f0 tx=0x7fcc18009290).ready entity=mon.0 client_cookie=c420bc8471f1a578 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:27.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc25ffb700 1 -- 172.21.15.32:0/8941458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1800b870 con 0x7fcc280b3630 2022-08-01T23:05:27.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc25ffb700 1 -- 172.21.15.32:0/8941458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1800b9d0 con 0x7fcc280b3630 2022-08-01T23:05:27.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.535+0000 7fcc25ffb700 1 -- 172.21.15.32:0/8941458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1800bb40 con 0x7fcc280b3630 2022-08-01T23:05:27.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- 172.21.15.32:0/8941458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280b3630 msgr2=0x7fcc28143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/8941458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280b3630 0x7fcc28143f00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fcc180032f0 tx=0x7fcc18009290).stop 2022-08-01T23:05:27.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- 172.21.15.32:0/8941458 shutdown_connections 2022-08-01T23:05:27.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/8941458 >> v2:172.21.15.55:3300/0 conn(0x7fcc280af6f0 0x7fcc2805f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/8941458 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280aa0d0 0x7fcc280af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/8941458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280b3630 0x7fcc28143f00 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- 172.21.15.32:0/8941458 >> 172.21.15.32:0/8941458 conn(0x7fcc28060950 msgr2=0x7fcc28060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:27.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- 172.21.15.32:0/8941458 shutdown_connections 2022-08-01T23:05:27.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- 172.21.15.32:0/8941458 wait complete. 2022-08-01T23:05:27.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 Processor -- start 2022-08-01T23:05:27.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- start start 2022-08-01T23:05:27.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 0x7fcc28138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc280af6f0 0x7fcc28146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280b3630 0x7fcc280b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc281535d0 con 0x7fcc280aa0d0 2022-08-01T23:05:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280b44c0 con 0x7fcc280af6f0 2022-08-01T23:05:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc2d58a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280b4760 con 0x7fcc280b3630 2022-08-01T23:05:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 0x7fcc28138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 0x7fcc28138f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50004/0 (socket says 172.21.15.32:50004) 2022-08-01T23:05:27.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280b3630 0x7fcc280b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc26ffd700 1 -- 172.21.15.32:0/886832001 learned_addr learned my addr 172.21.15.32:0/886832001 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.539+0000 7fcc267fc700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc280af6f0 0x7fcc28146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc26ffd700 1 -- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280b3630 msgr2=0x7fcc280b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc26ffd700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280b3630 0x7fcc280b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc26ffd700 1 -- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc280af6f0 msgr2=0x7fcc28146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc26ffd700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc280af6f0 0x7fcc28146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc26ffd700 1 -- 172.21.15.32:0/886832001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18009530 con 0x7fcc280aa0d0 2022-08-01T23:05:27.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc26ffd700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 0x7fcc28138f40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fcc180032f0 tx=0x7fcc18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1800b870 con 0x7fcc280aa0d0 2022-08-01T23:05:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc18021630 con 0x7fcc280aa0d0 2022-08-01T23:05:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc280b4a00 con 0x7fcc280aa0d0 2022-08-01T23:05:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc18011ea0 con 0x7fcc280aa0d0 2022-08-01T23:05:27.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc281531a0 con 0x7fcc280aa0d0 2022-08-01T23:05:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc280b4b90 con 0x7fcc280aa0d0 2022-08-01T23:05:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.543+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcc18027020 con 0x7fcc280aa0d0 2022-08-01T23:05:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.547+0000 7fcc1ffff700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc0c053b30 0x7fcc0c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.547+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcc1806d4f0 con 0x7fcc280aa0d0 2022-08-01T23:05:27.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.547+0000 7fcc267fc700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc0c053b30 0x7fcc0c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.547+0000 7fcc267fc700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc0c053b30 0x7fcc0c055ff0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fcc1000c000 tx=0x7fcc10008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:27.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.551+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc18044120 con 0x7fcc280aa0d0 2022-08-01T23:05:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.711+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc281535d0 con 0x7fcc280aa0d0 2022-08-01T23:05:27.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc1ffff700 1 -- 172.21.15.32:0/886832001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fcc18015090 con 0x7fcc280aa0d0 2022-08-01T23:05:27.721 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:27.721 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1983320963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1983320963},{"type":"v1","addr":"172.21.15.55:6835","nonce":1983320963}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78},{"gid":8754,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3801584266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3801584266},{"type":"v1","addr":"172.21.15.32:6833","nonce":3801584266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:25.720657+0000","modified":"2022-08-01T23:05:27.400282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8757":{"gid":8757,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/324530820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":324530820},{"type":"v1","addr":"172.21.15.32:6835","nonce":324530820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T23:05:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc0c053b30 msgr2=0x7fcc0c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc0c053b30 0x7fcc0c055ff0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fcc1000c000 tx=0x7fcc10008040).stop 2022-08-01T23:05:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 msgr2=0x7fcc28138f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 0x7fcc28138f40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fcc180032f0 tx=0x7fcc18016000).stop 2022-08-01T23:05:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 shutdown_connections 2022-08-01T23:05:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc0c053b30 0x7fcc0c055ff0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc280b3630 0x7fcc280b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc280af6f0 0x7fcc28146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 --2- 172.21.15.32:0/886832001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc280aa0d0 0x7fcc28138f40 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 >> 172.21.15.32:0/886832001 conn(0x7fcc28060950 msgr2=0x7fcc280b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 shutdown_connections 2022-08-01T23:05:27.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.715+0000 7fcc2d58a700 1 -- 172.21.15.32:0/886832001 wait complete. 2022-08-01T23:05:27.726 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2022-08-01T23:05:27.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 Processor -- start 2022-08-01T23:05:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 -- start start 2022-08-01T23:05:27.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 0x7f5110061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f5110140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51100aee80 con 0x7f51100614b0 2022-08-01T23:05:27.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51100b70b0 con 0x7f51100ad750 2022-08-01T23:05:27.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.831+0000 7f510ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51100af0b0 con 0x7f51101405d0 2022-08-01T23:05:27.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510e7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 0x7f5110061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f5110140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510e7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 0x7f5110061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:59798/0 (socket says 172.21.15.32:59798) 2022-08-01T23:05:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510e7fc700 1 -- 172.21.15.32:0/4280019428 learned_addr learned my addr 172.21.15.32:0/4280019428 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510effd700 1 --1- 172.21.15.32:0/4280019428 >> v1:172.21.15.55:6789/0 conn(0x7f51100614b0 0x7f51100b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.32:58460/0 (socket says 172.21.15.32:58460) 2022-08-01T23:05:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510f7fe700 1 -- 172.21.15.32:0/4280019428 >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 msgr2=0x7f5110061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510f7fe700 1 --2- 172.21.15.32:0/4280019428 >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 0x7f5110061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510f7fe700 1 -- 172.21.15.32:0/4280019428 >> v1:172.21.15.55:6789/0 conn(0x7f51100614b0 legacy=0x7f51100b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510f7fe700 1 -- 172.21.15.32:0/4280019428 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100af350 con 0x7f51101405d0 2022-08-01T23:05:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510e7fc700 1 --2- 172.21.15.32:0/4280019428 >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 0x7f5110061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510f7fe700 1 --2- 172.21.15.32:0/4280019428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f5110140a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f51080032f0 tx=0x7f51080091d0).ready entity=mon.2 client_cookie=7278c7dfe533246a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510dffb700 1 -- 172.21.15.32:0/4280019428 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510800b6c0 con 0x7f51101405d0 2022-08-01T23:05:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510dffb700 1 -- 172.21.15.32:0/4280019428 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510800b820 con 0x7f51101405d0 2022-08-01T23:05:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510dffb700 1 -- 172.21.15.32:0/4280019428 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510800b990 con 0x7f51101405d0 2022-08-01T23:05:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 -- 172.21.15.32:0/4280019428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 msgr2=0x7f5110140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 --2- 172.21.15.32:0/4280019428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f5110140a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f51080032f0 tx=0x7f51080091d0).stop 2022-08-01T23:05:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 -- 172.21.15.32:0/4280019428 shutdown_connections 2022-08-01T23:05:27.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 --2- 172.21.15.32:0/4280019428 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f5110140a80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 --2- 172.21.15.32:0/4280019428 >> v2:172.21.15.55:3300/0 conn(0x7f51100ad750 0x7f5110061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 -- 172.21.15.32:0/4280019428 >> 172.21.15.32:0/4280019428 conn(0x7f5110066f70 msgr2=0x7f5110067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:27.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 -- 172.21.15.32:0/4280019428 shutdown_connections 2022-08-01T23:05:27.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.835+0000 7f510ffff700 1 -- 172.21.15.32:0/4280019428 wait complete. 2022-08-01T23:05:27.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 Processor -- start 2022-08-01T23:05:27.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 -- start start 2022-08-01T23:05:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 0x7f5110155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51100ad750 0x7f51100b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f511013f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51101501a0 con 0x7f51100614b0 2022-08-01T23:05:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51100b4020 con 0x7f51100ad750 2022-08-01T23:05:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5110156440 con 0x7f51101405d0 2022-08-01T23:05:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 0x7f5110155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 0x7f5110155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50018/0 (socket says 172.21.15.32:50018) 2022-08-01T23:05:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 -- 172.21.15.32:0/1777424872 learned_addr learned my addr 172.21.15.32:0/1777424872 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510e7fc700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51100ad750 0x7f51100b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510f7fe700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f511013f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 -- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 msgr2=0x7f511013f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f511013f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 -- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51100ad750 msgr2=0x7f51100b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51100ad750 0x7f51100b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 -- 172.21.15.32:0/1777424872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510000b1e0 con 0x7f51100614b0 2022-08-01T23:05:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510f7fe700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f511013f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510effd700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 0x7f5110155d90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f5110146080 tx=0x7f5104007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510400ce20 con 0x7f51100614b0 2022-08-01T23:05:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51040072e0 con 0x7f51100614b0 2022-08-01T23:05:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5108009470 con 0x7f51100614b0 2022-08-01T23:05:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5104005740 con 0x7f51100614b0 2022-08-01T23:05:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.839+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51100b7f30 con 0x7f51100614b0 2022-08-01T23:05:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.843+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5110156280 con 0x7f51100614b0 2022-08-01T23:05:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.843+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5104010070 con 0x7f51100614b0 2022-08-01T23:05:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.843+0000 7f50fffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50ec053980 0x7f50ec055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.843+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f510406b720 con 0x7f51100614b0 2022-08-01T23:05:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.843+0000 7f510e7fc700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50ec053980 0x7f50ec055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.843+0000 7f510e7fc700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50ec053980 0x7f50ec055e40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f510000b000 tx=0x7f5100008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.851+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51040423d0 con 0x7f51100614b0 2022-08-01T23:05:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.975+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51101568e0 con 0x7f51100614b0 2022-08-01T23:05:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.975+0000 7f50fffff700 1 -- 172.21.15.32:0/1777424872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f51040459f0 con 0x7f51100614b0 2022-08-01T23:05:27.984 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:27.984 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:26.723454+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:24.145656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:24.725583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:27.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50ec053980 msgr2=0x7f50ec055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50ec053980 0x7f50ec055e40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f510000b000 tx=0x7f5100008040).stop 2022-08-01T23:05:27.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 msgr2=0x7f5110155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 0x7f5110155d90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f5110146080 tx=0x7f5104007c00).stop 2022-08-01T23:05:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 shutdown_connections 2022-08-01T23:05:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50ec053980 0x7f50ec055e40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51101405d0 0x7f511013f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51100ad750 0x7f51100b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 --2- 172.21.15.32:0/1777424872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51100614b0 0x7f5110155d90 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 >> 172.21.15.32:0/1777424872 conn(0x7f5110066f70 msgr2=0x7f5110142a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 shutdown_connections 2022-08-01T23:05:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:27.979+0000 7f510ffff700 1 -- 172.21.15.32:0/1777424872 wait complete. 2022-08-01T23:05:27.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:05:28.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bbc952700 1 Processor -- start 2022-08-01T23:05:28.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bbc952700 1 -- start start 2022-08-01T23:05:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bbc952700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7bb805ff60 0x7f7bb8145060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bbc952700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80b6e70 con 0x7f7bb80b6a90 2022-08-01T23:05:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bbc952700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80ac6e0 con 0x7f7bb80aa210 2022-08-01T23:05:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bbc952700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb80ac980 con 0x7f7bb805ff60 2022-08-01T23:05:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7bb80b6a90 0x7f7bb8144950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37568/0 (socket says 172.21.15.32:37568) 2022-08-01T23:05:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb659c700 1 -- 172.21.15.32:0/1360276905 learned_addr learned my addr 172.21.15.32:0/1360276905 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb6d9d700 1 --2- 172.21.15.32:0/1360276905 >> v2:172.21.15.55:3300/0 conn(0x7f7bb805ff60 0x7f7bb8145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb5d9b700 1 -- 172.21.15.32:0/1360276905 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253702315 0 0) 0x7f7bb80b6e70 con 0x7f7bb80b6a90 2022-08-01T23:05:28.110 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb5d9b700 1 -- 172.21.15.32:0/1360276905 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b98003260 con 0x7f7bb80b6a90 2022-08-01T23:05:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb6d9d700 1 -- 172.21.15.32:0/1360276905 >> v1:172.21.15.55:6789/0 conn(0x7f7bb80aa210 legacy=0x7f7bb814ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb6d9d700 1 -- 172.21.15.32:0/1360276905 >> v1:172.21.15.32:6789/0 conn(0x7f7bb80b6a90 legacy=0x7f7bb8144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb6d9d700 1 -- 172.21.15.32:0/1360276905 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80acc20 con 0x7f7bb805ff60 2022-08-01T23:05:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb6d9d700 1 --2- 172.21.15.32:0/1360276905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb805ff60 0x7f7bb8145060 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f7bb00032f0 tx=0x7f7bb00091d0).ready entity=mon.1 client_cookie=af4f80c15be0fd73 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb5d9b700 1 -- 172.21.15.32:0/1360276905 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bb000b6c0 con 0x7f7bb805ff60 2022-08-01T23:05:28.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb5d9b700 1 -- 172.21.15.32:0/1360276905 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb000b820 con 0x7f7bb805ff60 2022-08-01T23:05:28.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.099+0000 7f7bb5d9b700 1 -- 172.21.15.32:0/1360276905 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bb000b990 con 0x7f7bb805ff60 2022-08-01T23:05:28.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 -- 172.21.15.32:0/1360276905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb805ff60 msgr2=0x7f7bb8145060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.112 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 --2- 172.21.15.32:0/1360276905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb805ff60 0x7f7bb8145060 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f7bb00032f0 tx=0x7f7bb00091d0).stop 2022-08-01T23:05:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 -- 172.21.15.32:0/1360276905 shutdown_connections 2022-08-01T23:05:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 --2- 172.21.15.32:0/1360276905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb805ff60 0x7f7bb8145060 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 -- 172.21.15.32:0/1360276905 >> 172.21.15.32:0/1360276905 conn(0x7f7bb80b3610 msgr2=0x7f7bb80b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 -- 172.21.15.32:0/1360276905 shutdown_connections 2022-08-01T23:05:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 -- 172.21.15.32:0/1360276905 wait complete. 2022-08-01T23:05:28.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 Processor -- start 2022-08-01T23:05:28.116 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.103+0000 7f7bbc952700 1 -- start start 2022-08-01T23:05:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 0x7f7bb80b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7bb80aa210 0x7f7bb814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb80b6a90 0x7f7bb80af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8063540 con 0x7f7bb805ff60 2022-08-01T23:05:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb814b7b0 con 0x7f7bb80b6a90 2022-08-01T23:05:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb814ba50 con 0x7f7bb80aa210 2022-08-01T23:05:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 0x7f7bb80b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 0x7f7bb80b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50034/0 (socket says 172.21.15.32:50034) 2022-08-01T23:05:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 -- 172.21.15.32:0/3742865941 learned_addr learned my addr 172.21.15.32:0/3742865941 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7badd9b700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7bb80aa210 0x7f7bb814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb6d9d700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb80b6a90 0x7f7bb80af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 -- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7bb80aa210 msgr2=0x7f7bb814d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7bb80aa210 0x7f7bb814d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 -- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb80b6a90 msgr2=0x7f7bb80af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb80b6a90 0x7f7bb80af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 -- 172.21.15.32:0/3742865941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0009470 con 0x7f7bb805ff60 2022-08-01T23:05:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bb659c700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 0x7f7bb80b0100 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f7ba8001110 tx=0x7f7ba8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba80070d0 con 0x7f7bb805ff60 2022-08-01T23:05:28.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8007230 con 0x7f7bb805ff60 2022-08-01T23:05:28.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb80631c0 con 0x7f7bb805ff60 2022-08-01T23:05:28.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba8005610 con 0x7f7bb805ff60 2022-08-01T23:05:28.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.107+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb814a2a0 con 0x7f7bb805ff60 2022-08-01T23:05:28.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.111+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb814bc50 con 0x7f7bb805ff60 2022-08-01T23:05:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.111+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ba8010070 con 0x7f7bb805ff60 2022-08-01T23:05:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.111+0000 7f7baffff700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b9c053a70 0x7f7b9c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.111+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7ba806b660 con 0x7f7bb805ff60 2022-08-01T23:05:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.111+0000 7f7badd9b700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b9c053a70 0x7f7b9c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.111+0000 7f7badd9b700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b9c053a70 0x7f7b9c055f30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f7ba000d000 tx=0x7f7ba0007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.119+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ba8042380 con 0x7f7bb805ff60 2022-08-01T23:05:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.291+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7bb80acec0 con 0x7f7bb805ff60 2022-08-01T23:05:28.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7baffff700 1 -- 172.21.15.32:0/3742865941 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7ba8018030 con 0x7f7bb805ff60 2022-08-01T23:05:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b9c053a70 msgr2=0x7f7b9c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b9c053a70 0x7f7b9c055f30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f7ba000d000 tx=0x7f7ba0007b90).stop 2022-08-01T23:05:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 msgr2=0x7f7bb80b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 0x7f7bb80b0100 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f7ba8001110 tx=0x7f7ba8007aa0).stop 2022-08-01T23:05:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 shutdown_connections 2022-08-01T23:05:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f7b9c053a70 0x7f7b9c055f30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bb80b6a90 0x7f7bb80af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f7bb80aa210 0x7f7bb814d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 --2- 172.21.15.32:0/3742865941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb805ff60 0x7f7bb80b0100 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 >> 172.21.15.32:0/3742865941 conn(0x7f7bb80b3610 msgr2=0x7f7bb814e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 shutdown_connections 2022-08-01T23:05:28.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.299+0000 7f7bbc952700 1 -- 172.21.15.32:0/3742865941 wait complete. 2022-08-01T23:05:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2022-08-01T23:05:28.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T23:05:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.431+0000 7f14ca13d700 1 Processor -- start 2022-08-01T23:05:28.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.431+0000 7f14ca13d700 1 -- start start 2022-08-01T23:05:28.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14c40af550 0x7f14c40b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c41445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14c4144ad0 0x7f14c40a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c40b15e0 con 0x7f14c40af550 2022-08-01T23:05:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c40aa630 con 0x7f14c4144ad0 2022-08-01T23:05:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c405f840 con 0x7f14c4060330 2022-08-01T23:05:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14c40af550 0x7f14c40b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14c40af550 0x7f14c40b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50042/0 (socket says 172.21.15.32:50042) 2022-08-01T23:05:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 -- 172.21.15.32:0/240751615 learned_addr learned my addr 172.21.15.32:0/240751615 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c893a700 1 --2- 172.21.15.32:0/240751615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c41445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c993c700 1 --2- 172.21.15.32:0/240751615 >> v2:172.21.15.55:3300/0 conn(0x7f14c4144ad0 0x7f14c40a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 -- 172.21.15.32:0/240751615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 msgr2=0x7f14c41445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:28.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 --2- 172.21.15.32:0/240751615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c41445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 -- 172.21.15.32:0/240751615 >> v2:172.21.15.55:3300/0 conn(0x7f14c4144ad0 msgr2=0x7f14c40a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:28.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 --2- 172.21.15.32:0/240751615 >> v2:172.21.15.55:3300/0 conn(0x7f14c4144ad0 0x7f14c40a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 -- 172.21.15.32:0/240751615 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c405fd10 con 0x7f14c40af550 2022-08-01T23:05:28.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c993c700 1 --2- 172.21.15.32:0/240751615 >> v2:172.21.15.55:3300/0 conn(0x7f14c4144ad0 0x7f14c40a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:28.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14c913b700 1 --2- 172.21.15.32:0/240751615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c40af550 0x7f14c40b0e40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f14c00032f0 tx=0x7f14c0009290).ready entity=mon.0 client_cookie=efeeedaf9eb54b7a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14b3fff700 1 -- 172.21.15.32:0/240751615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c000b870 con 0x7f14c40af550 2022-08-01T23:05:28.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14b3fff700 1 -- 172.21.15.32:0/240751615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c000b9d0 con 0x7f14c40af550 2022-08-01T23:05:28.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14b3fff700 1 -- 172.21.15.32:0/240751615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14c000bb40 con 0x7f14c40af550 2022-08-01T23:05:28.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 -- 172.21.15.32:0/240751615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c40af550 msgr2=0x7f14c40b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- 172.21.15.32:0/240751615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c40af550 0x7f14c40b0e40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f14c00032f0 tx=0x7f14c0009290).stop 2022-08-01T23:05:28.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 -- 172.21.15.32:0/240751615 shutdown_connections 2022-08-01T23:05:28.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- 172.21.15.32:0/240751615 >> v2:172.21.15.55:3300/0 conn(0x7f14c4144ad0 0x7f14c40a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- 172.21.15.32:0/240751615 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c41445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 --2- 172.21.15.32:0/240751615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c40af550 0x7f14c40b0e40 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.435+0000 7f14ca13d700 1 -- 172.21.15.32:0/240751615 >> 172.21.15.32:0/240751615 conn(0x7f14c40612d0 msgr2=0x7f14c4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:28.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- 172.21.15.32:0/240751615 shutdown_connections 2022-08-01T23:05:28.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- 172.21.15.32:0/240751615 wait complete. 2022-08-01T23:05:28.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 Processor -- start 2022-08-01T23:05:28.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- start start 2022-08-01T23:05:28.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c4143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14c40af550 0x7f14c4141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c4144ad0 0x7f14c4141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40b5840 con 0x7f14c4144ad0 2022-08-01T23:05:28.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40a02c0 con 0x7f14c40af550 2022-08-01T23:05:28.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40a0560 con 0x7f14c4060330 2022-08-01T23:05:28.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c993c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c4144ad0 0x7f14c4141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c993c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c4144ad0 0x7f14c4141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50048/0 (socket says 172.21.15.32:50048) 2022-08-01T23:05:28.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c993c700 1 -- 172.21.15.32:0/1932937307 learned_addr learned my addr 172.21.15.32:0/1932937307 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:28.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14c40af550 0x7f14c4141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c913b700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c4143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 -- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 msgr2=0x7f14c4143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c4143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 -- 172.21.15.32:0/1932937307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c4144ad0 msgr2=0x7f14c4141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c4144ad0 0x7f14c4141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 -- 172.21.15.32:0/1932937307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0009530 con 0x7f14c40af550 2022-08-01T23:05:28.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c913b700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c4143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:28.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14c893a700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14c40af550 0x7f14c4141630 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f14b400c000 tx=0x7f14b4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b4016070 con 0x7f14c40af550 2022-08-01T23:05:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b4011750 con 0x7f14c40af550 2022-08-01T23:05:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b4011ab0 con 0x7f14c40af550 2022-08-01T23:05:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c40a0800 con 0x7f14c40af550 2022-08-01T23:05:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.439+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c41521f0 con 0x7f14c40af550 2022-08-01T23:05:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.443+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c40574e0 con 0x7f14c40af550 2022-08-01T23:05:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.443+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f14b4011c10 con 0x7f14c40af550 2022-08-01T23:05:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.443+0000 7f14b27fc700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14a4053950 0x7f14a4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.443+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f14b406c100 con 0x7f14c40af550 2022-08-01T23:05:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.443+0000 7f14c913b700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14a4053950 0x7f14a4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.443+0000 7f14c913b700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14a4053950 0x7f14a4055e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f14c001a000 tx=0x7f14c0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.451+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14b4042d30 con 0x7f14c40af550 2022-08-01T23:05:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.619+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f14c4069a70 con 0x7f14c40af550 2022-08-01T23:05:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.627+0000 7f14b27fc700 1 -- 172.21.15.32:0/1932937307 <== mon.1 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7f14b4046350 con 0x7f14c40af550 2022-08-01T23:05:28.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14a4053950 msgr2=0x7f14a4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.640 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14a4053950 0x7f14a4055e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f14c001a000 tx=0x7f14c0016000).stop 2022-08-01T23:05:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14c40af550 msgr2=0x7f14c4141630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14c40af550 0x7f14c4141630 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f14b400c000 tx=0x7f14b4006df0).stop 2022-08-01T23:05:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 shutdown_connections 2022-08-01T23:05:28.641 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f14a4053950 0x7f14a4055e10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c4144ad0 0x7f14c4141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14c40af550 0x7f14c4141630 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 --2- 172.21.15.32:0/1932937307 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14c4060330 0x7f14c4143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.642 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 >> 172.21.15.32:0/1932937307 conn(0x7f14c40612d0 msgr2=0x7f14c40abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:28.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 shutdown_connections 2022-08-01T23:05:28.643 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.631+0000 7f14ca13d700 1 -- 172.21.15.32:0/1932937307 wait complete. 2022-08-01T23:05:28.643 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.1 2022-08-01T23:05:28.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:28.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.751+0000 7f09dda9a700 1 Processor -- start 2022-08-01T23:05:28.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.751+0000 7f09dda9a700 1 -- start start 2022-08-01T23:05:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d8148530 0x7f09d80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09d80a7250 0x7f09d81448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80b04e0 con 0x7f09d80b1f40 2022-08-01T23:05:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80a9250 con 0x7f09d8148530 2022-08-01T23:05:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80b0780 con 0x7f09d80a7250 2022-08-01T23:05:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d8148530 0x7f09d80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d8148530 0x7f09d80b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50054/0 (socket says 172.21.15.32:50054) 2022-08-01T23:05:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 -- 172.21.15.32:0/3035437795 learned_addr learned my addr 172.21.15.32:0/3035437795 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d7fff700 1 --2- 172.21.15.32:0/3035437795 >> v2:172.21.15.55:3300/0 conn(0x7f09d80a7250 0x7f09d81448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 -- 172.21.15.32:0/3035437795 >> v2:172.21.15.55:3300/0 conn(0x7f09d80a7250 msgr2=0x7f09d81448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 --2- 172.21.15.32:0/3035437795 >> v2:172.21.15.55:3300/0 conn(0x7f09d80a7250 0x7f09d81448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 -- 172.21.15.32:0/3035437795 >> v1:172.21.15.32:6789/0 conn(0x7f09d80b1f40 legacy=0x7f09d80b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 -- 172.21.15.32:0/3035437795 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80b0a20 con 0x7f09d8148530 2022-08-01T23:05:28.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d7fff700 1 --2- 172.21.15.32:0/3035437795 >> v2:172.21.15.55:3300/0 conn(0x7f09d80a7250 0x7f09d81448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T23:05:28.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d6ffd700 1 --2- 172.21.15.32:0/3035437795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80b1160 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f09d81478c0 tx=0x7f09c8006eb0).ready entity=mon.0 client_cookie=8be328f9ada3dfa7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.766 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d67fc700 1 -- 172.21.15.32:0/3035437795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c800d610 con 0x7f09d8148530 2022-08-01T23:05:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d67fc700 1 -- 172.21.15.32:0/3035437795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c8000e90 con 0x7f09d8148530 2022-08-01T23:05:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09d67fc700 1 -- 172.21.15.32:0/3035437795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c800d960 con 0x7f09d8148530 2022-08-01T23:05:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- 172.21.15.32:0/3035437795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 msgr2=0x7f09d80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3035437795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80b1160 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f09d81478c0 tx=0x7f09c8006eb0).stop 2022-08-01T23:05:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- 172.21.15.32:0/3035437795 shutdown_connections 2022-08-01T23:05:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3035437795 >> v2:172.21.15.55:3300/0 conn(0x7f09d80a7250 0x7f09d81448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3035437795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80b1160 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- 172.21.15.32:0/3035437795 >> 172.21.15.32:0/3035437795 conn(0x7f09d805f840 msgr2=0x7f09d805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- 172.21.15.32:0/3035437795 shutdown_connections 2022-08-01T23:05:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- 172.21.15.32:0/3035437795 wait complete. 2022-08-01T23:05:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 Processor -- start 2022-08-01T23:05:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.755+0000 7f09dda9a700 1 -- start start 2022-08-01T23:05:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09d80a7250 0x7f09d814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09d80b1f40 0x7f09d80a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d814b300 con 0x7f09d8148530 2022-08-01T23:05:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d814a4d0 con 0x7f09d80a7250 2022-08-01T23:05:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d814b530 con 0x7f09d80b1f40 2022-08-01T23:05:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50062/0 (socket says 172.21.15.32:50062) 2022-08-01T23:05:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 -- 172.21.15.32:0/3875604466 learned_addr learned my addr 172.21.15.32:0/3875604466 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d6ffd700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09d80b1f40 0x7f09d80a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d77fe700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09d80a7250 0x7f09d814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 -- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09d80b1f40 msgr2=0x7f09d80a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09d80b1f40 0x7f09d80a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 -- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09d80a7250 msgr2=0x7f09d814d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09d80a7250 0x7f09d814d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 -- 172.21.15.32:0/3875604466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c800b1e0 con 0x7f09d8148530 2022-08-01T23:05:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d7fff700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80abbe0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f09d0007000 tx=0x7f09d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09d0015070 con 0x7f09d8148530 2022-08-01T23:05:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0010580 con 0x7f09d8148530 2022-08-01T23:05:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d814a730 con 0x7f09d8148530 2022-08-01T23:05:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09d00108e0 con 0x7f09d8148530 2022-08-01T23:05:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.759+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d80ab8d0 con 0x7f09d8148530 2022-08-01T23:05:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.763+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d80ac270 con 0x7f09d8148530 2022-08-01T23:05:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.763+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f09d0010a40 con 0x7f09d8148530 2022-08-01T23:05:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.763+0000 7f09d4ff9700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09b80539e0 0x7f09b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.763+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f09d006c2e0 con 0x7f09d8148530 2022-08-01T23:05:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.763+0000 7f09d77fe700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09b80539e0 0x7f09b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.763+0000 7f09d77fe700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09b80539e0 0x7f09b8055ea0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f09cc00d000 tx=0x7f09cc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.771+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09d0043000 con 0x7f09d8148530 2022-08-01T23:05:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.931+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09d80b0cc0 con 0x7f09d8148530 2022-08-01T23:05:28.940 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.931+0000 7f09d4ff9700 1 -- 172.21.15.32:0/3875604466 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f09d0046620 con 0x7f09d8148530 2022-08-01T23:05:28.940 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:28.941 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1983320963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1983320963},{"type":"v1","addr":"172.21.15.55:6835","nonce":1983320963}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78},{"gid":8754,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3801584266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3801584266},{"type":"v1","addr":"172.21.15.32:6833","nonce":3801584266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:25.720657+0000","modified":"2022-08-01T23:05:27.400282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8757":{"gid":8757,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/324530820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":324530820},{"type":"v1","addr":"172.21.15.32:6835","nonce":324530820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T23:05:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09b80539e0 msgr2=0x7f09b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09b80539e0 0x7f09b8055ea0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f09cc00d000 tx=0x7f09cc009040).stop 2022-08-01T23:05:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 msgr2=0x7f09d80abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80abbe0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f09d0007000 tx=0x7f09d0008040).stop 2022-08-01T23:05:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 shutdown_connections 2022-08-01T23:05:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f09b80539e0 0x7f09b8055ea0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d8148530 0x7f09d80abbe0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09d80b1f40 0x7f09d80a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 --2- 172.21.15.32:0/3875604466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09d80a7250 0x7f09d814d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:28.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 >> 172.21.15.32:0/3875604466 conn(0x7f09d805f840 msgr2=0x7f09d8144b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:28.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 shutdown_connections 2022-08-01T23:05:28.945 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:28.935+0000 7f09dda9a700 1 -- 172.21.15.32:0/3875604466 wait complete. 2022-08-01T23:05:28.946 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2022-08-01T23:05:29.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:05:25.720657+0000', 'modified': '2022-08-01T23:05:27.400282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8757}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8757': {'gid': 8757, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/324530820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 324530820}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 324530820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:05:29.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:05:29.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:05:29.376 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8757,"want_state":"up:active","state":"up:active","rank_uptime":3.6409863979999999,"mdsmap_epoch":81,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":8.0723884909999999} 2022-08-01T23:05:29.377 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8757, "mdsmap_epoch": 81, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 3.640986398, "state": "up:active", "uptime": 8.072388491, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:05:29.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 Processor -- start 2022-08-01T23:05:29.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 -- start start 2022-08-01T23:05:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c440614b0 0x7f6c44061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c440ad750 0x7f6c440b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c44140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440aee80 con 0x7f6c440ad750 2022-08-01T23:05:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440b70b0 con 0x7f6c440614b0 2022-08-01T23:05:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.467+0000 7f6c4af88700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440af0b0 con 0x7f6c441405d0 2022-08-01T23:05:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c440ad750 0x7f6c440b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c440ad750 0x7f6c440b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50068/0 (socket says 172.21.15.32:50068) 2022-08-01T23:05:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 -- 172.21.15.32:0/3498808961 learned_addr learned my addr 172.21.15.32:0/3498808961 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4a787700 1 --2- 172.21.15.32:0/3498808961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c44140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49f86700 1 --2- 172.21.15.32:0/3498808961 >> v2:172.21.15.55:3300/0 conn(0x7f6c440614b0 0x7f6c44061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 -- 172.21.15.32:0/3498808961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 msgr2=0x7f6c44140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:29.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 --2- 172.21.15.32:0/3498808961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c44140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 -- 172.21.15.32:0/3498808961 >> v2:172.21.15.55:3300/0 conn(0x7f6c440614b0 msgr2=0x7f6c44061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:29.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 --2- 172.21.15.32:0/3498808961 >> v2:172.21.15.55:3300/0 conn(0x7f6c440614b0 0x7f6c44061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 -- 172.21.15.32:0/3498808961 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440af350 con 0x7f6c440ad750 2022-08-01T23:05:29.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c49785700 1 --2- 172.21.15.32:0/3498808961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c440b9ff0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f6c440aed30 tx=0x7f6c40006eb0).ready entity=mon.0 client_cookie=4ec9eeac1b825922 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:29.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c48f84700 1 -- 172.21.15.32:0/3498808961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c4000d570 con 0x7f6c440ad750 2022-08-01T23:05:29.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c48f84700 1 -- 172.21.15.32:0/3498808961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40000e90 con 0x7f6c440ad750 2022-08-01T23:05:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c48f84700 1 -- 172.21.15.32:0/3498808961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c4000d900 con 0x7f6c440ad750 2022-08-01T23:05:29.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 -- 172.21.15.32:0/3498808961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 msgr2=0x7f6c440b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 --2- 172.21.15.32:0/3498808961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c440b9ff0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f6c440aed30 tx=0x7f6c40006eb0).stop 2022-08-01T23:05:29.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 -- 172.21.15.32:0/3498808961 shutdown_connections 2022-08-01T23:05:29.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 --2- 172.21.15.32:0/3498808961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c440b9ff0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 --2- 172.21.15.32:0/3498808961 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c44140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 --2- 172.21.15.32:0/3498808961 >> v2:172.21.15.55:3300/0 conn(0x7f6c440614b0 0x7f6c44061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 -- 172.21.15.32:0/3498808961 >> 172.21.15.32:0/3498808961 conn(0x7f6c44066f70 msgr2=0x7f6c44067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:29.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 -- 172.21.15.32:0/3498808961 shutdown_connections 2022-08-01T23:05:29.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.471+0000 7f6c4af88700 1 -- 172.21.15.32:0/3498808961 wait complete. 2022-08-01T23:05:29.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 Processor -- start 2022-08-01T23:05:29.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 -- start start 2022-08-01T23:05:29.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440614b0 0x7f6c440b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c44152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c440abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440b2f70 con 0x7f6c440ad750 2022-08-01T23:05:29.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440b01c0 con 0x7f6c440614b0 2022-08-01T23:05:29.490 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4af88700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440afac0 con 0x7f6c441405d0 2022-08-01T23:05:29.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c49785700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c44152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c49785700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c44152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50076/0 (socket says 172.21.15.32:50076) 2022-08-01T23:05:29.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c49785700 1 -- 172.21.15.32:0/2992311362 learned_addr learned my addr 172.21.15.32:0/2992311362 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:29.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c4a787700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c440abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.491 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.475+0000 7f6c49f86700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440614b0 0x7f6c440b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49785700 1 -- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 msgr2=0x7f6c440abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49785700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c440abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49785700 1 -- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440614b0 msgr2=0x7f6c440b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49785700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440614b0 0x7f6c440b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49785700 1 -- 172.21.15.32:0/2992311362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4000b1e0 con 0x7f6c440ad750 2022-08-01T23:05:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49f86700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440614b0 0x7f6c440b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c49785700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c44152e80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f6c440b0300 tx=0x7f6c40015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c40011860 con 0x7f6c440ad750 2022-08-01T23:05:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400119c0 con 0x7f6c440ad750 2022-08-01T23:05:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c440b0570 con 0x7f6c440ad750 2022-08-01T23:05:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c40011d20 con 0x7f6c440ad750 2022-08-01T23:05:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c440afd20 con 0x7f6c440ad750 2022-08-01T23:05:29.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c440af5f0 con 0x7f6c440ad750 2022-08-01T23:05:29.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.479+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c40025030 con 0x7f6c440ad750 2022-08-01T23:05:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.483+0000 7f6c32ffd700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c24053980 0x7f6c24055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.483+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c4006d160 con 0x7f6c440ad750 2022-08-01T23:05:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.483+0000 7f6c49f86700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c24053980 0x7f6c24055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.483+0000 7f6c49f86700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c24053980 0x7f6c24055e40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f6c3400c000 tx=0x7f6c34008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.487+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c40043f20 con 0x7f6c440ad750 2022-08-01T23:05:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.611+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c440574e0 con 0x7f6c440ad750 2022-08-01T23:05:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.615+0000 7f6c32ffd700 1 -- 172.21.15.32:0/2992311362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f6c40023020 con 0x7f6c440ad750 2022-08-01T23:05:29.622 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:29.622 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:26.723454+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:24.145656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:24.725583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:29.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c24053980 msgr2=0x7f6c24055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c24053980 0x7f6c24055e40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f6c3400c000 tx=0x7f6c34008040).stop 2022-08-01T23:05:29.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 msgr2=0x7f6c44152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c44152e80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f6c440b0300 tx=0x7f6c40015000).stop 2022-08-01T23:05:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 shutdown_connections 2022-08-01T23:05:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6c24053980 0x7f6c24055e40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6c441405d0 0x7f6c440abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c440ad750 0x7f6c44152e80 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 --2- 172.21.15.32:0/2992311362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440614b0 0x7f6c440b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 >> 172.21.15.32:0/2992311362 conn(0x7f6c44066f70 msgr2=0x7f6c44067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 shutdown_connections 2022-08-01T23:05:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.619+0000 7f6c4af88700 1 -- 172.21.15.32:0/2992311362 wait complete. 2022-08-01T23:05:29.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:05:24.145656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:05:24.725583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:05:29.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:05:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 Processor -- start 2022-08-01T23:05:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- start start 2022-08-01T23:05:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11140b2930 0x7f11140612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1114143400 0x7f1114143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11140671e0 con 0x7f11140b7840 2022-08-01T23:05:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1114065540 con 0x7f11140b2930 2022-08-01T23:05:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11140b7c20 con 0x7f1114143400 2022-08-01T23:05:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11140b2930 0x7f11140612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11140b2930 0x7f11140612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50084/0 (socket says 172.21.15.32:50084) 2022-08-01T23:05:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 -- 172.21.15.32:0/3101152625 learned_addr learned my addr 172.21.15.32:0/3101152625 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111a972700 1 --2- 172.21.15.32:0/3101152625 >> v2:172.21.15.55:3300/0 conn(0x7f1114143400 0x7f1114143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 -- 172.21.15.32:0/3101152625 >> v2:172.21.15.55:3300/0 conn(0x7f1114143400 msgr2=0x7f1114143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 --2- 172.21.15.32:0/3101152625 >> v2:172.21.15.55:3300/0 conn(0x7f1114143400 0x7f1114143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 -- 172.21.15.32:0/3101152625 >> v1:172.21.15.55:6789/0 conn(0x7f11140b7840 legacy=0x7f1114060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 -- 172.21.15.32:0/3101152625 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11140617e0 con 0x7f11140b2930 2022-08-01T23:05:29.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f1119970700 1 --2- 172.21.15.32:0/3101152625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11140b2930 0x7f11140612f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f1110002f70 tx=0x7f1110006eb0).ready entity=mon.0 client_cookie=d5cead3010610bad server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:29.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111916f700 1 -- 172.21.15.32:0/3101152625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111000d5c0 con 0x7f11140b2930 2022-08-01T23:05:29.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111916f700 1 -- 172.21.15.32:0/3101152625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110000e90 con 0x7f11140b2930 2022-08-01T23:05:29.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111916f700 1 -- 172.21.15.32:0/3101152625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f111000d950 con 0x7f11140b2930 2022-08-01T23:05:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- 172.21.15.32:0/3101152625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11140b2930 msgr2=0x7f11140612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 --2- 172.21.15.32:0/3101152625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11140b2930 0x7f11140612f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f1110002f70 tx=0x7f1110006eb0).stop 2022-08-01T23:05:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- 172.21.15.32:0/3101152625 shutdown_connections 2022-08-01T23:05:29.754 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 --2- 172.21.15.32:0/3101152625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11140b2930 0x7f11140612f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 --2- 172.21.15.32:0/3101152625 >> v2:172.21.15.55:3300/0 conn(0x7f1114143400 0x7f1114143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- 172.21.15.32:0/3101152625 >> 172.21.15.32:0/3101152625 conn(0x7f111405f840 msgr2=0x7f111405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:29.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- 172.21.15.32:0/3101152625 shutdown_connections 2022-08-01T23:05:29.755 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.739+0000 7f111b173700 1 -- 172.21.15.32:0/3101152625 wait complete. 2022-08-01T23:05:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 Processor -- start 2022-08-01T23:05:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 -- start start 2022-08-01T23:05:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11140b2930 0x7f1114144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.756 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11140b7840 0x7f11140ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 0x7f11141559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140acd80 con 0x7f1114143400 2022-08-01T23:05:29.757 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140ad020 con 0x7f11140b2930 2022-08-01T23:05:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11140adb30 con 0x7f11140b7840 2022-08-01T23:05:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 0x7f11141559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 0x7f11141559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50092/0 (socket says 172.21.15.32:50092) 2022-08-01T23:05:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 -- 172.21.15.32:0/2361933095 learned_addr learned my addr 172.21.15.32:0/2361933095 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:29.758 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a171700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11140b2930 0x7f1114144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f1119970700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11140b7840 0x7f11140ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 -- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11140b7840 msgr2=0x7f11140ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11140b7840 0x7f11140ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.759 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 -- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11140b2930 msgr2=0x7f1114144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11140b2930 0x7f1114144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 -- 172.21.15.32:0/2361933095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111000b1e0 con 0x7f1114143400 2022-08-01T23:05:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111a972700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 0x7f11141559e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f110c006e00 tx=0x7f110c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f110c00c640 con 0x7f1114143400 2022-08-01T23:05:29.760 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c00c7a0 con 0x7f1114143400 2022-08-01T23:05:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11140addd0 con 0x7f1114143400 2022-08-01T23:05:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f110c00cb00 con 0x7f1114143400 2022-08-01T23:05:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.743+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11140b09a0 con 0x7f1114143400 2022-08-01T23:05:29.761 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.747+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1114156070 con 0x7f1114143400 2022-08-01T23:05:29.762 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.747+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f110c00cc60 con 0x7f1114143400 2022-08-01T23:05:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.747+0000 7f11037fe700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10f4053a70 0x7f10f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.747+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f110c06c360 con 0x7f1114143400 2022-08-01T23:05:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.747+0000 7f111a171700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10f4053a70 0x7f10f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:29.763 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.747+0000 7f111a171700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10f4053a70 0x7f10f4055f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f110400d000 tx=0x7f1104007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:29.764 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.755+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f110c043010 con 0x7f1114143400 2022-08-01T23:05:29.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.875+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f11140adf60 con 0x7f1114143400 2022-08-01T23:05:29.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.875+0000 7f11037fe700 1 -- 172.21.15.32:0/2361933095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v158) v1 ==== 110+0+10 (secure 0 0 0) 0x7f110c046630 con 0x7f1114143400 2022-08-01T23:05:29.883 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:05:29.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10f4053a70 msgr2=0x7f10f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10f4053a70 0x7f10f4055f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f110400d000 tx=0x7f1104007b80).stop 2022-08-01T23:05:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 msgr2=0x7f11141559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 0x7f11141559e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f110c006e00 tx=0x7f110c0091d0).stop 2022-08-01T23:05:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 shutdown_connections 2022-08-01T23:05:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f10f4053a70 0x7f10f4055f30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1114143400 0x7f11141559e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11140b7840 0x7f11140ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 --2- 172.21.15.32:0/2361933095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11140b2930 0x7f1114144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 >> 172.21.15.32:0/2361933095 conn(0x7f111405f840 msgr2=0x7f111405c040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:29.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 shutdown_connections 2022-08-01T23:05:29.888 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.879+0000 7f111b173700 1 -- 172.21.15.32:0/2361933095 wait complete. 2022-08-01T23:05:29.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:05:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.999+0000 7fcadbfff700 1 Processor -- start 2022-08-01T23:05:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:29.999+0000 7fcadbfff700 1 -- start start 2022-08-01T23:05:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcadc066780 0x7fcadc066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc05f8e0 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc0b1630 con 0x7fcadc140e40 2022-08-01T23:05:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc05fb10 con 0x7fcadc066780 2022-08-01T23:05:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcad3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcadc0b1bc0 0x7fcadc05ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37640/0 (socket says 172.21.15.32:37640) 2022-08-01T23:05:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcad3fff700 1 -- 172.21.15.32:0/2029901588 learned_addr learned my addr 172.21.15.32:0/2029901588 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadaffd700 1 --2- 172.21.15.32:0/2029901588 >> v2:172.21.15.55:3300/0 conn(0x7fcadc066780 0x7fcadc066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcada7fc700 1 -- 172.21.15.32:0/2029901588 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298984458 0 0) 0x7fcadc05f8e0 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcada7fc700 1 -- 172.21.15.32:0/2029901588 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcabc003260 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadaffd700 1 -- 172.21.15.32:0/2029901588 >> v1:172.21.15.55:6789/0 conn(0x7fcadc140e40 legacy=0x7fcadc13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadaffd700 1 -- 172.21.15.32:0/2029901588 >> v1:172.21.15.32:6789/0 conn(0x7fcadc0b1bc0 legacy=0x7fcadc05ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadaffd700 1 -- 172.21.15.32:0/2029901588 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0b18d0 con 0x7fcadc066780 2022-08-01T23:05:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadaffd700 1 --2- 172.21.15.32:0/2029901588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc066780 0x7fcadc066b60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fcacc002f70 tx=0x7fcacc006df0).ready entity=mon.1 client_cookie=cfa3d6e95c590179 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcada7fc700 1 -- 172.21.15.32:0/2029901588 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcacc00d610 con 0x7fcadc066780 2022-08-01T23:05:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcada7fc700 1 -- 172.21.15.32:0/2029901588 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc000e90 con 0x7fcadc066780 2022-08-01T23:05:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcada7fc700 1 -- 172.21.15.32:0/2029901588 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcacc00d960 con 0x7fcadc066780 2022-08-01T23:05:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 -- 172.21.15.32:0/2029901588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc066780 msgr2=0x7fcadc066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 --2- 172.21.15.32:0/2029901588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc066780 0x7fcadc066b60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fcacc002f70 tx=0x7fcacc006df0).stop 2022-08-01T23:05:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 -- 172.21.15.32:0/2029901588 shutdown_connections 2022-08-01T23:05:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 --2- 172.21.15.32:0/2029901588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc066780 0x7fcadc066b60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.003+0000 7fcadbfff700 1 -- 172.21.15.32:0/2029901588 >> 172.21.15.32:0/2029901588 conn(0x7fcadc063870 msgr2=0x7fcadc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 -- 172.21.15.32:0/2029901588 shutdown_connections 2022-08-01T23:05:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 -- 172.21.15.32:0/2029901588 wait complete. 2022-08-01T23:05:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 Processor -- start 2022-08-01T23:05:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 -- start start 2022-08-01T23:05:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcadc066780 0x7fcadc09ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 0x7fcadc0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc140e40 0x7fcadc0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0adeb0 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0ad220 con 0x7fcadc140e40 2022-08-01T23:05:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadbfff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0ae080 con 0x7fcadc066780 2022-08-01T23:05:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcad3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 0x7fcadc0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcad3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 0x7fcadc0ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50104/0 (socket says 172.21.15.32:50104) 2022-08-01T23:05:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcad3fff700 1 -- 172.21.15.32:0/570858601 learned_addr learned my addr 172.21.15.32:0/570858601 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadaffd700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcadc066780 0x7fcadc09ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.007+0000 7fcadb7fe700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc140e40 0x7fcadc0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad3fff700 1 -- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcadc066780 msgr2=0x7fcadc09ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad3fff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcadc066780 0x7fcadc09ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad3fff700 1 -- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc140e40 msgr2=0x7fcadc0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad3fff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc140e40 0x7fcadc0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad3fff700 1 -- 172.21.15.32:0/570858601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc00b180 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad3fff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 0x7fcadc0ac840 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fcac4001110 tx=0x7fcac4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcac40070d0 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac4007230 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc0ae320 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcac4005610 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc0ae860 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcac4010070 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc0b12b0 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad8ff9700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcac0053950 0x7fcac0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcac406b720 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcadaffd700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcac0053950 0x7fcac0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.011+0000 7fcadaffd700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcac0053950 0x7fcac0055e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fcacc01a000 tx=0x7fcacc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.015+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcac4042400 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcadc0574e0 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcad8ff9700 1 -- 172.21.15.32:0/570858601 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcac4045a20 con 0x7fcadc0b1bc0 2022-08-01T23:05:30.151 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:30.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcac0053950 msgr2=0x7fcac0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcac0053950 0x7fcac0055e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fcacc01a000 tx=0x7fcacc014000).stop 2022-08-01T23:05:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 msgr2=0x7fcadc0ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 0x7fcadc0ac840 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fcac4001110 tx=0x7fcac4007ad0).stop 2022-08-01T23:05:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 shutdown_connections 2022-08-01T23:05:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcac0053950 0x7fcac0055e10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcadc140e40 0x7fcadc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0b1bc0 0x7fcadc0ac840 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 --2- 172.21.15.32:0/570858601 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcadc066780 0x7fcadc09ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.143+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 >> 172.21.15.32:0/570858601 conn(0x7fcadc063870 msgr2=0x7fcadc13db10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.147+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 shutdown_connections 2022-08-01T23:05:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.147+0000 7fcadbfff700 1 -- 172.21.15.32:0/570858601 wait complete. 2022-08-01T23:05:30.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:05:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.263+0000 7f6b3c13a700 1 Processor -- start 2022-08-01T23:05:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.263+0000 7f6b3c13a700 1 -- start start 2022-08-01T23:05:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b340b1460 0x7f6b340b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340a7960 con 0x7f6b340b2050 2022-08-01T23:05:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340b07e0 con 0x7f6b340b2430 2022-08-01T23:05:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b340b18d0 con 0x7f6b340b1460 2022-08-01T23:05:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b396d5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b340b1460 0x7f6b340b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b39ed6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b340b2050 0x7f6b340a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37652/0 (socket says 172.21.15.32:37652) 2022-08-01T23:05:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b39ed6700 1 -- 172.21.15.32:0/769556301 learned_addr learned my addr 172.21.15.32:0/769556301 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:30.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974985444 0 0) 0x7f6b340a7960 con 0x7f6b340b2050 2022-08-01T23:05:30.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b1c003260 con 0x7f6b340b2050 2022-08-01T23:05:30.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637167927 0 0) 0x7f6b340b07e0 con 0x7f6b340b2430 2022-08-01T23:05:30.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b340a7960 con 0x7f6b340b2430 2022-08-01T23:05:30.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b396d5700 1 -- 172.21.15.32:0/769556301 >> v1:172.21.15.55:6789/0 conn(0x7f6b340b2430 legacy=0x7f6b341413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b396d5700 1 -- 172.21.15.32:0/769556301 >> v1:172.21.15.32:6789/0 conn(0x7f6b340b2050 legacy=0x7f6b340a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b396d5700 1 -- 172.21.15.32:0/769556301 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340b0a80 con 0x7f6b340b1460 2022-08-01T23:05:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b396d5700 1 --2- 172.21.15.32:0/769556301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b340b0140 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f6b24002f70 tx=0x7f6b24006df0).ready entity=mon.0 client_cookie=ef3b418ff9b16b27 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2400d540 con 0x7f6b340b1460 2022-08-01T23:05:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24000e90 con 0x7f6b340b1460 2022-08-01T23:05:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b38ed4700 1 -- 172.21.15.32:0/769556301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2400d890 con 0x7f6b340b1460 2022-08-01T23:05:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- 172.21.15.32:0/769556301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 msgr2=0x7f6b340b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/769556301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b340b0140 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f6b24002f70 tx=0x7f6b24006df0).stop 2022-08-01T23:05:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- 172.21.15.32:0/769556301 shutdown_connections 2022-08-01T23:05:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/769556301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b340b0140 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- 172.21.15.32:0/769556301 >> 172.21.15.32:0/769556301 conn(0x7f6b3405f840 msgr2=0x7f6b3405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- 172.21.15.32:0/769556301 shutdown_connections 2022-08-01T23:05:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.267+0000 7f6b3c13a700 1 -- 172.21.15.32:0/769556301 wait complete. 2022-08-01T23:05:30.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 Processor -- start 2022-08-01T23:05:30.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 -- start start 2022-08-01T23:05:30.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b34147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b340b2050 0x7f6b34147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b340b2430 0x7f6b34148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3415a610 con 0x7f6b340b1460 2022-08-01T23:05:30.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3415a8b0 con 0x7f6b340b2050 2022-08-01T23:05:30.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3c13a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3415ab50 con 0x7f6b340b2430 2022-08-01T23:05:30.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b39ed6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b34147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b39ed6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b34147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50116/0 (socket says 172.21.15.32:50116) 2022-08-01T23:05:30.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b396d5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b340b2050 0x7f6b34147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b39ed6700 1 -- 172.21.15.32:0/1020238816 learned_addr learned my addr 172.21.15.32:0/1020238816 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:30.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.271+0000 7f6b3a6d7700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b340b2430 0x7f6b34148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b39ed6700 1 -- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b340b2430 msgr2=0x7f6b34148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b39ed6700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b340b2430 0x7f6b34148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b39ed6700 1 -- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b340b2050 msgr2=0x7f6b34147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b39ed6700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b340b2050 0x7f6b34147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b39ed6700 1 -- 172.21.15.32:0/1020238816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2400b180 con 0x7f6b340b1460 2022-08-01T23:05:30.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b3a6d7700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b340b2430 0x7f6b34148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b39ed6700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b34147950 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f6b300011d0 tx=0x7f6b30007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b300070d0 con 0x7f6b340b1460 2022-08-01T23:05:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30007230 con 0x7f6b340b1460 2022-08-01T23:05:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3415a290 con 0x7f6b340b1460 2022-08-01T23:05:30.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b30005610 con 0x7f6b340b1460 2022-08-01T23:05:30.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b34156ec0 con 0x7f6b340b1460 2022-08-01T23:05:30.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3415b1b0 con 0x7f6b340b1460 2022-08-01T23:05:30.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b30010070 con 0x7f6b340b1460 2022-08-01T23:05:30.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b22ffd700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b14053a70 0x7f6b14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6b3006b790 con 0x7f6b340b1460 2022-08-01T23:05:30.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.275+0000 7f6b396d5700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b14053a70 0x7f6b14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.279+0000 7f6b396d5700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b14053a70 0x7f6b14055f30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f6b2401a000 tx=0x7f6b24014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.292 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.283+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b30042550 con 0x7f6b340b1460 2022-08-01T23:05:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b340574e0 con 0x7f6b340b1460 2022-08-01T23:05:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b22ffd700 1 -- 172.21.15.32:0/1020238816 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6b30045b70 con 0x7f6b340b1460 2022-08-01T23:05:30.418 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:30.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b14053a70 msgr2=0x7f6b14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b14053a70 0x7f6b14055f30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f6b2401a000 tx=0x7f6b24014000).stop 2022-08-01T23:05:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 msgr2=0x7f6b34147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b34147950 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f6b300011d0 tx=0x7f6b30007b50).stop 2022-08-01T23:05:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 shutdown_connections 2022-08-01T23:05:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6b14053a70 0x7f6b14055f30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f6b340b2430 0x7f6b34148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b340b2050 0x7f6b34147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 --2- 172.21.15.32:0/1020238816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b340b1460 0x7f6b34147950 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 >> 172.21.15.32:0/1020238816 conn(0x7f6b3405f840 msgr2=0x7f6b3413da90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 shutdown_connections 2022-08-01T23:05:30.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.411+0000 7f6b3c13a700 1 -- 172.21.15.32:0/1020238816 wait complete. 2022-08-01T23:05:30.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:05:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 Processor -- start 2022-08-01T23:05:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 -- start start 2022-08-01T23:05:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11d40614b0 0x7f11d4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11d40ad750 0x7f11d40b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40aee80 con 0x7f11d41405d0 2022-08-01T23:05:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d40b70b0 con 0x7f11d40ad750 2022-08-01T23:05:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d90ec700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d40af0b0 con 0x7f11d40614b0 2022-08-01T23:05:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11d40ad750 0x7f11d40b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d88eb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f11d41405d0 0x7f11d41466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37670/0 (socket says 172.21.15.32:37670) 2022-08-01T23:05:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.531+0000 7f11d88eb700 1 -- 172.21.15.32:0/3804931878 learned_addr learned my addr 172.21.15.32:0/3804931878 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d3fff700 1 --2- 172.21.15.32:0/3804931878 >> v2:172.21.15.55:3300/0 conn(0x7f11d40614b0 0x7f11d4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d37fe700 1 -- 172.21.15.32:0/3804931878 >> v2:172.21.15.55:3300/0 conn(0x7f11d40614b0 msgr2=0x7f11d4061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d37fe700 1 --2- 172.21.15.32:0/3804931878 >> v2:172.21.15.55:3300/0 conn(0x7f11d40614b0 0x7f11d4061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d37fe700 1 -- 172.21.15.32:0/3804931878 >> v1:172.21.15.32:6789/0 conn(0x7f11d41405d0 legacy=0x7f11d41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d2ffd700 1 -- 172.21.15.32:0/3804931878 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811536497 0 0) 0x7f11d40aee80 con 0x7f11d41405d0 2022-08-01T23:05:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d37fe700 1 -- 172.21.15.32:0/3804931878 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40af350 con 0x7f11d40ad750 2022-08-01T23:05:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d37fe700 1 --2- 172.21.15.32:0/3804931878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b9ff0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f11d40aed30 tx=0x7f11c4006df0).ready entity=mon.0 client_cookie=4cdf0627d89f9503 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d2ffd700 1 -- 172.21.15.32:0/3804931878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11c400d5c0 con 0x7f11d40ad750 2022-08-01T23:05:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d2ffd700 1 -- 172.21.15.32:0/3804931878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4000e90 con 0x7f11d40ad750 2022-08-01T23:05:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d2ffd700 1 -- 172.21.15.32:0/3804931878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11c400d910 con 0x7f11d40ad750 2022-08-01T23:05:30.548 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 -- 172.21.15.32:0/3804931878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 msgr2=0x7f11d40b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 --2- 172.21.15.32:0/3804931878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b9ff0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f11d40aed30 tx=0x7f11c4006df0).stop 2022-08-01T23:05:30.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 -- 172.21.15.32:0/3804931878 shutdown_connections 2022-08-01T23:05:30.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 --2- 172.21.15.32:0/3804931878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b9ff0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 --2- 172.21.15.32:0/3804931878 >> v2:172.21.15.55:3300/0 conn(0x7f11d40614b0 0x7f11d4061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 -- 172.21.15.32:0/3804931878 >> 172.21.15.32:0/3804931878 conn(0x7f11d4066f70 msgr2=0x7f11d4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 -- 172.21.15.32:0/3804931878 shutdown_connections 2022-08-01T23:05:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 -- 172.21.15.32:0/3804931878 wait complete. 2022-08-01T23:05:30.550 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 Processor -- start 2022-08-01T23:05:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.535+0000 7f11d90ec700 1 -- start start 2022-08-01T23:05:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11d40614b0 0x7f11d4143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d41405d0 0x7f11d40b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40b1950 con 0x7f11d40ad750 2022-08-01T23:05:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40b0b10 con 0x7f11d41405d0 2022-08-01T23:05:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40b02d0 con 0x7f11d40614b0 2022-08-01T23:05:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50132/0 (socket says 172.21.15.32:50132) 2022-08-01T23:05:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 -- 172.21.15.32:0/680979109 learned_addr learned my addr 172.21.15.32:0/680979109 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d3fff700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11d40614b0 0x7f11d4143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d88eb700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d41405d0 0x7f11d40b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 -- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11d40614b0 msgr2=0x7f11d4143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11d40614b0 0x7f11d4143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 -- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d41405d0 msgr2=0x7f11d40b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d41405d0 0x7f11d40b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 -- 172.21.15.32:0/680979109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c400b180 con 0x7f11d40ad750 2022-08-01T23:05:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d37fe700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b0f70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f11d40aed30 tx=0x7f11c4012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11c4002f70 con 0x7f11d40ad750 2022-08-01T23:05:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400de70 con 0x7f11d40ad750 2022-08-01T23:05:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11c4013e90 con 0x7f11d40ad750 2022-08-01T23:05:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d40b0570 con 0x7f11d40ad750 2022-08-01T23:05:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d4142ad0 con 0x7f11d40ad750 2022-08-01T23:05:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f11c401c070 con 0x7f11d40ad750 2022-08-01T23:05:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.539+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d40b0730 con 0x7f11d40ad750 2022-08-01T23:05:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.543+0000 7f11d17fa700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f11b4053950 0x7f11b4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.543+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f11c406ccd0 con 0x7f11d40ad750 2022-08-01T23:05:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.543+0000 7f11d3fff700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f11b4053950 0x7f11b4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.543+0000 7f11d3fff700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f11b4053950 0x7f11b4055e10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f11c800c000 tx=0x7f11c8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.547+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11d40b0730 con 0x7f11d40ad750 2022-08-01T23:05:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.671+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11d40b0730 con 0x7f11d40ad750 2022-08-01T23:05:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d17fa700 1 -- 172.21.15.32:0/680979109 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+10 (secure 0 0 0) 0x7f11d40b0730 con 0x7f11d40ad750 2022-08-01T23:05:30.681 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f11b4053950 msgr2=0x7f11b4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f11b4053950 0x7f11b4055e10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f11c800c000 tx=0x7f11c8008040).stop 2022-08-01T23:05:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 msgr2=0x7f11d40b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b0f70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f11d40aed30 tx=0x7f11c4012000).stop 2022-08-01T23:05:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 shutdown_connections 2022-08-01T23:05:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f11b4053950 0x7f11b4055e10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d41405d0 0x7f11d40b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d40ad750 0x7f11d40b0f70 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 --2- 172.21.15.32:0/680979109 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f11d40614b0 0x7f11d4143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 >> 172.21.15.32:0/680979109 conn(0x7f11d4066f70 msgr2=0x7f11d413fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 shutdown_connections 2022-08-01T23:05:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.675+0000 7f11d90ec700 1 -- 172.21.15.32:0/680979109 wait complete. 2022-08-01T23:05:30.694 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:05:30.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:30.801 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 Processor -- start 2022-08-01T23:05:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 -- start start 2022-08-01T23:05:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c0b7500 0x7f952c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f952c0b31f0 0x7f952c13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0b2510 con 0x7f952c13e8a0 2022-08-01T23:05:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c1443e0 con 0x7f952c0b31f0 2022-08-01T23:05:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9532f8e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b3700 con 0x7f952c0b7500 2022-08-01T23:05:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953278d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f952c13e8a0 0x7f952c064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37684/0 (socket says 172.21.15.32:37684) 2022-08-01T23:05:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953278d700 1 -- 172.21.15.32:0/1276193694 learned_addr learned my addr 172.21.15.32:0/1276193694 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953178b700 1 --2- 172.21.15.32:0/1276193694 >> v2:172.21.15.55:3300/0 conn(0x7f952c0b31f0 0x7f952c13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9531f8c700 1 --2- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c0b7500 0x7f952c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9530f8a700 1 -- 172.21.15.32:0/1276193694 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035751278 0 0) 0x7f952c0b2510 con 0x7f952c13e8a0 2022-08-01T23:05:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9530f8a700 1 -- 172.21.15.32:0/1276193694 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9510003260 con 0x7f952c13e8a0 2022-08-01T23:05:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953178b700 1 -- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c0b7500 msgr2=0x7f952c0b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953178b700 1 --2- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c0b7500 0x7f952c0b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953178b700 1 -- 172.21.15.32:0/1276193694 >> v1:172.21.15.32:6789/0 conn(0x7f952c13e8a0 legacy=0x7f952c064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953178b700 1 -- 172.21.15.32:0/1276193694 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0b2740 con 0x7f952c0b31f0 2022-08-01T23:05:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f953178b700 1 --2- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b31f0 0x7f952c13e3b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f95240032f0 tx=0x7f95240091d0).ready entity=mon.1 client_cookie=cf208d1a07a07c77 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9530f8a700 1 -- 172.21.15.32:0/1276193694 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f952400b600 con 0x7f952c0b31f0 2022-08-01T23:05:30.808 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9530f8a700 1 -- 172.21.15.32:0/1276193694 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952400b760 con 0x7f952c0b31f0 2022-08-01T23:05:30.809 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.795+0000 7f9530f8a700 1 -- 172.21.15.32:0/1276193694 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f952400b8d0 con 0x7f952c0b31f0 2022-08-01T23:05:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b31f0 msgr2=0x7f952c13e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b31f0 0x7f952c13e3b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f95240032f0 tx=0x7f95240091d0).stop 2022-08-01T23:05:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- 172.21.15.32:0/1276193694 shutdown_connections 2022-08-01T23:05:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b31f0 0x7f952c13e3b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1276193694 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c0b7500 0x7f952c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- 172.21.15.32:0/1276193694 >> 172.21.15.32:0/1276193694 conn(0x7f952c05f840 msgr2=0x7f952c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- 172.21.15.32:0/1276193694 shutdown_connections 2022-08-01T23:05:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- 172.21.15.32:0/1276193694 wait complete. 2022-08-01T23:05:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 Processor -- start 2022-08-01T23:05:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- start start 2022-08-01T23:05:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 0x7f952c0abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b7500 0x7f952c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c13e8a0 0x7f952c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1541e0 con 0x7f952c0b31f0 2022-08-01T23:05:30.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0af6c0 con 0x7f952c0b7500 2022-08-01T23:05:30.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.799+0000 7f9532f8e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0ac9a0 con 0x7f952c13e8a0 2022-08-01T23:05:30.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 0x7f952c0abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.817 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 0x7f952c0abb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50144/0 (socket says 172.21.15.32:50144) 2022-08-01T23:05:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 -- 172.21.15.32:0/1202833582 learned_addr learned my addr 172.21.15.32:0/1202833582 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f953278d700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c13e8a0 0x7f952c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f953178b700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b7500 0x7f952c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 -- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c13e8a0 msgr2=0x7f952c153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c13e8a0 0x7f952c153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 -- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b7500 msgr2=0x7f952c153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b7500 0x7f952c153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 -- 172.21.15.32:0/1202833582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524009470 con 0x7f952c0b31f0 2022-08-01T23:05:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9531f8c700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 0x7f952c0abb10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f952000c000 tx=0x7f9520006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95200104c0 con 0x7f952c0b31f0 2022-08-01T23:05:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c0af8f0 con 0x7f952c0b31f0 2022-08-01T23:05:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9520010620 con 0x7f952c0b31f0 2022-08-01T23:05:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9520010980 con 0x7f952c0b31f0 2022-08-01T23:05:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c0ac080 con 0x7f952c0b31f0 2022-08-01T23:05:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c0574e0 con 0x7f952c0b31f0 2022-08-01T23:05:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f952001a030 con 0x7f952c0b31f0 2022-08-01T23:05:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f951effd700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9514053aa0 0x7f9514055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f952006bc30 con 0x7f952c0b31f0 2022-08-01T23:05:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.803+0000 7f953178b700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9514053aa0 0x7f9514055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.807+0000 7f953178b700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9514053aa0 0x7f9514055f60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f9524019000 tx=0x7f9524016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.815+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f95200429a0 con 0x7f952c0b31f0 2022-08-01T23:05:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f952c060330 con 0x7f952c0b31f0 2022-08-01T23:05:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f951effd700 1 -- 172.21.15.32:0/1202833582 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f9520045fc0 con 0x7f952c0b31f0 2022-08-01T23:05:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:30.983 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1983320963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1983320963},{"type":"v1","addr":"172.21.15.55:6835","nonce":1983320963}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78},{"gid":8754,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3801584266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3801584266},{"type":"v1","addr":"172.21.15.32:6833","nonce":3801584266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:25.720657+0000","modified":"2022-08-01T23:05:27.400282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8757":{"gid":8757,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/324530820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":324530820},{"type":"v1","addr":"172.21.15.32:6835","nonce":324530820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T23:05:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9514053aa0 msgr2=0x7f9514055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9514053aa0 0x7f9514055f60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f9524019000 tx=0x7f9524016000).stop 2022-08-01T23:05:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 msgr2=0x7f952c0abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 0x7f952c0abb10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f952000c000 tx=0x7f9520006d90).stop 2022-08-01T23:05:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 shutdown_connections 2022-08-01T23:05:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9514053aa0 0x7f9514055f60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f952c13e8a0 0x7f952c153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f952c0b7500 0x7f952c153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 --2- 172.21.15.32:0/1202833582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b31f0 0x7f952c0abb10 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.975+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 >> 172.21.15.32:0/1202833582 conn(0x7f952c05f840 msgr2=0x7f952c05c700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.979+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 shutdown_connections 2022-08-01T23:05:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:30.979+0000 7f9532f8e700 1 -- 172.21.15.32:0/1202833582 wait complete. 2022-08-01T23:05:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2022-08-01T23:05:31.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:05:25.720657+0000', 'modified': '2022-08-01T23:05:27.400282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8757}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8757': {'gid': 8757, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/324530820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 324530820}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 324530820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:05:31.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:05:31.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:05:31.415 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8757,"want_state":"up:active","state":"up:active","rank_uptime":5.6802902209999999,"mdsmap_epoch":81,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":10.111692226000001} 2022-08-01T23:05:31.416 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8757, "mdsmap_epoch": 81, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 5.680290221, "state": "up:active", "uptime": 10.111692226, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:05:31.417 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:05:31.417 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T23:05:31.418 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi130.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T23:05:31.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:31.419 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:31.427 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:05:31.485 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:31.485 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:31.502 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:31.503 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:31.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:31.504 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:05:31.509 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:05:31.509 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:31.554 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:05:31.570 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:31.570 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:31.579 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:31.580 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:31.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:31.582 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:05:31.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:2022-08-01T23:05:31.625+0000 7f5092755080 -1 init, newargv = 0x55e74539d020 newargc=9ceph-fuse[23733]: starting ceph client 2022-08-01T23:05:31.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr: 2022-08-01T23:05:31.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi130.stderr:ceph-fuse[23733]: starting fuse 2022-08-01T23:05:32.688 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2022-08-01T23:05:32.706 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:32.707 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:32.762 INFO:teuthology.orchestra.run.smithi130.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:32.763 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:32.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:32.765 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2022-08-01T23:05:32.770 INFO:teuthology.orchestra.run.smithi130.stdout:52 2022-08-01T23:05:32.771 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:05:32.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:32.772 DEBUG:teuthology.orchestra.run.smithi130:> sudo python3 -c ' 2022-08-01T23:05:32.772 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2022-08-01T23:05:32.773 DEBUG:teuthology.orchestra.run.smithi130:> import re 2022-08-01T23:05:32.773 DEBUG:teuthology.orchestra.run.smithi130:> import os 2022-08-01T23:05:32.773 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2022-08-01T23:05:32.774 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:32.774 DEBUG:teuthology.orchestra.run.smithi130:> def find_socket(client_name): 2022-08-01T23:05:32.775 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T23:05:32.775 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2022-08-01T23:05:32.775 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:32.776 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2022-08-01T23:05:32.776 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2022-08-01T23:05:32.777 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2022-08-01T23:05:32.777 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2022-08-01T23:05:32.777 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:32.778 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2022-08-01T23:05:32.778 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:05:32.779 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:05:32.779 DEBUG:teuthology.orchestra.run.smithi130:> return f 2022-08-01T23:05:32.780 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:05:32.780 DEBUG:teuthology.orchestra.run.smithi130:> 2022-08-01T23:05:32.780 DEBUG:teuthology.orchestra.run.smithi130:> print(find_socket("client.0")) 2022-08-01T23:05:32.781 DEBUG:teuthology.orchestra.run.smithi130:> ' 2022-08-01T23:05:32.858 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.0.23733.asok 2022-08-01T23:05:32.863 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23733.asok 2022-08-01T23:05:32.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:32.864 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23733.asok status 2022-08-01T23:05:32.952 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2022-08-01T23:05:32.953 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2022-08-01T23:05:32.953 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:05:32.953 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:05:32.954 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "0", 2022-08-01T23:05:32.954 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2022-08-01T23:05:32.954 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T23:05:32.954 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "23733", 2022-08-01T23:05:32.955 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2022-08-01T23:05:32.955 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:32.955 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2022-08-01T23:05:32.955 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2022-08-01T23:05:32.955 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 8919, 2022-08-01T23:05:32.956 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2022-08-01T23:05:32.956 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2022-08-01T23:05:32.956 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2022-08-01T23:05:32.956 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 8919 2022-08-01T23:05:32.957 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:32.957 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:05:32.957 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:05:32.957 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:05:32.957 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 322222041 2022-08-01T23:05:32.958 INFO:teuthology.orchestra.run.smithi130.stdout: } 2022-08-01T23:05:32.958 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:32.958 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2022-08-01T23:05:32.958 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2022-08-01T23:05:32.959 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "172.21.15.130:0", 2022-08-01T23:05:32.959 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 322222041 2022-08-01T23:05:32.959 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2022-08-01T23:05:32.959 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.8919 172.21.15.130:0/322222041", 2022-08-01T23:05:32.959 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "172.21.15.130:0/322222041", 2022-08-01T23:05:32.960 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2022-08-01T23:05:32.960 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 81, 2022-08-01T23:05:32.960 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 158, 2022-08-01T23:05:32.960 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:05:32.961 INFO:teuthology.orchestra.run.smithi130.stdout: "blacklisted": false, 2022-08-01T23:05:32.961 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2022-08-01T23:05:32.961 INFO:teuthology.orchestra.run.smithi130.stdout:} 2022-08-01T23:05:32.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:32.963 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:32.963 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:32.970 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2022-08-01T23:05:32.971 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T23:05:32.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:32.972 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:05:33.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:05:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 Processor -- start 2022-08-01T23:05:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 -- start start 2022-08-01T23:05:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09740b01a0 0x7f09740a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740afd60 0x7f09740a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09740a7250 con 0x7f09740b2710 2022-08-01T23:05:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09740a7480 con 0x7f09740b01a0 2022-08-01T23:05:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0973fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09740a7720 con 0x7f09740afd60 2022-08-01T23:05:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0972ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09740b2710 0x7f097414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37696/0 (socket says 172.21.15.32:37696) 2022-08-01T23:05:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0972ffd700 1 -- 172.21.15.32:0/3696955317 learned_addr learned my addr 172.21.15.32:0/3696955317 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f09727fc700 1 --2- 172.21.15.32:0/3696955317 >> v2:172.21.15.55:3300/0 conn(0x7f09740b01a0 0x7f09740a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f09737fe700 1 --2- 172.21.15.32:0/3696955317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740afd60 0x7f09740a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.135+0000 7f0971ffb700 1 -- 172.21.15.32:0/3696955317 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613397184 0 0) 0x7f09740a7250 con 0x7f09740b2710 2022-08-01T23:05:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0971ffb700 1 -- 172.21.15.32:0/3696955317 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0958003260 con 0x7f09740b2710 2022-08-01T23:05:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f09737fe700 1 -- 172.21.15.32:0/3696955317 >> v2:172.21.15.55:3300/0 conn(0x7f09740b01a0 msgr2=0x7f09740a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f09737fe700 1 --2- 172.21.15.32:0/3696955317 >> v2:172.21.15.55:3300/0 conn(0x7f09740b01a0 0x7f09740a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0971ffb700 1 -- 172.21.15.32:0/3696955317 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18530361 0 0) 0x7f0958003260 con 0x7f09740b2710 2022-08-01T23:05:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f09737fe700 1 -- 172.21.15.32:0/3696955317 >> v1:172.21.15.32:6789/0 conn(0x7f09740b2710 legacy=0x7f097414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f09737fe700 1 -- 172.21.15.32:0/3696955317 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09740a79c0 con 0x7f09740afd60 2022-08-01T23:05:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f09737fe700 1 --2- 172.21.15.32:0/3696955317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740afd60 0x7f09740a8a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f0968001800 tx=0x7f096c006df0).ready entity=mon.2 client_cookie=21dc0aff633c50bd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0971ffb700 1 -- 172.21.15.32:0/3696955317 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0968001660 con 0x7f09740afd60 2022-08-01T23:05:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0971ffb700 1 -- 172.21.15.32:0/3696955317 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c000e90 con 0x7f09740afd60 2022-08-01T23:05:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0971ffb700 1 -- 172.21.15.32:0/3696955317 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096c012460 con 0x7f09740afd60 2022-08-01T23:05:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 -- 172.21.15.32:0/3696955317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740afd60 msgr2=0x7f09740a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 --2- 172.21.15.32:0/3696955317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740afd60 0x7f09740a8a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f0968001800 tx=0x7f096c006df0).stop 2022-08-01T23:05:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 -- 172.21.15.32:0/3696955317 shutdown_connections 2022-08-01T23:05:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 --2- 172.21.15.32:0/3696955317 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740afd60 0x7f09740a8a30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 --2- 172.21.15.32:0/3696955317 >> v2:172.21.15.55:3300/0 conn(0x7f09740b01a0 0x7f09740a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 -- 172.21.15.32:0/3696955317 >> 172.21.15.32:0/3696955317 conn(0x7f0974060310 msgr2=0x7f09740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 -- 172.21.15.32:0/3696955317 shutdown_connections 2022-08-01T23:05:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.139+0000 7f0973fff700 1 -- 172.21.15.32:0/3696955317 wait complete. 2022-08-01T23:05:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 Processor -- start 2022-08-01T23:05:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 -- start start 2022-08-01T23:05:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09740afd60 0x7f097405c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740b01a0 0x7f0974159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 0x7f0974063d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0974063630 con 0x7f09740b2710 2022-08-01T23:05:33.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09740b3a30 con 0x7f09740afd60 2022-08-01T23:05:33.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09740b3790 con 0x7f09740b01a0 2022-08-01T23:05:33.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 0x7f0974063d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 0x7f0974063d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50162/0 (socket says 172.21.15.32:50162) 2022-08-01T23:05:33.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 -- 172.21.15.32:0/232672066 learned_addr learned my addr 172.21.15.32:0/232672066 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:33.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0972ffd700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09740afd60 0x7f097405c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09727fc700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740b01a0 0x7f0974159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 -- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740b01a0 msgr2=0x7f0974159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740b01a0 0x7f0974159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 -- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09740afd60 msgr2=0x7f097405c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09740afd60 0x7f097405c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 -- 172.21.15.32:0/232672066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c00b180 con 0x7f09740b2710 2022-08-01T23:05:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09727fc700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740b01a0 0x7f0974159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0972ffd700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09740afd60 0x7f097405c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:33.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f09737fe700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 0x7f0974063d30 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f0968001800 tx=0x7f096c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096c016070 con 0x7f09740b2710 2022-08-01T23:05:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c00d5e0 con 0x7f09740b2710 2022-08-01T23:05:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0974063890 con 0x7f09740b2710 2022-08-01T23:05:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096c00d940 con 0x7f09740b2710 2022-08-01T23:05:33.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.143+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0974063190 con 0x7f09740b2710 2022-08-01T23:05:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.147+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0974063a20 con 0x7f09740b2710 2022-08-01T23:05:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.147+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f096c025030 con 0x7f09740b2710 2022-08-01T23:05:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.147+0000 7f0963fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0950053a70 0x7f0950055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.147+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f096c06ce40 con 0x7f09740b2710 2022-08-01T23:05:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.147+0000 7f0972ffd700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0950053a70 0x7f0950055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.151+0000 7f0972ffd700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0950053a70 0x7f0950055f30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f096800d000 tx=0x7f0968001d40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.155+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f096c043c00 con 0x7f09740b2710 2022-08-01T23:05:33.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.283+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09740574e0 con 0x7f09740b2710 2022-08-01T23:05:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.283+0000 7f0963fff700 1 -- 172.21.15.32:0/232672066 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f096c023070 con 0x7f09740b2710 2022-08-01T23:05:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:33.292 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:26.723454+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T23:05:24.145656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T23:05:24.725583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:05:33.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0950053a70 msgr2=0x7f0950055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0950053a70 0x7f0950055f30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f096800d000 tx=0x7f0968001d40).stop 2022-08-01T23:05:33.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 msgr2=0x7f0974063d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 0x7f0974063d30 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f0968001800 tx=0x7f096c014000).stop 2022-08-01T23:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 shutdown_connections 2022-08-01T23:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0950053a70 0x7f0950055f30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09740b2710 0x7f0974063d30 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f09740b01a0 0x7f0974159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 --2- 172.21.15.32:0/232672066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09740afd60 0x7f097405c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 >> 172.21.15.32:0/232672066 conn(0x7f0974060310 msgr2=0x7f097405cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 shutdown_connections 2022-08-01T23:05:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.287+0000 7f0973fff700 1 -- 172.21.15.32:0/232672066 wait complete. 2022-08-01T23:05:33.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T22:55:34.277887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T22:55:43.120854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T23:05:24.145656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T23:05:24.725583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T23:05:33.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T23:05:33.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 Processor -- start 2022-08-01T23:05:33.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- start start 2022-08-01T23:05:33.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b2710 0x7f512014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51200b01a0 0x7f512014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51200afd60 0x7f51200a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51200a7250 con 0x7f51200afd60 2022-08-01T23:05:33.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51200a7480 con 0x7f51200b01a0 2022-08-01T23:05:33.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51200a7720 con 0x7f51200b2710 2022-08-01T23:05:33.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51200afd60 0x7f51200a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51200afd60 0x7f51200a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50166/0 (socket says 172.21.15.32:50166) 2022-08-01T23:05:33.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 -- 172.21.15.32:0/28777397 learned_addr learned my addr 172.21.15.32:0/28777397 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511effd700 1 --2- 172.21.15.32:0/28777397 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b2710 0x7f512014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511e7fc700 1 --2- 172.21.15.32:0/28777397 >> v2:172.21.15.55:3300/0 conn(0x7f51200b01a0 0x7f512014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 -- 172.21.15.32:0/28777397 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b2710 msgr2=0x7f512014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 --2- 172.21.15.32:0/28777397 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b2710 0x7f512014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 -- 172.21.15.32:0/28777397 >> v2:172.21.15.55:3300/0 conn(0x7f51200b01a0 msgr2=0x7f512014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 --2- 172.21.15.32:0/28777397 >> v2:172.21.15.55:3300/0 conn(0x7f51200b01a0 0x7f512014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 -- 172.21.15.32:0/28777397 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51200a79c0 con 0x7f51200afd60 2022-08-01T23:05:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511f7fe700 1 --2- 172.21.15.32:0/28777397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f51200a8540 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f51180032f0 tx=0x7f5118009290).ready entity=mon.0 client_cookie=7bcbba24092601a2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511dffb700 1 -- 172.21.15.32:0/28777397 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511800b870 con 0x7f51200afd60 2022-08-01T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511dffb700 1 -- 172.21.15.32:0/28777397 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511800b9d0 con 0x7f51200afd60 2022-08-01T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511dffb700 1 -- 172.21.15.32:0/28777397 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511800bb40 con 0x7f51200afd60 2022-08-01T23:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- 172.21.15.32:0/28777397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 msgr2=0x7f51200a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- 172.21.15.32:0/28777397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f51200a8540 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f51180032f0 tx=0x7f5118009290).stop 2022-08-01T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- 172.21.15.32:0/28777397 shutdown_connections 2022-08-01T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- 172.21.15.32:0/28777397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f51200a8540 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- 172.21.15.32:0/28777397 >> v2:172.21.15.55:3300/0 conn(0x7f51200b01a0 0x7f512014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 --2- 172.21.15.32:0/28777397 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b2710 0x7f512014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- 172.21.15.32:0/28777397 >> 172.21.15.32:0/28777397 conn(0x7f5120060310 msgr2=0x7f51200b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- 172.21.15.32:0/28777397 shutdown_connections 2022-08-01T23:05:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.415+0000 7f511ffff700 1 -- 172.21.15.32:0/28777397 wait complete. 2022-08-01T23:05:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 Processor -- start 2022-08-01T23:05:33.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 -- start start 2022-08-01T23:05:33.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f5120143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b01a0 0x7f51200aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51200b2710 0x7f51200a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120159b90 con 0x7f51200afd60 2022-08-01T23:05:33.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120159df0 con 0x7f51200b2710 2022-08-01T23:05:33.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512015a090 con 0x7f51200b01a0 2022-08-01T23:05:33.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f5120143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f5120143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50172/0 (socket says 172.21.15.32:50172) 2022-08-01T23:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.419+0000 7f511effd700 1 -- 172.21.15.32:0/1238054531 learned_addr learned my addr 172.21.15.32:0/1238054531 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511f7fe700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51200b2710 0x7f51200a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511e7fc700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b01a0 0x7f51200aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511effd700 1 -- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b01a0 msgr2=0x7f51200aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511effd700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b01a0 0x7f51200aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511effd700 1 -- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51200b2710 msgr2=0x7f51200a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511effd700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51200b2710 0x7f51200a9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511effd700 1 -- 172.21.15.32:0/1238054531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009530 con 0x7f51200afd60 2022-08-01T23:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511effd700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f5120143e80 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f51100030b0 tx=0x7f5110008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5110011780 con 0x7f51200afd60 2022-08-01T23:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51100118e0 con 0x7f51200afd60 2022-08-01T23:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512015a330 con 0x7f51200afd60 2022-08-01T23:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5110011c40 con 0x7f51200afd60 2022-08-01T23:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51200ab2a0 con 0x7f51200afd60 2022-08-01T23:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512014d660 con 0x7f51200afd60 2022-08-01T23:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5110012420 con 0x7f51200afd60 2022-08-01T23:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f510ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc053aa0 0x7f50fc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f511006bd60 con 0x7f51200afd60 2022-08-01T23:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.423+0000 7f511e7fc700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc053aa0 0x7f50fc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.427+0000 7f511e7fc700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc053aa0 0x7f50fc055f60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f511400c000 tx=0x7f5114008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.431+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f512014d660 con 0x7f51200afd60 2022-08-01T23:05:33.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.559+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f512014d660 con 0x7f51200afd60 2022-08-01T23:05:33.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.559+0000 7f510ffff700 1 -- 172.21.15.32:0/1238054531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v158) v1 ==== 110+0+10 (secure 0 0 0) 0x7f512014d660 con 0x7f51200afd60 2022-08-01T23:05:33.567 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2022-08-01T23:05:33.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.559+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc053aa0 msgr2=0x7f50fc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc053aa0 0x7f50fc055f60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f511400c000 tx=0x7f5114008040).stop 2022-08-01T23:05:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 msgr2=0x7f5120143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f5120143e80 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f51100030b0 tx=0x7f5110008040).stop 2022-08-01T23:05:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 shutdown_connections 2022-08-01T23:05:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f50fc053aa0 0x7f50fc055f60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51200b2710 0x7f51200a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f51200b01a0 0x7f51200aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 --2- 172.21.15.32:0/1238054531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51200afd60 0x7f5120143e80 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 >> 172.21.15.32:0/1238054531 conn(0x7f5120060310 msgr2=0x7f5120144c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 shutdown_connections 2022-08-01T23:05:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.563+0000 7f511ffff700 1 -- 172.21.15.32:0/1238054531 wait complete. 2022-08-01T23:05:33.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T23:05:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.683+0000 7fec9759e700 1 Processor -- start 2022-08-01T23:05:33.691 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.683+0000 7fec9759e700 1 -- start start 2022-08-01T23:05:33.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 0x7fec98061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980aee80 con 0x7fec980ad750 2022-08-01T23:05:33.693 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980b70b0 con 0x7fec981405d0 2022-08-01T23:05:33.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980af0b0 con 0x7fec980614b0 2022-08-01T23:05:33.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec95d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec980ad750 0x7fec980b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37724/0 (socket says 172.21.15.32:37724) 2022-08-01T23:05:33.694 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec95d9b700 1 -- 172.21.15.32:0/2950911497 learned_addr learned my addr 172.21.15.32:0/2950911497 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 0x7fec98061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198691592 0 0) 0x7fec980aee80 con 0x7fec980ad750 2022-08-01T23:05:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec78003260 con 0x7fec980ad750 2022-08-01T23:05:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523553435 0 0) 0x7fec980b70b0 con 0x7fec981405d0 2022-08-01T23:05:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec980aee80 con 0x7fec981405d0 2022-08-01T23:05:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435522848 0 0) 0x7fec78003260 con 0x7fec980ad750 2022-08-01T23:05:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 msgr2=0x7fec98061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 0x7fec98061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 >> v1:172.21.15.55:6789/0 conn(0x7fec981405d0 legacy=0x7fec981466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980af350 con 0x7fec980ad750 2022-08-01T23:05:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fec88001660 con 0x7fec980ad750 2022-08-01T23:05:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 >> v1:172.21.15.32:6789/0 conn(0x7fec980ad750 legacy=0x7fec980b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec78003910 0x7fec78003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 0x7fec78004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec780049f0 0x7fec78004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980af0b0 con 0x7fec78003910 2022-08-01T23:05:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7800b780 con 0x7fec780049f0 2022-08-01T23:05:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7800b940 con 0x7fec980614b0 2022-08-01T23:05:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec78003910 0x7fec78003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec96d9d700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec780049f0 0x7fec78004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec95d9b700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 0x7fec78004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 -- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 msgr2=0x7fec78004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec980614b0 0x7fec78004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 -- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec780049f0 msgr2=0x7fec78004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec780049f0 0x7fec78004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 -- 172.21.15.32:0/2950911497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec78003260 con 0x7fec78003910 2022-08-01T23:05:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9659c700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec78003910 0x7fec78003d10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fec8000c000 tx=0x7fec80006df0).ready entity=mon.0 client_cookie=3c19d8f2197d8e8c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8000e830 con 0x7fec78003910 2022-08-01T23:05:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8000e990 con 0x7fec78003910 2022-08-01T23:05:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9559a700 1 -- 172.21.15.32:0/2950911497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8000ecf0 con 0x7fec78003910 2022-08-01T23:05:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- 172.21.15.32:0/2950911497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec78003910 msgr2=0x7fec78003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 --2- 172.21.15.32:0/2950911497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec78003910 0x7fec78003d10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fec8000c000 tx=0x7fec80006df0).stop 2022-08-01T23:05:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec96d9d700 1 -- 172.21.15.32:0/2950911497 reap_dead start 2022-08-01T23:05:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- 172.21.15.32:0/2950911497 shutdown_connections 2022-08-01T23:05:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- 172.21.15.32:0/2950911497 >> 172.21.15.32:0/2950911497 conn(0x7fec98066f70 msgr2=0x7fec98067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- 172.21.15.32:0/2950911497 shutdown_connections 2022-08-01T23:05:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.687+0000 7fec9759e700 1 -- 172.21.15.32:0/2950911497 wait complete. 2022-08-01T23:05:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 Processor -- start 2022-08-01T23:05:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 -- start start 2022-08-01T23:05:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980ab390 0x7fec980b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 0x7fec98156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec98156860 0x7fec98156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec98157200 con 0x7fec980b3740 2022-08-01T23:05:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec98153940 con 0x7fec980ab390 2022-08-01T23:05:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec98153b70 con 0x7fec98156860 2022-08-01T23:05:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 0x7fec98156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec96d9d700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec98156860 0x7fec98156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 0x7fec98156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50196/0 (socket says 172.21.15.32:50196) 2022-08-01T23:05:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 -- 172.21.15.32:0/2671596522 learned_addr learned my addr 172.21.15.32:0/2671596522 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9659c700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980ab390 0x7fec980b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 -- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec98156860 msgr2=0x7fec98156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec98156860 0x7fec98156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 -- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980ab390 msgr2=0x7fec980b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980ab390 0x7fec980b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 -- 172.21.15.32:0/2671596522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980b16c0 con 0x7fec980b3740 2022-08-01T23:05:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec95d9b700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 0x7fec98156370 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7fec880011d0 tx=0x7fec980af4e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec880116b0 con 0x7fec980b3740 2022-08-01T23:05:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec88011810 con 0x7fec980b3740 2022-08-01T23:05:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec980ad3c0 con 0x7fec980b3740 2022-08-01T23:05:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec88011b70 con 0x7fec980b3740 2022-08-01T23:05:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.691+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec980ab770 con 0x7fec980b3740 2022-08-01T23:05:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.695+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec98153d30 con 0x7fec980b3740 2022-08-01T23:05:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.695+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec8801b030 con 0x7fec980b3740 2022-08-01T23:05:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.695+0000 7fec8f7fe700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec7c053b30 0x7fec7c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.695+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fec8806ba20 con 0x7fec980b3740 2022-08-01T23:05:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.695+0000 7fec9659c700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec7c053b30 0x7fec7c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.699+0000 7fec9659c700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec7c053b30 0x7fec7c055ff0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7fec8001d000 tx=0x7fec80019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.699+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec880427e0 con 0x7fec980b3740 2022-08-01T23:05:33.833 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.827+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fec980af5f0 con 0x7fec980b3740 2022-08-01T23:05:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.827+0000 7fec8f7fe700 1 -- 172.21.15.32:0/2671596522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7fec88019030 con 0x7fec980b3740 2022-08-01T23:05:33.835 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec7c053b30 msgr2=0x7fec7c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec7c053b30 0x7fec7c055ff0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7fec8001d000 tx=0x7fec80019000).stop 2022-08-01T23:05:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 msgr2=0x7fec98156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 0x7fec98156370 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7fec880011d0 tx=0x7fec980af4e0).stop 2022-08-01T23:05:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 shutdown_connections 2022-08-01T23:05:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fec7c053b30 0x7fec7c055ff0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fec98156860 0x7fec98156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec980b3740 0x7fec98156370 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 --2- 172.21.15.32:0/2671596522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980ab390 0x7fec980b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 >> 172.21.15.32:0/2671596522 conn(0x7fec98066f70 msgr2=0x7fec980b9260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 shutdown_connections 2022-08-01T23:05:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.831+0000 7fec9759e700 1 -- 172.21.15.32:0/2671596522 wait complete. 2022-08-01T23:05:33.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T23:05:33.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 Processor -- start 2022-08-01T23:05:33.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- start start 2022-08-01T23:05:33.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c4c0b01a0 0x7f1c4c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0a7250 con 0x7f1c4c0afd60 2022-08-01T23:05:33.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0a7480 con 0x7f1c4c0b2710 2022-08-01T23:05:33.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c0a7720 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c4c0b01a0 0x7f1c4c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c4c0b01a0 0x7f1c4c0a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50202/0 (socket says 172.21.15.32:50202) 2022-08-01T23:05:33.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3317344549 learned_addr learned my addr 172.21.15.32:0/3317344549 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:33.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c4bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c4c0afd60 0x7f1c4c151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37746/0 (socket says 172.21.15.32:37746) 2022-08-01T23:05:33.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3317344549 >> v1:172.21.15.55:6789/0 conn(0x7f1c4c0b2710 legacy=0x7f1c4c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3317344549 >> v1:172.21.15.32:6789/0 conn(0x7f1c4c0afd60 legacy=0x7f1c4c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3317344549 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0a79c0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c42ffd700 1 --2- 172.21.15.32:0/3317344549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c0a8540 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f1c34002f70 tx=0x7f1c34006eb0).ready entity=mon.0 client_cookie=ec9cb3a4953cdfe4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c4affd700 1 -- 172.21.15.32:0/3317344549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3400d660 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c4affd700 1 -- 172.21.15.32:0/3317344549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c34000e90 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c4affd700 1 -- 172.21.15.32:0/3317344549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3400d9f0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- 172.21.15.32:0/3317344549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 msgr2=0x7f1c4c0a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3317344549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c0a8540 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f1c34002f70 tx=0x7f1c34006eb0).stop 2022-08-01T23:05:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- 172.21.15.32:0/3317344549 shutdown_connections 2022-08-01T23:05:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3317344549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c0a8540 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- 172.21.15.32:0/3317344549 >> 172.21.15.32:0/3317344549 conn(0x7f1c4c060310 msgr2=0x7f1c4c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- 172.21.15.32:0/3317344549 shutdown_connections 2022-08-01T23:05:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.955+0000 7f1c5081b700 1 -- 172.21.15.32:0/3317344549 wait complete. 2022-08-01T23:05:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 Processor -- start 2022-08-01T23:05:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 -- start start 2022-08-01T23:05:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c4c0afd60 0x7f1c4c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c4c0b2710 0x7f1c4c157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b3ef0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c156860 con 0x7f1c4c0b2710 2022-08-01T23:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c5081b700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c066870 con 0x7f1c4c0afd60 2022-08-01T23:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50208/0 (socket says 172.21.15.32:50208) 2022-08-01T23:05:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3676101871 learned_addr learned my addr 172.21.15.32:0/3676101871 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c4b7fe700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c4c0afd60 0x7f1c4c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c4bfff700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c4c0b2710 0x7f1c4c157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c4c0afd60 msgr2=0x7f1c4c0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c4c0afd60 0x7f1c4c0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c4c0b2710 msgr2=0x7f1c4c157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c4c0b2710 0x7f1c4c157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:33.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c42ffd700 1 -- 172.21.15.32:0/3676101871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3400b1e0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.959+0000 7f1c4b7fe700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c4c0afd60 0x7f1c4c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:33.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c42ffd700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c156c50 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f1c34002f70 tx=0x7f1c34015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c34011cc0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c34011e20 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3401e650 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0b5620 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c066cb0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c3401a040 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0b08c0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c497fa700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c30053a70 0x7f1c30055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1c3406d1e0 con 0x7f1c4c0b01a0 2022-08-01T23:05:33.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.963+0000 7f1c4b7fe700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c30053a70 0x7f1c30055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:33.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.967+0000 7f1c4b7fe700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c30053a70 0x7f1c30055f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f1c3c00b000 tx=0x7f1c3c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:33.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:33.967+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c34043f10 con 0x7f1c4c0b01a0 2022-08-01T23:05:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c4c0574e0 con 0x7f1c4c0b01a0 2022-08-01T23:05:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c497fa700 1 -- 172.21.15.32:0/3676101871 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1c340474b0 con 0x7f1c4c0b01a0 2022-08-01T23:05:34.102 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c30053a70 msgr2=0x7f1c30055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c30053a70 0x7f1c30055f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f1c3c00b000 tx=0x7f1c3c009040).stop 2022-08-01T23:05:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 msgr2=0x7f1c4c156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c156c50 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f1c34002f70 tx=0x7f1c34015000).stop 2022-08-01T23:05:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 shutdown_connections 2022-08-01T23:05:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1c30053a70 0x7f1c30055f30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c4c0b2710 0x7f1c4c157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c4c0b01a0 0x7f1c4c156c50 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 --2- 172.21.15.32:0/3676101871 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1c4c0afd60 0x7f1c4c0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 >> 172.21.15.32:0/3676101871 conn(0x7f1c4c060310 msgr2=0x7f1c4c067740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 shutdown_connections 2022-08-01T23:05:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.095+0000 7f1c5081b700 1 -- 172.21.15.32:0/3676101871 wait complete. 2022-08-01T23:05:34.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T23:05:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.215+0000 7f4e270a2700 1 Processor -- start 2022-08-01T23:05:34.225 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.215+0000 7f4e270a2700 1 -- start start 2022-08-01T23:05:34.227 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4310 0x7f4e20140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e20143cf0 con 0x7f4e20146ae0 2022-08-01T23:05:34.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e20143f90 con 0x7f4e200b4820 2022-08-01T23:05:34.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201476a0 con 0x7f4e200b4310 2022-08-01T23:05:34.228 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e24e3e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e20146ae0 0x7f4e201403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37756/0 (socket says 172.21.15.32:37756) 2022-08-01T23:05:34.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e24e3e700 1 -- 172.21.15.32:0/1279043595 learned_addr learned my addr 172.21.15.32:0/1279043595 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:34.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1ffff700 1 --2- 172.21.15.32:0/1279043595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4310 0x7f4e20140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207045603 0 0) 0x7f4e20143cf0 con 0x7f4e20146ae0 2022-08-01T23:05:34.229 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e0c003260 con 0x7f4e20146ae0 2022-08-01T23:05:34.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553924086 0 0) 0x7f4e20143f90 con 0x7f4e200b4820 2022-08-01T23:05:34.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e20143cf0 con 0x7f4e200b4820 2022-08-01T23:05:34.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1ffff700 1 -- 172.21.15.32:0/1279043595 >> v1:172.21.15.55:6789/0 conn(0x7f4e200b4820 legacy=0x7f4e200b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.230 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1ffff700 1 -- 172.21.15.32:0/1279043595 >> v1:172.21.15.32:6789/0 conn(0x7f4e20146ae0 legacy=0x7f4e201403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1ffff700 1 -- 172.21.15.32:0/1279043595 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200b0e20 con 0x7f4e200b4310 2022-08-01T23:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1ffff700 1 --2- 172.21.15.32:0/1279043595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4310 0x7f4e20140ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f4e14002f70 tx=0x7f4e14006df0).ready entity=mon.2 client_cookie=288d7f2b20fa7053 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1400d5c0 con 0x7f4e200b4310 2022-08-01T23:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e14000e90 con 0x7f4e200b4310 2022-08-01T23:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e1f7fe700 1 -- 172.21.15.32:0/1279043595 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1400d910 con 0x7f4e200b4310 2022-08-01T23:05:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- 172.21.15.32:0/1279043595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4310 msgr2=0x7f4e20140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 --2- 172.21.15.32:0/1279043595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4310 0x7f4e20140ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f4e14002f70 tx=0x7f4e14006df0).stop 2022-08-01T23:05:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- 172.21.15.32:0/1279043595 shutdown_connections 2022-08-01T23:05:34.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 --2- 172.21.15.32:0/1279043595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4310 0x7f4e20140ad0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- 172.21.15.32:0/1279043595 >> 172.21.15.32:0/1279043595 conn(0x7f4e2005c620 msgr2=0x7f4e2005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:34.236 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- 172.21.15.32:0/1279043595 shutdown_connections 2022-08-01T23:05:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.219+0000 7f4e270a2700 1 -- 172.21.15.32:0/1279043595 wait complete. 2022-08-01T23:05:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 Processor -- start 2022-08-01T23:05:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 -- start start 2022-08-01T23:05:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 0x7f4e200b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4820 0x7f4e201560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e20146ae0 0x7f4e20156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200ae140 con 0x7f4e200b4310 2022-08-01T23:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20155c00 con 0x7f4e20146ae0 2022-08-01T23:05:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200b2cf0 con 0x7f4e200b4820 2022-08-01T23:05:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 0x7f4e200b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 0x7f4e200b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50222/0 (socket says 172.21.15.32:50222) 2022-08-01T23:05:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 -- 172.21.15.32:0/3032743708 learned_addr learned my addr 172.21.15.32:0/3032743708 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e1ffff700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4820 0x7f4e201560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e2563f700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e20146ae0 0x7f4e20156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 -- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4820 msgr2=0x7f4e201560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4820 0x7f4e201560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 -- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e20146ae0 msgr2=0x7f4e20156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e20146ae0 0x7f4e20156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 -- 172.21.15.32:0/3032743708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1400b180 con 0x7f4e200b4310 2022-08-01T23:05:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e2563f700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e20146ae0 0x7f4e20156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e24e3e700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 0x7f4e200b2800 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f4e100011d0 tx=0x7f4e10007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e100070d0 con 0x7f4e200b4310 2022-08-01T23:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10007230 con 0x7f4e200b4310 2022-08-01T23:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e200b8e00 con 0x7f4e200b4310 2022-08-01T23:05:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e10005610 con 0x7f4e200b4310 2022-08-01T23:05:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.223+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e200adbc0 con 0x7f4e200b4310 2022-08-01T23:05:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.227+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e20155dc0 con 0x7f4e200b4310 2022-08-01T23:05:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.227+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e1001a060 con 0x7f4e200b4310 2022-08-01T23:05:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.227+0000 7f4e1dffb700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e00053a70 0x7f4e00055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.227+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4e1006b790 con 0x7f4e200b4310 2022-08-01T23:05:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.231+0000 7f4e1ffff700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e00053a70 0x7f4e00055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.231+0000 7f4e1ffff700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e00053a70 0x7f4e00055f30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f4e14018000 tx=0x7f4e14014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.235+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e10042550 con 0x7f4e200b4310 2022-08-01T23:05:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.363+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e200574e0 con 0x7f4e200b4310 2022-08-01T23:05:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.363+0000 7f4e1dffb700 1 -- 172.21.15.32:0/3032743708 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4e10018020 con 0x7f4e200b4310 2022-08-01T23:05:34.372 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2022-08-01T23:05:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e00053a70 msgr2=0x7f4e00055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e00053a70 0x7f4e00055f30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f4e14018000 tx=0x7f4e14014000).stop 2022-08-01T23:05:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 msgr2=0x7f4e200b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 0x7f4e200b2800 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f4e100011d0 tx=0x7f4e10007b50).stop 2022-08-01T23:05:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 shutdown_connections 2022-08-01T23:05:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4e00053a70 0x7f4e00055f30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e20146ae0 0x7f4e20156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f4e200b4820 0x7f4e201560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 --2- 172.21.15.32:0/3032743708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200b4310 0x7f4e200b2800 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 >> 172.21.15.32:0/3032743708 conn(0x7f4e2005c620 msgr2=0x7f4e20145a90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 shutdown_connections 2022-08-01T23:05:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.367+0000 7f4e270a2700 1 -- 172.21.15.32:0/3032743708 wait complete. 2022-08-01T23:05:34.385 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T23:05:34.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 Processor -- start 2022-08-01T23:05:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- start start 2022-08-01T23:05:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c0b1f40 0x7f5d7c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c0b04e0 con 0x7f5d7c0a7250 2022-08-01T23:05:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c0b0780 con 0x7f5d7c148530 2022-08-01T23:05:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0a9250 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d80c69700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5d7c0a7250 0x7f5d7c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37774/0 (socket says 172.21.15.32:37774) 2022-08-01T23:05:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d80c69700 1 -- 172.21.15.32:0/3166673264 learned_addr learned my addr 172.21.15.32:0/3166673264 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7bfff700 1 --2- 172.21.15.32:0/3166673264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c0b1f40 0x7f5d7c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464973762 0 0) 0x7f5d7c0b04e0 con 0x7f5d7c0a7250 2022-08-01T23:05:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d68003260 con 0x7f5d7c0a7250 2022-08-01T23:05:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838889950 0 0) 0x7f5d7c0b0780 con 0x7f5d7c148530 2022-08-01T23:05:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c0b04e0 con 0x7f5d7c148530 2022-08-01T23:05:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7bfff700 1 -- 172.21.15.32:0/3166673264 >> v1:172.21.15.55:6789/0 conn(0x7f5d7c148530 legacy=0x7f5d7c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7bfff700 1 -- 172.21.15.32:0/3166673264 >> v1:172.21.15.32:6789/0 conn(0x7f5d7c0a7250 legacy=0x7f5d7c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7bfff700 1 -- 172.21.15.32:0/3166673264 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0b0a20 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7bfff700 1 --2- 172.21.15.32:0/3166673264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c0b1f40 0x7f5d7c0b1160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f5d6c0032f0 tx=0x7f5d6c0091d0).ready entity=mon.2 client_cookie=daf033eb3403f5a4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:34.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6c00b640 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c00b7a0 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d7affd700 1 -- 172.21.15.32:0/3166673264 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6c00b910 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- 172.21.15.32:0/3166673264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c0b1f40 msgr2=0x7f5d7c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.496 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 --2- 172.21.15.32:0/3166673264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c0b1f40 0x7f5d7c0b1160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f5d6c0032f0 tx=0x7f5d6c0091d0).stop 2022-08-01T23:05:34.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- 172.21.15.32:0/3166673264 shutdown_connections 2022-08-01T23:05:34.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 --2- 172.21.15.32:0/3166673264 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c0b1f40 0x7f5d7c0b1160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- 172.21.15.32:0/3166673264 >> 172.21.15.32:0/3166673264 conn(0x7f5d7c05f840 msgr2=0x7f5d7c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:34.497 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- 172.21.15.32:0/3166673264 shutdown_connections 2022-08-01T23:05:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.483+0000 7f5d826cc700 1 -- 172.21.15.32:0/3166673264 wait complete. 2022-08-01T23:05:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 Processor -- start 2022-08-01T23:05:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 -- start start 2022-08-01T23:05:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c0a7250 0x7f5d7c14b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 0x7f5d7c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c148530 0x7f5d7c0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ac4b0 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ab2e0 con 0x7f5d7c0a7250 2022-08-01T23:05:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c149ea0 con 0x7f5d7c148530 2022-08-01T23:05:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 0x7f5d7c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 0x7f5d7c0ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50236/0 (socket says 172.21.15.32:50236) 2022-08-01T23:05:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 -- 172.21.15.32:0/1426576713 learned_addr learned my addr 172.21.15.32:0/1426576713 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7bfff700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c0a7250 0x7f5d7c14b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d80c69700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c148530 0x7f5d7c0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 -- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c148530 msgr2=0x7f5d7c0abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c148530 0x7f5d7c0abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 -- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c0a7250 msgr2=0x7f5d7c14b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c0a7250 0x7f5d7c14b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 -- 172.21.15.32:0/1426576713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c009470 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d80c69700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c148530 0x7f5d7c0abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d7b7fe700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 0x7f5d7c0ae300 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f5d70001110 tx=0x7f5d70007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7000cde0 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.506 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70012400 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c14a140 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.487+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d70012760 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.491+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c14a680 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.507 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.491+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c0574e0 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.491+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d70019030 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.491+0000 7f5d797fa700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d5c053a70 0x7f5d5c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.491+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5d7006b2a0 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.508 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.491+0000 7f5d7bfff700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d5c053a70 0x7f5d5c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.495+0000 7f5d7bfff700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d5c053a70 0x7f5d5c055f30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f5d6c01b000 tx=0x7f5d6c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.499+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d70042060 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.659+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d7c0447e0 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.659+0000 7f5d797fa700 1 -- 172.21.15.32:0/1426576713 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f5d70045680 con 0x7f5d7c0b1f40 2022-08-01T23:05:34.667 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:34.667 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1983320963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1983320963},{"type":"v1","addr":"172.21.15.55:6835","nonce":1983320963}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78},{"gid":8754,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3801584266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3801584266},{"type":"v1","addr":"172.21.15.32:6833","nonce":3801584266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:25.720657+0000","modified":"2022-08-01T23:05:27.400282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8757":{"gid":8757,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/324530820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":324530820},{"type":"v1","addr":"172.21.15.32:6835","nonce":324530820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T23:05:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d5c053a70 msgr2=0x7f5d5c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d5c053a70 0x7f5d5c055f30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f5d6c01b000 tx=0x7f5d6c016000).stop 2022-08-01T23:05:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 msgr2=0x7f5d7c0ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 0x7f5d7c0ae300 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f5d70001110 tx=0x7f5d70007a60).stop 2022-08-01T23:05:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 shutdown_connections 2022-08-01T23:05:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5d5c053a70 0x7f5d5c055f30 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5d7c148530 0x7f5d7c0abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7c0b1f40 0x7f5d7c0ae300 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 --2- 172.21.15.32:0/1426576713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c0a7250 0x7f5d7c14b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 >> 172.21.15.32:0/1426576713 conn(0x7f5d7c05f840 msgr2=0x7f5d7c0b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 shutdown_connections 2022-08-01T23:05:34.672 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:34.663+0000 7f5d826cc700 1 -- 172.21.15.32:0/1426576713 wait complete. 2022-08-01T23:05:34.672 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2022-08-01T23:05:34.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T23:05:25.720657+0000', 'modified': '2022-08-01T23:05:27.400282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8757}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8757': {'gid': 8757, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/324530820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 324530820}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 324530820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T23:05:35.028 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T23:05:35.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T23:05:35.148 INFO:teuthology.orchestra.run.smithi032.stdout:{"cluster_fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","whoami":0,"id":8757,"want_state":"up:active","state":"up:active","rank_uptime":9.4133108579999991,"mdsmap_epoch":81,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":13.844713520999999} 2022-08-01T23:05:35.149 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", "id": 8757, "mdsmap_epoch": 81, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 9.413310858, "state": "up:active", "uptime": 13.844713521, "want_state": "up:active", "whoami": 0 } 2022-08-01T23:05:35.150 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T23:05:35.150 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T23:05:35.151 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi157.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T23:05:35.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:35.151 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:35.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:05:35.217 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:35.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:35.234 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:35.235 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:35.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:35.237 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:05:35.243 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T23:05:35.244 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:35.247 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:05:35.262 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:35.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:35.320 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:35.321 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2022-08-01T23:05:35.311+0000 7f65b3ba6080 -1 init, newargv = 0x56065ec60880 newargc=9 2022-08-01T23:05:35.321 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23278]: starting ceph client 2022-08-01T23:05:35.322 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:35.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:35.323 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:05:35.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23278]: starting fuse 2022-08-01T23:05:36.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2022-08-01T23:05:36.448 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T23:05:36.448 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T23:05:36.504 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T23:05:36.505 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T23:05:36.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.507 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2022-08-01T23:05:36.513 INFO:teuthology.orchestra.run.smithi157.stdout:52 2022-08-01T23:05:36.514 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T23:05:36.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo python3 -c ' 2022-08-01T23:05:36.515 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2022-08-01T23:05:36.515 DEBUG:teuthology.orchestra.run.smithi157:> import re 2022-08-01T23:05:36.516 DEBUG:teuthology.orchestra.run.smithi157:> import os 2022-08-01T23:05:36.516 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2022-08-01T23:05:36.517 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:36.517 DEBUG:teuthology.orchestra.run.smithi157:> def find_socket(client_name): 2022-08-01T23:05:36.517 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T23:05:36.518 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2022-08-01T23:05:36.518 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:36.519 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2022-08-01T23:05:36.519 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2022-08-01T23:05:36.519 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2022-08-01T23:05:36.520 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2022-08-01T23:05:36.520 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:36.521 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2022-08-01T23:05:36.521 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T23:05:36.521 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T23:05:36.522 DEBUG:teuthology.orchestra.run.smithi157:> return f 2022-08-01T23:05:36.522 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T23:05:36.523 DEBUG:teuthology.orchestra.run.smithi157:> 2022-08-01T23:05:36.523 DEBUG:teuthology.orchestra.run.smithi157:> print(find_socket("client.1")) 2022-08-01T23:05:36.523 DEBUG:teuthology.orchestra.run.smithi157:> ' 2022-08-01T23:05:36.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.23278.asok 2022-08-01T23:05:36.573 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23278.asok 2022-08-01T23:05:36.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23278.asok status 2022-08-01T23:05:36.664 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2022-08-01T23:05:36.664 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2022-08-01T23:05:36.665 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T23:05:36.665 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T23:05:36.665 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2022-08-01T23:05:36.665 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2022-08-01T23:05:36.666 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T23:05:36.666 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "23278", 2022-08-01T23:05:36.666 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2022-08-01T23:05:36.667 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:36.667 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2022-08-01T23:05:36.667 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2022-08-01T23:05:36.667 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7568, 2022-08-01T23:05:36.668 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2022-08-01T23:05:36.668 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2022-08-01T23:05:36.668 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2022-08-01T23:05:36.668 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7568 2022-08-01T23:05:36.669 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:36.669 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:05:36.669 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:05:36.669 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:05:36.670 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2690588748 2022-08-01T23:05:36.670 INFO:teuthology.orchestra.run.smithi157.stdout: } 2022-08-01T23:05:36.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:36.671 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2022-08-01T23:05:36.671 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2022-08-01T23:05:36.671 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "172.21.15.157:0", 2022-08-01T23:05:36.671 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 2690588748 2022-08-01T23:05:36.672 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2022-08-01T23:05:36.672 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7568 172.21.15.157:0/2690588748", 2022-08-01T23:05:36.672 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "172.21.15.157:0/2690588748", 2022-08-01T23:05:36.672 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2022-08-01T23:05:36.672 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 81, 2022-08-01T23:05:36.673 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 158, 2022-08-01T23:05:36.673 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2022-08-01T23:05:36.673 INFO:teuthology.orchestra.run.smithi157.stdout: "blacklisted": false, 2022-08-01T23:05:36.673 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2022-08-01T23:05:36.674 INFO:teuthology.orchestra.run.smithi157.stdout:} 2022-08-01T23:05:36.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.675 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:36.676 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:36.683 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:05:36.684 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:05:36.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.685 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:36.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.701 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:36.702 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:36.709 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:05:36.711 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:05:36.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.711 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:36.712 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:36.760 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2022-08-01T23:05:36.761 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T23:05:36.762 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2022-08-01T23:05:36.762 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T23:05:36.763 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:36.777 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[23278]: fuse finished with error 0 and tester_r 0 2022-08-01T23:05:36.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:05:36.780 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:05:36.781 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:36.786 INFO:teuthology.orchestra.run.smithi157.stdout:ext2/ext3 2022-08-01T23:05:36.787 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T23:05:36.787 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T23:05:42.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T23:05:42.890 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T23:05:42.898 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T23:05:42.899 DEBUG:teuthology.orchestra.run.smithi130:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-08-01T23:05:42.919 INFO:teuthology.orchestra.run.smithi130.stdout: File: "." 2022-08-01T23:05:42.919 INFO:teuthology.orchestra.run.smithi130.stdout: ID: 0 Namelen: 255 Type: fuseblk 2022-08-01T23:05:42.919 INFO:teuthology.orchestra.run.smithi130.stdout:Block size: 4194304 Fundamental block size: 4194304 2022-08-01T23:05:42.920 INFO:teuthology.orchestra.run.smithi130.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2022-08-01T23:05:42.920 INFO:teuthology.orchestra.run.smithi130.stdout:Inodes: Total: 1 Free: 0 2022-08-01T23:05:42.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T23:05:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.023+0000 7f8f293df700 1 Processor -- start 2022-08-01T23:05:43.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.023+0000 7f8f293df700 1 -- start start 2022-08-01T23:05:43.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8f240b4550 0x7f8f240b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f24144240 con 0x7f8f240b6af0 2022-08-01T23:05:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f24143d00 con 0x7f8f240b6710 2022-08-01T23:05:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f24143fa0 con 0x7f8f240b4550 2022-08-01T23:05:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f237fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f240b6af0 0x7f8f2412bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37788/0 (socket says 172.21.15.32:37788) 2022-08-01T23:05:43.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f237fe700 1 -- 172.21.15.32:0/4151617801 learned_addr learned my addr 172.21.15.32:0/4151617801 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:43.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f227fc700 1 --2- 172.21.15.32:0/4151617801 >> v2:172.21.15.55:3300/0 conn(0x7f8f240b4550 0x7f8f240b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f21ffb700 1 -- 172.21.15.32:0/4151617801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232744807 0 0) 0x7f8f24144240 con 0x7f8f240b6af0 2022-08-01T23:05:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f21ffb700 1 -- 172.21.15.32:0/4151617801 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f08003260 con 0x7f8f240b6af0 2022-08-01T23:05:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f227fc700 1 -- 172.21.15.32:0/4151617801 >> v1:172.21.15.55:6789/0 conn(0x7f8f240b6710 legacy=0x7f8f24061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f21ffb700 1 -- 172.21.15.32:0/4151617801 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3035812462 0 0) 0x7f8f08003260 con 0x7f8f240b6af0 2022-08-01T23:05:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f227fc700 1 -- 172.21.15.32:0/4151617801 >> v1:172.21.15.32:6789/0 conn(0x7f8f240b6af0 legacy=0x7f8f2412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f227fc700 1 -- 172.21.15.32:0/4151617801 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f24061780 con 0x7f8f240b4550 2022-08-01T23:05:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f227fc700 1 --2- 172.21.15.32:0/4151617801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b4550 0x7f8f240b5d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f8f1c001b80 tx=0x7f8f14006df0).ready entity=mon.1 client_cookie=7ad324038c77f9be server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f21ffb700 1 -- 172.21.15.32:0/4151617801 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1c0019e0 con 0x7f8f240b4550 2022-08-01T23:05:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f21ffb700 1 -- 172.21.15.32:0/4151617801 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14000e90 con 0x7f8f240b4550 2022-08-01T23:05:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f21ffb700 1 -- 172.21.15.32:0/4151617801 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1400d4f0 con 0x7f8f240b4550 2022-08-01T23:05:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 -- 172.21.15.32:0/4151617801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b4550 msgr2=0x7f8f240b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 --2- 172.21.15.32:0/4151617801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b4550 0x7f8f240b5d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f8f1c001b80 tx=0x7f8f14006df0).stop 2022-08-01T23:05:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 -- 172.21.15.32:0/4151617801 shutdown_connections 2022-08-01T23:05:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 --2- 172.21.15.32:0/4151617801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b4550 0x7f8f240b5d80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.027+0000 7f8f293df700 1 -- 172.21.15.32:0/4151617801 >> 172.21.15.32:0/4151617801 conn(0x7f8f2405c620 msgr2=0x7f8f2405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.031+0000 7f8f293df700 1 -- 172.21.15.32:0/4151617801 shutdown_connections 2022-08-01T23:05:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.031+0000 7f8f293df700 1 -- 172.21.15.32:0/4151617801 wait complete. 2022-08-01T23:05:43.044 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.031+0000 7f8f293df700 1 Processor -- start 2022-08-01T23:05:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.031+0000 7f8f293df700 1 -- start start 2022-08-01T23:05:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b4550 0x7f8f24127620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 0x7f8f240b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b6af0 0x7f8f240b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f241568e0 con 0x7f8f240b6710 2022-08-01T23:05:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240b7640 con 0x7f8f240b6af0 2022-08-01T23:05:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240b78e0 con 0x7f8f240b4550 2022-08-01T23:05:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 0x7f8f240b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 0x7f8f240b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50250/0 (socket says 172.21.15.32:50250) 2022-08-01T23:05:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 -- 172.21.15.32:0/1933989957 learned_addr learned my addr 172.21.15.32:0/1933989957 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f22ffd700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b4550 0x7f8f24127620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f237fe700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b6af0 0x7f8f240b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 -- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b4550 msgr2=0x7f8f24127620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b4550 0x7f8f24127620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 -- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b6af0 msgr2=0x7f8f240b9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b6af0 0x7f8f240b9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 -- 172.21.15.32:0/1933989957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1400b180 con 0x7f8f240b6710 2022-08-01T23:05:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f227fc700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 0x7f8f240b1cc0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f8f1c001b80 tx=0x7f8f14014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f14016070 con 0x7f8f240b6710 2022-08-01T23:05:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14013af0 con 0x7f8f240b6710 2022-08-01T23:05:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2412b0d0 con 0x7f8f240b6710 2022-08-01T23:05:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f1400d4f0 con 0x7f8f240b6710 2022-08-01T23:05:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.035+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f241565c0 con 0x7f8f240b6710 2022-08-01T23:05:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.039+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8f1400f030 con 0x7f8f240b6710 2022-08-01T23:05:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.039+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f240b7aa0 con 0x7f8f240b6710 2022-08-01T23:05:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.039+0000 7f8f13fff700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f00053aa0 0x7f8f00055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.039+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f1406c9e0 con 0x7f8f240b6710 2022-08-01T23:05:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.039+0000 7f8f22ffd700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f00053aa0 0x7f8f00055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.039+0000 7f8f22ffd700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f00053aa0 0x7f8f00055f60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f8f1800d000 tx=0x7f8f18009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.047+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f14043690 con 0x7f8f240b6710 2022-08-01T23:05:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.175+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8f24061a20 con 0x7f8f240b6710 2022-08-01T23:05:43.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.175+0000 7f8f13fff700 1 -- 172.21.15.32:0/1933989957 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v158) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f8f14022090 con 0x7f8f240b6710 2022-08-01T23:05:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:05:43.185 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 158, 2022-08-01T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e", 2022-08-01T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2022-08-01T22:55:31.351237+0000", 2022-08-01T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2022-08-01T23:05:26.723454+0000", 2022-08-01T23:05:43.186 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2022-08-01T22:55:39.069490+0000", 2022-08-01T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2022-08-01T23:05:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2022-08-01T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2022-08-01T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2022-08-01T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2022-08-01T23:05:43.188 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2022-08-01T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2022-08-01T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T23:05:43.189 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2022-08-01T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 30, 2022-08-01T23:05:43.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2022-08-01T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "jewel", 2022-08-01T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2022-08-01T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "octopus", 2022-08-01T23:05:43.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2022-08-01T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2022-08-01T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "device_health_metrics", 2022-08-01T23:05:43.192 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:34.277887+0000", 2022-08-01T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:43.193 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2022-08-01T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2022-08-01T23:05:43.195 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2022-08-01T23:05:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2022-08-01T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2022-08-01T23:05:43.197 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:43.198 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:43.199 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:43.200 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.201 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "15", 2022-08-01T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:43.202 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:43.203 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:43.204 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:43.205 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:43.206 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:43.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:43.208 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:43.209 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:43.210 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:43.211 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:43.212 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:43.213 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.215 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:43.216 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:43.217 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2022-08-01T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.218 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2022-08-01T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.219 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2022-08-01T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2022-08-01T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T22:55:43.120854+0000", 2022-08-01T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:43.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:43.221 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2022-08-01T23:05:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:43.239 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2022-08-01T23:05:43.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 29, 2022-08-01T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2022-08-01T23:05:43.241 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:05:24.145656+0000", 2022-08-01T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:43.242 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:43.243 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:43.244 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:43.244 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:43.244 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:43.244 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:43.245 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:43.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.247 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "157", 2022-08-01T23:05:43.247 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:43.247 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:43.247 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:43.247 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:43.248 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:43.248 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:43.248 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:43.248 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:43.249 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:43.249 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:43.249 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:43.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:43.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:43.250 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:43.250 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:43.250 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:43.250 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:43.251 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:43.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:43.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:43.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:43.251 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:43.252 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:43.252 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:43.252 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:43.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:43.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.253 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:43.253 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:43.253 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:43.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:43.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:43.254 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:43.254 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:43.254 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:43.254 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:43.255 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:43.255 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:43.255 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2022-08-01T23:05:43.255 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2022-08-01T23:05:43.256 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2022-08-01T23:05:43.256 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2022-08-01T23:05:43.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.256 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:43.256 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:05:43.257 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2022-08-01T23:05:43.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.258 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 30, 2022-08-01T23:05:43.258 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2022-08-01T23:05:43.258 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2022-08-01T23:05:24.725583+0000", 2022-08-01T23:05:43.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2022-08-01T23:05:43.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2022-08-01T23:05:43.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2022-08-01T23:05:43.259 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2022-08-01T23:05:43.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2022-08-01T23:05:43.260 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2022-08-01T23:05:43.260 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2022-08-01T23:05:43.260 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2022-08-01T23:05:43.260 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2022-08-01T23:05:43.261 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2022-08-01T23:05:43.261 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2022-08-01T23:05:43.261 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2022-08-01T23:05:43.261 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2022-08-01T23:05:43.262 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2022-08-01T23:05:43.262 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2022-08-01T23:05:43.262 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2022-08-01T23:05:43.262 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2022-08-01T23:05:43.263 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2022-08-01T23:05:43.263 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2022-08-01T23:05:43.263 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2022-08-01T23:05:43.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "158", 2022-08-01T23:05:43.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2022-08-01T23:05:43.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T23:05:43.264 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T23:05:43.265 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2022-08-01T23:05:43.265 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2022-08-01T23:05:43.265 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2022-08-01T23:05:43.265 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2022-08-01T23:05:43.265 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2022-08-01T23:05:43.266 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2022-08-01T23:05:43.266 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2022-08-01T23:05:43.266 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2022-08-01T23:05:43.266 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2022-08-01T23:05:43.267 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2022-08-01T23:05:43.267 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2022-08-01T23:05:43.267 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2022-08-01T23:05:43.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2022-08-01T23:05:43.267 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2022-08-01T23:05:43.268 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2022-08-01T23:05:43.268 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T23:05:43.268 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T23:05:43.268 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T23:05:43.268 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2022-08-01T23:05:43.269 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2022-08-01T23:05:43.269 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2022-08-01T23:05:43.269 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2022-08-01T23:05:43.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2022-08-01T23:05:43.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.270 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2022-08-01T23:05:43.270 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2022-08-01T23:05:43.270 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2022-08-01T23:05:43.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2022-08-01T23:05:43.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2022-08-01T23:05:43.271 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T23:05:43.271 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2022-08-01T23:05:43.272 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2022-08-01T23:05:43.272 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2022-08-01T23:05:43.272 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2022-08-01T23:05:43.272 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2022-08-01T23:05:43.273 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2022-08-01T23:05:43.273 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2022-08-01T23:05:43.273 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2022-08-01T23:05:43.273 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2022-08-01T23:05:43.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.274 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:43.274 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2022-08-01T23:05:43.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.275 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:05:43.275 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "047a2077-8683-4670-a43a-a616da639621", 2022-08-01T23:05:43.275 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.275 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.276 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.276 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.276 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.276 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.277 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.277 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 156, 2022-08-01T23:05:43.277 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.277 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.277 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.278 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2022-08-01T23:05:43.278 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.279 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2022-08-01T23:05:43.280 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.280 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.280 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.281 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2022-08-01T23:05:43.281 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.282 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2022-08-01T23:05:43.283 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.283 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.284 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2022-08-01T23:05:43.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2022-08-01T23:05:43.286 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.287 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2022-08-01T23:05:43.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.289 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2022-08-01T23:05:43.289 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23003 2022-08-01T23:05:43.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.290 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/23003", 2022-08-01T23:05:43.290 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/23003", 2022-08-01T23:05:43.290 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/23003", 2022-08-01T23:05:43.291 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/23003", 2022-08-01T23:05:43.291 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.291 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.291 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.291 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.292 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:05:43.292 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "200572dd-9760-43f7-9568-b2c7553721b3", 2022-08-01T23:05:43.293 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.293 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.293 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.293 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.293 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.294 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.294 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.294 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 155, 2022-08-01T23:05:43.294 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.294 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.295 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.295 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.296 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2022-08-01T23:05:43.296 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.297 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2022-08-01T23:05:43.297 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.298 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.298 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.299 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2022-08-01T23:05:43.299 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.300 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2022-08-01T23:05:43.300 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.301 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.301 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.302 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2022-08-01T23:05:43.302 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.303 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2022-08-01T23:05:43.303 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.304 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.304 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.305 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2022-08-01T23:05:43.305 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.306 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2022-08-01T23:05:43.306 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23005 2022-08-01T23:05:43.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.307 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/23005", 2022-08-01T23:05:43.307 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/23005", 2022-08-01T23:05:43.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/23005", 2022-08-01T23:05:43.308 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/23005", 2022-08-01T23:05:43.308 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.308 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.309 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.310 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:05:43.310 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a8c47037-5440-477f-8684-a1f3f8fb26f4", 2022-08-01T23:05:43.310 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.310 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.311 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.311 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.311 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.311 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.312 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 156, 2022-08-01T23:05:43.312 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.312 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.312 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.312 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.313 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2022-08-01T23:05:43.313 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.314 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2022-08-01T23:05:43.314 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.315 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.315 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.316 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2022-08-01T23:05:43.316 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.317 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2022-08-01T23:05:43.317 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.318 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.318 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.319 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2022-08-01T23:05:43.319 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.320 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2022-08-01T23:05:43.320 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.321 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.322 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2022-08-01T23:05:43.323 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.324 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2022-08-01T23:05:43.324 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23004 2022-08-01T23:05:43.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.325 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/23004", 2022-08-01T23:05:43.325 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/23004", 2022-08-01T23:05:43.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/23004", 2022-08-01T23:05:43.326 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/23004", 2022-08-01T23:05:43.326 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.326 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.326 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.327 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:05:43.328 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "645afcc6-0658-4477-9722-c6e21159c5b3", 2022-08-01T23:05:43.328 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.328 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.328 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.328 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 156, 2022-08-01T23:05:43.329 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.331 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2022-08-01T23:05:43.331 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.332 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2022-08-01T23:05:43.332 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.333 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.333 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.334 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2022-08-01T23:05:43.334 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.335 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2022-08-01T23:05:43.335 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.336 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.336 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.337 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2022-08-01T23:05:43.337 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.338 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2022-08-01T23:05:43.338 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.339 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.339 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.340 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2022-08-01T23:05:43.340 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.341 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2022-08-01T23:05:43.341 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23015 2022-08-01T23:05:43.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.342 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/23015", 2022-08-01T23:05:43.343 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/23015", 2022-08-01T23:05:43.343 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/23015", 2022-08-01T23:05:43.343 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/23015", 2022-08-01T23:05:43.343 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.343 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.344 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.345 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:05:43.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "8bb02127-2e21-4432-b767-9a7559abfe67", 2022-08-01T23:05:43.345 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.345 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.345 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.346 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.347 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.347 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 155, 2022-08-01T23:05:43.347 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.347 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.347 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.348 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6819", 2022-08-01T23:05:43.349 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.349 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6821", 2022-08-01T23:05:43.350 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.350 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.351 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6823", 2022-08-01T23:05:43.352 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.352 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6825", 2022-08-01T23:05:43.353 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.353 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.354 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6831", 2022-08-01T23:05:43.355 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.355 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6833", 2022-08-01T23:05:43.356 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.356 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.356 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.357 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6827", 2022-08-01T23:05:43.357 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.359 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6829", 2022-08-01T23:05:43.359 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23335 2022-08-01T23:05:43.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.360 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6821/23335", 2022-08-01T23:05:43.360 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6825/23335", 2022-08-01T23:05:43.360 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6833/23335", 2022-08-01T23:05:43.361 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6829/23335", 2022-08-01T23:05:43.361 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.361 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.361 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.362 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.362 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:05:43.362 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d961d2aa-c6df-4770-90e1-0d2a43f360ad", 2022-08-01T23:05:43.363 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.363 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.363 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.363 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.364 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.364 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.364 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.364 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 156, 2022-08-01T23:05:43.364 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.365 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.365 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.365 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.366 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6802", 2022-08-01T23:05:43.366 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.367 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6803", 2022-08-01T23:05:43.367 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.368 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6804", 2022-08-01T23:05:43.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6805", 2022-08-01T23:05:43.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.371 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6808", 2022-08-01T23:05:43.372 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6809", 2022-08-01T23:05:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.374 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.374 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6806", 2022-08-01T23:05:43.375 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6807", 2022-08-01T23:05:43.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23336 2022-08-01T23:05:43.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.377 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6803/23336", 2022-08-01T23:05:43.377 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6805/23336", 2022-08-01T23:05:43.377 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6809/23336", 2022-08-01T23:05:43.378 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6807/23336", 2022-08-01T23:05:43.378 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.378 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.378 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.379 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:05:43.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "162a1876-d2c7-4886-9434-b4114d80e92b", 2022-08-01T23:05:43.380 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.380 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.380 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.380 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.380 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.381 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.381 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.381 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 156, 2022-08-01T23:05:43.381 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.382 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.382 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6810", 2022-08-01T23:05:43.383 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.384 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6811", 2022-08-01T23:05:43.384 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.385 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.386 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.386 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6812", 2022-08-01T23:05:43.387 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6813", 2022-08-01T23:05:43.388 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.389 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.390 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6816", 2022-08-01T23:05:43.390 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.391 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6817", 2022-08-01T23:05:43.391 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.392 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.392 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.393 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6814", 2022-08-01T23:05:43.393 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.394 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6815", 2022-08-01T23:05:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23334 2022-08-01T23:05:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.396 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6811/23334", 2022-08-01T23:05:43.396 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6813/23334", 2022-08-01T23:05:43.396 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6817/23334", 2022-08-01T23:05:43.396 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6815/23334", 2022-08-01T23:05:43.397 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.397 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.397 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.398 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:05:43.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "d1c950ea-35ee-4ac6-8e31-e832328f349f", 2022-08-01T23:05:43.398 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2022-08-01T23:05:43.399 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2022-08-01T23:05:43.399 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2022-08-01T23:05:43.399 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2022-08-01T23:05:43.399 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2022-08-01T23:05:43.399 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2022-08-01T23:05:43.400 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 13, 2022-08-01T23:05:43.400 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 156, 2022-08-01T23:05:43.400 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2022-08-01T23:05:43.400 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2022-08-01T23:05:43.400 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2022-08-01T23:05:43.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6818", 2022-08-01T23:05:43.401 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.402 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6820", 2022-08-01T23:05:43.403 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.403 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.403 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2022-08-01T23:05:43.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6822", 2022-08-01T23:05:43.404 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.405 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6824", 2022-08-01T23:05:43.406 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.407 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2022-08-01T23:05:43.407 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.408 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6830", 2022-08-01T23:05:43.408 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.409 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6832", 2022-08-01T23:05:43.409 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.410 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2022-08-01T23:05:43.410 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2022-08-01T23:05:43.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2022-08-01T23:05:43.411 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6826", 2022-08-01T23:05:43.412 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2022-08-01T23:05:43.412 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.55:6828", 2022-08-01T23:05:43.413 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 23359 2022-08-01T23:05:43.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.413 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.55:6820/23359", 2022-08-01T23:05:43.414 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.55:6824/23359", 2022-08-01T23:05:43.414 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.55:6832/23359", 2022-08-01T23:05:43.414 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.55:6828/23359", 2022-08-01T23:05:43.414 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2022-08-01T23:05:43.414 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2022-08-01T23:05:43.415 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2022-08-01T23:05:43.415 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2022-08-01T23:05:43.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.415 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:43.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2022-08-01T23:05:43.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.416 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2022-08-01T23:05:43.416 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.416 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.417 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.417 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.417 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.417 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.411086+0000", 2022-08-01T23:05:43.417 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2022-08-01T23:05:43.418 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.419 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.419 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.419 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.419 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.419 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.153541+0000", 2022-08-01T23:05:43.420 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.420 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2022-08-01T23:05:43.421 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.421 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.421 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.422 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.422 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.422 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.354950+0000", 2022-08-01T23:05:43.422 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.423 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2022-08-01T23:05:43.423 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.423 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.424 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.424 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.424 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.424 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.423072+0000", 2022-08-01T23:05:43.425 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.425 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2022-08-01T23:05:43.426 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.426 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.426 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.426 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.426 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.427 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.377079+0000", 2022-08-01T23:05:43.427 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2022-08-01T23:05:43.428 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.428 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.428 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.428 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.429 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.429 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.170426+0000", 2022-08-01T23:05:43.429 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2022-08-01T23:05:43.430 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.430 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.430 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.431 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.431 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.431 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.325899+0000", 2022-08-01T23:05:43.431 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2022-08-01T23:05:43.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2022-08-01T23:05:43.432 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2022-08-01T23:05:43.432 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2022-08-01T23:05:43.433 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2022-08-01T23:05:43.433 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138292840890367, 2022-08-01T23:05:43.433 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2022-08-01T23:05:43.434 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2022-08-01T22:55:37.406820+0000", 2022-08-01T23:05:43.434 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2022-08-01T23:05:43.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.434 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2022-08-01T23:05:43.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2022-08-01T23:05:43.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2022-08-01T23:05:43.435 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2022-08-01T23:05:43.435 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2022-08-01T23:05:43.435 INFO:teuthology.orchestra.run.smithi032.stdout: "blacklist": {}, 2022-08-01T23:05:43.436 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2022-08-01T23:05:43.436 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_data": { 2022-08-01T23:05:43.436 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-device-class": "", 2022-08-01T23:05:43.436 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:05:43.436 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-root": "default", 2022-08-01T23:05:43.437 INFO:teuthology.orchestra.run.smithi032.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T23:05:43.437 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:05:43.437 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "2", 2022-08-01T23:05:43.437 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:05:43.437 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:05:43.438 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van", 2022-08-01T23:05:43.438 INFO:teuthology.orchestra.run.smithi032.stdout: "w": "8" 2022-08-01T23:05:43.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2022-08-01T23:05:43.439 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2022-08-01T23:05:43.439 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2022-08-01T23:05:43.439 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2022-08-01T23:05:43.439 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2022-08-01T23:05:43.439 INFO:teuthology.orchestra.run.smithi032.stdout: "ruleset-failure-domain": "osd", 2022-08-01T23:05:43.440 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2022-08-01T23:05:43.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2022-08-01T23:05:43.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2022-08-01T23:05:43.440 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2022-08-01T23:05:43.440 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2022-08-01T23:05:43.441 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2022-08-01T23:05:43.441 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2022-08-01T23:05:43.441 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {} 2022-08-01T23:05:43.441 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:05:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f00053aa0 msgr2=0x7f8f00055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f00053aa0 0x7f8f00055f60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f8f1800d000 tx=0x7f8f18009040).stop 2022-08-01T23:05:43.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 msgr2=0x7f8f240b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 0x7f8f240b1cc0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7f8f1c001b80 tx=0x7f8f14014000).stop 2022-08-01T23:05:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 shutdown_connections 2022-08-01T23:05:43.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f8f00053aa0 0x7f8f00055f60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f240b6af0 0x7f8f240b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f240b6710 0x7f8f240b1cc0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 --2- 172.21.15.32:0/1933989957 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f8f240b4550 0x7f8f24127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 >> 172.21.15.32:0/1933989957 conn(0x7f8f2405c620 msgr2=0x7f8f24061f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:43.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 shutdown_connections 2022-08-01T23:05:43.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.179+0000 7f8f293df700 1 -- 172.21.15.32:0/1933989957 wait complete. 2022-08-01T23:05:43.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 Processor -- start 2022-08-01T23:05:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 -- start start 2022-08-01T23:05:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e000b6a90 0x7f9e000acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e00060aa0 0x7f9e000aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e000b6e70 con 0x7f9e000aa540 2022-08-01T23:05:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e00061370 con 0x7f9e00060aa0 2022-08-01T23:05:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9e05b9c700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e00061610 con 0x7f9e000b6a90 2022-08-01T23:05:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9dfeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e00060aa0 0x7f9e000aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9dfffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e000aa540 0x7f9e000b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37802/0 (socket says 172.21.15.32:37802) 2022-08-01T23:05:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9dfffff700 1 -- 172.21.15.32:0/830096799 learned_addr learned my addr 172.21.15.32:0/830096799 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.551+0000 7f9dff7fe700 1 --2- 172.21.15.32:0/830096799 >> v2:172.21.15.55:3300/0 conn(0x7f9e000b6a90 0x7f9e000acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/830096799 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39996778 0 0) 0x7f9e000b6e70 con 0x7f9e000aa540 2022-08-01T23:05:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/830096799 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec003260 con 0x7f9e000aa540 2022-08-01T23:05:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfeffd700 1 -- 172.21.15.32:0/830096799 >> v2:172.21.15.55:3300/0 conn(0x7f9e000b6a90 msgr2=0x7f9e000acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfeffd700 1 --2- 172.21.15.32:0/830096799 >> v2:172.21.15.55:3300/0 conn(0x7f9e000b6a90 0x7f9e000acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfeffd700 1 -- 172.21.15.32:0/830096799 >> v1:172.21.15.32:6789/0 conn(0x7f9e000aa540 legacy=0x7f9e000b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfeffd700 1 -- 172.21.15.32:0/830096799 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000618b0 con 0x7f9e00060aa0 2022-08-01T23:05:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dff7fe700 1 --2- 172.21.15.32:0/830096799 >> v2:172.21.15.55:3300/0 conn(0x7f9e000b6a90 0x7f9e000acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfeffd700 1 --2- 172.21.15.32:0/830096799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e00060aa0 0x7f9e000aa050 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f9df4002f70 tx=0x7f9df4006df0).ready entity=mon.0 client_cookie=7f3b928bc337cb39 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/830096799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df400d570 con 0x7f9e00060aa0 2022-08-01T23:05:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/830096799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4000e90 con 0x7f9e00060aa0 2022-08-01T23:05:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/830096799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df400d8c0 con 0x7f9e00060aa0 2022-08-01T23:05:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 -- 172.21.15.32:0/830096799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e00060aa0 msgr2=0x7f9e000aa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/830096799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e00060aa0 0x7f9e000aa050 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f9df4002f70 tx=0x7f9df4006df0).stop 2022-08-01T23:05:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 -- 172.21.15.32:0/830096799 shutdown_connections 2022-08-01T23:05:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/830096799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e00060aa0 0x7f9e000aa050 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/830096799 >> v2:172.21.15.55:3300/0 conn(0x7f9e000b6a90 0x7f9e000acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 -- 172.21.15.32:0/830096799 >> 172.21.15.32:0/830096799 conn(0x7f9e000b3610 msgr2=0x7f9e000b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 -- 172.21.15.32:0/830096799 shutdown_connections 2022-08-01T23:05:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.555+0000 7f9e05b9c700 1 -- 172.21.15.32:0/830096799 wait complete. 2022-08-01T23:05:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 Processor -- start 2022-08-01T23:05:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 -- start start 2022-08-01T23:05:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e00060aa0 0x7f9e0013c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e000aa540 0x7f9e0014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 0x7f9e0014f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001527c0 con 0x7f9e000b6a90 2022-08-01T23:05:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001529f0 con 0x7f9e00060aa0 2022-08-01T23:05:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9e05b9c700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00152c90 con 0x7f9e000aa540 2022-08-01T23:05:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 0x7f9e0014f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 0x7f9e0014f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50266/0 (socket says 172.21.15.32:50266) 2022-08-01T23:05:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 -- 172.21.15.32:0/1563000452 learned_addr learned my addr 172.21.15.32:0/1563000452 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dff7fe700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e00060aa0 0x7f9e0013c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfeffd700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e000aa540 0x7f9e0014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 -- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e000aa540 msgr2=0x7f9e0014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e000aa540 0x7f9e0014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 -- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e00060aa0 msgr2=0x7f9e0013c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e00060aa0 0x7f9e0013c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 -- 172.21.15.32:0/1563000452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df400b180 con 0x7f9e000b6a90 2022-08-01T23:05:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfffff700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 0x7f9e0014f6e0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f9df8001490 tx=0x7f9df8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df8005ba0 con 0x7f9e000b6a90 2022-08-01T23:05:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.559+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8005d00 con 0x7f9e000b6a90 2022-08-01T23:05:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e00152f30 con 0x7f9e000b6a90 2022-08-01T23:05:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df80072c0 con 0x7f9e000b6a90 2022-08-01T23:05:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0013ba20 con 0x7f9e000b6a90 2022-08-01T23:05:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e001416f0 con 0x7f9e000b6a90 2022-08-01T23:05:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9df8007420 con 0x7f9e000b6a90 2022-08-01T23:05:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9dfcff9700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9de0053aa0 0x7f9de0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9df806dd90 con 0x7f9e000b6a90 2022-08-01T23:05:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9dff7fe700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9de0053aa0 0x7f9de0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.563+0000 7f9dff7fe700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9de0053aa0 0x7f9de0055f60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f9df000c000 tx=0x7f9df0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.567+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e001416f0 con 0x7f9e000b6a90 2022-08-01T23:05:43.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e001416f0 con 0x7f9e000b6a90 2022-08-01T23:05:43.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9dfcff9700 1 -- 172.21.15.32:0/1563000452 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f9e001416f0 con 0x7f9e000b6a90 2022-08-01T23:05:43.738 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:43.738 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1983320963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1983320963},{"type":"v1","addr":"172.21.15.55:6835","nonce":1983320963}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78},{"gid":8754,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3801584266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3801584266},{"type":"v1","addr":"172.21.15.32:6833","nonce":3801584266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:25.720657+0000","modified":"2022-08-01T23:05:27.400282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8757":{"gid":8757,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/324530820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":324530820},{"type":"v1","addr":"172.21.15.32:6835","nonce":324530820}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T23:05:43.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9de0053aa0 msgr2=0x7f9de0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9de0053aa0 0x7f9de0055f60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f9df000c000 tx=0x7f9df0008040).stop 2022-08-01T23:05:43.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 msgr2=0x7f9e0014f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 0x7f9e0014f6e0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f9df8001490 tx=0x7f9df8007dc0).stop 2022-08-01T23:05:43.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 shutdown_connections 2022-08-01T23:05:43.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9de0053aa0 0x7f9de0055f60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e000b6a90 0x7f9e0014f6e0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9e000aa540 0x7f9e0014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 --2- 172.21.15.32:0/1563000452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e00060aa0 0x7f9e0013c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 >> 172.21.15.32:0/1563000452 conn(0x7f9e000b3610 msgr2=0x7f9e00137120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:43.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 shutdown_connections 2022-08-01T23:05:43.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.731+0000 7f9e05b9c700 1 -- 172.21.15.32:0/1563000452 wait complete. 2022-08-01T23:05:43.743 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2022-08-01T23:05:43.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T23:05:43.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d88b0700 1 Processor -- start 2022-08-01T23:05:43.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d88b0700 1 -- start start 2022-08-01T23:05:43.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d88b0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b07a0 0x7f39d40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d88b0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d40b5a10 con 0x7f39d4147730 2022-08-01T23:05:43.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d88b0700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d40ac990 con 0x7f39d40b5630 2022-08-01T23:05:43.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d88b0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4145470 con 0x7f39d40b07a0 2022-08-01T23:05:43.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d359e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39d4147730 0x7f39d40acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37816/0 (socket says 172.21.15.32:37816) 2022-08-01T23:05:43.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d359e700 1 -- 172.21.15.32:0/411444888 learned_addr learned my addr 172.21.15.32:0/411444888 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:43.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d259c700 1 --2- 172.21.15.32:0/411444888 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b07a0 0x7f39d40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d1d9b700 1 -- 172.21.15.32:0/411444888 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951551734 0 0) 0x7f39d40b5a10 con 0x7f39d4147730 2022-08-01T23:05:43.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d1d9b700 1 -- 172.21.15.32:0/411444888 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b4003260 con 0x7f39d4147730 2022-08-01T23:05:43.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d259c700 1 -- 172.21.15.32:0/411444888 >> v1:172.21.15.55:6789/0 conn(0x7f39d40b5630 legacy=0x7f39d4067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d1d9b700 1 -- 172.21.15.32:0/411444888 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809589951 0 0) 0x7f39d40ac990 con 0x7f39d40b5630 2022-08-01T23:05:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d259c700 1 -- 172.21.15.32:0/411444888 >> v1:172.21.15.32:6789/0 conn(0x7f39d4147730 legacy=0x7f39d40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d259c700 1 -- 172.21.15.32:0/411444888 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40b1bc0 con 0x7f39d40b07a0 2022-08-01T23:05:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d259c700 1 --2- 172.21.15.32:0/411444888 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b07a0 0x7f39d40af6e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f39c40032f0 tx=0x7f39c40091d0).ready entity=mon.2 client_cookie=583bcdc99b98f9d9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d1d9b700 1 -- 172.21.15.32:0/411444888 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c400b6c0 con 0x7f39d40b07a0 2022-08-01T23:05:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d1d9b700 1 -- 172.21.15.32:0/411444888 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c400b820 con 0x7f39d40b07a0 2022-08-01T23:05:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.847+0000 7f39d1d9b700 1 -- 172.21.15.32:0/411444888 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c400b990 con 0x7f39d40b07a0 2022-08-01T23:05:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- 172.21.15.32:0/411444888 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b07a0 msgr2=0x7f39d40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 --2- 172.21.15.32:0/411444888 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b07a0 0x7f39d40af6e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f39c40032f0 tx=0x7f39c40091d0).stop 2022-08-01T23:05:43.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- 172.21.15.32:0/411444888 shutdown_connections 2022-08-01T23:05:43.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 --2- 172.21.15.32:0/411444888 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b07a0 0x7f39d40af6e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- 172.21.15.32:0/411444888 >> 172.21.15.32:0/411444888 conn(0x7f39d40626f0 msgr2=0x7f39d4062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:43.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- 172.21.15.32:0/411444888 shutdown_connections 2022-08-01T23:05:43.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- 172.21.15.32:0/411444888 wait complete. 2022-08-01T23:05:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 Processor -- start 2022-08-01T23:05:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- start start 2022-08-01T23:05:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 0x7f39d4149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b5630 0x7f39d4142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39d4147730 0x7f39d4142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40ae180 con 0x7f39d40b07a0 2022-08-01T23:05:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4063780 con 0x7f39d4147730 2022-08-01T23:05:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d88b0700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40b48d0 con 0x7f39d40b5630 2022-08-01T23:05:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 0x7f39d4149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 0x7f39d4149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50278/0 (socket says 172.21.15.32:50278) 2022-08-01T23:05:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 -- 172.21.15.32:0/3546630925 learned_addr learned my addr 172.21.15.32:0/3546630925 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d259c700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b5630 0x7f39d4142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d359e700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39d4147730 0x7f39d4142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 -- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b5630 msgr2=0x7f39d4142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b5630 0x7f39d4142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 -- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39d4147730 msgr2=0x7f39d4142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:43.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39d4147730 0x7f39d4142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:43.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d2d9d700 1 -- 172.21.15.32:0/3546630925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009470 con 0x7f39d40b07a0 2022-08-01T23:05:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.851+0000 7f39d359e700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39d4147730 0x7f39d4142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39d2d9d700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 0x7f39d4149b30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f39bc000ed0 tx=0x7f39bc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39bc012400 con 0x7f39d40b07a0 2022-08-01T23:05:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d4143100 con 0x7f39d40b07a0 2022-08-01T23:05:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc012560 con 0x7f39d40b07a0 2022-08-01T23:05:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39bc0128c0 con 0x7f39d40b07a0 2022-08-01T23:05:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40b50e0 con 0x7f39d40b07a0 2022-08-01T23:05:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d40b1680 con 0x7f39d40b07a0 2022-08-01T23:05:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f39bc019030 con 0x7f39d40b07a0 2022-08-01T23:05:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39cbfff700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39b8053b00 0x7f39b8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.855+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f39bc06b3f0 con 0x7f39d40b07a0 2022-08-01T23:05:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.859+0000 7f39d259c700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39b8053b00 0x7f39b8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.859+0000 7f39d259c700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39b8053b00 0x7f39b8055fc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f39c401b000 tx=0x7f39c4016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:43.867+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39bc042160 con 0x7f39d40b07a0 2022-08-01T23:05:44.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.027+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f39d40574e0 con 0x7f39d40b07a0 2022-08-01T23:05:44.475 INFO:tasks.ceph.mon.a.smithi032.stderr:2022-08-01T23:05:44.471+0000 7f9828561700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T23:05:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.475+0000 7f39cbfff700 1 -- 172.21.15.32:0/3546630925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f39bc045780 con 0x7f39d40b07a0 2022-08-01T23:05:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39b8053b00 msgr2=0x7f39b8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39b8053b00 0x7f39b8055fc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f39c401b000 tx=0x7f39c4016000).stop 2022-08-01T23:05:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 msgr2=0x7f39d4149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 0x7f39d4149b30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f39bc000ed0 tx=0x7f39bc007ac0).stop 2022-08-01T23:05:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 shutdown_connections 2022-08-01T23:05:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f39b8053b00 0x7f39b8055fc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39d4147730 0x7f39d4142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f39d40b5630 0x7f39d4142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 --2- 172.21.15.32:0/3546630925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d40b07a0 0x7f39d4149b30 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 >> 172.21.15.32:0/3546630925 conn(0x7f39d40626f0 msgr2=0x7f39d40acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 shutdown_connections 2022-08-01T23:05:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.479+0000 7f39d88b0700 1 -- 172.21.15.32:0/3546630925 wait complete. 2022-08-01T23:05:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T23:05:44.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T23:05:44.509 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2022-08-01T23:05:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.595+0000 7fbcb359e700 1 Processor -- start 2022-08-01T23:05:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.595+0000 7fbcb359e700 1 -- start start 2022-08-01T23:05:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcb40b3260 0x7fbcb41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbcb40ad000 0x7fbcb413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb4143ab0 con 0x7fbcb40b08a0 2022-08-01T23:05:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40b0c80 con 0x7fbcb40b3260 2022-08-01T23:05:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40ae810 con 0x7fbcb40ad000 2022-08-01T23:05:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcb40b3260 0x7fbcb41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbcb40b08a0 0x7fbcb4142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37838/0 (socket says 172.21.15.32:37838) 2022-08-01T23:05:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcb40b3260 0x7fbcb41435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50298/0 (socket says 172.21.15.32:50298) 2022-08-01T23:05:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb259c700 1 -- 172.21.15.32:0/3080279988 learned_addr learned my addr 172.21.15.32:0/3080279988 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb2d9d700 1 --2- 172.21.15.32:0/3080279988 >> v2:172.21.15.55:3300/0 conn(0x7fbcb40ad000 0x7fbcb413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 -- 172.21.15.32:0/3080279988 >> v2:172.21.15.55:3300/0 conn(0x7fbcb40ad000 msgr2=0x7fbcb413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 --2- 172.21.15.32:0/3080279988 >> v2:172.21.15.55:3300/0 conn(0x7fbcb40ad000 0x7fbcb413fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 -- 172.21.15.32:0/3080279988 >> v1:172.21.15.32:6789/0 conn(0x7fbcb40b08a0 legacy=0x7fbcb4142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 -- 172.21.15.32:0/3080279988 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40aeab0 con 0x7fbcb40b3260 2022-08-01T23:05:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb1d9b700 1 --2- 172.21.15.32:0/3080279988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb41435c0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7fbca8002f70 tx=0x7fbca8006df0).ready entity=mon.0 client_cookie=59b70baf19e591db server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb159a700 1 -- 172.21.15.32:0/3080279988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca800d5c0 con 0x7fbcb40b3260 2022-08-01T23:05:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb159a700 1 -- 172.21.15.32:0/3080279988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8000e90 con 0x7fbcb40b3260 2022-08-01T23:05:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb159a700 1 -- 172.21.15.32:0/3080279988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca800d910 con 0x7fbcb40b3260 2022-08-01T23:05:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 -- 172.21.15.32:0/3080279988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 msgr2=0x7fbcb41435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.599+0000 7fbcb359e700 1 --2- 172.21.15.32:0/3080279988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb41435c0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7fbca8002f70 tx=0x7fbca8006df0).stop 2022-08-01T23:05:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- 172.21.15.32:0/3080279988 shutdown_connections 2022-08-01T23:05:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 --2- 172.21.15.32:0/3080279988 >> v2:172.21.15.55:3300/0 conn(0x7fbcb40ad000 0x7fbcb413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 --2- 172.21.15.32:0/3080279988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb41435c0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- 172.21.15.32:0/3080279988 >> 172.21.15.32:0/3080279988 conn(0x7fbcb40612d0 msgr2=0x7fbcb4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- 172.21.15.32:0/3080279988 shutdown_connections 2022-08-01T23:05:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- 172.21.15.32:0/3080279988 wait complete. 2022-08-01T23:05:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 Processor -- start 2022-08-01T23:05:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- start start 2022-08-01T23:05:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcb40ad000 0x7fbcb4064980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcb40b08a0 0x7fbcb4152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb413e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40a9b30 con 0x7fbcb40b3260 2022-08-01T23:05:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb4065770 con 0x7fbcb40b08a0 2022-08-01T23:05:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb359e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb4065270 con 0x7fbcb40ad000 2022-08-01T23:05:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb413e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb413e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50306/0 (socket says 172.21.15.32:50306) 2022-08-01T23:05:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 -- 172.21.15.32:0/1538434138 learned_addr learned my addr 172.21.15.32:0/1538434138 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb259c700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcb40ad000 0x7fbcb4064980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb1d9b700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcb40b08a0 0x7fbcb4152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 -- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcb40ad000 msgr2=0x7fbcb4064980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcb40ad000 0x7fbcb4064980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 -- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcb40b08a0 msgr2=0x7fbcb4152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcb40b08a0 0x7fbcb4152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 -- 172.21.15.32:0/1538434138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca800b180 con 0x7fbcb40b3260 2022-08-01T23:05:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb259c700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcb40ad000 0x7fbcb4064980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbcb2d9d700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb413e210 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7fbcac00a000 tx=0x7fbcac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcac011780 con 0x7fbcb40b3260 2022-08-01T23:05:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.603+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac0118e0 con 0x7fbcb40b3260 2022-08-01T23:05:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb41534b0 con 0x7fbcb40b3260 2022-08-01T23:05:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcac011c40 con 0x7fbcb40b3260 2022-08-01T23:05:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb4065f40 con 0x7fbcb40b3260 2022-08-01T23:05:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb40574e0 con 0x7fbcb40b3260 2022-08-01T23:05:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbcac012420 con 0x7fbcb40b3260 2022-08-01T23:05:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbca37fe700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbc90053980 0x7fbc90055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fbcac06bef0 con 0x7fbcb40b3260 2022-08-01T23:05:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbcb259c700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbc90053980 0x7fbc90055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.607+0000 7fbcb259c700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbc90053980 0x7fbc90055e40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fbca400b000 tx=0x7fbca4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.611+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbcac042c00 con 0x7fbcb40b3260 2022-08-01T23:05:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.775+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb40b2e00 con 0x7fbcb40b3260 2022-08-01T23:05:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbca37fe700 1 -- 172.21.15.32:0/1538434138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fbcac046220 con 0x7fbcb40b3260 2022-08-01T23:05:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:05:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1983320963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1983320963},{"type":"v1","addr":"172.21.15.55:6835","nonce":1983320963}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78},{"gid":8754,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/3801584266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3801584266},{"type":"v1","addr":"172.21.15.32:6833","nonce":3801584266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T23:05:25.720657+0000","modified":"2022-08-01T23:05:44.473020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":159,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T23:05:44.786 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbc90053980 msgr2=0x7fbc90055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbc90053980 0x7fbc90055e40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fbca400b000 tx=0x7fbca4009040).stop 2022-08-01T23:05:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 msgr2=0x7fbcb413e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.787 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb413e210 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7fbcac00a000 tx=0x7fbcac008040).stop 2022-08-01T23:05:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 shutdown_connections 2022-08-01T23:05:44.788 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fbc90053980 0x7fbc90055e40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcb40b3260 0x7fbcb413e210 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcb40b08a0 0x7fbcb4152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 --2- 172.21.15.32:0/1538434138 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fbcb40ad000 0x7fbcb4064980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.789 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 >> 172.21.15.32:0/1538434138 conn(0x7fbcb40612d0 msgr2=0x7fbcb40ac490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 shutdown_connections 2022-08-01T23:05:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.779+0000 7fbcb359e700 1 -- 172.21.15.32:0/1538434138 wait complete. 2022-08-01T23:05:44.790 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 82 2022-08-01T23:05:44.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T23:05:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3b59e700 1 Processor -- start 2022-08-01T23:05:44.950 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3b59e700 1 -- start start 2022-08-01T23:05:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9b3c0b13f0 0x7f9b3c0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0adf10 con 0x7f9b3c0b1920 2022-08-01T23:05:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0ae1b0 con 0x7f9b3c0b2710 2022-08-01T23:05:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0a9520 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b3c0b1920 0x7f9b3c14a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37856/0 (socket says 172.21.15.32:37856) 2022-08-01T23:05:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b3ad9d700 1 -- 172.21.15.32:0/177001234 learned_addr learned my addr 172.21.15.32:0/177001234 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b31d9b700 1 --2- 172.21.15.32:0/177001234 >> v2:172.21.15.55:3300/0 conn(0x7f9b3c0b13f0 0x7f9b3c0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427177080 0 0) 0x7f9b3c0adf10 con 0x7f9b3c0b1920 2022-08-01T23:05:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b20003260 con 0x7f9b3c0b1920 2022-08-01T23:05:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212526960 0 0) 0x7f9b3c0ae1b0 con 0x7f9b3c0b2710 2022-08-01T23:05:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b3c0adf10 con 0x7f9b3c0b2710 2022-08-01T23:05:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3593823599 0 0) 0x7f9b20003260 con 0x7f9b3c0b1920 2022-08-01T23:05:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b31d9b700 1 -- 172.21.15.32:0/177001234 >> v1:172.21.15.55:6789/0 conn(0x7f9b3c0b2710 legacy=0x7f9b3c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b31d9b700 1 -- 172.21.15.32:0/177001234 >> v1:172.21.15.32:6789/0 conn(0x7f9b3c0b1920 legacy=0x7f9b3c14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.943+0000 7f9b31d9b700 1 -- 172.21.15.32:0/177001234 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0a99c0 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b31d9b700 1 --2- 172.21.15.32:0/177001234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b13f0 0x7f9b3c0afd60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f9b34001490 tx=0x7f9b24006df0).ready entity=mon.1 client_cookie=70b04c6fca152436 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b340019e0 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b24000e90 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b39d9b700 1 -- 172.21.15.32:0/177001234 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b24012460 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 -- 172.21.15.32:0/177001234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b13f0 msgr2=0x7f9b3c0afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/177001234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b13f0 0x7f9b3c0afd60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f9b34001490 tx=0x7f9b24006df0).stop 2022-08-01T23:05:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 -- 172.21.15.32:0/177001234 shutdown_connections 2022-08-01T23:05:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/177001234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b13f0 0x7f9b3c0afd60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 -- 172.21.15.32:0/177001234 >> 172.21.15.32:0/177001234 conn(0x7f9b3c060310 msgr2=0x7f9b3c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 -- 172.21.15.32:0/177001234 shutdown_connections 2022-08-01T23:05:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 -- 172.21.15.32:0/177001234 wait complete. 2022-08-01T23:05:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 Processor -- start 2022-08-01T23:05:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.947+0000 7f9b3b59e700 1 -- start start 2022-08-01T23:05:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b3c0b13f0 0x7f9b3c0634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 0x7f9b3c066c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b2710 0x7f9b3c067110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c141e20 con 0x7f9b3c0b1920 2022-08-01T23:05:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c143000 con 0x7f9b3c0b2710 2022-08-01T23:05:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c141b80 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b31d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 0x7f9b3c066c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b3c0b13f0 0x7f9b3c0634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b31d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 0x7f9b3c066c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50318/0 (socket says 172.21.15.32:50318) 2022-08-01T23:05:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b31d9b700 1 -- 172.21.15.32:0/4153830003 learned_addr learned my addr 172.21.15.32:0/4153830003 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3ad9d700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b2710 0x7f9b3c067110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 -- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b2710 msgr2=0x7f9b3c067110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b2710 0x7f9b3c067110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 -- 172.21.15.32:0/4153830003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 msgr2=0x7f9b3c066c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 0x7f9b3c066c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 -- 172.21.15.32:0/4153830003 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2400b180 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b31d9b700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 0x7f9b3c066c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:05:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3a59c700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b3c0b13f0 0x7f9b3c0634d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9b2c001110 tx=0x7f9b2c007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b2c00cd60 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c012400 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c143460 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b2c012760 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.951+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c142940 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.955+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c141830 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.955+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b2c019030 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.955+0000 7f9b33fff700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053950 0x7f9b14055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.955+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9b2c06b400 con 0x7f9b3c0b13f0 2022-08-01T23:05:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.955+0000 7f9b31d9b700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053950 0x7f9b14055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.955+0000 7f9b31d9b700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053950 0x7f9b14055e10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f9b2401a000 tx=0x7f9b24014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:44.963+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b2c042060 con 0x7f9b3c0b13f0 2022-08-01T23:05:45.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:45.127+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9b3c0574e0 con 0x7f9b3c0b13f0 2022-08-01T23:05:46.047 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.039+0000 7f9b33fff700 1 -- 172.21.15.32:0/4153830003 <== mon.2 v2:172.21.15.55:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9b2c045680 con 0x7f9b3c0b13f0 2022-08-01T23:05:46.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053950 msgr2=0x7f9b14055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:46.051 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053950 0x7f9b14055e10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f9b2401a000 tx=0x7f9b24014000).stop 2022-08-01T23:05:46.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b3c0b13f0 msgr2=0x7f9b3c0634d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:46.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b3c0b13f0 0x7f9b3c0634d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9b2c001110 tx=0x7f9b2c007a20).stop 2022-08-01T23:05:46.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 shutdown_connections 2022-08-01T23:05:46.052 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9b14053950 0x7f9b14055e10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b3c0b2710 0x7f9b3c067110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b3c0b1920 0x7f9b3c066c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 --2- 172.21.15.32:0/4153830003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9b3c0b13f0 0x7f9b3c0634d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.053 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 >> 172.21.15.32:0/4153830003 conn(0x7f9b3c060310 msgr2=0x7f9b3c144db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:46.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 shutdown_connections 2022-08-01T23:05:46.054 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.043+0000 7f9b3b59e700 1 -- 172.21.15.32:0/4153830003 wait complete. 2022-08-01T23:05:46.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T23:05:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.179+0000 7f61bb27f700 1 Processor -- start 2022-08-01T23:05:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.179+0000 7f61bb27f700 1 -- start start 2022-08-01T23:05:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f61b40b0d10 0x7f61b4146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61b40b0560 0x7f61b4140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b40b12d0 con 0x7f61b40a9320 2022-08-01T23:05:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40a9700 con 0x7f61b40b0560 2022-08-01T23:05:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40b1500 con 0x7f61b40b0d10 2022-08-01T23:05:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61b40b0560 0x7f61b4140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61b40b0560 0x7f61b4140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50326/0 (socket says 172.21.15.32:50326) 2022-08-01T23:05:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61ba27d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f61b40a9320 0x7f61b41466e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37866/0 (socket says 172.21.15.32:37866) 2022-08-01T23:05:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 -- 172.21.15.32:0/3298518566 learned_addr learned my addr 172.21.15.32:0/3298518566 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61b9a7c700 1 --2- 172.21.15.32:0/3298518566 >> v2:172.21.15.55:3300/0 conn(0x7f61b40b0d10 0x7f61b4146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 -- 172.21.15.32:0/3298518566 >> v2:172.21.15.55:3300/0 conn(0x7f61b40b0d10 msgr2=0x7f61b4146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:05:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 --2- 172.21.15.32:0/3298518566 >> v2:172.21.15.55:3300/0 conn(0x7f61b40b0d10 0x7f61b4146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 -- 172.21.15.32:0/3298518566 >> v1:172.21.15.32:6789/0 conn(0x7f61b40a9320 legacy=0x7f61b41466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 -- 172.21.15.32:0/3298518566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40b17a0 con 0x7f61b40b0560 2022-08-01T23:05:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61b927b700 1 -- 172.21.15.32:0/3298518566 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977430827 0 0) 0x7f61b40b12d0 con 0x7f61b40a9320 2022-08-01T23:05:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61baa7e700 1 --2- 172.21.15.32:0/3298518566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0560 0x7f61b4140350 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f61ac0032f0 tx=0x7f61ac009290).ready entity=mon.0 client_cookie=2811cfc0b2763b8b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61b927b700 1 -- 172.21.15.32:0/3298518566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61ac00b870 con 0x7f61b40b0560 2022-08-01T23:05:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61b927b700 1 -- 172.21.15.32:0/3298518566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac00b9d0 con 0x7f61b40b0560 2022-08-01T23:05:46.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61b927b700 1 -- 172.21.15.32:0/3298518566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61ac00bb40 con 0x7f61b40b0560 2022-08-01T23:05:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- 172.21.15.32:0/3298518566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0560 msgr2=0x7f61b4140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 --2- 172.21.15.32:0/3298518566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0560 0x7f61b4140350 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f61ac0032f0 tx=0x7f61ac009290).stop 2022-08-01T23:05:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- 172.21.15.32:0/3298518566 shutdown_connections 2022-08-01T23:05:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 --2- 172.21.15.32:0/3298518566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0560 0x7f61b4140350 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 --2- 172.21.15.32:0/3298518566 >> v2:172.21.15.55:3300/0 conn(0x7f61b40b0d10 0x7f61b4146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- 172.21.15.32:0/3298518566 >> 172.21.15.32:0/3298518566 conn(0x7f61b40612d0 msgr2=0x7f61b4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- 172.21.15.32:0/3298518566 shutdown_connections 2022-08-01T23:05:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.183+0000 7f61bb27f700 1 -- 172.21.15.32:0/3298518566 wait complete. 2022-08-01T23:05:46.201 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 Processor -- start 2022-08-01T23:05:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 -- start start 2022-08-01T23:05:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b40a9320 0x7f61b4145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f61b40b0560 0x7f61b40a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:46.202 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 0x7f61b4152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40aa3d0 con 0x7f61b40b0d10 2022-08-01T23:05:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40ab7d0 con 0x7f61b40a9320 2022-08-01T23:05:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40aba30 con 0x7f61b40b0560 2022-08-01T23:05:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 0x7f61b4152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 0x7f61b4152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50334/0 (socket says 172.21.15.32:50334) 2022-08-01T23:05:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 -- 172.21.15.32:0/1206699003 learned_addr learned my addr 172.21.15.32:0/1206699003 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61ba27d700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b40a9320 0x7f61b4145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61b9a7c700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f61b40b0560 0x7f61b40a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:46.204 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 -- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f61b40b0560 msgr2=0x7f61b40a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f61b40b0560 0x7f61b40a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 -- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b40a9320 msgr2=0x7f61b4145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b40a9320 0x7f61b4145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:46.205 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 -- 172.21.15.32:0/1206699003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac009530 con 0x7f61b40b0d10 2022-08-01T23:05:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61b9a7c700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f61b40b0560 0x7f61b40a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61ba27d700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b40a9320 0x7f61b4145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:05:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61baa7e700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 0x7f61b4152b30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f61ac0032f0 tx=0x7f61ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61ac00b870 con 0x7f61b40b0d10 2022-08-01T23:05:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac021670 con 0x7f61b40b0d10 2022-08-01T23:05:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b4152690 con 0x7f61b40b0d10 2022-08-01T23:05:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61ac0219d0 con 0x7f61b40b0d10 2022-08-01T23:05:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.187+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b4152380 con 0x7f61b40b0d10 2022-08-01T23:05:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.191+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b4152820 con 0x7f61b40b0d10 2022-08-01T23:05:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.191+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f61ac00f070 con 0x7f61b40b0d10 2022-08-01T23:05:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.191+0000 7f61a37fe700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6194053b00 0x7f6194055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:46.208 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.191+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f61ac06d470 con 0x7f61b40b0d10 2022-08-01T23:05:46.209 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.191+0000 7f61ba27d700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6194053b00 0x7f6194055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.195+0000 7f61ba27d700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6194053b00 0x7f6194055fc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f61b000b000 tx=0x7f61b00070e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:46.210 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.199+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61ac044180 con 0x7f61b40b0d10 2022-08-01T23:05:46.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:46.323+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61b40574e0 con 0x7f61b40b0d10 2022-08-01T23:05:47.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.283+0000 7f61a37fe700 1 -- 172.21.15.32:0/1206699003 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) v1 ==== 185+0+0 (secure 0 0 0) 0x7f61ac015070 con 0x7f61b40b0d10 2022-08-01T23:05:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6194053b00 msgr2=0x7f6194055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6194053b00 0x7f6194055fc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f61b000b000 tx=0x7f61b00070e0).stop 2022-08-01T23:05:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 msgr2=0x7f61b4152b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 0x7f61b4152b30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f61ac0032f0 tx=0x7f61ac016000).stop 2022-08-01T23:05:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 shutdown_connections 2022-08-01T23:05:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f6194053b00 0x7f6194055fc0 secure :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0x7f61b000b000 tx=0x7f61b00070e0).stop 2022-08-01T23:05:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b40b0d10 0x7f61b4152b30 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f61b40b0560 0x7f61b40a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 --2- 172.21.15.32:0/1206699003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61b40a9320 0x7f61b4145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 >> 172.21.15.32:0/1206699003 conn(0x7f61b40612d0 msgr2=0x7f61b4141cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:47.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 shutdown_connections 2022-08-01T23:05:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.287+0000 7f61bb27f700 1 -- 172.21.15.32:0/1206699003 wait complete. 2022-08-01T23:05:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2022-08-01T23:05:47.308 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T23:05:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.411+0000 7ff9cae3e700 1 Processor -- start 2022-08-01T23:05:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.411+0000 7ff9cae3e700 1 -- start start 2022-08-01T23:05:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c40b1280 0x7ff9c413f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c413fa10 con 0x7ff9c405d240 2022-08-01T23:05:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c40b16d0 con 0x7ff9c405c960 2022-08-01T23:05:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40ad6d0 con 0x7ff9c40b1280 2022-08-01T23:05:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c963b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9c405d240 0x7ff9c40aee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37880/0 (socket says 172.21.15.32:37880) 2022-08-01T23:05:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c963b700 1 -- 172.21.15.32:0/2696875280 learned_addr learned my addr 172.21.15.32:0/2696875280 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:05:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9ca63d700 1 --2- 172.21.15.32:0/2696875280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c40b1280 0x7ff9c413f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878272759 0 0) 0x7ff9b4001110 con 0x7ff9c405d240 2022-08-01T23:05:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9ac003260 con 0x7ff9c405d240 2022-08-01T23:05:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233845848 0 0) 0x7ff9c40b16d0 con 0x7ff9c405c960 2022-08-01T23:05:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b4001110 con 0x7ff9c405c960 2022-08-01T23:05:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3723906350 0 0) 0x7ff9ac003260 con 0x7ff9c405d240 2022-08-01T23:05:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9ca63d700 1 -- 172.21.15.32:0/2696875280 >> v1:172.21.15.55:6789/0 conn(0x7ff9c405c960 legacy=0x7ff9c40acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9ca63d700 1 -- 172.21.15.32:0/2696875280 >> v1:172.21.15.32:6789/0 conn(0x7ff9c405d240 legacy=0x7ff9c40aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9ca63d700 1 -- 172.21.15.32:0/2696875280 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c405cd40 con 0x7ff9c40b1280 2022-08-01T23:05:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9ca63d700 1 --2- 172.21.15.32:0/2696875280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c40b1280 0x7ff9c413f520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7ff9b40012a0 tx=0x7ff9bc0091d0).ready entity=mon.2 client_cookie=fed468405ac2fbac server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9b40017f0 con 0x7ff9c40b1280 2022-08-01T23:05:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9bc00f510 con 0x7ff9c40b1280 2022-08-01T23:05:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9c8e3a700 1 -- 172.21.15.32:0/2696875280 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9bc00f680 con 0x7ff9c40b1280 2022-08-01T23:05:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- 172.21.15.32:0/2696875280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c40b1280 msgr2=0x7ff9c413f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/2696875280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c40b1280 0x7ff9c413f520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7ff9b40012a0 tx=0x7ff9bc0091d0).stop 2022-08-01T23:05:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- 172.21.15.32:0/2696875280 shutdown_connections 2022-08-01T23:05:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/2696875280 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c40b1280 0x7ff9c413f520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- 172.21.15.32:0/2696875280 >> 172.21.15.32:0/2696875280 conn(0x7ff9c4060310 msgr2=0x7ff9c40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- 172.21.15.32:0/2696875280 shutdown_connections 2022-08-01T23:05:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.415+0000 7ff9cae3e700 1 -- 172.21.15.32:0/2696875280 wait complete. 2022-08-01T23:05:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 Processor -- start 2022-08-01T23:05:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 -- start start 2022-08-01T23:05:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c405c960 0x7ff9c40af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 0x7ff9c40a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9c40b1280 0x7ff9c412be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c412d990 con 0x7ff9c405d240 2022-08-01T23:05:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40a9d30 con 0x7ff9c40b1280 2022-08-01T23:05:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40afed0 con 0x7ff9c405c960 2022-08-01T23:05:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 0x7ff9c40a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 0x7ff9c40a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50344/0 (socket says 172.21.15.32:50344) 2022-08-01T23:05:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 -- 172.21.15.32:0/1759600595 learned_addr learned my addr 172.21.15.32:0/1759600595 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:05:47.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9ca63d700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9c40b1280 0x7ff9c412be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:47.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c9e3c700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c405c960 0x7ff9c40af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:47.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 -- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c405c960 msgr2=0x7ff9c40af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c405c960 0x7ff9c40af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:47.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 -- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9c40b1280 msgr2=0x7ff9c412be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9c40b1280 0x7ff9c412be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 -- 172.21.15.32:0/1759600595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc009470 con 0x7ff9c405d240 2022-08-01T23:05:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9c963b700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 0x7ff9c40a92d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7ff9b40019d0 tx=0x7ff9b4001c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9b4007170 con 0x7ff9c405d240 2022-08-01T23:05:47.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b40072d0 con 0x7ff9c405d240 2022-08-01T23:05:47.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c40b0ba0 con 0x7ff9c405d240 2022-08-01T23:05:47.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9b4004920 con 0x7ff9c405d240 2022-08-01T23:05:47.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.419+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c40b02c0 con 0x7ff9c405d240 2022-08-01T23:05:47.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.423+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c40a9ef0 con 0x7ff9c405d240 2022-08-01T23:05:47.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.423+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff9b401b070 con 0x7ff9c405d240 2022-08-01T23:05:47.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.423+0000 7ff9b2ffd700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9a4053b00 0x7ff9a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:05:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.423+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6068+0+0 (secure 0 0 0) 0x7ff9b406b800 con 0x7ff9c405d240 2022-08-01T23:05:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.423+0000 7ff9c9e3c700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9a4053b00 0x7ff9a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:05:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.427+0000 7ff9c9e3c700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9a4053b00 0x7ff9a4055fc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7ff9c000d000 tx=0x7ff9c0007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:05:47.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.431+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff9b4042610 con 0x7ff9c405d240 2022-08-01T23:05:47.564 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:47.555+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff9c413fa10 con 0x7ff9c405d240 2022-08-01T23:05:48.516 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.507+0000 7ff9b2ffd700 1 -- 172.21.15.32:0/1759600595 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff9c413fa10 con 0x7ff9c405d240 2022-08-01T23:05:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.511+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9a4053b00 msgr2=0x7ff9a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9a4053b00 0x7ff9a4055fc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7ff9c000d000 tx=0x7ff9c0007a20).stop 2022-08-01T23:05:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 msgr2=0x7ff9c40a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:05:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 0x7ff9c40a92d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7ff9b40019d0 tx=0x7ff9b4001c70).stop 2022-08-01T23:05:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 shutdown_connections 2022-08-01T23:05:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff9a4053b00 0x7ff9a4055fc0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9c40b1280 0x7ff9c412be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c405d240 0x7ff9c40a92d0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 --2- 172.21.15.32:0/1759600595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9c405c960 0x7ff9c40af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:05:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 >> 172.21.15.32:0/1759600595 conn(0x7ff9c4060310 msgr2=0x7ff9c413d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:05:48.523 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 shutdown_connections 2022-08-01T23:05:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:05:48.515+0000 7ff9cae3e700 1 -- 172.21.15.32:0/1759600595 wait complete. 2022-08-01T23:05:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2022-08-01T23:05:48.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2022-08-01T23:05:48.535 DEBUG:teuthology.orchestra.run.smithi130:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-08-01T23:05:54.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:54.907 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:54.911 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:05:54.914 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:18.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:06:18.641 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:06:18.642 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:06:24.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:25.010 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:25.021 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:25.027 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:48.547 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T23:06:48.552 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:06:55.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:55.088 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:55.092 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:06:55.095 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:25.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:25.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:25.154 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:25.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:55.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:55.220 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:55.233 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:07:55.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:25.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:25.304 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:25.307 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:25.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:55.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:55.379 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:55.383 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:08:55.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:25.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:25.450 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:25.453 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:25.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:55.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:55.541 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:55.552 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:09:55.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:25.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:25.614 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:25.620 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:25.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:55.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:55.686 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:55.689 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:10:55.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:25.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:25.771 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:25.775 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:25.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:55.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:55.834 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:55.837 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:11:55.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:25.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:25.891 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:25.895 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:25.898 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:56.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:56.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:56.043 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:12:56.046 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:26.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:26.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:26.110 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:26.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:56.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:56.169 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:56.172 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:13:56.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:26.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:26.230 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:26.233 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:26.236 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:56.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:56.302 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:56.306 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:14:56.309 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:26.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:26.364 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:26.366 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:26.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:56.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:56.444 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:56.447 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:15:56.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:26.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:26.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:26.534 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:26.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:56.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:56.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:56.590 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:16:56.592 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:26.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:26.650 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:26.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:26.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:56.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:56.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:56.729 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:17:56.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:26.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:26.788 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:26.791 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:26.795 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:56.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:56.863 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:56.866 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:18:56.869 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:26.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:26.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:26.938 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:26.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:57.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:57.016 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:57.018 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:19:57.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:27.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:27.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:27.076 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:27.078 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:57.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:57.135 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:57.138 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:20:57.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:18.646 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T23:21:18.648 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:21:18.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:21:18.654 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T23:21:18.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:21:18.676 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:21:18.698 DEBUG:teuthology.orchestra.run.smithi130:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:21:18.719 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T23:21:18.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T23:21:18.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:21:18.741 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:21:18.742 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:21:27.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:27.223 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:27.225 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:27.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:57.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:57.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:57.321 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:21:57.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:27.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:27.389 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:27.391 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:27.394 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:57.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:57.461 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:57.464 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:22:57.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:27.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:27.533 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:27.537 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:27.540 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:57.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:57.595 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:57.598 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:23:57.600 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:27.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:27.659 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:27.662 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:27.664 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:57.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:57.766 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:57.769 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:24:57.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:27.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:27.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:27.860 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:27.865 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:57.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:57.921 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:57.924 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:25:57.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:28.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:28.008 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:28.011 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:28.015 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:58.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:58.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:58.075 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:26:58.079 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:28.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:28.141 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:28.144 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:28.147 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:58.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:58.211 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:58.214 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:27:58.216 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:28.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:28.275 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:28.278 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:28.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:58.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:58.332 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:58.335 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:28:58.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:28.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:28.396 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:28.399 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:28.402 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:58.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:58.491 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:58.494 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:29:58.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:28.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:28.558 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:28.560 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:28.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:58.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:58.622 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:58.625 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:30:58.627 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:28.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:28.699 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:28.702 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:28.704 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:58.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:58.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:58.765 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:31:58.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:28.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:28.841 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:28.843 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:28.846 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:58.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:58.904 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:58.908 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:32:58.910 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:28.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:28.987 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:28.992 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:28.996 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:59.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:59.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:59.054 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:33:59.057 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:29.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:29.117 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:29.120 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:29.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:59.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:59.315 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:59.318 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:34:59.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:29.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:29.405 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:29.408 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:29.411 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:59.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:59.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:59.480 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:35:59.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:18.768 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T23:36:18.769 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:36:18.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:36:18.774 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2022-08-01T23:36:18.774 INFO:tasks.cephfs_test_runner:ERROR 2022-08-01T23:36:18.775 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.776 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T23:36:18.776 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:36:18.776 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T23:36:18.777 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.777 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T23:36:18.778 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T23:36:18.778 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T23:36:18.778 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T23:36:18.779 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T23:36:18.779 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.779 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T23:36:18.780 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.780 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.781 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-08-01T23:36:18.781 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-08-01T23:36:18.781 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-08-01T23:36:18.782 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-08-01T23:36:18.782 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T23:36:18.782 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T23:36:18.783 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T23:36:18.783 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T23:36:18.783 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T23:36:18.784 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T23:36:18.784 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T23:36:18.784 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T23:36:18.785 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T23:36:18.785 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T23:36:18.785 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T23:36:18.785 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T23:36:18.786 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T23:36:18.786 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T23:36:18.787 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T23:36:18.787 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T23:36:18.788 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T23:36:18.788 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T23:36:18.789 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T23:36:18.789 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T23:36:18.790 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T23:36:18.790 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T23:36:18.791 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.791 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T23:36:18.791 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:36:18.792 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T23:36:18.792 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.793 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T23:36:18.793 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T23:36:18.793 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T23:36:18.794 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T23:36:18.794 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T23:36:18.794 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.795 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T23:36:18.795 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.795 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.796 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-08-01T23:36:18.796 INFO:tasks.cephfs_test_runner: m.teardown() 2022-08-01T23:36:18.796 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-08-01T23:36:18.796 INFO:tasks.cephfs_test_runner: self.umount() 2022-08-01T23:36:18.797 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-08-01T23:36:18.797 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-08-01T23:36:18.797 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T23:36:18.798 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T23:36:18.798 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T23:36:18.798 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T23:36:18.799 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T23:36:18.799 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T23:36:18.799 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T23:36:18.799 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T23:36:18.800 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T23:36:18.800 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T23:36:18.800 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T23:36:18.801 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T23:36:18.801 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T23:36:18.801 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T23:36:18.801 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T23:36:18.802 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T23:36:18.802 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T23:36:18.802 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T23:36:18.803 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T23:36:18.803 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T23:36:18.803 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T23:36:18.803 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T23:36:18.804 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.804 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T23:36:18.804 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2402.757s 2022-08-01T23:36:18.805 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.805 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2022-08-01T23:36:18.805 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.806 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T23:36:18.806 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:36:18.806 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T23:36:18.807 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.807 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T23:36:18.807 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T23:36:18.808 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T23:36:18.808 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T23:36:18.808 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T23:36:18.808 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.809 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T23:36:18.809 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.809 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.810 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-08-01T23:36:18.810 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-08-01T23:36:18.810 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-08-01T23:36:18.810 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-08-01T23:36:18.811 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T23:36:18.811 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T23:36:18.811 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T23:36:18.812 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T23:36:18.812 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T23:36:18.812 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T23:36:18.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T23:36:18.813 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T23:36:18.813 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T23:36:18.813 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T23:36:18.814 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T23:36:18.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T23:36:18.814 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T23:36:18.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T23:36:18.815 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T23:36:18.815 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T23:36:18.815 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T23:36:18.816 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T23:36:18.816 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T23:36:18.816 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T23:36:18.816 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T23:36:18.817 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T23:36:18.817 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.817 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T23:36:18.818 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:36:18.818 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T23:36:18.818 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T23:36:18.819 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T23:36:18.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T23:36:18.819 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T23:36:18.820 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T23:36:18.820 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.820 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T23:36:18.821 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.821 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T23:36:18.821 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-08-01T23:36:18.821 INFO:tasks.cephfs_test_runner: m.teardown() 2022-08-01T23:36:18.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-08-01T23:36:18.822 INFO:tasks.cephfs_test_runner: self.umount() 2022-08-01T23:36:18.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-08-01T23:36:18.823 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-08-01T23:36:18.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T23:36:18.823 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T23:36:18.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T23:36:18.824 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T23:36:18.824 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T23:36:18.824 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T23:36:18.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T23:36:18.825 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T23:36:18.825 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T23:36:18.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T23:36:18.826 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T23:36:18.826 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T23:36:18.826 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T23:36:18.827 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T23:36:18.827 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T23:36:18.827 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T23:36:18.827 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T23:36:18.828 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T23:36:18.828 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T23:36:18.828 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T23:36:18.829 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T23:36:18.829 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T23:36:18.829 INFO:tasks.cephfs_test_runner: 2022-08-01T23:36:18.830 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:36:18.955 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=30bbd7878db147488a968c4464f287d4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T23:36:18.958 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2022-08-01T23:36:19.105 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2022-08-01T23:36:19.119 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2022-08-01T23:36:19.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T23:36:19.120 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T23:36:19.120 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T23:36:29.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:29.600 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:29.603 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:29.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:59.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:59.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:59.671 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:36:59.674 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:29.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:29.733 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:29.736 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:29.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:59.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:59.794 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:59.797 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:37:59.800 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:29.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:29.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:29.862 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:29.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:59.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:59.921 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:59.924 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:38:59.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:39:29.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:39:29.985 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:39:29.990 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:39:29.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:00.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:00.097 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:00.100 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:00.102 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:30.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:30.159 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:30.162 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:40:30.165 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:00.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:00.287 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:00.290 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:00.292 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:30.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:30.367 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:30.370 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:41:30.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:00.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:00.506 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:00.508 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:00.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:30.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:30.625 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:30.628 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:42:30.630 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:00.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:00.771 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:00.773 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:00.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:30.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:30.916 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:30.920 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:43:30.922 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:01.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:01.029 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:01.033 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:01.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:31.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:31.195 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:31.199 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:44:31.203 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:01.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:01.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:01.277 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:01.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:31.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:31.491 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:31.494 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:45:31.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:01.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:01.603 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:01.605 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:01.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:31.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:31.782 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:31.785 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:46:31.789 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:01.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:01.984 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:01.987 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:01.989 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:32.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:32.120 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:32.124 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:47:32.127 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:02.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:02.235 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:02.237 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:02.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:32.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:32.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:32.377 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:48:32.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:02.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:02.588 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:02.591 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:02.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:32.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:32.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:32.704 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:49:32.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:02.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:02.781 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:02.784 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:02.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:32.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:32.913 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:32.916 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:50:32.919 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:03.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:03.058 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:03.061 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:03.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:19.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:51:19.138 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:51:19.159 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2022-08-01T23:51:19.161 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:51:19.173 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2022-08-01T23:51:19.187 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2022-08-01T23:51:19.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:51:19.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 Processor -- start 2022-08-01T23:51:19.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 -- start start 2022-08-01T23:51:19.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f23c8040c90 0x7f23c8005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23c80400c0 0x7f23c80d8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c8041640 con 0x7f23c8042530 2022-08-01T23:51:19.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c803e450 con 0x7f23c80400c0 2022-08-01T23:51:19.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.312+0000 7f23d5ff8700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c803f1a0 con 0x7f23c8040c90 2022-08-01T23:51:19.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d57f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23c80400c0 0x7f23c80d8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d4ff6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23c8042530 0x7f23c803f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37938/0 (socket says 172.21.15.32:37938) 2022-08-01T23:51:19.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d4ff6700 1 -- 172.21.15.32:0/3110450113 learned_addr learned my addr 172.21.15.32:0/3110450113 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cffff700 1 --2- 172.21.15.32:0/3110450113 >> v2:172.21.15.55:3300/0 conn(0x7f23c8040c90 0x7f23c8005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cf7fe700 1 -- 172.21.15.32:0/3110450113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294223192 0 0) 0x7f23c8041640 con 0x7f23c8042530 2022-08-01T23:51:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cf7fe700 1 -- 172.21.15.32:0/3110450113 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b4003260 con 0x7f23c8042530 2022-08-01T23:51:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cf7fe700 1 -- 172.21.15.32:0/3110450113 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261983418 0 0) 0x7f23b4003260 con 0x7f23c8042530 2022-08-01T23:51:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d57f7700 1 -- 172.21.15.32:0/3110450113 >> v2:172.21.15.55:3300/0 conn(0x7f23c8040c90 msgr2=0x7f23c8005660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d57f7700 1 --2- 172.21.15.32:0/3110450113 >> v2:172.21.15.55:3300/0 conn(0x7f23c8040c90 0x7f23c8005660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d57f7700 1 -- 172.21.15.32:0/3110450113 >> v1:172.21.15.32:6789/0 conn(0x7f23c8042530 legacy=0x7f23c803f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d57f7700 1 -- 172.21.15.32:0/3110450113 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c803fca0 con 0x7f23c80400c0 2022-08-01T23:51:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d57f7700 1 --2- 172.21.15.32:0/3110450113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c80400c0 0x7f23c80d8950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f23bc001800 tx=0x7f23d005f750).ready entity=mon.0 client_cookie=d72829593ec82d8f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cf7fe700 1 -- 172.21.15.32:0/3110450113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d0085070 con 0x7f23c80400c0 2022-08-01T23:51:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cf7fe700 1 -- 172.21.15.32:0/3110450113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d00696f0 con 0x7f23c80400c0 2022-08-01T23:51:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23cf7fe700 1 -- 172.21.15.32:0/3110450113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23bc001660 con 0x7f23c80400c0 2022-08-01T23:51:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 -- 172.21.15.32:0/3110450113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c80400c0 msgr2=0x7f23c80d8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/3110450113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c80400c0 0x7f23c80d8950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f23bc001800 tx=0x7f23d005f750).stop 2022-08-01T23:51:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 -- 172.21.15.32:0/3110450113 shutdown_connections 2022-08-01T23:51:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/3110450113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c80400c0 0x7f23c80d8950 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/3110450113 >> v2:172.21.15.55:3300/0 conn(0x7f23c8040c90 0x7f23c8005660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 -- 172.21.15.32:0/3110450113 >> 172.21.15.32:0/3110450113 conn(0x7f23c80462f0 msgr2=0x7f23c8046700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:19.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 -- 172.21.15.32:0/3110450113 shutdown_connections 2022-08-01T23:51:19.349 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 -- 172.21.15.32:0/3110450113 wait complete. 2022-08-01T23:51:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 Processor -- start 2022-08-01T23:51:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.316+0000 7f23d5ff8700 1 -- start start 2022-08-01T23:51:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c80400c0 0x7f23c80eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23c8040c90 0x7f23c80a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 0x7f23c803bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c803a8a0 con 0x7f23c8042530 2022-08-01T23:51:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c803ab40 con 0x7f23c80400c0 2022-08-01T23:51:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c803ade0 con 0x7f23c8040c90 2022-08-01T23:51:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 0x7f23c803bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 0x7f23c803bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50406/0 (socket says 172.21.15.32:50406) 2022-08-01T23:51:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 -- 172.21.15.32:0/1107468614 learned_addr learned my addr 172.21.15.32:0/1107468614 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d4ff6700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c80400c0 0x7f23c80eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23cffff700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23c8040c90 0x7f23c80a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 -- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23c8040c90 msgr2=0x7f23c80a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23c8040c90 0x7f23c80a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 -- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c80400c0 msgr2=0x7f23c80eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c80400c0 0x7f23c80eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 -- 172.21.15.32:0/1107468614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c809f4e0 con 0x7f23c8042530 2022-08-01T23:51:19.356 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d57f7700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 0x7f23c803bc20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f23bc001800 tx=0x7f23d0083000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d0085040 con 0x7f23c8042530 2022-08-01T23:51:19.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c80a12b0 con 0x7f23c8042530 2022-08-01T23:51:19.357 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d005db00 con 0x7f23c8042530 2022-08-01T23:51:19.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d0069a70 con 0x7f23c8042530 2022-08-01T23:51:19.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.320+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c803c110 con 0x7f23c8042530 2022-08-01T23:51:19.358 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.324+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c803afa0 con 0x7f23c8042530 2022-08-01T23:51:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.324+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f23d0069bd0 con 0x7f23c8042530 2022-08-01T23:51:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.324+0000 7f23cdffb700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23b0053980 0x7f23b0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.359 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.324+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f23d00da520 con 0x7f23c8042530 2022-08-01T23:51:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.324+0000 7f23d4ff6700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23b0053980 0x7f23b0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.360 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.324+0000 7f23d4ff6700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23b0053980 0x7f23b0055e40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f23bc00d000 tx=0x7f23bc001cc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.332+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f23d00b1430 con 0x7f23c8042530 2022-08-01T23:51:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.460+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23c8041870 con 0x7f23c8042530 2022-08-01T23:51:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.460+0000 7f23cdffb700 1 -- 172.21.15.32:0/1107468614 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+13551 (secure 0 0 0) 0x7f23d00b4a50 con 0x7f23c8042530 2022-08-01T23:51:19.466 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:51:19.467 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":161,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:48.495851+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.32:6835/324530820":"2022-08-02T23:05:44.031828+0000","172.21.15.32:6834/324530820":"2022-08-02T23:05:44.031828+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:51:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23b0053980 msgr2=0x7f23b0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23b0053980 0x7f23b0055e40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f23bc00d000 tx=0x7f23bc001cc0).stop 2022-08-01T23:51:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 msgr2=0x7f23c803bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 0x7f23c803bc20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f23bc001800 tx=0x7f23d0083000).stop 2022-08-01T23:51:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 shutdown_connections 2022-08-01T23:51:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f23b0053980 0x7f23b0055e40 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c8042530 0x7f23c803bc20 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f23c8040c90 0x7f23c80a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 --2- 172.21.15.32:0/1107468614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c80400c0 0x7f23c80eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 >> 172.21.15.32:0/1107468614 conn(0x7f23c80462f0 msgr2=0x7f23c8044b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 shutdown_connections 2022-08-01T23:51:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.464+0000 7f23d5ff8700 1 -- 172.21.15.32:0/1107468614 wait complete. 2022-08-01T23:51:19.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T23:51:19.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 Processor -- start 2022-08-01T23:51:19.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 -- start start 2022-08-01T23:51:19.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e4064b10 0x7f37e4144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37e40b36e0 0x7f37e4141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f37e4141860 0x7f37e4066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e413ebf0 con 0x7f37e40b36e0 2022-08-01T23:51:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e4064ef0 con 0x7f37e4141860 2022-08-01T23:51:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e974a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4066690 con 0x7f37e4064b10 2022-08-01T23:51:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37e40b36e0 0x7f37e4141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37e40b36e0 0x7f37e4141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50410/0 (socket says 172.21.15.32:50410) 2022-08-01T23:51:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 -- 172.21.15.32:0/3880499116 learned_addr learned my addr 172.21.15.32:0/3880499116 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e8f49700 1 --2- 172.21.15.32:0/3880499116 >> v2:172.21.15.55:3300/0 conn(0x7f37e4141860 0x7f37e4066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e3fff700 1 --2- 172.21.15.32:0/3880499116 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e4064b10 0x7f37e4144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 -- 172.21.15.32:0/3880499116 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e4064b10 msgr2=0x7f37e4144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:19.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 --2- 172.21.15.32:0/3880499116 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e4064b10 0x7f37e4144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 -- 172.21.15.32:0/3880499116 >> v2:172.21.15.55:3300/0 conn(0x7f37e4141860 msgr2=0x7f37e4066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:19.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 --2- 172.21.15.32:0/3880499116 >> v2:172.21.15.55:3300/0 conn(0x7f37e4141860 0x7f37e4066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 -- 172.21.15.32:0/3880499116 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40b4850 con 0x7f37e40b36e0 2022-08-01T23:51:19.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.576+0000 7f37e37fe700 1 --2- 172.21.15.32:0/3880499116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40b36e0 0x7f37e4141370 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f37d4002f70 tx=0x7f37d4006eb0).ready entity=mon.0 client_cookie=c22c53984f9a4999 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e2ffd700 1 -- 172.21.15.32:0/3880499116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37d400d570 con 0x7f37e40b36e0 2022-08-01T23:51:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e2ffd700 1 -- 172.21.15.32:0/3880499116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d4000e90 con 0x7f37e40b36e0 2022-08-01T23:51:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e2ffd700 1 -- 172.21.15.32:0/3880499116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37d400d900 con 0x7f37e40b36e0 2022-08-01T23:51:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- 172.21.15.32:0/3880499116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40b36e0 msgr2=0x7f37e4141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- 172.21.15.32:0/3880499116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40b36e0 0x7f37e4141370 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f37d4002f70 tx=0x7f37d4006eb0).stop 2022-08-01T23:51:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- 172.21.15.32:0/3880499116 shutdown_connections 2022-08-01T23:51:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- 172.21.15.32:0/3880499116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40b36e0 0x7f37e4141370 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- 172.21.15.32:0/3880499116 >> v2:172.21.15.55:3300/0 conn(0x7f37e4141860 0x7f37e4066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- 172.21.15.32:0/3880499116 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e4064b10 0x7f37e4144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- 172.21.15.32:0/3880499116 >> 172.21.15.32:0/3880499116 conn(0x7f37e405f840 msgr2=0x7f37e405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- 172.21.15.32:0/3880499116 shutdown_connections 2022-08-01T23:51:19.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- 172.21.15.32:0/3880499116 wait complete. 2022-08-01T23:51:19.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 Processor -- start 2022-08-01T23:51:19.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- start start 2022-08-01T23:51:19.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37e4064b10 0x7f37e4138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e40b36e0 0x7f37e40b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 0x7f37e40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40b1340 con 0x7f37e4141860 2022-08-01T23:51:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40b5630 con 0x7f37e4064b10 2022-08-01T23:51:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e974a700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40b6550 con 0x7f37e40b36e0 2022-08-01T23:51:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e8f49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 0x7f37e40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e8f49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 0x7f37e40b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50420/0 (socket says 172.21.15.32:50420) 2022-08-01T23:51:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e8f49700 1 -- 172.21.15.32:0/3293132595 learned_addr learned my addr 172.21.15.32:0/3293132595 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e37fe700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e40b36e0 0x7f37e40b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.580+0000 7f37e3fff700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37e4064b10 0x7f37e4138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e8f49700 1 -- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e40b36e0 msgr2=0x7f37e40b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e8f49700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e40b36e0 0x7f37e40b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e8f49700 1 -- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37e4064b10 msgr2=0x7f37e4138ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.600 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e8f49700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37e4064b10 0x7f37e4138ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e8f49700 1 -- 172.21.15.32:0/3293132595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d400b1e0 con 0x7f37e4141860 2022-08-01T23:51:19.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e8f49700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 0x7f37e40b73e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f37dc006e00 tx=0x7f37dc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.601 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37dc012600 con 0x7f37e4141860 2022-08-01T23:51:19.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc012760 con 0x7f37e4141860 2022-08-01T23:51:19.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37dc012ac0 con 0x7f37e4141860 2022-08-01T23:51:19.602 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e40b5860 con 0x7f37e4141860 2022-08-01T23:51:19.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e41393d0 con 0x7f37e4141860 2022-08-01T23:51:19.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e41386f0 con 0x7f37e4141860 2022-08-01T23:51:19.603 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f37dc022070 con 0x7f37e4141860 2022-08-01T23:51:19.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e17fa700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37c4053980 0x7f37c4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f37dc06c310 con 0x7f37e4141860 2022-08-01T23:51:19.604 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.584+0000 7f37e3fff700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37c4053980 0x7f37c4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.588+0000 7f37e3fff700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37c4053980 0x7f37c4055e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f37d800c000 tx=0x7f37d8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.588+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f37dc043310 con 0x7f37e4141860 2022-08-01T23:51:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.708+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37e40574e0 con 0x7f37c4053980 2022-08-01T23:51:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e17fa700 1 -- 172.21.15.32:0/3293132595 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68036 (secure 0 0 0) 0x7f37e40574e0 con 0x7f37c4053980 2022-08-01T23:51:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37c4053980 msgr2=0x7f37c4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37c4053980 0x7f37c4055e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f37d800c000 tx=0x7f37d8008040).stop 2022-08-01T23:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 msgr2=0x7f37e40b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 0x7f37e40b73e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f37dc006e00 tx=0x7f37dc0091d0).stop 2022-08-01T23:51:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 shutdown_connections 2022-08-01T23:51:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f37c4053980 0x7f37c4055e40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4141860 0x7f37e40b73e0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f37e40b36e0 0x7f37e40b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 --2- 172.21.15.32:0/3293132595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37e4064b10 0x7f37e4138ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 >> 172.21.15.32:0/3293132595 conn(0x7f37e405f840 msgr2=0x7f37e405c700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 shutdown_connections 2022-08-01T23:51:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.716+0000 7f37e974a700 1 -- 172.21.15.32:0/3293132595 wait complete. 2022-08-01T23:51:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2022-08-01T23:51:19.732 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1826,"stamp":"2022-08-01T23:51:19.163578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404704,"kb_used_data":16032,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570016,"statfs":{"total":773094113280,"available":764487696384,"internally_reserved":8589934592,"allocated":16416768,"data_stored":8039656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003304"},"pg_stats":[{"pgid":"1.0","version":"14'2","reported_seq":145,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346420+0000","last_change":"2022-08-01T22:55:40.426793+0000","last_active":"2022-08-01T23:05:48.346420+0000","last_peered":"2022-08-01T23:05:48.346420+0000","last_clean":"2022-08-01T23:05:48.346420+0000","last_became_active":"2022-08-01T22:55:40.425307+0000","last_became_peered":"2022-08-01T22:55:40.425307+0000","last_unstale":"2022-08-01T23:05:48.346420+0000","last_undegraded":"2022-08-01T23:05:48.346420+0000","last_fullsized":"2022-08-01T23:05:48.346420+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_clean_scrub_stamp":"2022-08-01T22:55:34.279477+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.289895+0000","last_change":"2022-08-01T22:55:44.188728+0000","last_active":"2022-08-01T23:05:47.289895+0000","last_peered":"2022-08-01T23:05:47.289895+0000","last_clean":"2022-08-01T23:05:47.289895+0000","last_became_active":"2022-08-01T22:55:44.188196+0000","last_became_peered":"2022-08-01T22:55:44.188196+0000","last_unstale":"2022-08-01T23:05:47.289895+0000","last_undegraded":"2022-08-01T23:05:47.289895+0000","last_fullsized":"2022-08-01T23:05:47.289895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346419+0000","last_change":"2022-08-01T22:55:44.188357+0000","last_active":"2022-08-01T23:05:48.346419+0000","last_peered":"2022-08-01T23:05:48.346419+0000","last_clean":"2022-08-01T23:05:48.346419+0000","last_became_active":"2022-08-01T22:55:44.187581+0000","last_became_peered":"2022-08-01T22:55:44.187581+0000","last_unstale":"2022-08-01T23:05:48.346419+0000","last_undegraded":"2022-08-01T23:05:48.346419+0000","last_fullsized":"2022-08-01T23:05:48.346419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.345170+0000","last_change":"2022-08-01T22:55:44.555310+0000","last_active":"2022-08-01T23:05:48.345170+0000","last_peered":"2022-08-01T23:05:48.345170+0000","last_clean":"2022-08-01T23:05:48.345170+0000","last_became_active":"2022-08-01T22:55:44.554353+0000","last_became_peered":"2022-08-01T22:55:44.554353+0000","last_unstale":"2022-08-01T23:05:48.345170+0000","last_undegraded":"2022-08-01T23:05:48.345170+0000","last_fullsized":"2022-08-01T23:05:48.345170+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.289864+0000","last_change":"2022-08-01T22:55:44.188785+0000","last_active":"2022-08-01T23:05:47.289864+0000","last_peered":"2022-08-01T23:05:47.289864+0000","last_clean":"2022-08-01T23:05:47.289864+0000","last_became_active":"2022-08-01T22:55:44.188345+0000","last_became_peered":"2022-08-01T22:55:44.188345+0000","last_unstale":"2022-08-01T23:05:47.289864+0000","last_undegraded":"2022-08-01T23:05:47.289864+0000","last_fullsized":"2022-08-01T23:05:47.289864+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.286498+0000","last_change":"2022-08-01T22:55:44.414856+0000","last_active":"2022-08-01T23:05:47.286498+0000","last_peered":"2022-08-01T23:05:47.286498+0000","last_clean":"2022-08-01T23:05:47.286498+0000","last_became_active":"2022-08-01T22:55:44.413727+0000","last_became_peered":"2022-08-01T22:55:44.413727+0000","last_unstale":"2022-08-01T23:05:47.286498+0000","last_undegraded":"2022-08-01T23:05:47.286498+0000","last_fullsized":"2022-08-01T23:05:47.286498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346553+0000","last_change":"2022-08-01T22:55:44.415068+0000","last_active":"2022-08-01T23:05:48.346553+0000","last_peered":"2022-08-01T23:05:48.346553+0000","last_clean":"2022-08-01T23:05:48.346553+0000","last_became_active":"2022-08-01T22:55:44.414349+0000","last_became_peered":"2022-08-01T22:55:44.414349+0000","last_unstale":"2022-08-01T23:05:48.346553+0000","last_undegraded":"2022-08-01T23:05:48.346553+0000","last_fullsized":"2022-08-01T23:05:48.346553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.287516+0000","last_change":"2022-08-01T22:55:44.190143+0000","last_active":"2022-08-01T23:05:47.287516+0000","last_peered":"2022-08-01T23:05:47.287516+0000","last_clean":"2022-08-01T23:05:47.287516+0000","last_became_active":"2022-08-01T22:55:44.189745+0000","last_became_peered":"2022-08-01T22:55:44.189745+0000","last_unstale":"2022-08-01T23:05:47.287516+0000","last_undegraded":"2022-08-01T23:05:47.287516+0000","last_fullsized":"2022-08-01T23:05:47.287516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.593244+0000","last_change":"2022-08-01T22:55:44.191477+0000","last_active":"2022-08-01T23:05:47.593244+0000","last_peered":"2022-08-01T23:05:47.593244+0000","last_clean":"2022-08-01T23:05:47.593244+0000","last_became_active":"2022-08-01T22:55:44.191167+0000","last_became_peered":"2022-08-01T22:55:44.191167+0000","last_unstale":"2022-08-01T23:05:47.593244+0000","last_undegraded":"2022-08-01T23:05:47.593244+0000","last_fullsized":"2022-08-01T23:05:47.593244+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.873},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.1459999999999999,"15min":1.3069999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90500000000000003},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.173,"5min":1.222,"15min":1.27},"last":0.77900000000000003}]},{"osd":1,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.85099999999999998,"15min":0.85799999999999998},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.2589999999999999,"15min":1.7210000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.78200000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.155,"5min":1.1599999999999999,"15min":1.653},"last":1.032}]},{"osd":2,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86199999999999999,"15min":0.88200000000000001},"min":{"1min":0.66300000000000003,"5min":0.58099999999999996,"15min":0.57899999999999996},"max":{"1min":1.1140000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.57999999999999996},"max":{"1min":1.1140000000000001,"5min":1.2410000000000001,"15min":1.3999999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86199999999999999,"15min":0.85199999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.04,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.60999999999999999},"max":{"1min":1.0309999999999999,"5min":1.2130000000000001,"15min":1.3540000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.77700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.52900000000000003},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.155},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1279999999999999,"5min":1.323,"15min":1.323},"last":0.86699999999999999}]},{"osd":5,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94999999999999996,"15min":0.94799999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.53800000000000003},"max":{"1min":1.1719999999999999,"5min":1.2849999999999999,"15min":1.444},"last":1.093},{"interface":"front","average":{"1min":1.014,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2649999999999999,"5min":1.325,"15min":1.3420000000000001},"last":1.262}]},{"osd":6,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.65000000000000002,"5min":0.64600000000000002,"15min":0.628},"max":{"1min":1.22,"5min":1.2569999999999999,"15min":1.319},"last":1.216},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0229999999999999,"15min":1.002},"min":{"1min":0.78600000000000003,"5min":0.67700000000000005,"15min":0.627},"max":{"1min":1.2070000000000001,"5min":1.4079999999999999,"15min":1.446},"last":1.079}]}]},{"osd":6,"up_from":13,"seq":55834575517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.48899999999999999},"max":{"1min":1.21,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93400000000000005,"15min":0.92200000000000004},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.57999999999999996},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.056}]},{"osd":1,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59299999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.81100000000000005}]},{"osd":2,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.2270000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.879,"15min":0.88500000000000001},"min":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.60199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.284},"last":1.0289999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":0.90600000000000003},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.151,"5min":1.1639999999999999,"15min":1.274},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.72699999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.873}]},{"osd":4,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.252},"last":1.143}]},{"osd":5,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.57699999999999996},"max":{"1min":1.28,"5min":1.286,"15min":1.3999999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.95999999999999996,"15min":0.94099999999999995},"min":{"1min":0.59999999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.246,"5min":1.2789999999999999,"15min":1.3500000000000001},"last":1.042}]},{"osd":7,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0369999999999999,"15min":1.02},"min":{"1min":0.92100000000000004,"5min":0.69899999999999995,"15min":0.61399999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.071,"15min":1.077},"min":{"1min":0.92800000000000005,"5min":0.64200000000000002,"15min":0.61799999999999999},"max":{"1min":1.351,"5min":1.3600000000000001,"15min":1.546},"last":1.081}]}]},{"osd":5,"up_from":13,"seq":55834575517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92100000000000004,"15min":0.88300000000000001},"min":{"1min":0.81299999999999994,"5min":0.58499999999999996,"15min":0.55800000000000005},"max":{"1min":1.123,"5min":1.161,"15min":1.2729999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.81100000000000005,"5min":0.68799999999999994,"15min":0.64600000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3120000000000001},"last":0.94199999999999995}]},{"osd":1,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.876,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1459999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0569999999999999,"5min":1.25,"15min":1.25},"last":0.70999999999999996}]},{"osd":2,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.88500000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.157,"5min":1.1819999999999999,"15min":1.3480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94199999999999995,"15min":0.94399999999999995},"min":{"1min":0.70899999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.304},"last":1.0669999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"max":{"1min":1.139,"5min":1.1930000000000001,"15min":1.278},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0089999999999999,"15min":1.0049999999999999},"min":{"1min":0.68300000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3540000000000001},"last":0.97999999999999998}]},{"osd":4,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85199999999999998,"15min":0.85399999999999998},"min":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.3080000000000001},"last":0.62},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.93000000000000005},"min":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.57299999999999995},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.3120000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.60899999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.4279999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98299999999999998},"min":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.3,"5min":1.304,"15min":1.48},"last":0.89600000000000002}]},{"osd":7,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0269999999999999,"15min":1.0129999999999999},"min":{"1min":0.83799999999999997,"5min":0.71099999999999997,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.4990000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.165,"15min":1.153},"min":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1930000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.90400000000000003,"15min":0.90000000000000002},"min":{"1min":0.67500000000000004,"5min":0.60199999999999998,"15min":0.54400000000000004},"max":{"1min":1.129,"5min":1.171,"15min":1.2789999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.88800000000000001},"min":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.66300000000000003},"max":{"1min":0.95399999999999996,"5min":1.1479999999999999,"15min":1.2230000000000001},"last":0.89300000000000002}]},{"osd":1,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.54200000000000004},"max":{"1min":1.1870000000000001,"5min":1.194,"15min":1.194},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94899999999999995,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.94599999999999995}]},{"osd":2,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94199999999999995,"15min":0.94399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.55800000000000005},"max":{"1min":1.1479999999999999,"5min":1.2,"15min":1.2310000000000001},"last":1.018},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.93400000000000005,"15min":0.95299999999999996},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.21,"15min":1.3360000000000001},"last":1.1639999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.82199999999999995},"min":{"1min":0.69099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.98299999999999998,"5min":1.1459999999999999,"15min":1.169},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.91400000000000003,"15min":0.92500000000000004},"min":{"1min":0.73899999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.163,"5min":1.163,"15min":1.2390000000000001},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.64100000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1180000000000001,"5min":1.1819999999999999,"15min":1.3089999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1479999999999999,"5min":1.3029999999999999,"15min":1.3080000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.3500000000000001,"15min":1.4299999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.73699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.244,"5min":1.2470000000000001,"15min":1.3819999999999999},"last":1.2310000000000001}]},{"osd":7,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93100000000000005,"15min":0.93600000000000005},"min":{"1min":0.65200000000000002,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0900000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0509999999999999,"15min":1.0549999999999999},"min":{"1min":0.84899999999999998,"5min":0.70899999999999996,"15min":0.65500000000000003},"max":{"1min":1.202,"5min":1.4039999999999999,"15min":1.5049999999999999},"last":1.103}]}]},{"osd":3,"up_from":13,"seq":55834575517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55600000000000005},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.625}]},{"osd":1,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.135},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.75},"min":{"1min":0.56599999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.82099999999999995,"15min":0.80500000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.51600000000000001},"max":{"1min":1.04,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.88800000000000001},"min":{"1min":0.69199999999999995,"5min":0.58399999999999996,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.872,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93600000000000005,"15min":0.92900000000000005},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.66600000000000004},"max":{"1min":1.3180000000000001,"5min":1.331,"15min":1.522},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.98499999999999999,"15min":0.98699999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.628},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.014,"5min":0.96799999999999997,"15min":0.95699999999999996},"min":{"1min":0.72599999999999998,"5min":0.64400000000000002,"15min":0.63200000000000001},"max":{"1min":1.389,"5min":1.546,"15min":1.546},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0229999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.79100000000000004,"15min":0.71099999999999997},"max":{"1min":1.2470000000000001,"5min":1.5289999999999999,"15min":1.6919999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0609999999999999},"min":{"1min":0.80400000000000005,"5min":0.73399999999999999,"15min":0.72999999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.74},"last":1.0069999999999999}]},{"osd":7,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0569999999999999,"15min":1.05},"min":{"1min":0.88900000000000001,"5min":0.77000000000000002,"15min":0.75900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.726},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.105,"5min":1.099,"15min":1.087},"min":{"1min":0.92800000000000005,"5min":0.79900000000000004,"15min":0.72299999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.665},"last":1.548}]}]},{"osd":2,"up_from":13,"seq":55834575517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.45400000000000001,"5min":0.439,"15min":0.43099999999999999},"max":{"1min":1,"5min":1,"15min":1.0600000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58199999999999996},"max":{"1min":0.90400000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90200000000000002,"5min":0.98099999999999998,"15min":1.115},"last":0.624},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.71199999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.1160000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.505},"max":{"1min":0.90100000000000002,"5min":1.03,"15min":1.149},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.623,"5min":0.60599999999999998,"15min":0.52300000000000002},"max":{"1min":1.04,"5min":1.1100000000000001,"15min":1.1519999999999999},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.96699999999999997,"15min":0.93799999999999994},"min":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.65100000000000002},"max":{"1min":1.143,"5min":1.238,"15min":1.2609999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.90900000000000003},"min":{"1min":0.71199999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.038,"5min":1.2709999999999999,"15min":1.2789999999999999},"last":0.996}]},{"osd":5,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.0069999999999999,"15min":1.016},"min":{"1min":0.79500000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2989999999999999,"5min":1.4710000000000001,"15min":1.607},"last":1.085},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.995,"15min":0.98999999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.67900000000000005},"max":{"1min":1.2769999999999999,"5min":1.413,"15min":1.413},"last":1.232}]},{"osd":6,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.68700000000000006},"max":{"1min":1.2969999999999999,"5min":1.333,"15min":1.504},"last":1.038},{"interface":"front","average":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0780000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.3,"5min":1.524,"15min":1.77},"last":1.1870000000000001}]},{"osd":7,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0600000000000001,"15min":1.0660000000000001},"min":{"1min":0.82899999999999996,"5min":0.79800000000000004,"15min":0.77800000000000002},"max":{"1min":1.212,"5min":1.49,"15min":1.49},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0649999999999999,"15min":1.075},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.73699999999999999},"max":{"1min":1.284,"5min":1.498,"15min":1.575},"last":1.0680000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575518,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.76800000000000002,"5min":1.133,"15min":1.258},"last":0.503},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.72499999999999998},"min":{"1min":0.504,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.93899999999999995,"5min":1.004,"15min":1.147},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91400000000000003,"5min":1.05,"15min":1.3100000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.629,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.05,"5min":1.1830000000000001,"15min":1.2829999999999999},"last":1.0409999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.84299999999999997,"15min":0.82399999999999995},"min":{"1min":0.69099999999999995,"5min":0.59699999999999998,"15min":0.56200000000000006},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71599999999999997,"15min":0.72599999999999998},"min":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.92400000000000004}]},{"osd":4,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.90700000000000003,"15min":0.91700000000000004},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.63300000000000001},"max":{"1min":1.012,"5min":1.3129999999999999,"15min":1.381},"last":0.753},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98699999999999999,"15min":0.97599999999999998},"min":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.65100000000000002},"max":{"1min":1.129,"5min":1.423,"15min":1.573},"last":0.97999999999999998}]},{"osd":5,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93999999999999995,"15min":0.93200000000000005},"min":{"1min":0.82999999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"max":{"1min":1.272,"5min":1.3029999999999999,"15min":1.373},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":1.01},"min":{"1min":0.77900000000000003,"5min":0.69399999999999995,"15min":0.68799999999999994},"max":{"1min":1.236,"5min":1.657,"15min":1.7190000000000001},"last":0.83399999999999996}]},{"osd":6,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0229999999999999,"15min":1.042},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2310000000000001,"5min":1.72,"15min":1.72},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.024,"5min":0.99199999999999999,"15min":1.006},"min":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.256,"5min":1.2589999999999999,"15min":1.357},"last":0.877}]},{"osd":7,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.85999999999999999,"5min":0.754,"15min":0.74099999999999999},"max":{"1min":1.4430000000000001,"5min":1.4930000000000001,"15min":2.0369999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.0569999999999999,"15min":1.0629999999999999},"min":{"1min":0.97099999999999997,"5min":0.81000000000000005,"15min":0.755},"max":{"1min":1.29,"5min":1.3680000000000001,"15min":1.411},"last":1.004}]}]},{"osd":0,"up_from":13,"seq":55834575517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.5,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":0.76100000000000001,"5min":1.012,"15min":1.173},"last":0.497},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.89200000000000002,"5min":1.0409999999999999,"15min":1.2030000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.57899999999999996,"5min":0.50800000000000001,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.103,"15min":1.135},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.087,"5min":1.087,"15min":1.095},"last":0.66900000000000004}]},{"osd":3,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72899999999999998,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81799999999999995,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.57999999999999996,"15min":0.54100000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.26},"last":0.73399999999999999}]},{"osd":4,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94999999999999996,"15min":0.93700000000000006},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.321,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.64700000000000002},"max":{"1min":1.3080000000000001,"5min":1.331,"15min":1.331},"last":1.0109999999999999}]},{"osd":5,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93000000000000005,"15min":0.93899999999999995},"min":{"1min":0.75700000000000001,"5min":0.73399999999999999,"15min":0.67500000000000004},"max":{"1min":1.3899999999999999,"5min":1.403,"15min":1.4410000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.018,"15min":1.0409999999999999},"min":{"1min":0.80300000000000005,"5min":0.74099999999999999,"15min":0.70099999999999996},"max":{"1min":1.3360000000000001,"5min":1.4159999999999999,"15min":1.4299999999999999},"last":0.97999999999999998}]},{"osd":6,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.4670000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0740000000000001,"15min":1.0660000000000001},"min":{"1min":0.91200000000000003,"5min":0.80000000000000004,"15min":0.76700000000000002},"max":{"1min":1.329,"5min":1.369,"15min":1.4330000000000001},"last":0.89100000000000001}]},{"osd":7,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.103,"15min":1.101},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.423},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.117,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.94499999999999995,"5min":0.84999999999999998,"15min":0.75600000000000001},"max":{"1min":1.474,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.0640000000000001}]}]}]}} 2022-08-01T23:51:19.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T23:51:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 Processor -- start 2022-08-01T23:51:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 -- start start 2022-08-01T23:51:19.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f580b01a0 0x7f9f580a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.842 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9f580afd60 0x7f9f580a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f580a7250 con 0x7f9f580b2710 2022-08-01T23:51:19.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580a7480 con 0x7f9f580b01a0 2022-08-01T23:51:19.843 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d927700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580a7720 con 0x7f9f580afd60 2022-08-01T23:51:19.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f580b01a0 0x7f9f580a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f580b01a0 0x7f9f580a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50424/0 (socket says 172.21.15.32:50424) 2022-08-01T23:51:19.844 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 -- 172.21.15.32:0/1918666082 learned_addr learned my addr 172.21.15.32:0/1918666082 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:19.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f5d126700 1 --2- 172.21.15.32:0/1918666082 >> v2:172.21.15.55:3300/0 conn(0x7f9f580afd60 0x7f9f580a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 -- 172.21.15.32:0/1918666082 >> v2:172.21.15.55:3300/0 conn(0x7f9f580afd60 msgr2=0x7f9f580a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:19.845 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 --2- 172.21.15.32:0/1918666082 >> v2:172.21.15.55:3300/0 conn(0x7f9f580afd60 0x7f9f580a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 -- 172.21.15.32:0/1918666082 >> v1:172.21.15.55:6789/0 conn(0x7f9f580b2710 legacy=0x7f9f5814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 -- 172.21.15.32:0/1918666082 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580a79c0 con 0x7f9f580b01a0 2022-08-01T23:51:19.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.836+0000 7f9f57fff700 1 --2- 172.21.15.32:0/1918666082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580b01a0 0x7f9f580a8540 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f9f480032f0 tx=0x7f9f48009290).ready entity=mon.0 client_cookie=84c2d1e591bb7090 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f577fe700 1 -- 172.21.15.32:0/1918666082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f4800b870 con 0x7f9f580b01a0 2022-08-01T23:51:19.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f577fe700 1 -- 172.21.15.32:0/1918666082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4800b9d0 con 0x7f9f580b01a0 2022-08-01T23:51:19.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f577fe700 1 -- 172.21.15.32:0/1918666082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f4800bb40 con 0x7f9f580b01a0 2022-08-01T23:51:19.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 -- 172.21.15.32:0/1918666082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580b01a0 msgr2=0x7f9f580a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1918666082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580b01a0 0x7f9f580a8540 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f9f480032f0 tx=0x7f9f48009290).stop 2022-08-01T23:51:19.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 -- 172.21.15.32:0/1918666082 shutdown_connections 2022-08-01T23:51:19.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1918666082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580b01a0 0x7f9f580a8540 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1918666082 >> v2:172.21.15.55:3300/0 conn(0x7f9f580afd60 0x7f9f580a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 -- 172.21.15.32:0/1918666082 >> 172.21.15.32:0/1918666082 conn(0x7f9f58060310 msgr2=0x7f9f580b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 -- 172.21.15.32:0/1918666082 shutdown_connections 2022-08-01T23:51:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 -- 172.21.15.32:0/1918666082 wait complete. 2022-08-01T23:51:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 Processor -- start 2022-08-01T23:51:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.840+0000 7f9f5d927700 1 -- start start 2022-08-01T23:51:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 0x7f9f580ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f580b01a0 0x7f9f58159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580b2710 0x7f9f5815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58067270 con 0x7f9f580afd60 2022-08-01T23:51:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58067080 con 0x7f9f580b2710 2022-08-01T23:51:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580ad830 con 0x7f9f580b01a0 2022-08-01T23:51:19.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 0x7f9f580ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d126700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580b2710 0x7f9f5815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 0x7f9f580ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50430/0 (socket says 172.21.15.32:50430) 2022-08-01T23:51:19.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 -- 172.21.15.32:0/1112909401 learned_addr learned my addr 172.21.15.32:0/1112909401 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:19.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f57fff700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f580b01a0 0x7f9f58159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 -- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f580b01a0 msgr2=0x7f9f58159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f580b01a0 0x7f9f58159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 -- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580b2710 msgr2=0x7f9f5815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580b2710 0x7f9f5815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 -- 172.21.15.32:0/1112909401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48009530 con 0x7f9f580afd60 2022-08-01T23:51:19.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d126700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580b2710 0x7f9f5815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:19.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5c925700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 0x7f9f580ac170 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f9f5806b950 tx=0x7f9f4c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f4c007e70 con 0x7f9f580afd60 2022-08-01T23:51:19.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c007030 con 0x7f9f580afd60 2022-08-01T23:51:19.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f58063310 con 0x7f9f580afd60 2022-08-01T23:51:19.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f4c007390 con 0x7f9f580afd60 2022-08-01T23:51:19.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5805c620 con 0x7f9f580afd60 2022-08-01T23:51:19.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f4c010030 con 0x7f9f580afd60 2022-08-01T23:51:19.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.844+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f580574e0 con 0x7f9f580afd60 2022-08-01T23:51:19.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.848+0000 7f9f55ffb700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f38053a70 0x7f9f38055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:19.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.848+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9f4c06b8c0 con 0x7f9f580afd60 2022-08-01T23:51:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.848+0000 7f9f57fff700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f38053a70 0x7f9f38055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.848+0000 7f9f57fff700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f38053a70 0x7f9f38055f30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f9f4801b000 tx=0x7f9f48016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.852+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f4c042960 con 0x7f9f580afd60 2022-08-01T23:51:19.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.976+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f58069a70 con 0x7f9f580afd60 2022-08-01T23:51:19.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f55ffb700 1 -- 172.21.15.32:0/1112909401 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+13551 (secure 0 0 0) 0x7f9f4c045f80 con 0x7f9f580afd60 2022-08-01T23:51:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:51:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":161,"fsid":"a552eeb0-b3d6-42dc-b595-9e4ba3b3a09e","created":"2022-08-01T22:55:31.351237+0000","modified":"2022-08-01T23:05:48.495851+0000","last_up_change":"2022-08-01T22:55:39.069490+0000","last_in_change":"2022-08-01T22:55:39.069490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T22:55:34.277887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T22:55:43.120854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"047a2077-8683-4670-a43a-a616da639621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":23003},{"type":"v1","addr":"172.21.15.32:6825","nonce":23003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":23003},{"type":"v1","addr":"172.21.15.32:6827","nonce":23003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":23003},{"type":"v1","addr":"172.21.15.32:6831","nonce":23003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":23003},{"type":"v1","addr":"172.21.15.32:6829","nonce":23003}]},"public_addr":"172.21.15.32:6825/23003","cluster_addr":"172.21.15.32:6827/23003","heartbeat_back_addr":"172.21.15.32:6831/23003","heartbeat_front_addr":"172.21.15.32:6829/23003","state":["exists","up"]},{"osd":1,"uuid":"200572dd-9760-43f7-9568-b2c7553721b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":23005},{"type":"v1","addr":"172.21.15.32:6801","nonce":23005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":23005},{"type":"v1","addr":"172.21.15.32:6803","nonce":23005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":23005},{"type":"v1","addr":"172.21.15.32:6807","nonce":23005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":23005},{"type":"v1","addr":"172.21.15.32:6805","nonce":23005}]},"public_addr":"172.21.15.32:6801/23005","cluster_addr":"172.21.15.32:6803/23005","heartbeat_back_addr":"172.21.15.32:6807/23005","heartbeat_front_addr":"172.21.15.32:6805/23005","state":["exists","up"]},{"osd":2,"uuid":"a8c47037-5440-477f-8684-a1f3f8fb26f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":23004},{"type":"v1","addr":"172.21.15.32:6809","nonce":23004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":23004},{"type":"v1","addr":"172.21.15.32:6811","nonce":23004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":23004},{"type":"v1","addr":"172.21.15.32:6815","nonce":23004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":23004},{"type":"v1","addr":"172.21.15.32:6813","nonce":23004}]},"public_addr":"172.21.15.32:6809/23004","cluster_addr":"172.21.15.32:6811/23004","heartbeat_back_addr":"172.21.15.32:6815/23004","heartbeat_front_addr":"172.21.15.32:6813/23004","state":["exists","up"]},{"osd":3,"uuid":"645afcc6-0658-4477-9722-c6e21159c5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":23015},{"type":"v1","addr":"172.21.15.32:6817","nonce":23015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":23015},{"type":"v1","addr":"172.21.15.32:6819","nonce":23015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":23015},{"type":"v1","addr":"172.21.15.32:6823","nonce":23015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":23015},{"type":"v1","addr":"172.21.15.32:6821","nonce":23015}]},"public_addr":"172.21.15.32:6817/23015","cluster_addr":"172.21.15.32:6819/23015","heartbeat_back_addr":"172.21.15.32:6823/23015","heartbeat_front_addr":"172.21.15.32:6821/23015","state":["exists","up"]},{"osd":4,"uuid":"8bb02127-2e21-4432-b767-9a7559abfe67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6819","nonce":23335},{"type":"v1","addr":"172.21.15.55:6821","nonce":23335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":23335},{"type":"v1","addr":"172.21.15.55:6825","nonce":23335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6831","nonce":23335},{"type":"v1","addr":"172.21.15.55:6833","nonce":23335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6827","nonce":23335},{"type":"v1","addr":"172.21.15.55:6829","nonce":23335}]},"public_addr":"172.21.15.55:6821/23335","cluster_addr":"172.21.15.55:6825/23335","heartbeat_back_addr":"172.21.15.55:6833/23335","heartbeat_front_addr":"172.21.15.55:6829/23335","state":["exists","up"]},{"osd":5,"uuid":"d961d2aa-c6df-4770-90e1-0d2a43f360ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":23336},{"type":"v1","addr":"172.21.15.55:6803","nonce":23336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":23336},{"type":"v1","addr":"172.21.15.55:6805","nonce":23336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":23336},{"type":"v1","addr":"172.21.15.55:6809","nonce":23336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":23336},{"type":"v1","addr":"172.21.15.55:6807","nonce":23336}]},"public_addr":"172.21.15.55:6803/23336","cluster_addr":"172.21.15.55:6805/23336","heartbeat_back_addr":"172.21.15.55:6809/23336","heartbeat_front_addr":"172.21.15.55:6807/23336","state":["exists","up"]},{"osd":6,"uuid":"162a1876-d2c7-4886-9434-b4114d80e92b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":23334},{"type":"v1","addr":"172.21.15.55:6811","nonce":23334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":23334},{"type":"v1","addr":"172.21.15.55:6813","nonce":23334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":23334},{"type":"v1","addr":"172.21.15.55:6817","nonce":23334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":23334},{"type":"v1","addr":"172.21.15.55:6815","nonce":23334}]},"public_addr":"172.21.15.55:6811/23334","cluster_addr":"172.21.15.55:6813/23334","heartbeat_back_addr":"172.21.15.55:6817/23334","heartbeat_front_addr":"172.21.15.55:6815/23334","state":["exists","up"]},{"osd":7,"uuid":"d1c950ea-35ee-4ac6-8e31-e832328f349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":23359},{"type":"v1","addr":"172.21.15.55:6820","nonce":23359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":23359},{"type":"v1","addr":"172.21.15.55:6824","nonce":23359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":23359},{"type":"v1","addr":"172.21.15.55:6832","nonce":23359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":23359},{"type":"v1","addr":"172.21.15.55:6828","nonce":23359}]},"public_addr":"172.21.15.55:6820/23359","cluster_addr":"172.21.15.55:6824/23359","heartbeat_back_addr":"172.21.15.55:6832/23359","heartbeat_front_addr":"172.21.15.55:6828/23359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.411086+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.153541+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.354950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.423072+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.377079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.170426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.325899+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T22:55:37.406820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.32:6835/324530820":"2022-08-02T23:05:44.031828+0000","172.21.15.32:6834/324530820":"2022-08-02T23:05:44.031828+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T23:51:19.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f38053a70 msgr2=0x7f9f38055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f38053a70 0x7f9f38055f30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f9f4801b000 tx=0x7f9f48016000).stop 2022-08-01T23:51:19.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 msgr2=0x7f9f580ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 0x7f9f580ac170 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f9f5806b950 tx=0x7f9f4c007c00).stop 2022-08-01T23:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 shutdown_connections 2022-08-01T23:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9f38053a70 0x7f9f38055f30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580b2710 0x7f9f5815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9f580b01a0 0x7f9f58159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 --2- 172.21.15.32:0/1112909401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f580afd60 0x7f9f580ac170 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:19.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.980+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 >> 172.21.15.32:0/1112909401 conn(0x7f9f58060310 msgr2=0x7f9f580aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:19.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.984+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 shutdown_connections 2022-08-01T23:51:19.988 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:19.984+0000 7f9f5d927700 1 -- 172.21.15.32:0/1112909401 wait complete. 2022-08-01T23:51:21.000 INFO:tasks.ceph:Scrubbing osd.0 2022-08-01T23:51:21.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cfde4700 1 Processor -- start 2022-08-01T23:51:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cfde4700 1 -- start start 2022-08-01T23:51:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cfde4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05c80b6710 0x7f05c80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cfde4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c8144240 con 0x7f05c80b6af0 2022-08-01T23:51:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cfde4700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c8143d00 con 0x7f05c80b4550 2022-08-01T23:51:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cfde4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c8143fa0 con 0x7f05c80b6710 2022-08-01T23:51:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cdb80700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05c80b6710 0x7f05c80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cdb80700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05c80b6710 0x7f05c80b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50438/0 (socket says 172.21.15.32:50438) 2022-08-01T23:51:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cdb80700 1 -- 172.21.15.32:0/566736201 learned_addr learned my addr 172.21.15.32:0/566736201 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05ce381700 1 --1- 172.21.15.32:0/566736201 >> v1:172.21.15.32:6789/0 conn(0x7f05c80b6af0 0x7f05c812bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37984/0 (socket says 172.21.15.32:37984) 2022-08-01T23:51:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.116+0000 7f05cdb80700 1 -- 172.21.15.32:0/566736201 >> v1:172.21.15.55:6789/0 conn(0x7f05c80b4550 legacy=0x7f05c8061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cdb80700 1 -- 172.21.15.32:0/566736201 >> v1:172.21.15.32:6789/0 conn(0x7f05c80b6af0 legacy=0x7f05c812bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05ccb7e700 1 -- 172.21.15.32:0/566736201 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503885800 0 0) 0x7f05c8144240 con 0x7f05c80b6af0 2022-08-01T23:51:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cdb80700 1 -- 172.21.15.32:0/566736201 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8061780 con 0x7f05c80b6710 2022-08-01T23:51:21.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cdb80700 1 --2- 172.21.15.32:0/566736201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6710 0x7f05c80b5d80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7f05c81432b0 tx=0x7f05c4006eb0).ready entity=mon.0 client_cookie=7a36a7a7cb99342b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05ccb7e700 1 -- 172.21.15.32:0/566736201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05c400d660 con 0x7f05c80b6710 2022-08-01T23:51:21.126 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05ccb7e700 1 -- 172.21.15.32:0/566736201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c4000e90 con 0x7f05c80b6710 2022-08-01T23:51:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05ccb7e700 1 -- 172.21.15.32:0/566736201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05c400d9b0 con 0x7f05c80b6710 2022-08-01T23:51:21.133 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 -- 172.21.15.32:0/566736201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6710 msgr2=0x7f05c80b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 --2- 172.21.15.32:0/566736201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6710 0x7f05c80b5d80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7f05c81432b0 tx=0x7f05c4006eb0).stop 2022-08-01T23:51:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 -- 172.21.15.32:0/566736201 shutdown_connections 2022-08-01T23:51:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 --2- 172.21.15.32:0/566736201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6710 0x7f05c80b5d80 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 -- 172.21.15.32:0/566736201 >> 172.21.15.32:0/566736201 conn(0x7f05c805c620 msgr2=0x7f05c805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 -- 172.21.15.32:0/566736201 shutdown_connections 2022-08-01T23:51:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 -- 172.21.15.32:0/566736201 wait complete. 2022-08-01T23:51:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 Processor -- start 2022-08-01T23:51:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.120+0000 7f05cfde4700 1 -- start start 2022-08-01T23:51:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80b4550 0x7f05c8142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05c80b6710 0x7f05c8147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6af0 0x7f05c8148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81491f0 con 0x7f05c80b6af0 2022-08-01T23:51:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8155a40 con 0x7f05c80b4550 2022-08-01T23:51:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c80b7720 con 0x7f05c80b6710 2022-08-01T23:51:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05ce381700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6af0 0x7f05c8148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80b4550 0x7f05c8142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05ce381700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6af0 0x7f05c8148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50448/0 (socket says 172.21.15.32:50448) 2022-08-01T23:51:21.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05ce381700 1 -- 172.21.15.32:0/2148748066 learned_addr learned my addr 172.21.15.32:0/2148748066 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80b4550 0x7f05c8142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.32:60230/0 (socket says 172.21.15.32:60230) 2022-08-01T23:51:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cd37f700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05c80b6710 0x7f05c8147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 -- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05c80b6710 msgr2=0x7f05c8147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05c80b6710 0x7f05c8147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 -- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6af0 msgr2=0x7f05c8148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6af0 0x7f05c8148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 -- 172.21.15.32:0/2148748066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c400b1e0 con 0x7f05c80b4550 2022-08-01T23:51:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cd37f700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f05c80b6710 0x7f05c8147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05ce381700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c80b6af0 0x7f05c8148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cdb80700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80b4550 0x7f05c8142370 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f05c8149330 tx=0x7f05c4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05c4021070 con 0x7f05c80b4550 2022-08-01T23:51:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c400bdf0 con 0x7f05c80b4550 2022-08-01T23:51:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c80b79c0 con 0x7f05c80b4550 2022-08-01T23:51:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05c4011cf0 con 0x7f05c80b4550 2022-08-01T23:51:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c812b720 con 0x7f05c80b4550 2022-08-01T23:51:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.124+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05c80b75a0 con 0x7f05c80b4550 2022-08-01T23:51:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f05c400f070 con 0x7f05c80b4550 2022-08-01T23:51:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05a8053b00 0x7f05a8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f05c406dc50 con 0x7f05c80b4550 2022-08-01T23:51:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7f05a805a790 0x7f05a805cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 --> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05a805d210 con 0x7f05a805a790 2022-08-01T23:51:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05c406e030 con 0x7f05c80b4550 2022-08-01T23:51:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05cd37f700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05a8053b00 0x7f05a8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05ce381700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7f05a805a790 0x7f05a805cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05cd37f700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05a8053b00 0x7f05a8055fc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f05bc00d000 tx=0x7f05bc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05ce381700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7f05a805a790 0x7f05a805cc30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.128+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== osd.0 v2:172.21.15.32:6824/23003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f05a805d210 con 0x7f05a805a790 2022-08-01T23:51:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 --> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f05c8061a20 con 0x7f05a805a790 2022-08-01T23:51:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05baffd700 1 -- 172.21.15.32:0/2148748066 <== osd.0 v2:172.21.15.32:6824/23003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f05c8061a20 con 0x7f05a805a790 2022-08-01T23:51:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:21.155 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7f05a805a790 msgr2=0x7f05a805cc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.32:6824/23003,v1:172.21.15.32:6825/23003] conn(0x7f05a805a790 0x7f05a805cc30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05a8053b00 msgr2=0x7f05a8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f05a8053b00 0x7f05a8055fc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f05bc00d000 tx=0x7f05bc009040).stop 2022-08-01T23:51:21.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80b4550 msgr2=0x7f05c8142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 --2- 172.21.15.32:0/2148748066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80b4550 0x7f05c8142370 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f05c8149330 tx=0x7f05c4015000).stop 2022-08-01T23:51:21.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05ce381700 1 -- 172.21.15.32:0/2148748066 reap_dead start 2022-08-01T23:51:21.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 shutdown_connections 2022-08-01T23:51:21.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 >> 172.21.15.32:0/2148748066 conn(0x7f05c805c620 msgr2=0x7f05c8128c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 shutdown_connections 2022-08-01T23:51:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.148+0000 7f05cfde4700 1 -- 172.21.15.32:0/2148748066 wait complete. 2022-08-01T23:51:21.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2022-08-01T23:51:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 Processor -- start 2022-08-01T23:51:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 -- start start 2022-08-01T23:51:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff80b38a0 0x7fdff8064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff81470d0 0x7fdff81403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80b4f60 con 0x7fdff80647d0 2022-08-01T23:51:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff80b4a60 con 0x7fdff80b38a0 2022-08-01T23:51:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80b4cc0 con 0x7fdff81470d0 2022-08-01T23:51:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff80b38a0 0x7fdff8064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff80b38a0 0x7fdff8064bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50456/0 (socket says 172.21.15.32:50456) 2022-08-01T23:51:21.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 -- 172.21.15.32:0/1818857783 learned_addr learned my addr 172.21.15.32:0/1818857783 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff6d9d700 1 --2- 172.21.15.32:0/1818857783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff81470d0 0x7fdff81403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff5d9b700 1 -- 172.21.15.32:0/1818857783 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892959439 0 0) 0x7fdff80b4f60 con 0x7fdff80647d0 2022-08-01T23:51:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff5d9b700 1 -- 172.21.15.32:0/1818857783 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd8003260 con 0x7fdff80647d0 2022-08-01T23:51:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 -- 172.21.15.32:0/1818857783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff81470d0 msgr2=0x7fdff81403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 --2- 172.21.15.32:0/1818857783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff81470d0 0x7fdff81403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 -- 172.21.15.32:0/1818857783 >> v1:172.21.15.55:6789/0 conn(0x7fdff80647d0 legacy=0x7fdff81436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 -- 172.21.15.32:0/1818857783 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8143250 con 0x7fdff80b38a0 2022-08-01T23:51:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdfedd9b700 1 --2- 172.21.15.32:0/1818857783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b38a0 0x7fdff8064bb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fdfe0002f70 tx=0x7fdfe0006eb0).ready entity=mon.0 client_cookie=38b0cfaadd63532 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff5d9b700 1 -- 172.21.15.32:0/1818857783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfe000d5c0 con 0x7fdff80b38a0 2022-08-01T23:51:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff5d9b700 1 -- 172.21.15.32:0/1818857783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0000e90 con 0x7fdff80b38a0 2022-08-01T23:51:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.268+0000 7fdff5d9b700 1 -- 172.21.15.32:0/1818857783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfe000d950 con 0x7fdff80b38a0 2022-08-01T23:51:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 -- 172.21.15.32:0/1818857783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b38a0 msgr2=0x7fdff8064bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 --2- 172.21.15.32:0/1818857783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b38a0 0x7fdff8064bb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fdfe0002f70 tx=0x7fdfe0006eb0).stop 2022-08-01T23:51:21.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 -- 172.21.15.32:0/1818857783 shutdown_connections 2022-08-01T23:51:21.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 --2- 172.21.15.32:0/1818857783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff80b38a0 0x7fdff8064bb0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.289 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 --2- 172.21.15.32:0/1818857783 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff81470d0 0x7fdff81403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 -- 172.21.15.32:0/1818857783 >> 172.21.15.32:0/1818857783 conn(0x7fdff805f840 msgr2=0x7fdff805fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 -- 172.21.15.32:0/1818857783 shutdown_connections 2022-08-01T23:51:21.290 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 -- 172.21.15.32:0/1818857783 wait complete. 2022-08-01T23:51:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 Processor -- start 2022-08-01T23:51:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.272+0000 7fdff759e700 1 -- start start 2022-08-01T23:51:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80647d0 0x7fdff8146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b38a0 0x7fdff813f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 0x7fdff813b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80b5ef0 con 0x7fdff81470d0 2022-08-01T23:51:21.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80b8270 con 0x7fdff80b38a0 2022-08-01T23:51:21.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80b5510 con 0x7fdff80647d0 2022-08-01T23:51:21.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 0x7fdff813b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 0x7fdff813b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50464/0 (socket says 172.21.15.32:50464) 2022-08-01T23:51:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 -- 172.21.15.32:0/3520688171 learned_addr learned my addr 172.21.15.32:0/3520688171 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff659c700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80647d0 0x7fdff8146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdfedd9b700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b38a0 0x7fdff813f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 -- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80647d0 msgr2=0x7fdff8146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80647d0 0x7fdff8146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 -- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b38a0 msgr2=0x7fdff813f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b38a0 0x7fdff813f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 -- 172.21.15.32:0/3520688171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe000b1e0 con 0x7fdff81470d0 2022-08-01T23:51:21.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff6d9d700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 0x7fdff813b800 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fdff0006e00 tx=0x7fdff00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdff0012600 con 0x7fdff81470d0 2022-08-01T23:51:21.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff0012760 con 0x7fdff81470d0 2022-08-01T23:51:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff80b57b0 con 0x7fdff81470d0 2022-08-01T23:51:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdff0012ac0 con 0x7fdff81470d0 2022-08-01T23:51:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.276+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff80b7f80 con 0x7fdff81470d0 2022-08-01T23:51:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.280+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff8147560 con 0x7fdff81470d0 2022-08-01T23:51:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.280+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdff0022070 con 0x7fdff81470d0 2022-08-01T23:51:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.280+0000 7fdfeffff700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfdc0539e0 0x7fdfdc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.280+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fdff006c340 con 0x7fdff81470d0 2022-08-01T23:51:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.280+0000 7fdff659c700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfdc0539e0 0x7fdfdc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.280+0000 7fdff659c700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfdc0539e0 0x7fdfdc055ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fdfe800d000 tx=0x7fdfe8007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.288+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdff0043300 con 0x7fdff81470d0 2022-08-01T23:51:21.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fdff80574e0 con 0x7fdfdc0539e0 2022-08-01T23:51:21.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdfeffff700 1 -- 172.21.15.32:0/3520688171 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdff80574e0 con 0x7fdfdc0539e0 2022-08-01T23:51:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfdc0539e0 msgr2=0x7fdfdc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfdc0539e0 0x7fdfdc055ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fdfe800d000 tx=0x7fdfe8007a20).stop 2022-08-01T23:51:21.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 msgr2=0x7fdff813b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 0x7fdff813b800 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fdff0006e00 tx=0x7fdff00091d0).stop 2022-08-01T23:51:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 shutdown_connections 2022-08-01T23:51:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdfdc0539e0 0x7fdfdc055ea0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff81470d0 0x7fdff813b800 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff80b38a0 0x7fdff813f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 --2- 172.21.15.32:0/3520688171 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdff80647d0 0x7fdff8146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.416+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 >> 172.21.15.32:0/3520688171 conn(0x7fdff805f840 msgr2=0x7fdff8141090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.424 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.420+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 shutdown_connections 2022-08-01T23:51:21.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.420+0000 7fdff759e700 1 -- 172.21.15.32:0/3520688171 wait complete. 2022-08-01T23:51:21.425 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2022-08-01T23:51:21.432 INFO:tasks.ceph:Scrubbing osd.1 2022-08-01T23:51:21.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.540+0000 7faf9cffb700 1 Processor -- start 2022-08-01T23:51:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.540+0000 7faf9cffb700 1 -- start start 2022-08-01T23:51:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf98066780 0x7faf98066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9805f8e0 con 0x7faf98140e40 2022-08-01T23:51:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf980b1630 con 0x7faf980b1bc0 2022-08-01T23:51:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9805fb10 con 0x7faf98066780 2022-08-01T23:51:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf98066780 0x7faf98066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf97fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faf98140e40 0x7faf9813e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38014/0 (socket says 172.21.15.32:38014) 2022-08-01T23:51:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf97fff700 1 -- 172.21.15.32:0/3793046524 learned_addr learned my addr 172.21.15.32:0/3793046524 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf977fe700 1 -- 172.21.15.32:0/3793046524 >> v1:172.21.15.55:6789/0 conn(0x7faf980b1bc0 legacy=0x7faf9805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf977fe700 1 -- 172.21.15.32:0/3793046524 >> v1:172.21.15.32:6789/0 conn(0x7faf98140e40 legacy=0x7faf9813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf977fe700 1 -- 172.21.15.32:0/3793046524 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980b18d0 con 0x7faf98066780 2022-08-01T23:51:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf96ffd700 1 -- 172.21.15.32:0/3793046524 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112900159 0 0) 0x7faf9805f8e0 con 0x7faf98140e40 2022-08-01T23:51:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf977fe700 1 --2- 172.21.15.32:0/3793046524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf98066b60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7faf88002f70 tx=0x7faf88006df0).ready entity=mon.0 client_cookie=a2bdc72127c426a9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf96ffd700 1 -- 172.21.15.32:0/3793046524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf8800d610 con 0x7faf98066780 2022-08-01T23:51:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf96ffd700 1 -- 172.21.15.32:0/3793046524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf88000e90 con 0x7faf98066780 2022-08-01T23:51:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf96ffd700 1 -- 172.21.15.32:0/3793046524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf8800d960 con 0x7faf98066780 2022-08-01T23:51:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- 172.21.15.32:0/3793046524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 msgr2=0x7faf98066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 --2- 172.21.15.32:0/3793046524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf98066b60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7faf88002f70 tx=0x7faf88006df0).stop 2022-08-01T23:51:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- 172.21.15.32:0/3793046524 shutdown_connections 2022-08-01T23:51:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 --2- 172.21.15.32:0/3793046524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf98066b60 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- 172.21.15.32:0/3793046524 >> 172.21.15.32:0/3793046524 conn(0x7faf98063870 msgr2=0x7faf98063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- 172.21.15.32:0/3793046524 shutdown_connections 2022-08-01T23:51:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.544+0000 7faf9cffb700 1 -- 172.21.15.32:0/3793046524 wait complete. 2022-08-01T23:51:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 Processor -- start 2022-08-01T23:51:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 -- start start 2022-08-01T23:51:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf9813d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf980b1bc0 0x7faf98155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faf98140e40 0x7faf98155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b0720 con 0x7faf98066780 2022-08-01T23:51:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98142000 con 0x7faf980b1bc0 2022-08-01T23:51:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf9cffb700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf981422a0 con 0x7faf98140e40 2022-08-01T23:51:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf9813d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf9813d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50474/0 (socket says 172.21.15.32:50474) 2022-08-01T23:51:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf977fe700 1 -- 172.21.15.32:0/1670926277 learned_addr learned my addr 172.21.15.32:0/1670926277 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf8ffff700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf980b1bc0 0x7faf98155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.548+0000 7faf97fff700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faf98140e40 0x7faf98155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf977fe700 1 -- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faf98140e40 msgr2=0x7faf98155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf977fe700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7faf98140e40 0x7faf98155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf977fe700 1 -- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf980b1bc0 msgr2=0x7faf98155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf977fe700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf980b1bc0 0x7faf98155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf977fe700 1 -- 172.21.15.32:0/1670926277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8800b180 con 0x7faf98066780 2022-08-01T23:51:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf977fe700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf9813d1d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7faf88002f70 tx=0x7faf88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf8800d290 con 0x7faf98066780 2022-08-01T23:51:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf88011b80 con 0x7faf98066780 2022-08-01T23:51:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf98142540 con 0x7faf98066780 2022-08-01T23:51:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf8801e460 con 0x7faf98066780 2022-08-01T23:51:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf98064710 con 0x7faf98066780 2022-08-01T23:51:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf981429e0 con 0x7faf98066780 2022-08-01T23:51:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faf88025030 con 0x7faf98066780 2022-08-01T23:51:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.552+0000 7faf957fa700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faf70053aa0 0x7faf70055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7faf8806d040 con 0x7faf98066780 2022-08-01T23:51:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf957fa700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7faf7005a5c0 0x7faf7005ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 --> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf7005d040 con 0x7faf7005a5c0 2022-08-01T23:51:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf8806f6d0 con 0x7faf98066780 2022-08-01T23:51:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf97fff700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7faf7005a5c0 0x7faf7005ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf8ffff700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faf70053aa0 0x7faf70055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf97fff700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7faf7005a5c0 0x7faf7005ca60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf8ffff700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faf70053aa0 0x7faf70055f60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7faf8000d000 tx=0x7faf80007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.556+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== osd.1 v2:172.21.15.32:6800/23005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7faf7005d040 con 0x7faf7005a5c0 2022-08-01T23:51:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 --> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faf98156730 con 0x7faf7005a5c0 2022-08-01T23:51:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf957fa700 1 -- 172.21.15.32:0/1670926277 <== osd.1 v2:172.21.15.32:6800/23005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faf98156730 con 0x7faf7005a5c0 2022-08-01T23:51:21.585 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:21.586 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7faf7005a5c0 msgr2=0x7faf7005ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:6800/23005,v1:172.21.15.32:6801/23005] conn(0x7faf7005a5c0 0x7faf7005ca60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faf70053aa0 msgr2=0x7faf70055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7faf70053aa0 0x7faf70055f60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7faf8000d000 tx=0x7faf80007b90).stop 2022-08-01T23:51:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 msgr2=0x7faf9813d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 --2- 172.21.15.32:0/1670926277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf98066780 0x7faf9813d1d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7faf88002f70 tx=0x7faf88015000).stop 2022-08-01T23:51:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf97fff700 1 -- 172.21.15.32:0/1670926277 reap_dead start 2022-08-01T23:51:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 shutdown_connections 2022-08-01T23:51:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 >> 172.21.15.32:0/1670926277 conn(0x7faf98063870 msgr2=0x7faf98066b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 shutdown_connections 2022-08-01T23:51:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.568+0000 7faf9cffb700 1 -- 172.21.15.32:0/1670926277 wait complete. 2022-08-01T23:51:21.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2022-08-01T23:51:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.704+0000 7f43f1252700 1 Processor -- start 2022-08-01T23:51:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.704+0000 7f43f1252700 1 -- start start 2022-08-01T23:51:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.704+0000 7f43f1252700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43ec148530 0x7f43ec0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.704+0000 7f43f1252700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.704+0000 7f43f1252700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec0b04e0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0a9250 con 0x7f43ec148530 2022-08-01T23:51:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0b0780 con 0x7f43ec0a7250 2022-08-01T23:51:21.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ead9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43ec0b1f40 0x7f43ec0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38026/0 (socket says 172.21.15.32:38026) 2022-08-01T23:51:21.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ead9d700 1 -- 172.21.15.32:0/759698349 learned_addr learned my addr 172.21.15.32:0/759698349 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:21.713 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ea59c700 1 --2- 172.21.15.32:0/759698349 >> v2:172.21.15.55:3300/0 conn(0x7f43ec148530 0x7f43ec0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43eb59e700 1 --2- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43e9d9b700 1 -- 172.21.15.32:0/759698349 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370587394 0 0) 0x7f43ec0b04e0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43e9d9b700 1 -- 172.21.15.32:0/759698349 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d0003260 con 0x7f43ec0b1f40 2022-08-01T23:51:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ea59c700 1 -- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 msgr2=0x7f43ec1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ea59c700 1 --2- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ea59c700 1 -- 172.21.15.32:0/759698349 >> v1:172.21.15.32:6789/0 conn(0x7f43ec0b1f40 legacy=0x7f43ec0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ea59c700 1 -- 172.21.15.32:0/759698349 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0b0a20 con 0x7f43ec148530 2022-08-01T23:51:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43ea59c700 1 --2- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec0b1160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f43ec1478c0 tx=0x7f43dc006df0).ready entity=mon.1 client_cookie=3f21e39197404243 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43e9d9b700 1 -- 172.21.15.32:0/759698349 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc00d570 con 0x7f43ec148530 2022-08-01T23:51:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43e9d9b700 1 -- 172.21.15.32:0/759698349 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc000e90 con 0x7f43ec148530 2022-08-01T23:51:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43e9d9b700 1 -- 172.21.15.32:0/759698349 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc00d8c0 con 0x7f43ec148530 2022-08-01T23:51:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 msgr2=0x7f43ec0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 --2- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec0b1160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f43ec1478c0 tx=0x7f43dc006df0).stop 2022-08-01T23:51:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- 172.21.15.32:0/759698349 shutdown_connections 2022-08-01T23:51:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 --2- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 --2- 172.21.15.32:0/759698349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec0b1160 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- 172.21.15.32:0/759698349 >> 172.21.15.32:0/759698349 conn(0x7f43ec05f840 msgr2=0x7f43ec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- 172.21.15.32:0/759698349 shutdown_connections 2022-08-01T23:51:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.708+0000 7f43f1252700 1 -- 172.21.15.32:0/759698349 wait complete. 2022-08-01T23:51:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 Processor -- start 2022-08-01T23:51:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 -- start start 2022-08-01T23:51:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 0x7f43ec146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0abf40 con 0x7f43ec0b1f40 2022-08-01T23:51:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0ac170 con 0x7f43ec148530 2022-08-01T23:51:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43f1252700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0ac410 con 0x7f43ec0a7250 2022-08-01T23:51:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 0x7f43ec146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 0x7f43ec146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50490/0 (socket says 172.21.15.32:50490) 2022-08-01T23:51:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43ea59c700 1 -- 172.21.15.32:0/3149741255 learned_addr learned my addr 172.21.15.32:0/3149741255 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43ead9d700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.712+0000 7f43eb59e700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ea59c700 1 -- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 msgr2=0x7f43ec14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ea59c700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ea59c700 1 -- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 msgr2=0x7f43ec14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ea59c700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec14b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ea59c700 1 -- 172.21.15.32:0/3149741255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc00b180 con 0x7f43ec0b1f40 2022-08-01T23:51:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ea59c700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 0x7f43ec146240 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f43ec1478c0 tx=0x7f43dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc016070 con 0x7f43ec0b1f40 2022-08-01T23:51:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc013780 con 0x7f43ec0b1f40 2022-08-01T23:51:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc013ae0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec0ac6b0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec0aabc0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec0574e0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f43dc00f030 con 0x7f43ec0b1f40 2022-08-01T23:51:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43dbfff700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43c8053a70 0x7f43c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f43dc06c8a0 con 0x7f43ec0b1f40 2022-08-01T23:51:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.716+0000 7f43ead9d700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43c8053a70 0x7f43c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.720+0000 7f43ead9d700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43c8053a70 0x7f43c8055f30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f43e000d000 tx=0x7f43e0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.724+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43dc043940 con 0x7f43ec0b1f40 2022-08-01T23:51:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.844+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f43ec14b3d0 con 0x7f43c8053a70 2022-08-01T23:51:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.844+0000 7f43dbfff700 1 -- 172.21.15.32:0/3149741255 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f43e00133f0 con 0x7f43c8053a70 2022-08-01T23:51:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43c8053a70 msgr2=0x7f43c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43c8053a70 0x7f43c8055f30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f43e000d000 tx=0x7f43e0009040).stop 2022-08-01T23:51:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 msgr2=0x7f43ec146240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 0x7f43ec146240 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f43ec1478c0 tx=0x7f43dc014000).stop 2022-08-01T23:51:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 shutdown_connections 2022-08-01T23:51:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f43c8053a70 0x7f43c8055f30 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43ec148530 0x7f43ec14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ec0b1f40 0x7f43ec146240 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 --2- 172.21.15.32:0/3149741255 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f43ec0a7250 0x7f43ec14a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 >> 172.21.15.32:0/3149741255 conn(0x7f43ec05f840 msgr2=0x7f43ec147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 shutdown_connections 2022-08-01T23:51:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.848+0000 7f43f1252700 1 -- 172.21.15.32:0/3149741255 wait complete. 2022-08-01T23:51:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2022-08-01T23:51:21.864 INFO:tasks.ceph:Scrubbing osd.2 2022-08-01T23:51:21.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.968+0000 7f9bcd4ad700 1 Processor -- start 2022-08-01T23:51:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.968+0000 7f9bcd4ad700 1 -- start start 2022-08-01T23:51:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bc80b4550 0x7f9bc80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc8144240 con 0x7f9bc8061ec0 2022-08-01T23:51:21.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc80b6fe0 con 0x7f9bc80b4550 2022-08-01T23:51:21.977 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8143d00 con 0x7f9bc80b6710 2022-08-01T23:51:21.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bc80b4550 0x7f9bc80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.978 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9bc8061ec0 0x7f9bc8146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38044/0 (socket says 172.21.15.32:38044) 2022-08-01T23:51:21.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc77fe700 1 -- 172.21.15.32:0/2294660841 learned_addr learned my addr 172.21.15.32:0/2294660841 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:21.979 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc6ffd700 1 --2- 172.21.15.32:0/2294660841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc5ffb700 1 -- 172.21.15.32:0/2294660841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441491025 0 0) 0x7f9bc8144240 con 0x7f9bc8061ec0 2022-08-01T23:51:21.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc5ffb700 1 -- 172.21.15.32:0/2294660841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba8003260 con 0x7f9bc8061ec0 2022-08-01T23:51:21.980 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc67fc700 1 -- 172.21.15.32:0/2294660841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 msgr2=0x7f9bc80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc67fc700 1 --2- 172.21.15.32:0/2294660841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc67fc700 1 -- 172.21.15.32:0/2294660841 >> v1:172.21.15.32:6789/0 conn(0x7f9bc8061ec0 legacy=0x7f9bc8146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc67fc700 1 -- 172.21.15.32:0/2294660841 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8143fa0 con 0x7f9bc80b4550 2022-08-01T23:51:21.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc67fc700 1 --2- 172.21.15.32:0/2294660841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc80b6af0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f9bc0001b80 tx=0x7f9bb0006df0).ready entity=mon.0 client_cookie=71aa7cce99277c07 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.982 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc5ffb700 1 -- 172.21.15.32:0/2294660841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bc00019e0 con 0x7f9bc80b4550 2022-08-01T23:51:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc5ffb700 1 -- 172.21.15.32:0/2294660841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb0000e90 con 0x7f9bc80b4550 2022-08-01T23:51:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bc5ffb700 1 -- 172.21.15.32:0/2294660841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb0011400 con 0x7f9bc80b4550 2022-08-01T23:51:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/2294660841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 msgr2=0x7f9bc80b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 --2- 172.21.15.32:0/2294660841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc80b6af0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f9bc0001b80 tx=0x7f9bb0006df0).stop 2022-08-01T23:51:21.984 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/2294660841 shutdown_connections 2022-08-01T23:51:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 --2- 172.21.15.32:0/2294660841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc80b6af0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 --2- 172.21.15.32:0/2294660841 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/2294660841 >> 172.21.15.32:0/2294660841 conn(0x7f9bc805c620 msgr2=0x7f9bc805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/2294660841 shutdown_connections 2022-08-01T23:51:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.972+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/2294660841 wait complete. 2022-08-01T23:51:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 Processor -- start 2022-08-01T23:51:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 -- start start 2022-08-01T23:51:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc8061ec0 0x7f9bc812ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc81449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc812a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8129c10 con 0x7f9bc80b4550 2022-08-01T23:51:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b2bf0 con 0x7f9bc8061ec0 2022-08-01T23:51:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b2e90 con 0x7f9bc80b6710 2022-08-01T23:51:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc81449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc81449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50506/0 (socket says 172.21.15.32:50506) 2022-08-01T23:51:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc8061ec0 0x7f9bc812ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 -- 172.21.15.32:0/1772350354 learned_addr learned my addr 172.21.15.32:0/1772350354 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc77fe700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc812a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 -- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 msgr2=0x7f9bc812a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f9bc80b6710 0x7f9bc812a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 -- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc8061ec0 msgr2=0x7f9bc812ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc8061ec0 0x7f9bc812ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 -- 172.21.15.32:0/1772350354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb000b180 con 0x7f9bc80b4550 2022-08-01T23:51:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc6ffd700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc8061ec0 0x7f9bc812ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bc67fc700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc81449c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f9bc0001b80 tx=0x7f9bc80615c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb0015070 con 0x7f9bc80b4550 2022-08-01T23:51:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb000d730 con 0x7f9bc80b4550 2022-08-01T23:51:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb000da90 con 0x7f9bc80b4550 2022-08-01T23:51:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.976+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc80b3130 con 0x7f9bc80b4550 2022-08-01T23:51:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc8152160 con 0x7f9bc80b4550 2022-08-01T23:51:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9bc8129da0 con 0x7f9bc80b4550 2022-08-01T23:51:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9bb000f070 con 0x7f9bc80b4550 2022-08-01T23:51:21.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bbffff700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bac053b00 0x7f9bac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9bb006c7d0 con 0x7f9bc80b4550 2022-08-01T23:51:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bbffff700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f9bac05a790 0x7f9bac05cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 --> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bac05d210 con 0x7f9bac05a790 2022-08-01T23:51:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bb0071240 con 0x7f9bc80b4550 2022-08-01T23:51:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bc6ffd700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bac053b00 0x7f9bac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bc77fe700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f9bac05a790 0x7f9bac05cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bc77fe700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f9bac05a790 0x7f9bac05cc30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.980+0000 7f9bc6ffd700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bac053b00 0x7f9bac055fc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f9bb800b000 tx=0x7f9bb8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:21.984+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== osd.2 v2:172.21.15.32:6808/23004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f9bac05d210 con 0x7f9bac05a790 2022-08-01T23:51:22.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 --> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9bc812a7f0 con 0x7f9bac05a790 2022-08-01T23:51:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bbffff700 1 -- 172.21.15.32:0/1772350354 <== osd.2 v2:172.21.15.32:6808/23004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9bc812a7f0 con 0x7f9bac05a790 2022-08-01T23:51:22.006 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:22.007 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f9bac05a790 msgr2=0x7f9bac05cc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:6808/23004,v1:172.21.15.32:6809/23004] conn(0x7f9bac05a790 0x7f9bac05cc30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bac053b00 msgr2=0x7f9bac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f9bac053b00 0x7f9bac055fc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f9bb800b000 tx=0x7f9bb8006df0).stop 2022-08-01T23:51:22.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 msgr2=0x7f9bc81449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 --2- 172.21.15.32:0/1772350354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80b4550 0x7f9bc81449c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f9bc0001b80 tx=0x7f9bc80615c0).stop 2022-08-01T23:51:22.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bc77fe700 1 -- 172.21.15.32:0/1772350354 reap_dead start 2022-08-01T23:51:22.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 shutdown_connections 2022-08-01T23:51:22.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 >> 172.21.15.32:0/1772350354 conn(0x7f9bc805c620 msgr2=0x7f9bc81420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 shutdown_connections 2022-08-01T23:51:22.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.004+0000 7f9bcd4ad700 1 -- 172.21.15.32:0/1772350354 wait complete. 2022-08-01T23:51:22.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2022-08-01T23:51:22.130 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc05f10700 1 Processor -- start 2022-08-01T23:51:22.131 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc05f10700 1 -- start start 2022-08-01T23:51:22.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc05f10700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc000b3070 0x7fcc00065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.134 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc05f10700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc000b2c20 con 0x7fcc0014b040 2022-08-01T23:51:22.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc05f10700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc000b7c20 con 0x7fcc000b7840 2022-08-01T23:51:22.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc05f10700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc00061830 con 0x7fcc000b3070 2022-08-01T23:51:22.135 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc0570f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc0014b040 0x7fcc0014b480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38060/0 (socket says 172.21.15.32:38060) 2022-08-01T23:51:22.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc0570f700 1 -- 172.21.15.32:0/4011219056 learned_addr learned my addr 172.21.15.32:0/4011219056 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:22.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbfffff700 1 --2- 172.21.15.32:0/4011219056 >> v2:172.21.15.55:3300/0 conn(0x7fcc000b3070 0x7fcc00065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.136 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057725037 0 0) 0x7fcc000b2c20 con 0x7fcc0014b040 2022-08-01T23:51:22.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe4003260 con 0x7fcc0014b040 2022-08-01T23:51:22.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709704127 0 0) 0x7fcc000b7c20 con 0x7fcc000b7840 2022-08-01T23:51:22.137 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc000b2c20 con 0x7fcc000b7840 2022-08-01T23:51:22.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 125016770 0 0) 0x7fcbe4003260 con 0x7fcc0014b040 2022-08-01T23:51:22.138 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 >> v2:172.21.15.55:3300/0 conn(0x7fcc000b3070 msgr2=0x7fcc00065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:22.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 --2- 172.21.15.32:0/4011219056 >> v2:172.21.15.55:3300/0 conn(0x7fcc000b3070 0x7fcc00065720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 >> v1:172.21.15.55:6789/0 conn(0x7fcc000b7840 legacy=0x7fcc00061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.139 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc000b2980 con 0x7fcc0014b040 2022-08-01T23:51:22.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fcbf8001660 con 0x7fcc0014b040 2022-08-01T23:51:22.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbfffff700 1 --2- 172.21.15.32:0/4011219056 >> v2:172.21.15.55:3300/0 conn(0x7fcc000b3070 0x7fcc00065720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:22.140 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 >> v1:172.21.15.32:6789/0 conn(0x7fcc0014b040 legacy=0x7fcc0014b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcbe4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbe4004500 0x7fcbe4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe4004e70 0x7fcbe400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00061830 con 0x7fcbe40038e0 2022-08-01T23:51:22.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe400bd20 con 0x7fcbe4004e70 2022-08-01T23:51:22.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe400bee0 con 0x7fcbe4004500 2022-08-01T23:51:22.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcbe4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcbfffff700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbe4004500 0x7fcbe4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.143 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.128+0000 7fcc0570f700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe4004e70 0x7fcbe400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc04f0e700 1 -- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbe4004500 msgr2=0x7fcbe4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcbe4004500 0x7fcbe4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc04f0e700 1 -- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe4004e70 msgr2=0x7fcbe400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe4004e70 0x7fcbe400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc04f0e700 1 -- 172.21.15.32:0/4011219056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe4003260 con 0x7fcbe40038e0 2022-08-01T23:51:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc0570f700 1 -- 172.21.15.32:0/4011219056 reap_dead start 2022-08-01T23:51:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcbe4003d00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7fcbe80014e0 tx=0x7fcbe8007dd0).ready entity=mon.0 client_cookie=e2dd08e22f73f22b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbe80017b0 con 0x7fcbe40038e0 2022-08-01T23:51:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe80068e0 con 0x7fcbe40038e0 2022-08-01T23:51:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcbff7fe700 1 -- 172.21.15.32:0/4011219056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbe8006c40 con 0x7fcbe40038e0 2022-08-01T23:51:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 -- 172.21.15.32:0/4011219056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 msgr2=0x7fcbe4003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcbe4003d00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7fcbe80014e0 tx=0x7fcbe8007dd0).stop 2022-08-01T23:51:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 -- 172.21.15.32:0/4011219056 shutdown_connections 2022-08-01T23:51:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 --2- 172.21.15.32:0/4011219056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcbe4003d00 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 -- 172.21.15.32:0/4011219056 >> 172.21.15.32:0/4011219056 conn(0x7fcc0005f840 msgr2=0x7fcc0005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 -- 172.21.15.32:0/4011219056 shutdown_connections 2022-08-01T23:51:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 -- 172.21.15.32:0/4011219056 wait complete. 2022-08-01T23:51:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 Processor -- start 2022-08-01T23:51:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.132+0000 7fcc05f10700 1 -- start start 2022-08-01T23:51:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcc00065c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc000b07d0 0x7fcc000a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc000afcb0 0x7fcc000a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000a10d0 con 0x7fcbe40038e0 2022-08-01T23:51:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0009eae0 con 0x7fcc000b07d0 2022-08-01T23:51:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0009ed80 con 0x7fcc000afcb0 2022-08-01T23:51:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcc00065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcc00065c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50526/0 (socket says 172.21.15.32:50526) 2022-08-01T23:51:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 -- 172.21.15.32:0/244924537 learned_addr learned my addr 172.21.15.32:0/244924537 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcbfffff700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc000b07d0 0x7fcc000a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc0570f700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc000afcb0 0x7fcc000a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 -- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc000afcb0 msgr2=0x7fcc000a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc000afcb0 0x7fcc000a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 -- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc000b07d0 msgr2=0x7fcc000a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc000b07d0 0x7fcc000a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 -- 172.21.15.32:0/244924537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0009f020 con 0x7fcbe40038e0 2022-08-01T23:51:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcbfffff700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc000b07d0 0x7fcc000a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:51:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc04f0e700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcc00065c30 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7fcbe80014e0 tx=0x7fcbe8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbe80017b0 con 0x7fcbe40038e0 2022-08-01T23:51:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe800c3f0 con 0x7fcbe40038e0 2022-08-01T23:51:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbe801f5f0 con 0x7fcbe40038e0 2022-08-01T23:51:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc000b0e10 con 0x7fcbe40038e0 2022-08-01T23:51:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.136+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc00149fe0 con 0x7fcbe40038e0 2022-08-01T23:51:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.140+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc000609e0 con 0x7fcbe40038e0 2022-08-01T23:51:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.140+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcbe8026030 con 0x7fcbe40038e0 2022-08-01T23:51:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.140+0000 7fcbfdffb700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbd8053aa0 0x7fcbd8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.140+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcbe806cde0 con 0x7fcbe40038e0 2022-08-01T23:51:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.140+0000 7fcbfffff700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbd8053aa0 0x7fcbd8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.140+0000 7fcbfffff700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbd8053aa0 0x7fcbd8055f60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fcbf000b000 tx=0x7fcbf0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.144+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcbe8043f30 con 0x7fcbe40038e0 2022-08-01T23:51:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.272+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc0009f1b0 con 0x7fcbd8053aa0 2022-08-01T23:51:22.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.272+0000 7fcbfdffb700 1 -- 172.21.15.32:0/244924537 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcbf0013aa0 con 0x7fcbd8053aa0 2022-08-01T23:51:22.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbd8053aa0 msgr2=0x7fcbd8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbd8053aa0 0x7fcbd8055f60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fcbf000b000 tx=0x7fcbf0008040).stop 2022-08-01T23:51:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 msgr2=0x7fcc00065c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcc00065c30 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7fcbe80014e0 tx=0x7fcbe8015000).stop 2022-08-01T23:51:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 shutdown_connections 2022-08-01T23:51:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcbd8053aa0 0x7fcbd8055f60 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc000afcb0 0x7fcc000a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc000b07d0 0x7fcc000a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 --2- 172.21.15.32:0/244924537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbe40038e0 0x7fcc00065c30 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 >> 172.21.15.32:0/244924537 conn(0x7fcc0005f840 msgr2=0x7fcc0005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 shutdown_connections 2022-08-01T23:51:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.276+0000 7fcc05f10700 1 -- 172.21.15.32:0/244924537 wait complete. 2022-08-01T23:51:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2022-08-01T23:51:22.294 INFO:tasks.ceph:Scrubbing osd.3 2022-08-01T23:51:22.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.404+0000 7febb192f700 1 Processor -- start 2022-08-01T23:51:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.404+0000 7febb192f700 1 -- start start 2022-08-01T23:51:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.404+0000 7febb192f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7febac0614b0 0x7febac061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.404+0000 7febb192f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac1405d0 0x7febac140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.404+0000 7febb192f700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0aee80 con 0x7febac0ad750 2022-08-01T23:51:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0b70b0 con 0x7febac0614b0 2022-08-01T23:51:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0af0b0 con 0x7febac1405d0 2022-08-01T23:51:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7febac0614b0 0x7febac061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7febac0614b0 0x7febac061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50534/0 (socket says 172.21.15.32:50534) 2022-08-01T23:51:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 -- 172.21.15.32:0/2479745625 learned_addr learned my addr 172.21.15.32:0/2479745625 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb112e700 1 --2- 172.21.15.32:0/2479745625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac1405d0 0x7febac140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 -- 172.21.15.32:0/2479745625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac1405d0 msgr2=0x7febac140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 --2- 172.21.15.32:0/2479745625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac1405d0 0x7febac140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 -- 172.21.15.32:0/2479745625 >> v1:172.21.15.55:6789/0 conn(0x7febac0ad750 legacy=0x7febac0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.416 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 -- 172.21.15.32:0/2479745625 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0af350 con 0x7febac0614b0 2022-08-01T23:51:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb092d700 1 --2- 172.21.15.32:0/2479745625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac0614b0 0x7febac061890 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7febac0aed30 tx=0x7feb9c006eb0).ready entity=mon.0 client_cookie=baac3db3195f7d0f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febab7fe700 1 -- 172.21.15.32:0/2479745625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb9c00d5c0 con 0x7febac0614b0 2022-08-01T23:51:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febab7fe700 1 -- 172.21.15.32:0/2479745625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c000e90 con 0x7febac0614b0 2022-08-01T23:51:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febab7fe700 1 -- 172.21.15.32:0/2479745625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb9c00d950 con 0x7febac0614b0 2022-08-01T23:51:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- 172.21.15.32:0/2479745625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac0614b0 msgr2=0x7febac061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 --2- 172.21.15.32:0/2479745625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac0614b0 0x7febac061890 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7febac0aed30 tx=0x7feb9c006eb0).stop 2022-08-01T23:51:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- 172.21.15.32:0/2479745625 shutdown_connections 2022-08-01T23:51:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 --2- 172.21.15.32:0/2479745625 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac1405d0 0x7febac140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 --2- 172.21.15.32:0/2479745625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac0614b0 0x7febac061890 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- 172.21.15.32:0/2479745625 >> 172.21.15.32:0/2479745625 conn(0x7febac066f70 msgr2=0x7febac067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- 172.21.15.32:0/2479745625 shutdown_connections 2022-08-01T23:51:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- 172.21.15.32:0/2479745625 wait complete. 2022-08-01T23:51:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 Processor -- start 2022-08-01T23:51:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.408+0000 7febb192f700 1 -- start start 2022-08-01T23:51:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac0614b0 0x7febac0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febac0ad750 0x7febac09eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac1405d0 0x7febac155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac153690 con 0x7febac1405d0 2022-08-01T23:51:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0b0010 con 0x7febac0ad750 2022-08-01T23:51:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac1533f0 con 0x7febac0614b0 2022-08-01T23:51:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac1405d0 0x7febac155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac1405d0 0x7febac155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50544/0 (socket says 172.21.15.32:50544) 2022-08-01T23:51:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 -- 172.21.15.32:0/1397178178 learned_addr learned my addr 172.21.15.32:0/1397178178 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febabfff700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febac0ad750 0x7febac09eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb092d700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac0614b0 0x7febac0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 -- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac0614b0 msgr2=0x7febac0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7febac0614b0 0x7febac0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 -- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febac0ad750 msgr2=0x7febac09eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febac0ad750 0x7febac09eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 -- 172.21.15.32:0/1397178178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c00b1e0 con 0x7febac1405d0 2022-08-01T23:51:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb112e700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac1405d0 0x7febac155ca0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7feba4006e00 tx=0x7feba40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba4012600 con 0x7febac1405d0 2022-08-01T23:51:22.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba4012760 con 0x7febac1405d0 2022-08-01T23:51:22.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0b0fc0 con 0x7febac1405d0 2022-08-01T23:51:22.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feba4012ac0 con 0x7febac1405d0 2022-08-01T23:51:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac0b0b00 con 0x7febac1405d0 2022-08-01T23:51:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7febac0b1460 con 0x7febac1405d0 2022-08-01T23:51:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.412+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feba4022070 con 0x7febac1405d0 2022-08-01T23:51:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7feba9ffb700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb8c0539e0 0x7feb8c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7feba406c3d0 con 0x7febac1405d0 2022-08-01T23:51:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7feba9ffb700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7feb8c05a670 0x7feb8c05cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 --> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb8c05d190 con 0x7feb8c05a670 2022-08-01T23:51:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba406e9e0 con 0x7febac1405d0 2022-08-01T23:51:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7febb092d700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb8c0539e0 0x7feb8c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7febabfff700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7feb8c05a670 0x7feb8c05cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7febb092d700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb8c0539e0 0x7feb8c055ea0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7feb9c01c000 tx=0x7feb9c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7febabfff700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7feb8c05a670 0x7feb8c05cb10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.416+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== osd.3 v2:172.21.15.32:6816/23015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7feb8c05d190 con 0x7feb8c05a670 2022-08-01T23:51:22.442 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 --> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7febac0af5f0 con 0x7feb8c05a670 2022-08-01T23:51:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7feba9ffb700 1 -- 172.21.15.32:0/1397178178 <== osd.3 v2:172.21.15.32:6816/23015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7febac0af5f0 con 0x7feb8c05a670 2022-08-01T23:51:22.443 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:22.444 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7feb8c05a670 msgr2=0x7feb8c05cb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:6816/23015,v1:172.21.15.32:6817/23015] conn(0x7feb8c05a670 0x7feb8c05cb10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb8c0539e0 msgr2=0x7feb8c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7feb8c0539e0 0x7feb8c055ea0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7feb9c01c000 tx=0x7feb9c015000).stop 2022-08-01T23:51:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac1405d0 msgr2=0x7febac155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 --2- 172.21.15.32:0/1397178178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febac1405d0 0x7febac155ca0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7feba4006e00 tx=0x7feba40091d0).stop 2022-08-01T23:51:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb112e700 1 -- 172.21.15.32:0/1397178178 reap_dead start 2022-08-01T23:51:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 shutdown_connections 2022-08-01T23:51:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 >> 172.21.15.32:0/1397178178 conn(0x7febac066f70 msgr2=0x7febac143790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 shutdown_connections 2022-08-01T23:51:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.436+0000 7febb192f700 1 -- 172.21.15.32:0/1397178178 wait complete. 2022-08-01T23:51:22.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2022-08-01T23:51:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 Processor -- start 2022-08-01T23:51:22.562 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- start start 2022-08-01T23:51:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f033c0afd60 0x7f033c0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0a7250 con 0x7f033c0b2710 2022-08-01T23:51:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0a7480 con 0x7f033c0afd60 2022-08-01T23:51:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0a7720 con 0x7f033c0b01a0 2022-08-01T23:51:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03408b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f033c0afd60 0x7f033c0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f033c0b2710 0x7f033c14afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38092/0 (socket says 172.21.15.32:38092) 2022-08-01T23:51:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033bfff700 1 -- 172.21.15.32:0/3680756380 learned_addr learned my addr 172.21.15.32:0/3680756380 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:22.567 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03337fe700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033b7fe700 1 -- 172.21.15.32:0/3680756380 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100805771 0 0) 0x7f033c0a7250 con 0x7f033c0b2710 2022-08-01T23:51:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033b7fe700 1 -- 172.21.15.32:0/3680756380 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f031c003260 con 0x7f033c0b2710 2022-08-01T23:51:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03408b2700 1 -- 172.21.15.32:0/3680756380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 msgr2=0x7f033c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03408b2700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03408b2700 1 -- 172.21.15.32:0/3680756380 >> v1:172.21.15.32:6789/0 conn(0x7f033c0b2710 legacy=0x7f033c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03408b2700 1 -- 172.21.15.32:0/3680756380 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c0a79c0 con 0x7f033c0afd60 2022-08-01T23:51:22.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03337fe700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:22.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03408b2700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0afd60 0x7f033c0a8a30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f03340032f0 tx=0x7f03340091d0).ready entity=mon.0 client_cookie=161f93990da84d92 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033b7fe700 1 -- 172.21.15.32:0/3680756380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033400b600 con 0x7f033c0afd60 2022-08-01T23:51:22.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033b7fe700 1 -- 172.21.15.32:0/3680756380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033400b760 con 0x7f033c0afd60 2022-08-01T23:51:22.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f033b7fe700 1 -- 172.21.15.32:0/3680756380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033400b8d0 con 0x7f033c0afd60 2022-08-01T23:51:22.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- 172.21.15.32:0/3680756380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0afd60 msgr2=0x7f033c0a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0afd60 0x7f033c0a8a30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f03340032f0 tx=0x7f03340091d0).stop 2022-08-01T23:51:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- 172.21.15.32:0/3680756380 shutdown_connections 2022-08-01T23:51:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0afd60 0x7f033c0a8a30 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 --2- 172.21.15.32:0/3680756380 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- 172.21.15.32:0/3680756380 >> 172.21.15.32:0/3680756380 conn(0x7f033c060310 msgr2=0x7f033c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- 172.21.15.32:0/3680756380 shutdown_connections 2022-08-01T23:51:22.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.560+0000 7f03410b3700 1 -- 172.21.15.32:0/3680756380 wait complete. 2022-08-01T23:51:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 Processor -- start 2022-08-01T23:51:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 -- start start 2022-08-01T23:51:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033c0afd60 0x7f033c14cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 0x7f033c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b30e0 con 0x7f033c0b2710 2022-08-01T23:51:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0ade00 con 0x7f033c0afd60 2022-08-01T23:51:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0ae0a0 con 0x7f033c0b01a0 2022-08-01T23:51:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 0x7f033c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 0x7f033c156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50560/0 (socket says 172.21.15.32:50560) 2022-08-01T23:51:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 -- 172.21.15.32:0/208830595 learned_addr learned my addr 172.21.15.32:0/208830595 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f033bfff700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033c0afd60 0x7f033c14cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03337fe700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 -- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 msgr2=0x7f033c0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 -- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033c0afd60 msgr2=0x7f033c14cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033c0afd60 0x7f033c14cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 -- 172.21.15.32:0/208830595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032400b180 con 0x7f033c0b2710 2022-08-01T23:51:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03408b2700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 0x7f033c156490 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f03340032f0 tx=0x7f0334016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033400b600 con 0x7f033c0b2710 2022-08-01T23:51:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334011b60 con 0x7f033c0b2710 2022-08-01T23:51:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f033401f3f0 con 0x7f033c0b2710 2022-08-01T23:51:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0334009470 con 0x7f033c0b2710 2022-08-01T23:51:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.564+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c0a99c0 con 0x7f033c0b2710 2022-08-01T23:51:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.568+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c0574e0 con 0x7f033c0b2710 2022-08-01T23:51:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.568+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0334026030 con 0x7f033c0b2710 2022-08-01T23:51:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.568+0000 7f0339ffb700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0320053aa0 0x7f0320055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.568+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f033406c690 con 0x7f033c0b2710 2022-08-01T23:51:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.568+0000 7f033bfff700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0320053aa0 0x7f0320055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.568+0000 7f033bfff700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0320053aa0 0x7f0320055f60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f032c00d000 tx=0x7f032c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.572+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0334043730 con 0x7f033c0b2710 2022-08-01T23:51:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f033c0b3220 con 0x7f0320053aa0 2022-08-01T23:51:22.703 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f0339ffb700 1 -- 172.21.15.32:0/208830595 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f033c0b3220 con 0x7f0320053aa0 2022-08-01T23:51:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0320053aa0 msgr2=0x7f0320055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0320053aa0 0x7f0320055f60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f032c00d000 tx=0x7f032c009040).stop 2022-08-01T23:51:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 msgr2=0x7f033c156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.706 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 0x7f033c156490 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f03340032f0 tx=0x7f0334016000).stop 2022-08-01T23:51:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 shutdown_connections 2022-08-01T23:51:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0320053aa0 0x7f0320055f60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.707 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f033c0b2710 0x7f033c156490 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f033c0b01a0 0x7f033c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 --2- 172.21.15.32:0/208830595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033c0afd60 0x7f033c14cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.700+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 >> 172.21.15.32:0/208830595 conn(0x7f033c060310 msgr2=0x7f033c0672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.704+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 shutdown_connections 2022-08-01T23:51:22.709 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.704+0000 7f03410b3700 1 -- 172.21.15.32:0/208830595 wait complete. 2022-08-01T23:51:22.710 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2022-08-01T23:51:22.716 INFO:tasks.ceph:Scrubbing osd.4 2022-08-01T23:51:22.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5d5e3700 1 Processor -- start 2022-08-01T23:51:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5d5e3700 1 -- start start 2022-08-01T23:51:22.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5d5e3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5a580614b0 0x7f5a58061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.834 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5d5e3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a580aee80 con 0x7f5a581405d0 2022-08-01T23:51:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5d5e3700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a580b70b0 con 0x7f5a580ad750 2022-08-01T23:51:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5d5e3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a580af0b0 con 0x7f5a580614b0 2022-08-01T23:51:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5cde2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5a581405d0 0x7f5a581466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38108/0 (socket says 172.21.15.32:38108) 2022-08-01T23:51:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a5cde2700 1 -- 172.21.15.32:0/3810345481 learned_addr learned my addr 172.21.15.32:0/3810345481 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a57fff700 1 --2- 172.21.15.32:0/3810345481 >> v2:172.21.15.55:3300/0 conn(0x7f5a580614b0 0x7f5a58061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993706136 0 0) 0x7f5a580aee80 con 0x7f5a581405d0 2022-08-01T23:51:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a38003260 con 0x7f5a581405d0 2022-08-01T23:51:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452739551 0 0) 0x7f5a580b70b0 con 0x7f5a580ad750 2022-08-01T23:51:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.828+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a580aee80 con 0x7f5a580ad750 2022-08-01T23:51:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a57fff700 1 -- 172.21.15.32:0/3810345481 >> v1:172.21.15.55:6789/0 conn(0x7f5a580ad750 legacy=0x7f5a580b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a57fff700 1 -- 172.21.15.32:0/3810345481 >> v1:172.21.15.32:6789/0 conn(0x7f5a581405d0 legacy=0x7f5a581466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4075699299 0 0) 0x7f5a38003260 con 0x7f5a581405d0 2022-08-01T23:51:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a57fff700 1 -- 172.21.15.32:0/3810345481 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a580af350 con 0x7f5a580614b0 2022-08-01T23:51:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a57fff700 1 --2- 172.21.15.32:0/3810345481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580614b0 0x7f5a58061890 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f5a50001110 tx=0x7f5a480091d0).ready entity=mon.1 client_cookie=3d38e5fa50226c02 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a50001660 con 0x7f5a580614b0 2022-08-01T23:51:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4800f510 con 0x7f5a580614b0 2022-08-01T23:51:22.841 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a56ffd700 1 -- 172.21.15.32:0/3810345481 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a4800f680 con 0x7f5a580614b0 2022-08-01T23:51:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3810345481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580614b0 msgr2=0x7f5a58061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 --2- 172.21.15.32:0/3810345481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580614b0 0x7f5a58061890 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f5a50001110 tx=0x7f5a480091d0).stop 2022-08-01T23:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3810345481 shutdown_connections 2022-08-01T23:51:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 --2- 172.21.15.32:0/3810345481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580614b0 0x7f5a58061890 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3810345481 >> 172.21.15.32:0/3810345481 conn(0x7f5a58066f70 msgr2=0x7f5a58067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3810345481 shutdown_connections 2022-08-01T23:51:22.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3810345481 wait complete. 2022-08-01T23:51:22.850 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 Processor -- start 2022-08-01T23:51:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- start start 2022-08-01T23:51:22.852 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a580614b0 0x7f5a580aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580ad750 0x7f5a58141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a581405d0 0x7f5a58141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.853 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a58156950 con 0x7f5a581405d0 2022-08-01T23:51:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580b2fd0 con 0x7f5a580ad750 2022-08-01T23:51:22.854 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5d5e3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580b3270 con 0x7f5a580614b0 2022-08-01T23:51:22.855 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5cde2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a581405d0 0x7f5a58141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5cde2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a581405d0 0x7f5a58141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50574/0 (socket says 172.21.15.32:50574) 2022-08-01T23:51:22.856 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a577fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580ad750 0x7f5a58141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a5cde2700 1 -- 172.21.15.32:0/3770355734 learned_addr learned my addr 172.21.15.32:0/3770355734 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:22.857 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.832+0000 7f5a57fff700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a580614b0 0x7f5a580aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a577fe700 1 -- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a580614b0 msgr2=0x7f5a580aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a577fe700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f5a580614b0 0x7f5a580aa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a577fe700 1 -- 172.21.15.32:0/3770355734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a581405d0 msgr2=0x7f5a58141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a577fe700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a581405d0 0x7f5a58141c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a577fe700 1 -- 172.21.15.32:0/3770355734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48009470 con 0x7f5a580ad750 2022-08-01T23:51:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a577fe700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580ad750 0x7f5a58141770 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f5a40001110 tx=0x7f5a40007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== mon.1 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a4000ce80 con 0x7f5a580ad750 2022-08-01T23:51:22.861 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== mon.1 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40012400 con 0x7f5a580ad750 2022-08-01T23:51:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a580b3510 con 0x7f5a580ad750 2022-08-01T23:51:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== mon.1 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a40012760 con 0x7f5a580ad750 2022-08-01T23:51:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a581414b0 con 0x7f5a580ad750 2022-08-01T23:51:22.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a580b36a0 con 0x7f5a580ad750 2022-08-01T23:51:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== mon.1 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a40019030 con 0x7f5a580ad750 2022-08-01T23:51:22.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a3c0539e0 0x7f5a3c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== mon.1 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5a4006b0a0 con 0x7f5a580ad750 2022-08-01T23:51:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7f5a3c05a670 0x7f5a3c05cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 --> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a3c05d190 con 0x7f5a3c05a670 2022-08-01T23:51:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== mon.1 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a4006d710 con 0x7f5a580ad750 2022-08-01T23:51:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a57fff700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a3c0539e0 0x7f5a3c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.836+0000 7f5a5cde2700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7f5a3c05a670 0x7f5a3c05cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.840+0000 7f5a57fff700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a3c0539e0 0x7f5a3c055ea0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5a4801c000 tx=0x7f5a48016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.840+0000 7f5a5cde2700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7f5a3c05a670 0x7f5a3c05cb10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.840+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== osd.4 v2:172.21.15.55:6819/23335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f5a3c05d190 con 0x7f5a3c05a670 2022-08-01T23:51:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 --> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a580af5f0 con 0x7f5a3c05a670 2022-08-01T23:51:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a557fa700 1 -- 172.21.15.32:0/3770355734 <== osd.4 v2:172.21.15.55:6819/23335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5a580af5f0 con 0x7f5a3c05a670 2022-08-01T23:51:22.873 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:22.874 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:22.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7f5a3c05a670 msgr2=0x7f5a3c05cb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6819/23335,v1:172.21.15.55:6821/23335] conn(0x7f5a3c05a670 0x7f5a3c05cb10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:22.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a3c0539e0 msgr2=0x7f5a3c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f5a3c0539e0 0x7f5a3c055ea0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5a4801c000 tx=0x7f5a48016000).stop 2022-08-01T23:51:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580ad750 msgr2=0x7f5a58141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 --2- 172.21.15.32:0/3770355734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a580ad750 0x7f5a58141770 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f5a40001110 tx=0x7f5a40007a20).stop 2022-08-01T23:51:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5cde2700 1 -- 172.21.15.32:0/3770355734 reap_dead start 2022-08-01T23:51:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 shutdown_connections 2022-08-01T23:51:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 >> 172.21.15.32:0/3770355734 conn(0x7f5a58066f70 msgr2=0x7f5a58067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 shutdown_connections 2022-08-01T23:51:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.860+0000 7f5a5d5e3700 1 -- 172.21.15.32:0/3770355734 wait complete. 2022-08-01T23:51:22.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2022-08-01T23:51:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3146589700 1 Processor -- start 2022-08-01T23:51:22.998 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3146589700 1 -- start start 2022-08-01T23:51:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3146589700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31400b3260 0x7f31401435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3146589700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3140143ab0 con 0x7f31400b08a0 2022-08-01T23:51:22.999 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3146589700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31400b0c80 con 0x7f31400ad000 2022-08-01T23:51:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3146589700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314005cd30 con 0x7f31400b3260 2022-08-01T23:51:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31400b08a0 0x7f3140142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38122/0 (socket says 172.21.15.32:38122) 2022-08-01T23:51:23.000 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 -- 172.21.15.32:0/4207890328 learned_addr learned my addr 172.21.15.32:0/4207890328 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3144d86700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31400b3260 0x7f31401435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454207783 0 0) 0x7f3140143ab0 con 0x7f31400b08a0 2022-08-01T23:51:23.001 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3124003260 con 0x7f31400b08a0 2022-08-01T23:51:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104124242 0 0) 0x7f31400b0c80 con 0x7f31400ad000 2022-08-01T23:51:23.002 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3140143ab0 con 0x7f31400ad000 2022-08-01T23:51:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534936538 0 0) 0x7f3124003260 con 0x7f31400b08a0 2022-08-01T23:51:23.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31400b3260 msgr2=0x7f31401435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31400b3260 0x7f31401435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.004 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 >> v1:172.21.15.55:6789/0 conn(0x7f31400ad000 legacy=0x7f314005c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31400ae810 con 0x7f31400b08a0 2022-08-01T23:51:23.005 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7f3130001660 con 0x7f31400b08a0 2022-08-01T23:51:23.006 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 >> v1:172.21.15.32:6789/0 conn(0x7f31400b08a0 legacy=0x7f3140142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124003910 0x7f3124003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.007 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124004500 0x7f3124004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.008 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3124004e70 0x7f312400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314005cd30 con 0x7f3124003910 2022-08-01T23:51:23.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312400de50 con 0x7f3124004e70 2022-08-01T23:51:23.010 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312400e010 con 0x7f3124004500 2022-08-01T23:51:23.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3144d86700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f31400b3260 0x7f31401435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:23.011 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124003910 0x7f3124003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145d88700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3124004e70 0x7f312400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3144d86700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124004500 0x7f3124004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 -- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124004500 msgr2=0x7f3124004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124004500 0x7f3124004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 -- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3124004e70 msgr2=0x7f312400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.992+0000 7f3145587700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3124004e70 0x7f312400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3145587700 1 -- 172.21.15.32:0/4207890328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124003260 con 0x7f3124003910 2022-08-01T23:51:23.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3145d88700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3124004e70 0x7f312400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:51:23.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3145587700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124003910 0x7f3124003d10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f3130001430 tx=0x7f31400a95b0).ready entity=mon.0 client_cookie=bc5050b00884313a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.017 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31300077e0 con 0x7f3124003910 2022-08-01T23:51:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31300070d0 con 0x7f3124003910 2022-08-01T23:51:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3137fff700 1 -- 172.21.15.32:0/4207890328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3130007430 con 0x7f3124003910 2022-08-01T23:51:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3146589700 1 -- 172.21.15.32:0/4207890328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124003910 msgr2=0x7f3124003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3146589700 1 --2- 172.21.15.32:0/4207890328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124003910 0x7f3124003d10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f3130001430 tx=0x7f31400a95b0).stop 2022-08-01T23:51:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3145d88700 1 -- 172.21.15.32:0/4207890328 reap_dead start 2022-08-01T23:51:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3146589700 1 -- 172.21.15.32:0/4207890328 shutdown_connections 2022-08-01T23:51:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3146589700 1 -- 172.21.15.32:0/4207890328 >> 172.21.15.32:0/4207890328 conn(0x7f31400612d0 msgr2=0x7f3140062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3146589700 1 -- 172.21.15.32:0/4207890328 shutdown_connections 2022-08-01T23:51:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:22.996+0000 7f3146589700 1 -- 172.21.15.32:0/4207890328 wait complete. 2022-08-01T23:51:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 Processor -- start 2022-08-01T23:51:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 -- start start 2022-08-01T23:51:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124003910 0x7f314014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3140066fc0 0x7f3140140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 0x7f314009f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31400a24e0 con 0x7f3140141160 2022-08-01T23:51:23.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314014f420 con 0x7f3140066fc0 2022-08-01T23:51:23.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31400a1d00 con 0x7f3124003910 2022-08-01T23:51:23.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 0x7f314009f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 0x7f314009f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50596/0 (socket says 172.21.15.32:50596) 2022-08-01T23:51:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 -- 172.21.15.32:0/2206287677 learned_addr learned my addr 172.21.15.32:0/2206287677 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:23.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3144d86700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3140066fc0 0x7f3140140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145587700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124003910 0x7f314014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.032 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 -- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124003910 msgr2=0x7f314014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124003910 0x7f314014ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 -- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3140066fc0 msgr2=0x7f3140140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3140066fc0 0x7f3140140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 -- 172.21.15.32:0/2206287677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31400a1fa0 con 0x7f3140141160 2022-08-01T23:51:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3144d86700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3140066fc0 0x7f3140140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:51:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3145d88700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 0x7f314009f830 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f313c000e90 tx=0x7f313c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f313c007030 con 0x7f3140141160 2022-08-01T23:51:23.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c007190 con 0x7f3140141160 2022-08-01T23:51:23.035 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31400a2240 con 0x7f3140141160 2022-08-01T23:51:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f313c006510 con 0x7f3140141160 2022-08-01T23:51:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.000+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31400a19f0 con 0x7f3140141160 2022-08-01T23:51:23.036 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.004+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314009f520 con 0x7f3140141160 2022-08-01T23:51:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.004+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f313c006670 con 0x7f3140141160 2022-08-01T23:51:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.004+0000 7f31367fc700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3128053a10 0x7f3128055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.037 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.004+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f313c06c190 con 0x7f3140141160 2022-08-01T23:51:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.004+0000 7f3145587700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3128053a10 0x7f3128055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.004+0000 7f3145587700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3128053a10 0x7f3128055ed0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f3130014000 tx=0x7f3130015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.012+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f313c043040 con 0x7f3140141160 2022-08-01T23:51:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.140+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f31400675c0 con 0x7f3128053a10 2022-08-01T23:51:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.140+0000 7f31367fc700 1 -- 172.21.15.32:0/2206287677 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f31400675c0 con 0x7f3128053a10 2022-08-01T23:51:23.146 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3128053a10 msgr2=0x7f3128055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3128053a10 0x7f3128055ed0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f3130014000 tx=0x7f3130015000).stop 2022-08-01T23:51:23.147 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 msgr2=0x7f314009f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 0x7f314009f830 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f313c000e90 tx=0x7f313c009040).stop 2022-08-01T23:51:23.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 shutdown_connections 2022-08-01T23:51:23.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f3128053a10 0x7f3128055ed0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140141160 0x7f314009f830 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3140066fc0 0x7f3140140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 --2- 172.21.15.32:0/2206287677 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f3124003910 0x7f314014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 >> 172.21.15.32:0/2206287677 conn(0x7f31400612d0 msgr2=0x7f31400a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 shutdown_connections 2022-08-01T23:51:23.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.144+0000 7f3146589700 1 -- 172.21.15.32:0/2206287677 wait complete. 2022-08-01T23:51:23.153 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2022-08-01T23:51:23.160 INFO:tasks.ceph:Scrubbing osd.5 2022-08-01T23:51:23.161 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.268+0000 7ffa748e1700 1 Processor -- start 2022-08-01T23:51:23.272 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.268+0000 7ffa748e1700 1 -- start start 2022-08-01T23:51:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa700b2710 0x7ffa7014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffa700afd60 0x7ffa700a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700a7250 con 0x7ffa700b2710 2022-08-01T23:51:23.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700a7480 con 0x7ffa700afd60 2022-08-01T23:51:23.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700a7720 con 0x7ffa700b01a0 2022-08-01T23:51:23.277 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa700b2710 0x7ffa7014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa700b2710 0x7ffa7014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50602/0 (socket says 172.21.15.32:50602) 2022-08-01T23:51:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6effd700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.278 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffa700afd60 0x7ffa700a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 -- 172.21.15.32:0/484111876 learned_addr learned my addr 172.21.15.32:0/484111876 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 -- 172.21.15.32:0/484111876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 msgr2=0x7ffa7014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 --2- 172.21.15.32:0/484111876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.281 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 -- 172.21.15.32:0/484111876 >> v2:172.21.15.55:3300/0 conn(0x7ffa700afd60 msgr2=0x7ffa700a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 --2- 172.21.15.32:0/484111876 >> v2:172.21.15.55:3300/0 conn(0x7ffa700afd60 0x7ffa700a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.282 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 -- 172.21.15.32:0/484111876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700a79c0 con 0x7ffa700b2710 2022-08-01T23:51:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6ffff700 1 --2- 172.21.15.32:0/484111876 >> v2:172.21.15.55:3300/0 conn(0x7ffa700afd60 0x7ffa700a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6effd700 1 --2- 172.21.15.32:0/484111876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6f7fe700 1 --2- 172.21.15.32:0/484111876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014afd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7ffa64002f70 tx=0x7ffa64006eb0).ready entity=mon.0 client_cookie=2d73070b7fdb9dbe server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6e7fc700 1 -- 172.21.15.32:0/484111876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa6400d570 con 0x7ffa700b2710 2022-08-01T23:51:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6e7fc700 1 -- 172.21.15.32:0/484111876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64000e90 con 0x7ffa700b2710 2022-08-01T23:51:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa6e7fc700 1 -- 172.21.15.32:0/484111876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa6400d900 con 0x7ffa700b2710 2022-08-01T23:51:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- 172.21.15.32:0/484111876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 msgr2=0x7ffa7014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- 172.21.15.32:0/484111876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014afd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7ffa64002f70 tx=0x7ffa64006eb0).stop 2022-08-01T23:51:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- 172.21.15.32:0/484111876 shutdown_connections 2022-08-01T23:51:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- 172.21.15.32:0/484111876 >> v2:172.21.15.55:3300/0 conn(0x7ffa700afd60 0x7ffa700a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- 172.21.15.32:0/484111876 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 --2- 172.21.15.32:0/484111876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014afd0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- 172.21.15.32:0/484111876 >> 172.21.15.32:0/484111876 conn(0x7ffa70060310 msgr2=0x7ffa700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- 172.21.15.32:0/484111876 shutdown_connections 2022-08-01T23:51:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.272+0000 7ffa748e1700 1 -- 172.21.15.32:0/484111876 wait complete. 2022-08-01T23:51:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.276+0000 7ffa748e1700 1 Processor -- start 2022-08-01T23:51:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.276+0000 7ffa748e1700 1 -- start start 2022-08-01T23:51:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa700afd60 0x7ffa700aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7014d500 con 0x7ffa700b2710 2022-08-01T23:51:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7014cf00 con 0x7ffa700afd60 2022-08-01T23:51:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b60e0 con 0x7ffa700b01a0 2022-08-01T23:51:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50610/0 (socket says 172.21.15.32:50610) 2022-08-01T23:51:23.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 -- 172.21.15.32:0/3290398776 learned_addr learned my addr 172.21.15.32:0/3290398776 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6effd700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6f7fe700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa700afd60 0x7ffa700aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 -- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 msgr2=0x7ffa7014bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ffa700b01a0 0x7ffa7014bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 -- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa700afd60 msgr2=0x7ffa700aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa700afd60 0x7ffa700aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 -- 172.21.15.32:0/3290398776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6800b180 con 0x7ffa700b2710 2022-08-01T23:51:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6ffff700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014c090 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7ffa6800c000 tx=0x7ffa68006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa68010780 con 0x7ffa700b2710 2022-08-01T23:51:23.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa680108e0 con 0x7ffa700b2710 2022-08-01T23:51:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa60009470 con 0x7ffa700b2710 2022-08-01T23:51:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa68010c40 con 0x7ffa700b2710 2022-08-01T23:51:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6400b1e0 con 0x7ffa700b2710 2022-08-01T23:51:23.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.280+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa700574e0 con 0x7ffa700b2710 2022-08-01T23:51:23.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa6801c460 con 0x7ffa700b2710 2022-08-01T23:51:23.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6cff9700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa50053b30 0x7ffa50055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ffa6806c340 con 0x7ffa700b2710 2022-08-01T23:51:23.320 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6cff9700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7ffa5005a7c0 0x7ffa5005cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 --> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa5005d240 con 0x7ffa5005a7c0 2022-08-01T23:51:23.321 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa6806e8c0 con 0x7ffa700b2710 2022-08-01T23:51:23.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6f7fe700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa50053b30 0x7ffa50055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6effd700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7ffa5005a7c0 0x7ffa5005cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.322 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6f7fe700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa50053b30 0x7ffa50055ff0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7ffa6401c000 tx=0x7ffa64015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.284+0000 7ffa6effd700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7ffa5005a7c0 0x7ffa5005cc60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.323 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.288+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== osd.5 v2:172.21.15.55:6802/23336 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ffa5005d240 con 0x7ffa5005a7c0 2022-08-01T23:51:23.324 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 --> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ffa700a9ed0 con 0x7ffa5005a7c0 2022-08-01T23:51:23.325 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa6cff9700 1 -- 172.21.15.32:0/3290398776 <== osd.5 v2:172.21.15.55:6802/23336 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ffa700a9ed0 con 0x7ffa5005a7c0 2022-08-01T23:51:23.325 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:23.326 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:23.326 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:23.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7ffa5005a7c0 msgr2=0x7ffa5005cc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6802/23336,v1:172.21.15.55:6803/23336] conn(0x7ffa5005a7c0 0x7ffa5005cc60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa50053b30 msgr2=0x7ffa50055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ffa50053b30 0x7ffa50055ff0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7ffa6401c000 tx=0x7ffa64015000).stop 2022-08-01T23:51:23.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 msgr2=0x7ffa7014c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa748e1700 1 --2- 172.21.15.32:0/3290398776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700b2710 0x7ffa7014c090 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7ffa6800c000 tx=0x7ffa68006df0).stop 2022-08-01T23:51:23.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.304+0000 7ffa6ffff700 1 -- 172.21.15.32:0/3290398776 reap_dead start 2022-08-01T23:51:23.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.308+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 shutdown_connections 2022-08-01T23:51:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.308+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 >> 172.21.15.32:0/3290398776 conn(0x7ffa70060310 msgr2=0x7ffa70149a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.308+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 shutdown_connections 2022-08-01T23:51:23.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.308+0000 7ffa748e1700 1 -- 172.21.15.32:0/3290398776 wait complete. 2022-08-01T23:51:23.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2022-08-01T23:51:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf63fff700 1 Processor -- start 2022-08-01T23:51:23.443 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf63fff700 1 -- start start 2022-08-01T23:51:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf63fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdf640a9da0 0x7fdf64060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf63fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf64067030 con 0x7fdf640b31d0 2022-08-01T23:51:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf63fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf640b35b0 con 0x7fdf641402d0 2022-08-01T23:51:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf63fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf64064a70 con 0x7fdf640a9da0 2022-08-01T23:51:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf62ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdf640b31d0 0x7fdf64066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38158/0 (socket says 172.21.15.32:38158) 2022-08-01T23:51:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf62ffd700 1 -- 172.21.15.32:0/3442992447 learned_addr learned my addr 172.21.15.32:0/3442992447 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.440+0000 7fdf627fc700 1 --2- 172.21.15.32:0/3442992447 >> v2:172.21.15.55:3300/0 conn(0x7fdf640a9da0 0x7fdf64060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859671768 0 0) 0x7fdf64067030 con 0x7fdf640b31d0 2022-08-01T23:51:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf44003260 con 0x7fdf640b31d0 2022-08-01T23:51:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369803651 0 0) 0x7fdf640b35b0 con 0x7fdf641402d0 2022-08-01T23:51:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf64067030 con 0x7fdf641402d0 2022-08-01T23:51:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3876902459 0 0) 0x7fdf44003260 con 0x7fdf640b31d0 2022-08-01T23:51:23.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 >> v2:172.21.15.55:3300/0 conn(0x7fdf640a9da0 msgr2=0x7fdf64060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 --2- 172.21.15.32:0/3442992447 >> v2:172.21.15.55:3300/0 conn(0x7fdf640a9da0 0x7fdf64060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 >> v1:172.21.15.55:6789/0 conn(0x7fdf641402d0 legacy=0x7fdf64140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64066af0 con 0x7fdf640b31d0 2022-08-01T23:51:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3339217273 0 0) 0x7fdf4c001660 con 0x7fdf640b31d0 2022-08-01T23:51:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 >> v1:172.21.15.32:6789/0 conn(0x7fdf640b31d0 legacy=0x7fdf64066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf440038e0 0x7fdf44003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf44004500 0x7fdf44004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004e70 0x7fdf4400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4400bcf0 con 0x7fdf44004500 2022-08-01T23:51:23.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4400be30 con 0x7fdf440038e0 2022-08-01T23:51:23.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4400bff0 con 0x7fdf44004e70 2022-08-01T23:51:23.464 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 --2- 172.21.15.32:0/3442992447 >> v2:172.21.15.55:3300/0 conn(0x7fdf640a9da0 0x7fdf64060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T23:51:23.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf44004500 0x7fdf44004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.465 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf62ffd700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf440038e0 0x7fdf44003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf637fe700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004e70 0x7fdf4400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.466 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 -- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004e70 msgr2=0x7fdf4400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004e70 0x7fdf4400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 -- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf440038e0 msgr2=0x7fdf44003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf440038e0 0x7fdf44003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 -- 172.21.15.32:0/3442992447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44003260 con 0x7fdf44004500 2022-08-01T23:51:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf637fe700 1 -- 172.21.15.32:0/3442992447 reap_dead start 2022-08-01T23:51:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf627fc700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf44004500 0x7fdf44004980 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7fdf5400c000 tx=0x7fdf54006df0).ready entity=mon.0 client_cookie=35e6a0c818f18c66 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.469 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5400e800 con 0x7fdf44004500 2022-08-01T23:51:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400e960 con 0x7fdf44004500 2022-08-01T23:51:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf61ffb700 1 -- 172.21.15.32:0/3442992447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5400ecc0 con 0x7fdf44004500 2022-08-01T23:51:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 -- 172.21.15.32:0/3442992447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf44004500 msgr2=0x7fdf44004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf44004500 0x7fdf44004980 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7fdf5400c000 tx=0x7fdf54006df0).stop 2022-08-01T23:51:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 -- 172.21.15.32:0/3442992447 shutdown_connections 2022-08-01T23:51:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 --2- 172.21.15.32:0/3442992447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf44004500 0x7fdf44004980 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 -- 172.21.15.32:0/3442992447 >> 172.21.15.32:0/3442992447 conn(0x7fdf64063870 msgr2=0x7fdf64063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 -- 172.21.15.32:0/3442992447 shutdown_connections 2022-08-01T23:51:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.444+0000 7fdf63fff700 1 -- 172.21.15.32:0/3442992447 wait complete. 2022-08-01T23:51:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 Processor -- start 2022-08-01T23:51:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 -- start start 2022-08-01T23:51:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004500 0x7fdf640aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf640aa720 0x7fdf6414df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 0x7fdf6414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64152130 con 0x7fdf6414e420 2022-08-01T23:51:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6413e480 con 0x7fdf640aa720 2022-08-01T23:51:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.448+0000 7fdf63fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6413e720 con 0x7fdf44004500 2022-08-01T23:51:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 0x7fdf6414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 0x7fdf6414e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50632/0 (socket says 172.21.15.32:50632) 2022-08-01T23:51:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 -- 172.21.15.32:0/4216981208 learned_addr learned my addr 172.21.15.32:0/4216981208 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf627fc700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf640aa720 0x7fdf6414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf62ffd700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004500 0x7fdf640aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 -- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004500 msgr2=0x7fdf640aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004500 0x7fdf640aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 -- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf640aa720 msgr2=0x7fdf6414df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.482 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf640aa720 0x7fdf6414df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 -- 172.21.15.32:0/4216981208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6413e9c0 con 0x7fdf6414e420 2022-08-01T23:51:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf637fe700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 0x7fdf6414e8a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7fdf5c001210 tx=0x7fdf5c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5c003220 con 0x7fdf6414e420 2022-08-01T23:51:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c003380 con 0x7fdf6414e420 2022-08-01T23:51:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf64151db0 con 0x7fdf6414e420 2022-08-01T23:51:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf5c0016e0 con 0x7fdf6414e420 2022-08-01T23:51:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.452+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640aa4f0 con 0x7fdf6414e420 2022-08-01T23:51:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.456+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6413eb50 con 0x7fdf6414e420 2022-08-01T23:51:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.456+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf5c01c060 con 0x7fdf6414e420 2022-08-01T23:51:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.456+0000 7fdf5bfff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf48053b00 0x7fdf48055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.456+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fdf5c06b860 con 0x7fdf6414e420 2022-08-01T23:51:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.456+0000 7fdf62ffd700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf48053b00 0x7fdf48055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.456+0000 7fdf62ffd700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf48053b00 0x7fdf48055fc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fdf4c005000 tx=0x7fdf64065990).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.464+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf5c0427f0 con 0x7fdf6414e420 2022-08-01T23:51:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.588+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf64066d90 con 0x7fdf48053b00 2022-08-01T23:51:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.588+0000 7fdf5bfff700 1 -- 172.21.15.32:0/4216981208 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdf64066d90 con 0x7fdf48053b00 2022-08-01T23:51:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf48053b00 msgr2=0x7fdf48055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf48053b00 0x7fdf48055fc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7fdf4c005000 tx=0x7fdf64065990).stop 2022-08-01T23:51:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 msgr2=0x7fdf6414e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 0x7fdf6414e8a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7fdf5c001210 tx=0x7fdf5c009040).stop 2022-08-01T23:51:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 shutdown_connections 2022-08-01T23:51:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fdf48053b00 0x7fdf48055fc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf6414e420 0x7fdf6414e8a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf640aa720 0x7fdf6414df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 --2- 172.21.15.32:0/4216981208 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fdf44004500 0x7fdf640aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 >> 172.21.15.32:0/4216981208 conn(0x7fdf64063870 msgr2=0x7fdf6413f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 shutdown_connections 2022-08-01T23:51:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.592+0000 7fdf63fff700 1 -- 172.21.15.32:0/4216981208 wait complete. 2022-08-01T23:51:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2022-08-01T23:51:23.604 INFO:tasks.ceph:Scrubbing osd.6 2022-08-01T23:51:23.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:23.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc58e53700 1 Processor -- start 2022-08-01T23:51:23.715 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc58e53700 1 -- start start 2022-08-01T23:51:23.717 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc58e53700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc540b68f0 0x7fcc5413ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc58e53700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc540ac6e0 con 0x7fcc540aa210 2022-08-01T23:51:23.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc58e53700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc540ac980 con 0x7fcc5405ff40 2022-08-01T23:51:23.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc58e53700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc540acc20 con 0x7fcc540b68f0 2022-08-01T23:51:23.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc5259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc540b68f0 0x7fcc5413ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc51d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc540aa210 0x7fcc54146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38180/0 (socket says 172.21.15.32:38180) 2022-08-01T23:51:23.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc51d9b700 1 -- 172.21.15.32:0/1204022478 learned_addr learned my addr 172.21.15.32:0/1204022478 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:23.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc5259c700 1 -- 172.21.15.32:0/1204022478 >> v1:172.21.15.55:6789/0 conn(0x7fcc5405ff40 legacy=0x7fcc5414dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc5259c700 1 -- 172.21.15.32:0/1204022478 >> v1:172.21.15.32:6789/0 conn(0x7fcc540aa210 legacy=0x7fcc54146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc5259c700 1 -- 172.21.15.32:0/1204022478 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc540acec0 con 0x7fcc540b68f0 2022-08-01T23:51:23.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.712+0000 7fcc5159a700 1 -- 172.21.15.32:0/1204022478 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457419676 0 0) 0x7fcc540ac6e0 con 0x7fcc540aa210 2022-08-01T23:51:23.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc5259c700 1 --2- 172.21.15.32:0/1204022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540b68f0 0x7fcc5413ce80 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7fcc3c002f70 tx=0x7fcc3c006df0).ready entity=mon.0 client_cookie=4efe80d3117e8d9e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc5159a700 1 -- 172.21.15.32:0/1204022478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc3c00d610 con 0x7fcc540b68f0 2022-08-01T23:51:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc5159a700 1 -- 172.21.15.32:0/1204022478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c000e90 con 0x7fcc540b68f0 2022-08-01T23:51:23.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc5159a700 1 -- 172.21.15.32:0/1204022478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc3c00d960 con 0x7fcc540b68f0 2022-08-01T23:51:23.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 -- 172.21.15.32:0/1204022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540b68f0 msgr2=0x7fcc5413ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.727 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 --2- 172.21.15.32:0/1204022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540b68f0 0x7fcc5413ce80 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7fcc3c002f70 tx=0x7fcc3c006df0).stop 2022-08-01T23:51:23.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 -- 172.21.15.32:0/1204022478 shutdown_connections 2022-08-01T23:51:23.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 --2- 172.21.15.32:0/1204022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540b68f0 0x7fcc5413ce80 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.728 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 -- 172.21.15.32:0/1204022478 >> 172.21.15.32:0/1204022478 conn(0x7fcc540b3610 msgr2=0x7fcc540b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 -- 172.21.15.32:0/1204022478 shutdown_connections 2022-08-01T23:51:23.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 -- 172.21.15.32:0/1204022478 wait complete. 2022-08-01T23:51:23.729 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 Processor -- start 2022-08-01T23:51:23.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.716+0000 7fcc58e53700 1 -- start start 2022-08-01T23:51:23.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc5405ff40 0x7fcc540b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.730 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540aa210 0x7fcc540ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.731 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc540b68f0 0x7fcc540aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54067100 con 0x7fcc540aa210 2022-08-01T23:51:23.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541530a0 con 0x7fcc5405ff40 2022-08-01T23:51:23.732 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54153340 con 0x7fcc540b68f0 2022-08-01T23:51:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540aa210 0x7fcc540ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc5259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc5405ff40 0x7fcc540b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540aa210 0x7fcc540ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50646/0 (socket says 172.21.15.32:50646) 2022-08-01T23:51:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 -- 172.21.15.32:0/2426688847 learned_addr learned my addr 172.21.15.32:0/2426688847 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc52d9d700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc540b68f0 0x7fcc540aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 -- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc540b68f0 msgr2=0x7fcc540aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fcc540b68f0 0x7fcc540aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 -- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc5405ff40 msgr2=0x7fcc540b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc5405ff40 0x7fcc540b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 -- 172.21.15.32:0/2426688847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c00b180 con 0x7fcc540aa210 2022-08-01T23:51:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc51d9b700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540aa210 0x7fcc540ae7a0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7fcc44000bc0 tx=0x7fcc44007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc440070d0 con 0x7fcc540aa210 2022-08-01T23:51:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc54066000 con 0x7fcc540aa210 2022-08-01T23:51:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc54066bc0 con 0x7fcc540aa210 2022-08-01T23:51:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44007230 con 0x7fcc540aa210 2022-08-01T23:51:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc44005590 con 0x7fcc540aa210 2022-08-01T23:51:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.720+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc54066f80 con 0x7fcc540aa210 2022-08-01T23:51:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcc4401a060 con 0x7fcc540aa210 2022-08-01T23:51:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc38053b00 0x7fcc38055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcc4406b570 con 0x7fcc540aa210 2022-08-01T23:51:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7fcc3805a790 0x7fcc3805cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 --> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc3805d210 con 0x7fcc3805a790 2022-08-01T23:51:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc4406dbe0 con 0x7fcc540aa210 2022-08-01T23:51:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc5259c700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc38053b00 0x7fcc38055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc52d9d700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7fcc3805a790 0x7fcc3805cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc5259c700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc38053b00 0x7fcc38055fc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fcc3c01c000 tx=0x7fcc3c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc52d9d700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7fcc3805a790 0x7fcc3805cc30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.745 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.724+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== osd.6 v2:172.21.15.55:6810/23334 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fcc3805d210 con 0x7fcc3805a790 2022-08-01T23:51:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 --> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcc540672c0 con 0x7fcc3805a790 2022-08-01T23:51:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/2426688847 <== osd.6 v2:172.21.15.55:6810/23334 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcc4c002800 con 0x7fcc3805a790 2022-08-01T23:51:23.747 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:23.748 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:23.748 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7fcc3805a790 msgr2=0x7fcc3805cc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6810/23334,v1:172.21.15.55:6811/23334] conn(0x7fcc3805a790 0x7fcc3805cc30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc38053b00 msgr2=0x7fcc38055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fcc38053b00 0x7fcc38055fc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fcc3c01c000 tx=0x7fcc3c015000).stop 2022-08-01T23:51:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540aa210 msgr2=0x7fcc540ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 --2- 172.21.15.32:0/2426688847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc540aa210 0x7fcc540ae7a0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7fcc44000bc0 tx=0x7fcc44007a60).stop 2022-08-01T23:51:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc52d9d700 1 -- 172.21.15.32:0/2426688847 reap_dead start 2022-08-01T23:51:23.752 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 shutdown_connections 2022-08-01T23:51:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 >> 172.21.15.32:0/2426688847 conn(0x7fcc540b3610 msgr2=0x7fcc54142460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 shutdown_connections 2022-08-01T23:51:23.753 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.744+0000 7fcc58e53700 1 -- 172.21.15.32:0/2426688847 wait complete. 2022-08-01T23:51:23.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2022-08-01T23:51:23.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 Processor -- start 2022-08-01T23:51:23.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 -- start start 2022-08-01T23:51:23.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10040b1f40 0x7f10040b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10040a7250 0x7f10041448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040b04e0 con 0x7f1004148530 2022-08-01T23:51:23.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040a9250 con 0x7f10040b1f40 2022-08-01T23:51:23.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100c2bf700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040b0780 con 0x7f10040a7250 2022-08-01T23:51:23.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100a05b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10040b1f40 0x7f10040b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100985a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1004148530 0x7f10040b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38194/0 (socket says 172.21.15.32:38194) 2022-08-01T23:51:23.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100985a700 1 -- 172.21.15.32:0/1685152904 learned_addr learned my addr 172.21.15.32:0/1685152904 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:23.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.872+0000 7f100a85c700 1 --2- 172.21.15.32:0/1685152904 >> v2:172.21.15.55:3300/0 conn(0x7f10040a7250 0x7f10041448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f1009059700 1 -- 172.21.15.32:0/1685152904 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631774087 0 0) 0x7f10040b04e0 con 0x7f1004148530 2022-08-01T23:51:23.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100a05b700 1 -- 172.21.15.32:0/1685152904 >> v2:172.21.15.55:3300/0 conn(0x7f10040a7250 msgr2=0x7f10041448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:23.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100a05b700 1 --2- 172.21.15.32:0/1685152904 >> v2:172.21.15.55:3300/0 conn(0x7f10040a7250 0x7f10041448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100a05b700 1 -- 172.21.15.32:0/1685152904 >> v1:172.21.15.32:6789/0 conn(0x7f1004148530 legacy=0x7f10040b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100a05b700 1 -- 172.21.15.32:0/1685152904 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040b0a20 con 0x7f10040b1f40 2022-08-01T23:51:23.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100a05b700 1 --2- 172.21.15.32:0/1685152904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040b1160 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f10041478c0 tx=0x7f0ff4006df0).ready entity=mon.0 client_cookie=714988e5e05100ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f1009059700 1 -- 172.21.15.32:0/1685152904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ff400d5c0 con 0x7f10040b1f40 2022-08-01T23:51:23.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f1009059700 1 -- 172.21.15.32:0/1685152904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff4000e90 con 0x7f10040b1f40 2022-08-01T23:51:23.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f1009059700 1 -- 172.21.15.32:0/1685152904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ff400d910 con 0x7f10040b1f40 2022-08-01T23:51:23.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 -- 172.21.15.32:0/1685152904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 msgr2=0x7f10040b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1685152904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040b1160 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f10041478c0 tx=0x7f0ff4006df0).stop 2022-08-01T23:51:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 -- 172.21.15.32:0/1685152904 shutdown_connections 2022-08-01T23:51:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1685152904 >> v2:172.21.15.55:3300/0 conn(0x7f10040a7250 0x7f10041448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1685152904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040b1160 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 -- 172.21.15.32:0/1685152904 >> 172.21.15.32:0/1685152904 conn(0x7f100405f840 msgr2=0x7f100405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 -- 172.21.15.32:0/1685152904 shutdown_connections 2022-08-01T23:51:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.876+0000 7f100c2bf700 1 -- 172.21.15.32:0/1685152904 wait complete. 2022-08-01T23:51:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 Processor -- start 2022-08-01T23:51:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 -- start start 2022-08-01T23:51:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10040a7250 0x7f10040ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1004148530 0x7f10041439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040af4e0 con 0x7f10040b1f40 2022-08-01T23:51:23.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040af780 con 0x7f10040a7250 2022-08-01T23:51:23.903 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040afa20 con 0x7f1004148530 2022-08-01T23:51:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040a95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50660/0 (socket says 172.21.15.32:50660) 2022-08-01T23:51:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 -- 172.21.15.32:0/1472268925 learned_addr learned my addr 172.21.15.32:0/1472268925 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:23.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100a05b700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10040a7250 0x7f10040ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100a85c700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1004148530 0x7f10041439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.905 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 -- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1004148530 msgr2=0x7f10041439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1004148530 0x7f10041439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 -- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10040a7250 msgr2=0x7f10040ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10040a7250 0x7f10040ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 -- 172.21.15.32:0/1472268925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff400b180 con 0x7f10040b1f40 2022-08-01T23:51:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100a05b700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10040a7250 0x7f10040ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:51:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100985a700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040a95d0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f1000000ed0 tx=0x7f1000007cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1000001d30 con 0x7f10040b1f40 2022-08-01T23:51:23.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10000072e0 con 0x7f10040b1f40 2022-08-01T23:51:23.908 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.880+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10040afcc0 con 0x7f10040b1f40 2022-08-01T23:51:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1000005610 con 0x7f10040b1f40 2022-08-01T23:51:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10040a9c60 con 0x7f10040b1f40 2022-08-01T23:51:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10040afe50 con 0x7f10040b1f40 2022-08-01T23:51:23.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1000010070 con 0x7f10040b1f40 2022-08-01T23:51:23.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f0ff37fe700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fe4053b00 0x7f0fe4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:23.910 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f100006b520 con 0x7f10040b1f40 2022-08-01T23:51:23.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.884+0000 7f100a05b700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fe4053b00 0x7f0fe4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:23.911 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.888+0000 7f100a05b700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fe4053b00 0x7f0fe4055fc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f0ff4018000 tx=0x7f0ff4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:23.892+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1000042520 con 0x7f10040b1f40 2022-08-01T23:51:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.016+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f10040b0cc0 con 0x7f0fe4053b00 2022-08-01T23:51:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1472268925 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f10040b0cc0 con 0x7f0fe4053b00 2022-08-01T23:51:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fe4053b00 msgr2=0x7f0fe4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fe4053b00 0x7f0fe4055fc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f0ff4018000 tx=0x7f0ff4014000).stop 2022-08-01T23:51:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 msgr2=0x7f10040a95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040a95d0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f1000000ed0 tx=0x7f1000007cb0).stop 2022-08-01T23:51:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 shutdown_connections 2022-08-01T23:51:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f0fe4053b00 0x7f0fe4055fc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f1004148530 0x7f10041439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10040b1f40 0x7f10040a95d0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 --2- 172.21.15.32:0/1472268925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10040a7250 0x7f10040ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 >> 172.21.15.32:0/1472268925 conn(0x7f100405f840 msgr2=0x7f10040b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 shutdown_connections 2022-08-01T23:51:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.020+0000 7f100c2bf700 1 -- 172.21.15.32:0/1472268925 wait complete. 2022-08-01T23:51:24.031 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2022-08-01T23:51:24.035 INFO:tasks.ceph:Scrubbing osd.7 2022-08-01T23:51:24.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2022-08-01T23:51:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.144+0000 7f14bdf95700 1 Processor -- start 2022-08-01T23:51:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.144+0000 7f14bdf95700 1 -- start start 2022-08-01T23:51:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14b80614b0 0x7f14b8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80aee80 con 0x7f14b81405d0 2022-08-01T23:51:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80b70b0 con 0x7f14b80ad750 2022-08-01T23:51:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80af0b0 con 0x7f14b80614b0 2022-08-01T23:51:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bcf93700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14b80614b0 0x7f14b8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bd794700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14b81405d0 0x7f14b81466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38212/0 (socket says 172.21.15.32:38212) 2022-08-01T23:51:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bd794700 1 -- 172.21.15.32:0/3960370099 learned_addr learned my addr 172.21.15.32:0/3960370099 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:24.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14b77fe700 1 -- 172.21.15.32:0/3960370099 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405511480 0 0) 0x7f14b80aee80 con 0x7f14b81405d0 2022-08-01T23:51:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14b77fe700 1 -- 172.21.15.32:0/3960370099 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a4003260 con 0x7f14b81405d0 2022-08-01T23:51:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bcf93700 1 -- 172.21.15.32:0/3960370099 >> v1:172.21.15.55:6789/0 conn(0x7f14b80ad750 legacy=0x7f14b80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14b77fe700 1 -- 172.21.15.32:0/3960370099 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852539540 0 0) 0x7f14b80b70b0 con 0x7f14b80ad750 2022-08-01T23:51:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bcf93700 1 -- 172.21.15.32:0/3960370099 >> v1:172.21.15.32:6789/0 conn(0x7f14b81405d0 legacy=0x7f14b81466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bcf93700 1 -- 172.21.15.32:0/3960370099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80af350 con 0x7f14b80614b0 2022-08-01T23:51:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bcf93700 1 --2- 172.21.15.32:0/3960370099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b8061890 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f14b80aed30 tx=0x7f14a8006df0).ready entity=mon.0 client_cookie=344aabe0538a4955 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14b77fe700 1 -- 172.21.15.32:0/3960370099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14a800d610 con 0x7f14b80614b0 2022-08-01T23:51:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14b77fe700 1 -- 172.21.15.32:0/3960370099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8000e90 con 0x7f14b80614b0 2022-08-01T23:51:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14b77fe700 1 -- 172.21.15.32:0/3960370099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14a800d960 con 0x7f14b80614b0 2022-08-01T23:51:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- 172.21.15.32:0/3960370099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 msgr2=0x7f14b8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 --2- 172.21.15.32:0/3960370099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b8061890 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f14b80aed30 tx=0x7f14a8006df0).stop 2022-08-01T23:51:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- 172.21.15.32:0/3960370099 shutdown_connections 2022-08-01T23:51:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 --2- 172.21.15.32:0/3960370099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b8061890 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- 172.21.15.32:0/3960370099 >> 172.21.15.32:0/3960370099 conn(0x7f14b8066f70 msgr2=0x7f14b8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- 172.21.15.32:0/3960370099 shutdown_connections 2022-08-01T23:51:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.148+0000 7f14bdf95700 1 -- 172.21.15.32:0/3960370099 wait complete. 2022-08-01T23:51:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 Processor -- start 2022-08-01T23:51:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 -- start start 2022-08-01T23:51:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b81431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14b80ad750 0x7f14b80b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b81405d0 0x7f14b80b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80b03b0 con 0x7f14b80614b0 2022-08-01T23:51:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80b1750 con 0x7f14b81405d0 2022-08-01T23:51:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8142f50 con 0x7f14b80ad750 2022-08-01T23:51:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b81431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b81431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50672/0 (socket says 172.21.15.32:50672) 2022-08-01T23:51:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 -- 172.21.15.32:0/810102529 learned_addr learned my addr 172.21.15.32:0/810102529 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14b7fff700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14b80ad750 0x7f14b80b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bd794700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b81405d0 0x7f14b80b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 -- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14b80ad750 msgr2=0x7f14b80b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f14b80ad750 0x7f14b80b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 -- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b81405d0 msgr2=0x7f14b80b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b81405d0 0x7f14b80b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 -- 172.21.15.32:0/810102529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a800b180 con 0x7f14b80614b0 2022-08-01T23:51:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bd794700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b81405d0 0x7f14b80b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:51:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bcf93700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b81431b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7f14b80aed30 tx=0x7f14a8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14a8013e90 con 0x7f14b80614b0 2022-08-01T23:51:24.176 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8002f70 con 0x7f14b80614b0 2022-08-01T23:51:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8141aa0 con 0x7f14b80614b0 2022-08-01T23:51:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.152+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14a801e5f0 con 0x7f14b80614b0 2022-08-01T23:51:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b8142280 con 0x7f14b80614b0 2022-08-01T23:51:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14b8142480 con 0x7f14b80614b0 2022-08-01T23:51:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f14a8025030 con 0x7f14b80614b0 2022-08-01T23:51:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b5ffb700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1498053b30 0x7f1498055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f14a806cbe0 con 0x7f14b80614b0 2022-08-01T23:51:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b7fff700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1498053b30 0x7f1498055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b5ffb700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f149805a650 0x7f149805caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 --> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f149805d0d0 con 0x7f149805a650 2022-08-01T23:51:24.183 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a8071240 con 0x7f14b80614b0 2022-08-01T23:51:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.156+0000 7f14bd794700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f149805a650 0x7f149805caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.160+0000 7f14b7fff700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1498053b30 0x7f1498055ff0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f14ac00d000 tx=0x7f14ac007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.185 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.160+0000 7f14bd794700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f149805a650 0x7f149805caf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.160+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== osd.7 v2:172.21.15.55:6818/23359 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f149805d0d0 con 0x7f149805a650 2022-08-01T23:51:24.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 --> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f14b81429c0 con 0x7f149805a650 2022-08-01T23:51:24.187 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14b5ffb700 1 -- 172.21.15.32:0/810102529 <== osd.7 v2:172.21.15.55:6818/23359 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f14b81429c0 con 0x7f149805a650 2022-08-01T23:51:24.188 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2022-08-01T23:51:24.188 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T23:51:24.189 INFO:teuthology.orchestra.run.smithi032.stdout:} 2022-08-01T23:51:24.190 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f149805a650 msgr2=0x7f149805caf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6818/23359,v1:172.21.15.55:6820/23359] conn(0x7f149805a650 0x7f149805caf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1498053b30 msgr2=0x7f1498055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.191 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f1498053b30 0x7f1498055ff0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f14ac00d000 tx=0x7f14ac007a00).stop 2022-08-01T23:51:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 msgr2=0x7f14b81431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.176+0000 7f14bdf95700 1 --2- 172.21.15.32:0/810102529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80614b0 0x7f14b81431b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7f14b80aed30 tx=0x7f14a8014000).stop 2022-08-01T23:51:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.180+0000 7f14bd794700 1 -- 172.21.15.32:0/810102529 reap_dead start 2022-08-01T23:51:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.180+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 shutdown_connections 2022-08-01T23:51:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.180+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 >> 172.21.15.32:0/810102529 conn(0x7f14b8066f70 msgr2=0x7f14b8146300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.180+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 shutdown_connections 2022-08-01T23:51:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.180+0000 7f14bdf95700 1 -- 172.21.15.32:0/810102529 wait complete. 2022-08-01T23:51:24.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2022-08-01T23:51:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 Processor -- start 2022-08-01T23:51:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 -- start start 2022-08-01T23:51:24.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b2710 0x7fd3600abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3600b13f0 0x7fd3600ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.310 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3600adf10 con 0x7fd3600b1920 2022-08-01T23:51:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600ae1b0 con 0x7fd3600b13f0 2022-08-01T23:51:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd3657b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600a9520 con 0x7fd3600b2710 2022-08-01T23:51:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd357fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3600b13f0 0x7fd3600ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd364fb2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3600b1920 0x7fd3600afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38228/0 (socket says 172.21.15.32:38228) 2022-08-01T23:51:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd364fb2700 1 -- 172.21.15.32:0/2817423042 learned_addr learned my addr 172.21.15.32:0/2817423042 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd357fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3600b13f0 0x7fd3600ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50682/0 (socket says 172.21.15.32:50682) 2022-08-01T23:51:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.304+0000 7fd35ffff700 1 --2- 172.21.15.32:0/2817423042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b2710 0x7fd3600abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd357fff700 1 -- 172.21.15.32:0/2817423042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b2710 msgr2=0x7fd3600abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:24.314 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd357fff700 1 --2- 172.21.15.32:0/2817423042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b2710 0x7fd3600abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.315 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd357fff700 1 -- 172.21.15.32:0/2817423042 >> v1:172.21.15.32:6789/0 conn(0x7fd3600b1920 legacy=0x7fd3600afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd357fff700 1 -- 172.21.15.32:0/2817423042 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600a99c0 con 0x7fd3600b13f0 2022-08-01T23:51:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd35f7fe700 1 -- 172.21.15.32:0/2817423042 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42649861 0 0) 0x7fd3600adf10 con 0x7fd3600b1920 2022-08-01T23:51:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd357fff700 1 --2- 172.21.15.32:0/2817423042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd3600ac190 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fd360067bc0 tx=0x7fd348006df0).ready entity=mon.0 client_cookie=5ffd4848db35a929 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd35f7fe700 1 -- 172.21.15.32:0/2817423042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34800d570 con 0x7fd3600b13f0 2022-08-01T23:51:24.318 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd35f7fe700 1 -- 172.21.15.32:0/2817423042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348000e90 con 0x7fd3600b13f0 2022-08-01T23:51:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd35f7fe700 1 -- 172.21.15.32:0/2817423042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd34800d8c0 con 0x7fd3600b13f0 2022-08-01T23:51:24.326 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 -- 172.21.15.32:0/2817423042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 msgr2=0x7fd3600ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 --2- 172.21.15.32:0/2817423042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd3600ac190 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fd360067bc0 tx=0x7fd348006df0).stop 2022-08-01T23:51:24.327 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 -- 172.21.15.32:0/2817423042 shutdown_connections 2022-08-01T23:51:24.328 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 --2- 172.21.15.32:0/2817423042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd3600ac190 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 --2- 172.21.15.32:0/2817423042 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b2710 0x7fd3600abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.329 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 -- 172.21.15.32:0/2817423042 >> 172.21.15.32:0/2817423042 conn(0x7fd360060310 msgr2=0x7fd3600b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.330 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 -- 172.21.15.32:0/2817423042 shutdown_connections 2022-08-01T23:51:24.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.308+0000 7fd3657b3700 1 -- 172.21.15.32:0/2817423042 wait complete. 2022-08-01T23:51:24.331 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 Processor -- start 2022-08-01T23:51:24.332 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 -- start start 2022-08-01T23:51:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd36005c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.333 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b1920 0x7fd360064260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3600b2710 0x7fd360064750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600676a0 con 0x7fd3600b13f0 2022-08-01T23:51:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600b5a80 con 0x7fd3600b2710 2022-08-01T23:51:24.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600b5d20 con 0x7fd3600b1920 2022-08-01T23:51:24.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd36005c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.336 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd36005c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50688/0 (socket says 172.21.15.32:50688) 2022-08-01T23:51:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 -- 172.21.15.32:0/211445726 learned_addr learned my addr 172.21.15.32:0/211445726 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd364fb2700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3600b2710 0x7fd360064750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd357fff700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b1920 0x7fd360064260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 -- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b1920 msgr2=0x7fd360064260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b1920 0x7fd360064260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 -- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3600b2710 msgr2=0x7fd360064750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3600b2710 0x7fd360064750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 -- 172.21.15.32:0/211445726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34800b180 con 0x7fd3600b13f0 2022-08-01T23:51:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35ffff700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd36005c1b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7fd35000c000 tx=0x7fd350006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd350010780 con 0x7fd3600b13f0 2022-08-01T23:51:24.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3500108e0 con 0x7fd3600b13f0 2022-08-01T23:51:24.340 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3600b5fc0 con 0x7fd3600b13f0 2022-08-01T23:51:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.312+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd350010c40 con 0x7fd3600b13f0 2022-08-01T23:51:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd360067180 con 0x7fd3600b13f0 2022-08-01T23:51:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3600574e0 con 0x7fd3600b13f0 2022-08-01T23:51:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd35001c460 con 0x7fd3600b13f0 2022-08-01T23:51:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd35dffb700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3440539e0 0x7fd344055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd35006c2b0 con 0x7fd3600b13f0 2022-08-01T23:51:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd357fff700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3440539e0 0x7fd344055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.316+0000 7fd357fff700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3440539e0 0x7fd344055ea0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fd348018000 tx=0x7fd348014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.343 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.320+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3500432f0 con 0x7fd3600b13f0 2022-08-01T23:51:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.448+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd360069a70 con 0x7fd3440539e0 2022-08-01T23:51:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.448+0000 7fd35dffb700 1 -- 172.21.15.32:0/211445726 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd360069a70 con 0x7fd3440539e0 2022-08-01T23:51:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3440539e0 msgr2=0x7fd344055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3440539e0 0x7fd344055ea0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fd348018000 tx=0x7fd348014000).stop 2022-08-01T23:51:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 msgr2=0x7fd36005c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd36005c1b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7fd35000c000 tx=0x7fd350006df0).stop 2022-08-01T23:51:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 shutdown_connections 2022-08-01T23:51:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fd3440539e0 0x7fd344055ea0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3600b2710 0x7fd360064750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fd3600b1920 0x7fd360064260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 --2- 172.21.15.32:0/211445726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3600b13f0 0x7fd36005c1b0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 >> 172.21.15.32:0/211445726 conn(0x7fd360060310 msgr2=0x7fd36005ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 shutdown_connections 2022-08-01T23:51:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.452+0000 7fd3657b3700 1 -- 172.21.15.32:0/211445726 wait complete. 2022-08-01T23:51:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2022-08-01T23:51:24.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T23:51:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 Processor -- start 2022-08-01T23:51:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 -- start start 2022-08-01T23:51:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080b08a0 0x7ff908142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9080b3260 0x7ff9081433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9080ad000 0x7ff90805f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080b0c80 con 0x7ff9080ad000 2022-08-01T23:51:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080ae810 con 0x7ff9080b3260 2022-08-01T23:51:24.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90ddd3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080aeab0 con 0x7ff9080b08a0 2022-08-01T23:51:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9080ad000 0x7ff90805f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9080ad000 0x7ff90805f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50700/0 (socket says 172.21.15.32:50700) 2022-08-01T23:51:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90cdd1700 1 --2- 172.21.15.32:0/3532880833 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080b08a0 0x7ff908142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 -- 172.21.15.32:0/3532880833 learned_addr learned my addr 172.21.15.32:0/3532880833 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff907fff700 1 --2- 172.21.15.32:0/3532880833 >> v2:172.21.15.55:3300/0 conn(0x7ff9080b3260 0x7ff9081433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 -- 172.21.15.32:0/3532880833 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080b08a0 msgr2=0x7ff908142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 --2- 172.21.15.32:0/3532880833 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080b08a0 0x7ff908142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 -- 172.21.15.32:0/3532880833 >> v2:172.21.15.55:3300/0 conn(0x7ff9080b3260 msgr2=0x7ff9081433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:51:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 --2- 172.21.15.32:0/3532880833 >> v2:172.21.15.55:3300/0 conn(0x7ff9080b3260 0x7ff9081433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 -- 172.21.15.32:0/3532880833 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080aed50 con 0x7ff9080ad000 2022-08-01T23:51:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.568+0000 7ff90d5d2700 1 --2- 172.21.15.32:0/3532880833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080ad000 0x7ff90805f840 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7ff9000032f0 tx=0x7ff900009290).ready entity=mon.0 client_cookie=66845964dc01f5f3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff9077fe700 1 -- 172.21.15.32:0/3532880833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90000b680 con 0x7ff9080ad000 2022-08-01T23:51:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff9077fe700 1 -- 172.21.15.32:0/3532880833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000b7e0 con 0x7ff9080ad000 2022-08-01T23:51:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff9077fe700 1 -- 172.21.15.32:0/3532880833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90000b950 con 0x7ff9080ad000 2022-08-01T23:51:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- 172.21.15.32:0/3532880833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080ad000 msgr2=0x7ff90805f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/3532880833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080ad000 0x7ff90805f840 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7ff9000032f0 tx=0x7ff900009290).stop 2022-08-01T23:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- 172.21.15.32:0/3532880833 shutdown_connections 2022-08-01T23:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/3532880833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080ad000 0x7ff90805f840 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/3532880833 >> v2:172.21.15.55:3300/0 conn(0x7ff9080b3260 0x7ff9081433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/3532880833 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080b08a0 0x7ff908142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- 172.21.15.32:0/3532880833 >> 172.21.15.32:0/3532880833 conn(0x7ff9080612d0 msgr2=0x7ff908062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- 172.21.15.32:0/3532880833 shutdown_connections 2022-08-01T23:51:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- 172.21.15.32:0/3532880833 wait complete. 2022-08-01T23:51:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 Processor -- start 2022-08-01T23:51:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- start start 2022-08-01T23:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080ad000 0x7ff9080ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 0x7ff908156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9080b3260 0x7ff908152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908155c60 con 0x7ff9080b08a0 2022-08-01T23:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908155f00 con 0x7ff9080b3260 2022-08-01T23:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90ddd3700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081561a0 con 0x7ff9080ad000 2022-08-01T23:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff907fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 0x7ff908156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff907fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 0x7ff908156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50702/0 (socket says 172.21.15.32:50702) 2022-08-01T23:51:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff907fff700 1 -- 172.21.15.32:0/2256532445 learned_addr learned my addr 172.21.15.32:0/2256532445 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90cdd1700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080ad000 0x7ff9080ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.572+0000 7ff90d5d2700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9080b3260 0x7ff908152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff907fff700 1 -- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080ad000 msgr2=0x7ff9080ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff907fff700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080ad000 0x7ff9080ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.591 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff907fff700 1 -- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9080b3260 msgr2=0x7ff908152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff907fff700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9080b3260 0x7ff908152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.592 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff907fff700 1 -- 172.21.15.32:0/2256532445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900009530 con 0x7ff9080b08a0 2022-08-01T23:51:24.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff907fff700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 0x7ff908156640 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7ff8fc00d000 tx=0x7ff8fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8fc00aa20 con 0x7ff9080b08a0 2022-08-01T23:51:24.593 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00ab80 con 0x7ff9080b08a0 2022-08-01T23:51:24.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff908065770 con 0x7ff9080b08a0 2022-08-01T23:51:24.594 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8fc0113f0 con 0x7ff9080b08a0 2022-08-01T23:51:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908152d80 con 0x7ff9080b08a0 2022-08-01T23:51:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9080574e0 con 0x7ff9080b08a0 2022-08-01T23:51:24.595 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.576+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff8fc021070 con 0x7ff9080b08a0 2022-08-01T23:51:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.580+0000 7ff905ffb700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff8e8053950 0x7ff8e8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.580+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff8fc06c060 con 0x7ff9080b08a0 2022-08-01T23:51:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.580+0000 7ff90cdd1700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff8e8053950 0x7ff8e8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:24.597 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.580+0000 7ff90cdd1700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff8e8053950 0x7ff8e8055e10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7ff8f800b000 tx=0x7ff8f8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.584+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8fc042ff0 con 0x7ff9080b08a0 2022-08-01T23:51:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.708+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff908156360 con 0x7ff8e8053950 2022-08-01T23:51:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff905ffb700 1 -- 172.21.15.32:0/2256532445 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67956 (secure 0 0 0) 0x7ff908156360 con 0x7ff8e8053950 2022-08-01T23:51:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:51:24.718 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff8e8053950 msgr2=0x7ff8e8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff8e8053950 0x7ff8e8055e10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7ff8f800b000 tx=0x7ff8f8008040).stop 2022-08-01T23:51:24.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 msgr2=0x7ff908156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:24.719 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 0x7ff908156640 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7ff8fc00d000 tx=0x7ff8fc008040).stop 2022-08-01T23:51:24.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 shutdown_connections 2022-08-01T23:51:24.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7ff8e8053950 0x7ff8e8055e10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.720 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9080b3260 0x7ff908152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9080b08a0 0x7ff908156640 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 --2- 172.21.15.32:0/2256532445 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7ff9080ad000 0x7ff9080ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:24.721 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 >> 172.21.15.32:0/2256532445 conn(0x7ff9080612d0 msgr2=0x7ff908142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:24.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 shutdown_connections 2022-08-01T23:51:24.722 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:24.712+0000 7ff90ddd3700 1 -- 172.21.15.32:0/2256532445 wait complete. 2022-08-01T23:51:24.723 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2022-08-01T23:51:24.729 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1828,"stamp":"2022-08-01T23:51:23.164740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404704,"kb_used_data":16032,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570016,"statfs":{"total":773094113280,"available":764487696384,"internally_reserved":8589934592,"allocated":16416768,"data_stored":8039656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"1.0","version":"14'2","reported_seq":145,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346420+0000","last_change":"2022-08-01T22:55:40.426793+0000","last_active":"2022-08-01T23:05:48.346420+0000","last_peered":"2022-08-01T23:05:48.346420+0000","last_clean":"2022-08-01T23:05:48.346420+0000","last_became_active":"2022-08-01T22:55:40.425307+0000","last_became_peered":"2022-08-01T22:55:40.425307+0000","last_unstale":"2022-08-01T23:05:48.346420+0000","last_undegraded":"2022-08-01T23:05:48.346420+0000","last_fullsized":"2022-08-01T23:05:48.346420+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:34.279477+0000","last_clean_scrub_stamp":"2022-08-01T22:55:34.279477+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.289895+0000","last_change":"2022-08-01T22:55:44.188728+0000","last_active":"2022-08-01T23:05:47.289895+0000","last_peered":"2022-08-01T23:05:47.289895+0000","last_clean":"2022-08-01T23:05:47.289895+0000","last_became_active":"2022-08-01T22:55:44.188196+0000","last_became_peered":"2022-08-01T22:55:44.188196+0000","last_unstale":"2022-08-01T23:05:47.289895+0000","last_undegraded":"2022-08-01T23:05:47.289895+0000","last_fullsized":"2022-08-01T23:05:47.289895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346419+0000","last_change":"2022-08-01T22:55:44.188357+0000","last_active":"2022-08-01T23:05:48.346419+0000","last_peered":"2022-08-01T23:05:48.346419+0000","last_clean":"2022-08-01T23:05:48.346419+0000","last_became_active":"2022-08-01T22:55:44.187581+0000","last_became_peered":"2022-08-01T22:55:44.187581+0000","last_unstale":"2022-08-01T23:05:48.346419+0000","last_undegraded":"2022-08-01T23:05:48.346419+0000","last_fullsized":"2022-08-01T23:05:48.346419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.345170+0000","last_change":"2022-08-01T22:55:44.555310+0000","last_active":"2022-08-01T23:05:48.345170+0000","last_peered":"2022-08-01T23:05:48.345170+0000","last_clean":"2022-08-01T23:05:48.345170+0000","last_became_active":"2022-08-01T22:55:44.554353+0000","last_became_peered":"2022-08-01T22:55:44.554353+0000","last_unstale":"2022-08-01T23:05:48.345170+0000","last_undegraded":"2022-08-01T23:05:48.345170+0000","last_fullsized":"2022-08-01T23:05:48.345170+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.289864+0000","last_change":"2022-08-01T22:55:44.188785+0000","last_active":"2022-08-01T23:05:47.289864+0000","last_peered":"2022-08-01T23:05:47.289864+0000","last_clean":"2022-08-01T23:05:47.289864+0000","last_became_active":"2022-08-01T22:55:44.188345+0000","last_became_peered":"2022-08-01T22:55:44.188345+0000","last_unstale":"2022-08-01T23:05:47.289864+0000","last_undegraded":"2022-08-01T23:05:47.289864+0000","last_fullsized":"2022-08-01T23:05:47.289864+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.286498+0000","last_change":"2022-08-01T22:55:44.414856+0000","last_active":"2022-08-01T23:05:47.286498+0000","last_peered":"2022-08-01T23:05:47.286498+0000","last_clean":"2022-08-01T23:05:47.286498+0000","last_became_active":"2022-08-01T22:55:44.413727+0000","last_became_peered":"2022-08-01T22:55:44.413727+0000","last_unstale":"2022-08-01T23:05:47.286498+0000","last_undegraded":"2022-08-01T23:05:47.286498+0000","last_fullsized":"2022-08-01T23:05:47.286498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346553+0000","last_change":"2022-08-01T22:55:44.415068+0000","last_active":"2022-08-01T23:05:48.346553+0000","last_peered":"2022-08-01T23:05:48.346553+0000","last_clean":"2022-08-01T23:05:48.346553+0000","last_became_active":"2022-08-01T22:55:44.414349+0000","last_became_peered":"2022-08-01T22:55:44.414349+0000","last_unstale":"2022-08-01T23:05:48.346553+0000","last_undegraded":"2022-08-01T23:05:48.346553+0000","last_fullsized":"2022-08-01T23:05:48.346553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.287516+0000","last_change":"2022-08-01T22:55:44.190143+0000","last_active":"2022-08-01T23:05:47.287516+0000","last_peered":"2022-08-01T23:05:47.287516+0000","last_clean":"2022-08-01T23:05:47.287516+0000","last_became_active":"2022-08-01T22:55:44.189745+0000","last_became_peered":"2022-08-01T22:55:44.189745+0000","last_unstale":"2022-08-01T23:05:47.287516+0000","last_undegraded":"2022-08-01T23:05:47.287516+0000","last_fullsized":"2022-08-01T23:05:47.287516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:47.593244+0000","last_change":"2022-08-01T22:55:44.191477+0000","last_active":"2022-08-01T23:05:47.593244+0000","last_peered":"2022-08-01T23:05:47.593244+0000","last_clean":"2022-08-01T23:05:47.593244+0000","last_became_active":"2022-08-01T22:55:44.191167+0000","last_became_peered":"2022-08-01T22:55:44.191167+0000","last_unstale":"2022-08-01T23:05:47.593244+0000","last_undegraded":"2022-08-01T23:05:47.593244+0000","last_fullsized":"2022-08-01T23:05:47.593244+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575518,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.873},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.1459999999999999,"15min":1.3069999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.90500000000000003},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.56200000000000006},"max":{"1min":1.173,"5min":1.222,"15min":1.27},"last":0.77900000000000003}]},{"osd":1,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.85099999999999998,"15min":0.85799999999999998},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.2589999999999999,"15min":1.7210000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.78200000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.155,"5min":1.1599999999999999,"15min":1.653},"last":1.032}]},{"osd":2,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86199999999999999,"15min":0.88200000000000001},"min":{"1min":0.66300000000000003,"5min":0.58099999999999996,"15min":0.57899999999999996},"max":{"1min":1.1140000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.57999999999999996},"max":{"1min":1.1140000000000001,"5min":1.2410000000000001,"15min":1.3999999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86199999999999999,"15min":0.85199999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.04,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88700000000000001,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.60999999999999999},"max":{"1min":1.0309999999999999,"5min":1.2130000000000001,"15min":1.3540000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.77700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.52900000000000003},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.155},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1279999999999999,"5min":1.323,"15min":1.323},"last":0.86699999999999999}]},{"osd":5,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94999999999999996,"15min":0.94799999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.53800000000000003},"max":{"1min":1.1719999999999999,"5min":1.2849999999999999,"15min":1.444},"last":1.093},{"interface":"front","average":{"1min":1.014,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2649999999999999,"5min":1.325,"15min":1.3420000000000001},"last":1.262}]},{"osd":6,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.65000000000000002,"5min":0.64600000000000002,"15min":0.628},"max":{"1min":1.22,"5min":1.2569999999999999,"15min":1.319},"last":1.216},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0229999999999999,"15min":1.002},"min":{"1min":0.78600000000000003,"5min":0.67700000000000005,"15min":0.627},"max":{"1min":1.2070000000000001,"5min":1.4079999999999999,"15min":1.446},"last":1.079}]}]},{"osd":6,"up_from":13,"seq":55834575518,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.48899999999999999},"max":{"1min":1.21,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93400000000000005,"15min":0.92200000000000004},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.57999999999999996},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.042}]},{"osd":1,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59299999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.2270000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.879,"15min":0.88500000000000001},"min":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.60199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.284},"last":0.875}]},{"osd":3,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":0.90600000000000003},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.151,"5min":1.1639999999999999,"15min":1.274},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.72699999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.94699999999999995}]},{"osd":4,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.252},"last":1.131}]},{"osd":5,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.57699999999999996},"max":{"1min":1.28,"5min":1.286,"15min":1.3999999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.95999999999999996,"15min":0.94099999999999995},"min":{"1min":0.59999999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.246,"5min":1.2789999999999999,"15min":1.3500000000000001},"last":1.177}]},{"osd":7,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0369999999999999,"15min":1.02},"min":{"1min":0.92100000000000004,"5min":0.69899999999999995,"15min":0.61399999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.071,"15min":1.077},"min":{"1min":0.92800000000000005,"5min":0.64200000000000002,"15min":0.61799999999999999},"max":{"1min":1.351,"5min":1.3600000000000001,"15min":1.546},"last":1.077}]}]},{"osd":5,"up_from":13,"seq":55834575518,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92100000000000004,"15min":0.88300000000000001},"min":{"1min":0.81299999999999994,"5min":0.58499999999999996,"15min":0.55800000000000005},"max":{"1min":1.123,"5min":1.161,"15min":1.2729999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.81100000000000005,"5min":0.68799999999999994,"15min":0.64600000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3120000000000001},"last":0.89000000000000001}]},{"osd":1,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.876,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1459999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0569999999999999,"5min":1.25,"15min":1.25},"last":0.65100000000000002}]},{"osd":2,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.88500000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.157,"5min":1.1819999999999999,"15min":1.3480000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94199999999999995,"15min":0.94399999999999995},"min":{"1min":0.70899999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.304},"last":0.73599999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"max":{"1min":1.139,"5min":1.1930000000000001,"15min":1.278},"last":1.042},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0089999999999999,"15min":1.0049999999999999},"min":{"1min":0.68300000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3540000000000001},"last":1.169}]},{"osd":4,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85199999999999998,"15min":0.85399999999999998},"min":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.3080000000000001},"last":1.008},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.93000000000000005},"min":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.57299999999999995},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.3120000000000001},"last":1.0980000000000001}]},{"osd":6,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.60899999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.4279999999999999},"last":1.119},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98299999999999998},"min":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.3,"5min":1.304,"15min":1.48},"last":1.087}]},{"osd":7,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0269999999999999,"15min":1.0129999999999999},"min":{"1min":0.83799999999999997,"5min":0.71099999999999997,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.4990000000000001},"last":1.151},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.165,"15min":1.153},"min":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.90400000000000003,"15min":0.90000000000000002},"min":{"1min":0.67500000000000004,"5min":0.60199999999999998,"15min":0.54400000000000004},"max":{"1min":1.129,"5min":1.171,"15min":1.2789999999999999},"last":1.036},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.89200000000000002,"15min":0.88800000000000001},"min":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.66300000000000003},"max":{"1min":0.95399999999999996,"5min":1.1479999999999999,"15min":1.2230000000000001},"last":1.0649999999999999}]},{"osd":1,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83499999999999996,"15min":0.82599999999999996},"min":{"1min":0.65800000000000003,"5min":0.60399999999999998,"15min":0.54200000000000004},"max":{"1min":1.1870000000000001,"5min":1.194,"15min":1.194},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94899999999999995,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.111}]},{"osd":2,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94199999999999995,"15min":0.94399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.55800000000000005},"max":{"1min":1.1479999999999999,"5min":1.2,"15min":1.2310000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.93400000000000005,"15min":0.95299999999999996},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1839999999999999,"5min":1.21,"15min":1.3360000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.82199999999999995},"min":{"1min":0.69099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.98299999999999998,"5min":1.1459999999999999,"15min":1.169},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.91400000000000003,"15min":0.92500000000000004},"min":{"1min":0.73899999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.163,"5min":1.163,"15min":1.2390000000000001},"last":0.89600000000000002}]},{"osd":5,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.64100000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1180000000000001,"5min":1.1819999999999999,"15min":1.3089999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1479999999999999,"5min":1.3029999999999999,"15min":1.3080000000000001},"last":0.97499999999999998}]},{"osd":6,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.3500000000000001,"15min":1.4299999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.73699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.244,"5min":1.2470000000000001,"15min":1.3819999999999999},"last":0.753}]},{"osd":7,"last update":"Mon Aug 1 23:50:20 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93100000000000005,"15min":0.93600000000000005},"min":{"1min":0.65200000000000002,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0900000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0509999999999999,"15min":1.0549999999999999},"min":{"1min":0.84899999999999998,"5min":0.70899999999999996,"15min":0.65500000000000003},"max":{"1min":1.202,"5min":1.4039999999999999,"15min":1.5049999999999999},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55600000000000005},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.135},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.75},"min":{"1min":0.56599999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.82099999999999995,"15min":0.80500000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.51600000000000001},"max":{"1min":1.04,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.88800000000000001},"min":{"1min":0.69199999999999995,"5min":0.58399999999999996,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.028}]},{"osd":4,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.872,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93600000000000005,"15min":0.92900000000000005},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.66600000000000004},"max":{"1min":1.3180000000000001,"5min":1.331,"15min":1.522},"last":0.90100000000000002}]},{"osd":5,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.98499999999999999,"15min":0.98699999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.628},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.96799999999999997,"15min":0.95699999999999996},"min":{"1min":0.72599999999999998,"5min":0.64400000000000002,"15min":0.63200000000000001},"max":{"1min":1.389,"5min":1.546,"15min":1.546},"last":0.81899999999999995}]},{"osd":6,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0229999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.79100000000000004,"15min":0.71099999999999997},"max":{"1min":1.2470000000000001,"5min":1.5289999999999999,"15min":1.6919999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0609999999999999},"min":{"1min":0.80400000000000005,"5min":0.73399999999999999,"15min":0.72999999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.74},"last":1.1890000000000001}]},{"osd":7,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0569999999999999,"15min":1.05},"min":{"1min":0.88900000000000001,"5min":0.77000000000000002,"15min":0.75900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.726},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.105,"5min":1.099,"15min":1.087},"min":{"1min":0.92800000000000005,"5min":0.79900000000000004,"15min":0.72299999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.665},"last":1.0620000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575518,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.45400000000000001,"5min":0.439,"15min":0.43099999999999999},"max":{"1min":1,"5min":1,"15min":1.0600000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58199999999999996},"max":{"1min":0.90400000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90200000000000002,"5min":0.98099999999999998,"15min":1.115},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.71199999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.1160000000000001},"last":0.74299999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.55100000000000005,"5min":0.50700000000000001,"15min":0.505},"max":{"1min":0.90100000000000002,"5min":1.03,"15min":1.149},"last":0.627},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.623,"5min":0.60599999999999998,"15min":0.52300000000000002},"max":{"1min":1.04,"5min":1.1100000000000001,"15min":1.1519999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.96699999999999997,"15min":0.93799999999999994},"min":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.65100000000000002},"max":{"1min":1.143,"5min":1.238,"15min":1.2609999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89700000000000002,"15min":0.90900000000000003},"min":{"1min":0.71199999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.038,"5min":1.2709999999999999,"15min":1.2789999999999999},"last":1.0600000000000001}]},{"osd":5,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.0069999999999999,"15min":1.016},"min":{"1min":0.79500000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2989999999999999,"5min":1.4710000000000001,"15min":1.607},"last":1.171},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.995,"15min":0.98999999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.67900000000000005},"max":{"1min":1.2769999999999999,"5min":1.413,"15min":1.413},"last":0.84999999999999998}]},{"osd":6,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.68700000000000006},"max":{"1min":1.2969999999999999,"5min":1.333,"15min":1.504},"last":1.262},{"interface":"front","average":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0780000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.3,"5min":1.524,"15min":1.77},"last":1.274}]},{"osd":7,"last update":"Mon Aug 1 23:50:24 2022","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0600000000000001,"15min":1.0660000000000001},"min":{"1min":0.82899999999999996,"5min":0.79800000000000004,"15min":0.77800000000000002},"max":{"1min":1.212,"5min":1.49,"15min":1.49},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0649999999999999,"15min":1.075},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.73699999999999999},"max":{"1min":1.284,"5min":1.498,"15min":1.575},"last":1.2}]}]},{"osd":1,"up_from":13,"seq":55834575519,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.76800000000000002,"5min":1.133,"15min":1.258},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.72499999999999998},"min":{"1min":0.504,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.93899999999999995,"5min":1.004,"15min":1.147},"last":0.83899999999999997}]},{"osd":2,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91400000000000003,"5min":1.05,"15min":1.3100000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.629,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.05,"5min":1.1830000000000001,"15min":1.2829999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.84299999999999997,"15min":0.82399999999999995},"min":{"1min":0.69099999999999995,"5min":0.59699999999999998,"15min":0.56200000000000006},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71599999999999997,"15min":0.72599999999999998},"min":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.90700000000000003,"15min":0.91700000000000004},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.63300000000000001},"max":{"1min":1.012,"5min":1.3129999999999999,"15min":1.381},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98699999999999999,"15min":0.97599999999999998},"min":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.65100000000000002},"max":{"1min":1.129,"5min":1.423,"15min":1.573},"last":1.099}]},{"osd":5,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93999999999999995,"15min":0.93200000000000005},"min":{"1min":0.82999999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"max":{"1min":1.272,"5min":1.3029999999999999,"15min":1.373},"last":1.071},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":1.01},"min":{"1min":0.77900000000000003,"5min":0.69399999999999995,"15min":0.68799999999999994},"max":{"1min":1.236,"5min":1.657,"15min":1.7190000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0229999999999999,"15min":1.042},"min":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2310000000000001,"5min":1.72,"15min":1.72},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.99199999999999999,"15min":1.006},"min":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.256,"5min":1.2589999999999999,"15min":1.357},"last":0.95899999999999996}]},{"osd":7,"last update":"Mon Aug 1 23:50:32 2022","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.073,"15min":1.0820000000000001},"min":{"1min":0.85999999999999999,"5min":0.754,"15min":0.74099999999999999},"max":{"1min":1.4430000000000001,"5min":1.4930000000000001,"15min":2.0369999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.0569999999999999,"15min":1.0629999999999999},"min":{"1min":0.97099999999999997,"5min":0.81000000000000005,"15min":0.755},"max":{"1min":1.29,"5min":1.3680000000000001,"15min":1.411},"last":0.80700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575518,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.5,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":0.76100000000000001,"5min":1.012,"15min":1.173},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.89200000000000002,"5min":1.0409999999999999,"15min":1.2030000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.57899999999999996,"5min":0.50800000000000001,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.103,"15min":1.135},"last":0.752},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.087,"5min":1.087,"15min":1.095},"last":0.66300000000000003}]},{"osd":3,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72899999999999998,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81799999999999995,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.57999999999999996,"15min":0.54100000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.26},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94999999999999996,"15min":0.93700000000000006},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.321,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.92500000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.64700000000000002},"max":{"1min":1.3080000000000001,"5min":1.331,"15min":1.331},"last":0.94699999999999995}]},{"osd":5,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93000000000000005,"15min":0.93899999999999995},"min":{"1min":0.75700000000000001,"5min":0.73399999999999999,"15min":0.67500000000000004},"max":{"1min":1.3899999999999999,"5min":1.403,"15min":1.4410000000000001},"last":1.048},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.018,"15min":1.0409999999999999},"min":{"1min":0.80300000000000005,"5min":0.74099999999999999,"15min":0.70099999999999996},"max":{"1min":1.3360000000000001,"5min":1.4159999999999999,"15min":1.4299999999999999},"last":1.04}]},{"osd":6,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.4670000000000001},"last":1.155},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0740000000000001,"15min":1.0660000000000001},"min":{"1min":0.91200000000000003,"5min":0.80000000000000004,"15min":0.76700000000000002},"max":{"1min":1.329,"5min":1.369,"15min":1.4330000000000001},"last":1.1799999999999999}]},{"osd":7,"last update":"Mon Aug 1 23:50:33 2022","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.103,"15min":1.101},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.423},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.94499999999999995,"5min":0.84999999999999998,"15min":0.75600000000000001},"max":{"1min":1.474,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1339999999999999}]}]}]}} 2022-08-01T23:51:24.740 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2022-08-01T22:55:34.279477+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=34, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.741 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.742 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.743 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.743 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.744 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.745 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.746 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.746 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:24.747 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2022-08-01T23:51:33.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:33.346 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:33.348 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:33.351 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:51:44.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T23:51:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b3fff700 1 Processor -- start 2022-08-01T23:51:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b3fff700 1 -- start start 2022-08-01T23:51:44.862 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48b40b28b0 0x7f48b413e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b3fff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40aac20 con 0x7f48b40b1180 2022-08-01T23:51:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b3fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40b2c90 con 0x7f48b4141920 2022-08-01T23:51:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b3fff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b4141db0 con 0x7f48b40b28b0 2022-08-01T23:51:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b27fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f48b40b1180 0x7f48b4141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38252/0 (socket says 172.21.15.32:38252) 2022-08-01T23:51:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b27fc700 1 -- 172.21.15.32:0/1520421384 learned_addr learned my addr 172.21.15.32:0/1520421384 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:51:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.855+0000 7f48b2ffd700 1 --2- 172.21.15.32:0/1520421384 >> v2:172.21.15.32:3300/0 conn(0x7f48b40b28b0 0x7f48b413e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b1ffb700 1 -- 172.21.15.32:0/1520421384 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863080097 0 0) 0x7f48a4001110 con 0x7f48b40b1180 2022-08-01T23:51:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b1ffb700 1 -- 172.21.15.32:0/1520421384 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4898003260 con 0x7f48b40b1180 2022-08-01T23:51:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b1ffb700 1 -- 172.21.15.32:0/1520421384 <== mon.1 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137597618 0 0) 0x7f48b40b2c90 con 0x7f48b4141920 2022-08-01T23:51:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b2ffd700 1 -- 172.21.15.32:0/1520421384 >> v1:172.21.15.55:6789/0 conn(0x7f48b4141920 legacy=0x7f48b41443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b2ffd700 1 -- 172.21.15.32:0/1520421384 >> v1:172.21.15.32:6789/0 conn(0x7f48b40b1180 legacy=0x7f48b4141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:44.867 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b2ffd700 1 -- 172.21.15.32:0/1520421384 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40aa440 con 0x7f48b40b28b0 2022-08-01T23:51:44.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b2ffd700 1 --2- 172.21.15.32:0/1520421384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b413e1c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7f48a8002f70 tx=0x7f48a8006df0).ready entity=mon.0 client_cookie=7b1dbfd9cfc7c98c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:44.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b1ffb700 1 -- 172.21.15.32:0/1520421384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a800d610 con 0x7f48b40b28b0 2022-08-01T23:51:44.868 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b1ffb700 1 -- 172.21.15.32:0/1520421384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8000e90 con 0x7f48b40b28b0 2022-08-01T23:51:44.869 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b1ffb700 1 -- 172.21.15.32:0/1520421384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a800d960 con 0x7f48b40b28b0 2022-08-01T23:51:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 -- 172.21.15.32:0/1520421384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 msgr2=0x7f48b413e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 --2- 172.21.15.32:0/1520421384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b413e1c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7f48a8002f70 tx=0x7f48a8006df0).stop 2022-08-01T23:51:44.873 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 -- 172.21.15.32:0/1520421384 shutdown_connections 2022-08-01T23:51:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 --2- 172.21.15.32:0/1520421384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b413e1c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 -- 172.21.15.32:0/1520421384 >> 172.21.15.32:0/1520421384 conn(0x7f48b4060310 msgr2=0x7f48b40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:44.874 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 -- 172.21.15.32:0/1520421384 shutdown_connections 2022-08-01T23:51:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.859+0000 7f48b3fff700 1 -- 172.21.15.32:0/1520421384 wait complete. 2022-08-01T23:51:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 Processor -- start 2022-08-01T23:51:44.875 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 -- start start 2022-08-01T23:51:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b40b1180 0x7f48b4138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b40b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:44.876 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48b4141920 0x7f48b40b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b414ffd0 con 0x7f48b40b28b0 2022-08-01T23:51:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b413fa20 con 0x7f48b40b1180 2022-08-01T23:51:44.877 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b3fff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41501c0 con 0x7f48b4141920 2022-08-01T23:51:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b40b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b40b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50718/0 (socket says 172.21.15.32:50718) 2022-08-01T23:51:44.878 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 -- 172.21.15.32:0/759015186 learned_addr learned my addr 172.21.15.32:0/759015186 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:51:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b2ffd700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b40b1180 0x7f48b4138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b37fe700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48b4141920 0x7f48b40b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:44.879 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 -- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48b4141920 msgr2=0x7f48b40b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48b4141920 0x7f48b40b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 -- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b40b1180 msgr2=0x7f48b4138950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:44.880 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.863+0000 7f48b27fc700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b40b1180 0x7f48b4138950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b27fc700 1 -- 172.21.15.32:0/759015186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a800b180 con 0x7f48b40b28b0 2022-08-01T23:51:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b2ffd700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b40b1180 0x7f48b4138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T23:51:44.881 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b27fc700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b40b3580 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f48a40012a0 tx=0x7f48a4007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a400ce30 con 0x7f48b40b28b0 2022-08-01T23:51:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a40072e0 con 0x7f48b40b28b0 2022-08-01T23:51:44.882 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b40b30e0 con 0x7f48b40b28b0 2022-08-01T23:51:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a4005610 con 0x7f48b40b28b0 2022-08-01T23:51:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b40b49a0 con 0x7f48b40b28b0 2022-08-01T23:51:44.883 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b40574e0 con 0x7f48b40b28b0 2022-08-01T23:51:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f48a401a070 con 0x7f48b40b28b0 2022-08-01T23:51:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48a3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4890053a70 0x7f4890055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:51:44.884 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f48a406af70 con 0x7f48b40b28b0 2022-08-01T23:51:44.885 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.867+0000 7f48b2ffd700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4890053a70 0x7f4890055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:51:44.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.871+0000 7f48b2ffd700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4890053a70 0x7f4890055f30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f48a8018000 tx=0x7f48a8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:51:44.886 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.871+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48a4041f90 con 0x7f48b40b28b0 2022-08-01T23:51:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:44.999+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48b40b2f60 con 0x7f4890053a70 2022-08-01T23:51:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48a3fff700 1 -- 172.21.15.32:0/759015186 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68262 (secure 0 0 0) 0x7f48a800d690 con 0x7f4890053a70 2022-08-01T23:51:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:51:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4890053a70 msgr2=0x7f4890055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4890053a70 0x7f4890055f30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f48a8018000 tx=0x7f48a8014000).stop 2022-08-01T23:51:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 msgr2=0x7f48b40b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:51:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b40b3580 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f48a40012a0 tx=0x7f48a4007900).stop 2022-08-01T23:51:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 shutdown_connections 2022-08-01T23:51:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7f4890053a70 0x7f4890055f30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7f48b4141920 0x7f48b40b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b40b28b0 0x7f48b40b3580 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 --2- 172.21.15.32:0/759015186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b40b1180 0x7f48b4138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:51:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 >> 172.21.15.32:0/759015186 conn(0x7f48b4060310 msgr2=0x7f48b413a010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:51:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 shutdown_connections 2022-08-01T23:51:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:51:45.007+0000 7f48b3fff700 1 -- 172.21.15.32:0/759015186 wait complete. 2022-08-01T23:51:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2022-08-01T23:51:45.023 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1838,"stamp":"2022-08-01T23:51:43.170416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404704,"kb_used_data":16032,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570016,"statfs":{"total":773094113280,"available":764487696384,"internally_reserved":8589934592,"allocated":16416768,"data_stored":8039656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003438"},"pg_stats":[{"pgid":"1.0","version":"161'4","reported_seq":150,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:30.288449+0000","last_change":"2022-08-01T23:51:30.288626+0000","last_active":"2022-08-01T23:51:30.288449+0000","last_peered":"2022-08-01T23:51:30.288449+0000","last_clean":"2022-08-01T23:51:30.288449+0000","last_became_active":"2022-08-01T22:55:40.425307+0000","last_became_peered":"2022-08-01T22:55:40.425307+0000","last_unstale":"2022-08-01T23:51:30.288449+0000","last_undegraded":"2022-08-01T23:51:30.288449+0000","last_fullsized":"2022-08-01T23:51:30.288449+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'4","last_scrub_stamp":"2022-08-01T23:51:30.288448+0000","last_deep_scrub":"161'4","last_deep_scrub_stamp":"2022-08-01T23:51:30.288448+0000","last_clean_scrub_stamp":"2022-08-01T23:51:30.288448+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:30.644448+0000","last_change":"2022-08-01T23:51:30.644558+0000","last_active":"2022-08-01T23:51:30.644448+0000","last_peered":"2022-08-01T23:51:30.644448+0000","last_clean":"2022-08-01T23:51:30.644448+0000","last_became_active":"2022-08-01T22:55:44.188196+0000","last_became_peered":"2022-08-01T22:55:44.188196+0000","last_unstale":"2022-08-01T23:51:30.644448+0000","last_undegraded":"2022-08-01T23:51:30.644448+0000","last_fullsized":"2022-08-01T23:51:30.644448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:30.644447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:30.644447+0000","last_clean_scrub_stamp":"2022-08-01T23:51:30.644447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346419+0000","last_change":"2022-08-01T22:55:44.188357+0000","last_active":"2022-08-01T23:05:48.346419+0000","last_peered":"2022-08-01T23:05:48.346419+0000","last_clean":"2022-08-01T23:05:48.346419+0000","last_became_active":"2022-08-01T22:55:44.187581+0000","last_became_peered":"2022-08-01T22:55:44.187581+0000","last_unstale":"2022-08-01T23:05:48.346419+0000","last_undegraded":"2022-08-01T23:05:48.346419+0000","last_fullsized":"2022-08-01T23:05:48.346419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":143,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:23.207399+0000","last_change":"2022-08-01T23:51:23.207520+0000","last_active":"2022-08-01T23:51:23.207399+0000","last_peered":"2022-08-01T23:51:23.207399+0000","last_clean":"2022-08-01T23:51:23.207399+0000","last_became_active":"2022-08-01T22:55:44.554353+0000","last_became_peered":"2022-08-01T22:55:44.554353+0000","last_unstale":"2022-08-01T23:51:23.207399+0000","last_undegraded":"2022-08-01T23:51:23.207399+0000","last_fullsized":"2022-08-01T23:51:23.207399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:23.207398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:23.207398+0000","last_clean_scrub_stamp":"2022-08-01T23:51:23.207398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:28.641034+0000","last_change":"2022-08-01T23:51:28.641235+0000","last_active":"2022-08-01T23:51:28.641034+0000","last_peered":"2022-08-01T23:51:28.641034+0000","last_clean":"2022-08-01T23:51:28.641034+0000","last_became_active":"2022-08-01T22:55:44.188345+0000","last_became_peered":"2022-08-01T22:55:44.188345+0000","last_unstale":"2022-08-01T23:51:28.641034+0000","last_undegraded":"2022-08-01T23:51:28.641034+0000","last_fullsized":"2022-08-01T23:51:28.641034+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:28.641033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:28.641033+0000","last_clean_scrub_stamp":"2022-08-01T23:51:28.641033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:26.477028+0000","last_change":"2022-08-01T23:51:26.477225+0000","last_active":"2022-08-01T23:51:26.477028+0000","last_peered":"2022-08-01T23:51:26.477028+0000","last_clean":"2022-08-01T23:51:26.477028+0000","last_became_active":"2022-08-01T22:55:44.413727+0000","last_became_peered":"2022-08-01T22:55:44.413727+0000","last_unstale":"2022-08-01T23:51:26.477028+0000","last_undegraded":"2022-08-01T23:51:26.477028+0000","last_fullsized":"2022-08-01T23:51:26.477028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:26.477027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:26.477027+0000","last_clean_scrub_stamp":"2022-08-01T23:51:26.477027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T23:05:48.346553+0000","last_change":"2022-08-01T22:55:44.415068+0000","last_active":"2022-08-01T23:05:48.346553+0000","last_peered":"2022-08-01T23:05:48.346553+0000","last_clean":"2022-08-01T23:05:48.346553+0000","last_became_active":"2022-08-01T22:55:44.414349+0000","last_became_peered":"2022-08-01T22:55:44.414349+0000","last_unstale":"2022-08-01T23:05:48.346553+0000","last_undegraded":"2022-08-01T23:05:48.346553+0000","last_fullsized":"2022-08-01T23:05:48.346553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T22:55:43.171033+0000","last_clean_scrub_stamp":"2022-08-01T22:55:43.171033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":146,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:28.495358+0000","last_change":"2022-08-01T23:51:28.495643+0000","last_active":"2022-08-01T23:51:28.495358+0000","last_peered":"2022-08-01T23:51:28.495358+0000","last_clean":"2022-08-01T23:51:28.495358+0000","last_became_active":"2022-08-01T22:55:44.189745+0000","last_became_peered":"2022-08-01T22:55:44.189745+0000","last_unstale":"2022-08-01T23:51:28.495358+0000","last_undegraded":"2022-08-01T23:51:28.495358+0000","last_fullsized":"2022-08-01T23:51:28.495358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:28.495357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:28.495357+0000","last_clean_scrub_stamp":"2022-08-01T23:51:28.495357+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:24.682067+0000","last_change":"2022-08-01T23:51:24.682146+0000","last_active":"2022-08-01T23:51:24.682067+0000","last_peered":"2022-08-01T23:51:24.682067+0000","last_clean":"2022-08-01T23:51:24.682067+0000","last_became_active":"2022-08-01T22:55:44.191167+0000","last_became_peered":"2022-08-01T22:55:44.191167+0000","last_unstale":"2022-08-01T23:51:24.682067+0000","last_undegraded":"2022-08-01T23:51:24.682067+0000","last_fullsized":"2022-08-01T23:51:24.682067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:24.682067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:24.682067+0000","last_clean_scrub_stamp":"2022-08-01T23:51:24.682067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3069999999999999},"last":1.133},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.069,"5min":1.222,"15min":1.27},"last":0.85899999999999999}]},{"osd":1,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.101,"5min":1.2589999999999999,"15min":1.7210000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93200000000000005,"15min":0.93000000000000005},"min":{"1min":0.755,"5min":0.66100000000000003,"15min":0.61899999999999999},"max":{"1min":1.147,"5min":1.1599999999999999,"15min":1.653},"last":0.93100000000000005}]},{"osd":2,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1799999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.163},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92300000000000004,"15min":0.93300000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.22,"15min":1.3999999999999999},"last":1.3069999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.135,"5min":1.1799999999999999,"15min":1.2729999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.71799999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2130000000000001,"15min":1.3540000000000001},"last":1.1919999999999999}]},{"osd":4,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.58899999999999997,"5min":0.53300000000000003,"15min":0.52900000000000003},"max":{"1min":1.155,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.88100000000000001,"15min":0.875},"min":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.58099999999999996},"max":{"1min":1.2370000000000001,"5min":1.288,"15min":1.323},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":1.444},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.296,"5min":1.325,"15min":1.3420000000000001},"last":1.0269999999999999}]},{"osd":6,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98899999999999999,"15min":0.97799999999999998},"min":{"1min":0.73499999999999999,"5min":0.65000000000000002,"15min":0.628},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.319},"last":1.087},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.006,"15min":0.999},"min":{"1min":0.72899999999999998,"5min":0.67700000000000005,"15min":0.627},"max":{"1min":1.2869999999999999,"5min":1.349,"15min":1.446},"last":1.002}]}]},{"osd":6,"up_from":13,"seq":55834575522,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.48899999999999999},"max":{"1min":1.21,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93400000000000005,"15min":0.92200000000000004},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.57999999999999996},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.92700000000000005}]},{"osd":1,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59299999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.032}]},{"osd":2,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.2270000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.879,"15min":0.88500000000000001},"min":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.60199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.284},"last":0.89700000000000002}]},{"osd":3,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":0.90600000000000003},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.151,"5min":1.1639999999999999,"15min":1.274},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.72699999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.86699999999999999}]},{"osd":4,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.252},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.57699999999999996},"max":{"1min":1.28,"5min":1.286,"15min":1.3999999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.95999999999999996,"15min":0.94099999999999995},"min":{"1min":0.59999999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.246,"5min":1.2789999999999999,"15min":1.3500000000000001},"last":1.103}]},{"osd":7,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0369999999999999,"15min":1.02},"min":{"1min":0.92100000000000004,"5min":0.69899999999999995,"15min":0.61399999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.071,"15min":1.077},"min":{"1min":0.92800000000000005,"5min":0.64200000000000002,"15min":0.61799999999999999},"max":{"1min":1.351,"5min":1.3600000000000001,"15min":1.546},"last":0.93600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575522,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92100000000000004,"15min":0.88300000000000001},"min":{"1min":0.81299999999999994,"5min":0.58499999999999996,"15min":0.55800000000000005},"max":{"1min":1.123,"5min":1.161,"15min":1.2729999999999999},"last":0.876},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.81100000000000005,"5min":0.68799999999999994,"15min":0.64600000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3120000000000001},"last":1.0449999999999999}]},{"osd":1,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.876,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1459999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0569999999999999,"5min":1.25,"15min":1.25},"last":0.78700000000000003}]},{"osd":2,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.88500000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.157,"5min":1.1819999999999999,"15min":1.3480000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94199999999999995,"15min":0.94399999999999995},"min":{"1min":0.70899999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.304},"last":0.90100000000000002}]},{"osd":3,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"max":{"1min":1.139,"5min":1.1930000000000001,"15min":1.278},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0089999999999999,"15min":1.0049999999999999},"min":{"1min":0.68300000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3540000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85199999999999998,"15min":0.85399999999999998},"min":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.3080000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.93000000000000005},"min":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.57299999999999995},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.3120000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.60899999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.4279999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98299999999999998},"min":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.3,"5min":1.304,"15min":1.48},"last":0.72699999999999998}]},{"osd":7,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0269999999999999,"15min":1.0129999999999999},"min":{"1min":0.83799999999999997,"5min":0.71099999999999997,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.4990000000000001},"last":0.999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.165,"15min":1.153},"min":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.94299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.91300000000000003,"15min":0.90200000000000002},"min":{"1min":0.76300000000000001,"5min":0.60199999999999998,"15min":0.54400000000000004},"max":{"1min":1.1659999999999999,"5min":1.171,"15min":1.274},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91500000000000004,"15min":0.89800000000000002},"min":{"1min":0.82599999999999996,"5min":0.72199999999999998,"15min":0.66300000000000003},"max":{"1min":1.143,"5min":1.1479999999999999,"15min":1.2230000000000001},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83999999999999997,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.54200000000000004},"max":{"1min":1.083,"5min":1.194,"15min":1.194},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.073}]},{"osd":2,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.94799999999999995,"15min":0.94499999999999995},"min":{"1min":0.746,"5min":0.61399999999999999,"15min":0.55800000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2310000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93700000000000006,"15min":0.95299999999999996},"min":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.56599999999999995},"max":{"1min":1.1639999999999999,"5min":1.1839999999999999,"15min":1.3360000000000001},"last":0.88600000000000001}]},{"osd":3,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.032,"5min":1.0429999999999999,"15min":1.169},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92300000000000004},"min":{"1min":0.73099999999999998,"5min":0.70499999999999996,"15min":0.66700000000000004},"max":{"1min":1.0660000000000001,"5min":1.163,"15min":1.2390000000000001},"last":1.143}]},{"osd":5,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.58499999999999996,"5min":0.57399999999999995,"15min":0.52000000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.3089999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.63500000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89700000000000002,"15min":0.91700000000000004},"min":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"max":{"1min":1.1559999999999999,"5min":1.3500000000000001,"15min":1.4299999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.66600000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2310000000000001,"5min":1.244,"15min":1.3819999999999999},"last":0.999}]},{"osd":7,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.73599999999999999,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.1910000000000001,"5min":1.27,"15min":1.4219999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.042,"15min":1.0509999999999999},"min":{"1min":0.79900000000000004,"5min":0.71699999999999997,"15min":0.65500000000000003},"max":{"1min":1.2290000000000001,"5min":1.4039999999999999,"15min":1.5049999999999999},"last":1.224}]}]},{"osd":3,"up_from":13,"seq":55834575522,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.335,"15min":1.335},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.202,"15min":1.202},"last":0.84899999999999998}]},{"osd":1,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.495,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.135},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":1.3380000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.79300000000000004}]},{"osd":2,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.51600000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.88600000000000001},"min":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.50800000000000001},"max":{"1min":1.401,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.90500000000000003}]},{"osd":4,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.92800000000000005},"min":{"1min":0.73699999999999999,"5min":0.70399999999999996,"15min":0.66600000000000004},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.98099999999999998,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.628},"max":{"1min":1.5720000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97199999999999998,"15min":0.95499999999999996},"min":{"1min":0.73699999999999999,"5min":0.72599999999999998,"15min":0.63200000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.002}]},{"osd":6,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0189999999999999,"15min":1.006},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71099999999999997},"max":{"1min":1.645,"5min":1.645,"15min":1.6919999999999999},"last":1.034},{"interface":"front","average":{"1min":1.034,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.78900000000000003,"5min":0.75,"15min":0.72999999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.84699999999999998,"5min":0.77000000000000002,"15min":0.75900000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.726},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.099,"15min":1.085},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.72299999999999998},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.23}]}]},{"osd":2,"up_from":13,"seq":55834575522,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64700000000000002,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.439,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":1,"15min":1.0580000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.66000000000000003,"5min":0.58799999999999997,"15min":0.58199999999999996},"max":{"1min":0.90100000000000002,"5min":1.0780000000000001,"15min":1.1479999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.64800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.1160000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.149},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84699999999999998,"15min":0.84199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.0880000000000001,"5min":1.1100000000000001,"15min":1.1519999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.97199999999999998,"15min":0.94399999999999995},"min":{"1min":0.80800000000000005,"5min":0.71399999999999997,"15min":0.65100000000000002},"max":{"1min":1.1639999999999999,"5min":1.238,"15min":1.2450000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90000000000000002,"15min":0.91400000000000003},"min":{"1min":0.81899999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.2789999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0149999999999999,"15min":1.016},"min":{"1min":0.76600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.607},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0009999999999999,"15min":0.99199999999999999},"min":{"1min":0.84999999999999998,"5min":0.72799999999999998,"15min":0.71699999999999997},"max":{"1min":1.232,"5min":1.413,"15min":1.413},"last":0.88}]},{"osd":6,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.96699999999999997},"min":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"max":{"1min":1.262,"5min":1.333,"15min":1.504},"last":1.143},{"interface":"front","average":{"1min":1.129,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.81999999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.77},"last":1.131}]},{"osd":7,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.85699999999999998,"5min":0.79800000000000004,"15min":0.77800000000000002},"max":{"1min":1.2789999999999999,"5min":1.49,"15min":1.49},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.056,"5min":1.0589999999999999,"15min":1.0740000000000001},"min":{"1min":0.92600000000000005,"5min":0.80100000000000005,"15min":0.73699999999999999},"max":{"1min":1.2,"5min":1.498,"15min":1.575},"last":1.2230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575523,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.63400000000000001},"min":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":1.133,"15min":1.258},"last":0.504},{"interface":"front","average":{"1min":0.749,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.82799999999999996,"5min":1.05,"15min":1.3100000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78200000000000003,"15min":0.77100000000000002},"min":{"1min":0.58799999999999997,"5min":0.53400000000000003,"15min":0.51900000000000002},"max":{"1min":1.085,"5min":1.1830000000000001,"15min":1.2829999999999999},"last":0.90000000000000002}]},{"osd":3,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83999999999999997,"15min":0.82299999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56200000000000006},"max":{"1min":1.0029999999999999,"5min":1.552,"15min":1.552},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.58299999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93200000000000005,"5min":1.427,"15min":1.427},"last":0.77600000000000002}]},{"osd":4,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.88600000000000001,"15min":0.91300000000000003},"min":{"1min":0.67100000000000004,"5min":0.66200000000000003,"15min":0.63300000000000001},"max":{"1min":1.2130000000000001,"5min":1.3129999999999999,"15min":1.381},"last":0.745},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.3029999999999999,"15min":1.573},"last":1.165}]},{"osd":5,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"max":{"1min":1.155,"5min":1.272,"15min":1.373},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.008},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.68799999999999994},"max":{"1min":1.2569999999999999,"5min":1.657,"15min":1.7190000000000001},"last":0.85099999999999998}]},{"osd":6,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.02,"15min":1.0409999999999999},"min":{"1min":0.85999999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.29,"5min":1.72,"15min":1.72},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.75800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.129,"5min":1.2589999999999999,"15min":1.357},"last":1.121}]},{"osd":7,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.86699999999999999,"5min":0.754,"15min":0.74099999999999999},"max":{"1min":1.518,"5min":1.518,"15min":2.0369999999999999},"last":1.242},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.755},"max":{"1min":1.1890000000000001,"5min":1.3320000000000001,"15min":1.411},"last":0.98899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575522,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.42499999999999999},"max":{"1min":0.71299999999999997,"5min":1.012,"15min":1.173},"last":0.625},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.438},"max":{"1min":0.90400000000000003,"5min":1.0409999999999999,"15min":1.2030000000000001},"last":0.875}]},{"osd":2,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47799999999999998},"max":{"1min":0.79500000000000004,"5min":1.103,"15min":1.135},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.73799999999999999},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":0.89500000000000002,"5min":1.087,"15min":1.095},"last":0.78800000000000003}]},{"osd":3,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.48699999999999999},"max":{"1min":0.90300000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97899999999999998,"5min":1.042,"15min":1.26},"last":0.90600000000000003}]},{"osd":4,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.67000000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2629999999999999,"5min":1.321,"15min":1.4950000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92900000000000005,"15min":0.92200000000000004},"min":{"1min":0.73199999999999998,"5min":0.68999999999999995,"15min":0.64700000000000002},"max":{"1min":1.1020000000000001,"5min":1.3080000000000001,"15min":1.331},"last":0.85599999999999998}]},{"osd":5,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93200000000000005,"15min":0.93999999999999995},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.68300000000000005},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.016,"15min":1.038},"min":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.70099999999999996},"max":{"1min":1.323,"5min":1.3360000000000001,"15min":1.4299999999999999},"last":1.0980000000000001}]},{"osd":6,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.86899999999999999,"5min":0.72799999999999998,"15min":0.70699999999999996},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.075,"15min":1.0669999999999999},"min":{"1min":0.84999999999999998,"5min":0.80000000000000004,"15min":0.76700000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.4330000000000001},"last":1.3839999999999999}]},{"osd":7,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1040000000000001,"15min":1.1000000000000001},"min":{"1min":0.84099999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.3680000000000001,"5min":1.4179999999999999,"15min":1.423},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.75600000000000001},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.7210000000000001},"last":1.042}]}]}]}} 2022-08-01T23:51:45.026 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:45.027 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2022-08-01T22:55:43.171033+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=22, tm_min=55, tm_sec=43, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=23, tm_min=51, tm_sec=19, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T23:51:45.028 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2022-08-01T23:52:03.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:03.475 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:03.477 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:03.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:05.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T23:52:05.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 Processor -- start 2022-08-01T23:52:05.144 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 -- start start 2022-08-01T23:52:05.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7d4064b10 0x7fc7d4144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.149 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d4064ef0 con 0x7fc7d40b36e0 2022-08-01T23:52:05.150 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d413ebf0 con 0x7fc7d4064b10 2022-08-01T23:52:05.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d8d49700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4066690 con 0x7fc7d4141a80 2022-08-01T23:52:05.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7d4064b10 0x7fc7d4144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.151 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7d40b36e0 0x7fc7d4141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38266/0 (socket says 172.21.15.32:38266) 2022-08-01T23:52:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.139+0000 7fc7d2ffd700 1 -- 172.21.15.32:0/2285885345 learned_addr learned my addr 172.21.15.32:0/2285885345 (peer_addr_for_me v1:172.21.15.32:0/0) 2022-08-01T23:52:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d3fff700 1 --2- 172.21.15.32:0/2285885345 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d27fc700 1 -- 172.21.15.32:0/2285885345 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477481099 0 0) 0x7fc7d4064ef0 con 0x7fc7d40b36e0 2022-08-01T23:52:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d27fc700 1 -- 172.21.15.32:0/2285885345 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b4003260 con 0x7fc7d40b36e0 2022-08-01T23:52:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d37fe700 1 -- 172.21.15.32:0/2285885345 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 msgr2=0x7fc7d4066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:52:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d37fe700 1 --2- 172.21.15.32:0/2285885345 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d37fe700 1 -- 172.21.15.32:0/2285885345 >> v1:172.21.15.32:6789/0 conn(0x7fc7d40b36e0 legacy=0x7fc7d4141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d27fc700 1 -- 172.21.15.32:0/2285885345 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828786394 0 0) 0x7fc7b4003260 con 0x7fc7d40b36e0 2022-08-01T23:52:05.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d37fe700 1 -- 172.21.15.32:0/2285885345 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40b4850 con 0x7fc7d4064b10 2022-08-01T23:52:05.154 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d37fe700 1 --2- 172.21.15.32:0/2285885345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d4064b10 0x7fc7d4144e60 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7fc7bc001800 tx=0x7fc7c4006df0).ready entity=mon.0 client_cookie=f4592e6f448e4782 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:52:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d27fc700 1 -- 172.21.15.32:0/2285885345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7bc001660 con 0x7fc7d4064b10 2022-08-01T23:52:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d27fc700 1 -- 172.21.15.32:0/2285885345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4000e90 con 0x7fc7d4064b10 2022-08-01T23:52:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d27fc700 1 -- 172.21.15.32:0/2285885345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7c4012460 con 0x7fc7d4064b10 2022-08-01T23:52:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 -- 172.21.15.32:0/2285885345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d4064b10 msgr2=0x7fc7d4144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/2285885345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d4064b10 0x7fc7d4144e60 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7fc7bc001800 tx=0x7fc7c4006df0).stop 2022-08-01T23:52:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 -- 172.21.15.32:0/2285885345 shutdown_connections 2022-08-01T23:52:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/2285885345 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/2285885345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d4064b10 0x7fc7d4144e60 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 -- 172.21.15.32:0/2285885345 >> 172.21.15.32:0/2285885345 conn(0x7fc7d405f840 msgr2=0x7fc7d405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:52:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 -- 172.21.15.32:0/2285885345 shutdown_connections 2022-08-01T23:52:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 -- 172.21.15.32:0/2285885345 wait complete. 2022-08-01T23:52:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 Processor -- start 2022-08-01T23:52:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.143+0000 7fc7d8d49700 1 -- start start 2022-08-01T23:52:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7d4064b10 0x7fc7d4140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 0x7fc7d4131550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4131a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4135350 con 0x7fc7d40b36e0 2022-08-01T23:52:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4131f30 con 0x7fc7d4064b10 2022-08-01T23:52:05.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4136840 con 0x7fc7d4141a80 2022-08-01T23:52:05.164 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 0x7fc7d4131550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 0x7fc7d4131550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50732/0 (socket says 172.21.15.32:50732) 2022-08-01T23:52:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 -- 172.21.15.32:0/115054593 learned_addr learned my addr 172.21.15.32:0/115054593 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:52:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d37fe700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7d4064b10 0x7fc7d4140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d3fff700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 -- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 msgr2=0x7fc7d4131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4131a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 -- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7d4064b10 msgr2=0x7fc7d4140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7d4064b10 0x7fc7d4140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 -- 172.21.15.32:0/115054593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c400b180 con 0x7fc7d40b36e0 2022-08-01T23:52:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d2ffd700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 0x7fc7d4131550 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fc7d4132b50 tx=0x7fc7bc001230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:52:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7bc007170 con 0x7fc7d40b36e0 2022-08-01T23:52:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc0072d0 con 0x7fc7d40b36e0 2022-08-01T23:52:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d4147140 con 0x7fc7d40b36e0 2022-08-01T23:52:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7bc004710 con 0x7fc7d40b36e0 2022-08-01T23:52:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.147+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d4136de0 con 0x7fc7d40b36e0 2022-08-01T23:52:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.151+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d4136fa0 con 0x7fc7d40b36e0 2022-08-01T23:52:05.170 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.151+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc7bc004870 con 0x7fc7d40b36e0 2022-08-01T23:52:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.151+0000 7fc7d0ff9700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc7b8053950 0x7fc7b8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.151+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc7bc06b750 con 0x7fc7d40b36e0 2022-08-01T23:52:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.151+0000 7fc7d37fe700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc7b8053950 0x7fc7b8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.151+0000 7fc7d37fe700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc7b8053950 0x7fc7b8055e10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fc7c401a000 tx=0x7fc7c4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:52:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.155+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7bc0426e0 con 0x7fc7d40b36e0 2022-08-01T23:52:05.286 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.283+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 --> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7d4069d90 con 0x7fc7b8053950 2022-08-01T23:52:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d0ff9700 1 -- 172.21.15.32:0/115054593 <== mgr.4102 v2:172.21.15.55:6800/22889 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68280 (secure 0 0 0) 0x7fc7d4069d90 con 0x7fc7b8053950 2022-08-01T23:52:05.295 INFO:teuthology.orchestra.run.smithi032.stdout: 2022-08-01T23:52:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc7b8053950 msgr2=0x7fc7b8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc7b8053950 0x7fc7b8055e10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fc7c401a000 tx=0x7fc7c4014000).stop 2022-08-01T23:52:05.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 msgr2=0x7fc7d4131550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.298 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 0x7fc7d4131550 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7fc7d4132b50 tx=0x7fc7bc001230).stop 2022-08-01T23:52:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 shutdown_connections 2022-08-01T23:52:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fc7b8053950 0x7fc7b8055e10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fc7d4141a80 0x7fc7d4131a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d40b36e0 0x7fc7d4131550 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 --2- 172.21.15.32:0/115054593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7d4064b10 0x7fc7d4140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 >> 172.21.15.32:0/115054593 conn(0x7fc7d405f840 msgr2=0x7fc7d4147b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:52:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 shutdown_connections 2022-08-01T23:52:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.291+0000 7fc7d8d49700 1 -- 172.21.15.32:0/115054593 wait complete. 2022-08-01T23:52:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2022-08-01T23:52:05.313 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":1849,"stamp":"2022-08-01T23:52:05.176515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404704,"kb_used_data":16032,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570016,"statfs":{"total":773094113280,"available":764487696384,"internally_reserved":8589934592,"allocated":16416768,"data_stored":8039656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003316"},"pg_stats":[{"pgid":"1.0","version":"161'4","reported_seq":150,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:30.288449+0000","last_change":"2022-08-01T23:51:30.288626+0000","last_active":"2022-08-01T23:51:30.288449+0000","last_peered":"2022-08-01T23:51:30.288449+0000","last_clean":"2022-08-01T23:51:30.288449+0000","last_became_active":"2022-08-01T22:55:40.425307+0000","last_became_peered":"2022-08-01T22:55:40.425307+0000","last_unstale":"2022-08-01T23:51:30.288449+0000","last_undegraded":"2022-08-01T23:51:30.288449+0000","last_fullsized":"2022-08-01T23:51:30.288449+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'4","last_scrub_stamp":"2022-08-01T23:51:30.288448+0000","last_deep_scrub":"161'4","last_deep_scrub_stamp":"2022-08-01T23:51:30.288448+0000","last_clean_scrub_stamp":"2022-08-01T23:51:30.288448+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:30.644448+0000","last_change":"2022-08-01T23:51:30.644558+0000","last_active":"2022-08-01T23:51:30.644448+0000","last_peered":"2022-08-01T23:51:30.644448+0000","last_clean":"2022-08-01T23:51:30.644448+0000","last_became_active":"2022-08-01T22:55:44.188196+0000","last_became_peered":"2022-08-01T22:55:44.188196+0000","last_unstale":"2022-08-01T23:51:30.644448+0000","last_undegraded":"2022-08-01T23:51:30.644448+0000","last_fullsized":"2022-08-01T23:51:30.644448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:30.644447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:30.644447+0000","last_clean_scrub_stamp":"2022-08-01T23:51:30.644447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:40.414672+0000","last_change":"2022-08-01T23:51:40.414861+0000","last_active":"2022-08-01T23:51:40.414672+0000","last_peered":"2022-08-01T23:51:40.414672+0000","last_clean":"2022-08-01T23:51:40.414672+0000","last_became_active":"2022-08-01T22:55:44.187581+0000","last_became_peered":"2022-08-01T22:55:44.187581+0000","last_unstale":"2022-08-01T23:51:40.414672+0000","last_undegraded":"2022-08-01T23:51:40.414672+0000","last_fullsized":"2022-08-01T23:51:40.414672+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:40.414671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:40.414671+0000","last_clean_scrub_stamp":"2022-08-01T23:51:40.414671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":143,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:23.207399+0000","last_change":"2022-08-01T23:51:23.207520+0000","last_active":"2022-08-01T23:51:23.207399+0000","last_peered":"2022-08-01T23:51:23.207399+0000","last_clean":"2022-08-01T23:51:23.207399+0000","last_became_active":"2022-08-01T22:55:44.554353+0000","last_became_peered":"2022-08-01T22:55:44.554353+0000","last_unstale":"2022-08-01T23:51:23.207399+0000","last_undegraded":"2022-08-01T23:51:23.207399+0000","last_fullsized":"2022-08-01T23:51:23.207399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:23.207398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:23.207398+0000","last_clean_scrub_stamp":"2022-08-01T23:51:23.207398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:28.641034+0000","last_change":"2022-08-01T23:51:28.641235+0000","last_active":"2022-08-01T23:51:28.641034+0000","last_peered":"2022-08-01T23:51:28.641034+0000","last_clean":"2022-08-01T23:51:28.641034+0000","last_became_active":"2022-08-01T22:55:44.188345+0000","last_became_peered":"2022-08-01T22:55:44.188345+0000","last_unstale":"2022-08-01T23:51:28.641034+0000","last_undegraded":"2022-08-01T23:51:28.641034+0000","last_fullsized":"2022-08-01T23:51:28.641034+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:28.641033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:28.641033+0000","last_clean_scrub_stamp":"2022-08-01T23:51:28.641033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:26.477028+0000","last_change":"2022-08-01T23:51:26.477225+0000","last_active":"2022-08-01T23:51:26.477028+0000","last_peered":"2022-08-01T23:51:26.477028+0000","last_clean":"2022-08-01T23:51:26.477028+0000","last_became_active":"2022-08-01T22:55:44.413727+0000","last_became_peered":"2022-08-01T22:55:44.413727+0000","last_unstale":"2022-08-01T23:51:26.477028+0000","last_undegraded":"2022-08-01T23:51:26.477028+0000","last_fullsized":"2022-08-01T23:51:26.477028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:26.477027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:26.477027+0000","last_clean_scrub_stamp":"2022-08-01T23:51:26.477027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":143,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:41.404874+0000","last_change":"2022-08-01T23:51:41.405082+0000","last_active":"2022-08-01T23:51:41.404874+0000","last_peered":"2022-08-01T23:51:41.404874+0000","last_clean":"2022-08-01T23:51:41.404874+0000","last_became_active":"2022-08-01T22:55:44.414349+0000","last_became_peered":"2022-08-01T22:55:44.414349+0000","last_unstale":"2022-08-01T23:51:41.404874+0000","last_undegraded":"2022-08-01T23:51:41.404874+0000","last_fullsized":"2022-08-01T23:51:41.404874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:41.404873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:41.404873+0000","last_clean_scrub_stamp":"2022-08-01T23:51:41.404873+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":146,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:28.495358+0000","last_change":"2022-08-01T23:51:28.495643+0000","last_active":"2022-08-01T23:51:28.495358+0000","last_peered":"2022-08-01T23:51:28.495358+0000","last_clean":"2022-08-01T23:51:28.495358+0000","last_became_active":"2022-08-01T22:55:44.189745+0000","last_became_peered":"2022-08-01T22:55:44.189745+0000","last_unstale":"2022-08-01T23:51:28.495358+0000","last_undegraded":"2022-08-01T23:51:28.495358+0000","last_fullsized":"2022-08-01T23:51:28.495358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:28.495357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:28.495357+0000","last_clean_scrub_stamp":"2022-08-01T23:51:28.495357+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T23:51:24.682067+0000","last_change":"2022-08-01T23:51:24.682146+0000","last_active":"2022-08-01T23:51:24.682067+0000","last_peered":"2022-08-01T23:51:24.682067+0000","last_clean":"2022-08-01T23:51:24.682067+0000","last_became_active":"2022-08-01T22:55:44.191167+0000","last_became_peered":"2022-08-01T22:55:44.191167+0000","last_unstale":"2022-08-01T23:51:24.682067+0000","last_undegraded":"2022-08-01T23:51:24.682067+0000","last_fullsized":"2022-08-01T23:51:24.682067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T23:51:24.682067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T23:51:24.682067+0000","last_clean_scrub_stamp":"2022-08-01T23:51:24.682067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9733,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575527,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84099999999999997,"15min":0.871},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3069999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.069,"5min":1.222,"15min":1.27},"last":1.028}]},{"osd":1,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60999999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.101,"5min":1.2589999999999999,"15min":1.7210000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93200000000000005,"15min":0.93000000000000005},"min":{"1min":0.755,"5min":0.66100000000000003,"15min":0.61899999999999999},"max":{"1min":1.147,"5min":1.1599999999999999,"15min":1.653},"last":1.125}]},{"osd":2,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1799999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92300000000000004,"15min":0.93300000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.22,"15min":1.3999999999999999},"last":1.0860000000000001}]},{"osd":3,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.135,"5min":1.1799999999999999,"15min":1.2729999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.71799999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2130000000000001,"15min":1.3540000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.58899999999999997,"5min":0.53300000000000003,"15min":0.52900000000000003},"max":{"1min":1.155,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.88100000000000001,"15min":0.875},"min":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.58099999999999996},"max":{"1min":1.2370000000000001,"5min":1.288,"15min":1.323},"last":0.91800000000000004}]},{"osd":5,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":1.444},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0189999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.296,"5min":1.325,"15min":1.3420000000000001},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Aug 1 23:51:33 2022","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98899999999999999,"15min":0.97799999999999998},"min":{"1min":0.73499999999999999,"5min":0.65000000000000002,"15min":0.628},"max":{"1min":1.234,"5min":1.2569999999999999,"15min":1.319},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.006,"15min":0.999},"min":{"1min":0.72899999999999998,"5min":0.67700000000000005,"15min":0.627},"max":{"1min":1.2869999999999999,"5min":1.349,"15min":1.446},"last":0.98699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575527,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.48899999999999999},"max":{"1min":1.21,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93400000000000005,"15min":0.92200000000000004},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.57999999999999996},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.98999999999999999}]},{"osd":1,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59299999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.84399999999999997},"min":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.82899999999999996}]},{"osd":2,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.2270000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.879,"15min":0.88500000000000001},"min":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.60199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.284},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":0.90600000000000003},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.151,"5min":1.1639999999999999,"15min":1.274},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.72699999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.95399999999999996}]},{"osd":4,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.252},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.57699999999999996},"max":{"1min":1.28,"5min":1.286,"15min":1.3999999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.95999999999999996,"15min":0.94099999999999995},"min":{"1min":0.59999999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.246,"5min":1.2789999999999999,"15min":1.3500000000000001},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Aug 1 23:51:02 2022","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0369999999999999,"15min":1.02},"min":{"1min":0.92100000000000004,"5min":0.69899999999999995,"15min":0.61399999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.071,"15min":1.077},"min":{"1min":0.92800000000000005,"5min":0.64200000000000002,"15min":0.61799999999999999},"max":{"1min":1.351,"5min":1.3600000000000001,"15min":1.546},"last":0.96799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575527,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92100000000000004,"15min":0.88300000000000001},"min":{"1min":0.81299999999999994,"5min":0.58499999999999996,"15min":0.55800000000000005},"max":{"1min":1.123,"5min":1.161,"15min":1.2729999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.81100000000000005,"5min":0.68799999999999994,"15min":0.64600000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3120000000000001},"last":0.86799999999999999}]},{"osd":1,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.876,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1459999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.131},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0569999999999999,"5min":1.25,"15min":1.25},"last":0.67700000000000005}]},{"osd":2,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.88500000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.157,"5min":1.1819999999999999,"15min":1.3480000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94199999999999995,"15min":0.94399999999999995},"min":{"1min":0.70899999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.304},"last":0.81999999999999995}]},{"osd":3,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"max":{"1min":1.139,"5min":1.1930000000000001,"15min":1.278},"last":0.749},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0089999999999999,"15min":1.0049999999999999},"min":{"1min":0.68300000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3540000000000001},"last":0.92800000000000005}]},{"osd":4,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85199999999999998,"15min":0.85399999999999998},"min":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.3080000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.93000000000000005},"min":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.57299999999999995},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.3120000000000001},"last":0.95999999999999996}]},{"osd":6,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.60899999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.4279999999999999},"last":1.085},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98299999999999998},"min":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.3,"5min":1.304,"15min":1.48},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Aug 1 23:51:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0269999999999999,"15min":1.0129999999999999},"min":{"1min":0.83799999999999997,"5min":0.71099999999999997,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.4990000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.165,"15min":1.153},"min":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1719999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.91300000000000003,"15min":0.90200000000000002},"min":{"1min":0.76300000000000001,"5min":0.60199999999999998,"15min":0.54400000000000004},"max":{"1min":1.1659999999999999,"5min":1.171,"15min":1.274},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91500000000000004,"15min":0.89800000000000002},"min":{"1min":0.82599999999999996,"5min":0.72199999999999998,"15min":0.66300000000000003},"max":{"1min":1.143,"5min":1.1479999999999999,"15min":1.2230000000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83999999999999997,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.54200000000000004},"max":{"1min":1.083,"5min":1.194,"15min":1.194},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.747}]},{"osd":2,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.94799999999999995,"15min":0.94499999999999995},"min":{"1min":0.746,"5min":0.61399999999999999,"15min":0.55800000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2310000000000001},"last":1.002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93700000000000006,"15min":0.95299999999999996},"min":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.56599999999999995},"max":{"1min":1.1639999999999999,"5min":1.1839999999999999,"15min":1.3360000000000001},"last":1.038}]},{"osd":3,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.032,"5min":1.0429999999999999,"15min":1.169},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.92000000000000004,"15min":0.92300000000000004},"min":{"1min":0.73099999999999998,"5min":0.70499999999999996,"15min":0.66700000000000004},"max":{"1min":1.0660000000000001,"5min":1.163,"15min":1.2390000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.58499999999999996,"5min":0.57399999999999995,"15min":0.52000000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.3089999999999999},"last":0.877},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.63500000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89700000000000002,"15min":0.91700000000000004},"min":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"max":{"1min":1.1559999999999999,"5min":1.3500000000000001,"15min":1.4299999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.66600000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2310000000000001,"5min":1.244,"15min":1.3819999999999999},"last":1.083}]},{"osd":7,"last update":"Mon Aug 1 23:51:21 2022","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.73599999999999999,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.1910000000000001,"5min":1.27,"15min":1.4219999999999999},"last":1.016},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.042,"15min":1.0509999999999999},"min":{"1min":0.79900000000000004,"5min":0.71699999999999997,"15min":0.65500000000000003},"max":{"1min":1.2290000000000001,"5min":1.4039999999999999,"15min":1.5049999999999999},"last":1.117}]}]},{"osd":3,"up_from":13,"seq":55834575527,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.335,"15min":1.335},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.202,"15min":1.202},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.495,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.135},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":1.3380000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.51600000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.88600000000000001},"min":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.50800000000000001},"max":{"1min":1.401,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.85899999999999999}]},{"osd":4,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.92800000000000005},"min":{"1min":0.73699999999999999,"5min":0.70399999999999996,"15min":0.66600000000000004},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.98099999999999998,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.628},"max":{"1min":1.5720000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.048},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97199999999999998,"15min":0.95499999999999996},"min":{"1min":0.73699999999999999,"5min":0.72599999999999998,"15min":0.63200000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.747}]},{"osd":6,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0189999999999999,"15min":1.006},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71099999999999997},"max":{"1min":1.645,"5min":1.645,"15min":1.6919999999999999},"last":1.103},{"interface":"front","average":{"1min":1.034,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.78900000000000003,"5min":0.75,"15min":0.72999999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.125}]},{"osd":7,"last update":"Mon Aug 1 23:51:28 2022","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.84699999999999998,"5min":0.77000000000000002,"15min":0.75900000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.726},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.099,"15min":1.085},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.72299999999999998},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.145}]}]},{"osd":2,"up_from":13,"seq":55834575527,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64700000000000002,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.439,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":1,"15min":1.0580000000000001},"last":0.495},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.66000000000000003,"5min":0.58799999999999997,"15min":0.58199999999999996},"max":{"1min":0.90100000000000002,"5min":1.0780000000000001,"15min":1.1479999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.64800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.1160000000000001},"last":0.78400000000000003}]},{"osd":3,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.149},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84699999999999998,"15min":0.84199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.0880000000000001,"5min":1.1100000000000001,"15min":1.1519999999999999},"last":0.72699999999999998}]},{"osd":4,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.97199999999999998,"15min":0.94399999999999995},"min":{"1min":0.80800000000000005,"5min":0.71399999999999997,"15min":0.65100000000000002},"max":{"1min":1.1639999999999999,"5min":1.238,"15min":1.2450000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90000000000000002,"15min":0.91400000000000003},"min":{"1min":0.81899999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.2789999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0149999999999999,"15min":1.016},"min":{"1min":0.76600000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.607},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0009999999999999,"15min":0.99199999999999999},"min":{"1min":0.84999999999999998,"5min":0.72799999999999998,"15min":0.71699999999999997},"max":{"1min":1.232,"5min":1.413,"15min":1.413},"last":1.042}]},{"osd":6,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.96699999999999997},"min":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"max":{"1min":1.262,"5min":1.333,"15min":1.504},"last":1.012},{"interface":"front","average":{"1min":1.129,"5min":1.075,"15min":1.0800000000000001},"min":{"1min":0.81999999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.77},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Aug 1 23:51:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.85699999999999998,"5min":0.79800000000000004,"15min":0.77800000000000002},"max":{"1min":1.2789999999999999,"5min":1.49,"15min":1.49},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.0589999999999999,"15min":1.0740000000000001},"min":{"1min":0.92600000000000005,"5min":0.80100000000000005,"15min":0.73699999999999999},"max":{"1min":1.2,"5min":1.498,"15min":1.575},"last":1.125}]}]},{"osd":1,"up_from":13,"seq":55834575528,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.63400000000000001},"min":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84899999999999998,"5min":1.133,"15min":1.258},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.82799999999999996,"5min":1.05,"15min":1.3100000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78200000000000003,"15min":0.77100000000000002},"min":{"1min":0.58799999999999997,"5min":0.53400000000000003,"15min":0.51900000000000002},"max":{"1min":1.085,"5min":1.1830000000000001,"15min":1.2829999999999999},"last":0.72399999999999998}]},{"osd":3,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83999999999999997,"15min":0.82299999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56200000000000006},"max":{"1min":1.0029999999999999,"5min":1.552,"15min":1.552},"last":0.629},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.58299999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93200000000000005,"5min":1.427,"15min":1.427},"last":0.80200000000000005}]},{"osd":4,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.88600000000000001,"15min":0.91300000000000003},"min":{"1min":0.67100000000000004,"5min":0.66200000000000003,"15min":0.63300000000000001},"max":{"1min":1.2130000000000001,"5min":1.3129999999999999,"15min":1.381},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.3029999999999999,"15min":1.573},"last":0.97899999999999998}]},{"osd":5,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"max":{"1min":1.155,"5min":1.272,"15min":1.373},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.008},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.68799999999999994},"max":{"1min":1.2569999999999999,"5min":1.657,"15min":1.7190000000000001},"last":1.0940000000000001}]},{"osd":6,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.02,"15min":1.0409999999999999},"min":{"1min":0.85999999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.29,"5min":1.72,"15min":1.72},"last":1.204},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.75800000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.129,"5min":1.2589999999999999,"15min":1.357},"last":0.996}]},{"osd":7,"last update":"Mon Aug 1 23:51:36 2022","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.86699999999999999,"5min":0.754,"15min":0.74099999999999999},"max":{"1min":1.518,"5min":1.518,"15min":2.0369999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.755},"max":{"1min":1.1890000000000001,"5min":1.3320000000000001,"15min":1.411},"last":1.077}]}]},{"osd":0,"up_from":13,"seq":55834575527,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050588,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321252,"statfs":{"total":96636764160,"available":95560962048,"internally_reserved":1073741824,"allocated":2052096,"data_stored":1004957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.42499999999999999},"max":{"1min":0.71299999999999997,"5min":1.012,"15min":1.173},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.438},"max":{"1min":0.90400000000000003,"5min":1.0409999999999999,"15min":1.2030000000000001},"last":0.85699999999999998}]},{"osd":2,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47799999999999998},"max":{"1min":0.79500000000000004,"5min":1.103,"15min":1.135},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.73799999999999999},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":0.89500000000000002,"5min":1.087,"15min":1.095},"last":0.59199999999999997}]},{"osd":3,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.48699999999999999},"max":{"1min":0.90300000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97899999999999998,"5min":1.042,"15min":1.26},"last":0.67600000000000005}]},{"osd":4,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.67000000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2629999999999999,"5min":1.321,"15min":1.4950000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92900000000000005,"15min":0.92200000000000004},"min":{"1min":0.73199999999999998,"5min":0.68999999999999995,"15min":0.64700000000000002},"max":{"1min":1.1020000000000001,"5min":1.3080000000000001,"15min":1.331},"last":0.91400000000000003}]},{"osd":5,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93200000000000005,"15min":0.93999999999999995},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.68300000000000005},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.016,"15min":1.038},"min":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.70099999999999996},"max":{"1min":1.323,"5min":1.3360000000000001,"15min":1.4299999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.86899999999999999,"5min":0.72799999999999998,"15min":0.70699999999999996},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.075,"15min":1.0669999999999999},"min":{"1min":0.84999999999999998,"5min":0.80000000000000004,"15min":0.76700000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.4330000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Aug 1 23:51:38 2022","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1040000000000001,"15min":1.1000000000000001},"min":{"1min":0.84099999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.3680000000000001,"5min":1.4179999999999999,"15min":1.423},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.75600000000000001},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.7210000000000001},"last":1.3720000000000001}]}]}]}} 2022-08-01T23:52:05.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2022-08-01T23:52:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.423+0000 7fa43ffff700 1 Processor -- start 2022-08-01T23:52:05.428 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.423+0000 7fa43ffff700 1 -- start start 2022-08-01T23:52:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4400b3190 0x7fa440060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4400b2730 0x7fa440061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.431 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400666b0 0x7fa440066af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa440062190 con 0x7fa4400b3190 2022-08-01T23:52:05.432 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4400654c0 con 0x7fa4400b2730 2022-08-01T23:52:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4400b3570 con 0x7fa4400666b0 2022-08-01T23:52:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4400b3190 0x7fa440060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400666b0 0x7fa440066af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.434 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4400b3190 0x7fa440060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50738/0 (socket says 172.21.15.32:50738) 2022-08-01T23:52:05.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa437fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4400b2730 0x7fa440061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.435 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43effd700 1 -- 172.21.15.32:0/2012895177 learned_addr learned my addr 172.21.15.32:0/2012895177 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:52:05.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 -- 172.21.15.32:0/2012895177 >> v2:172.21.15.55:3300/0 conn(0x7fa4400b2730 msgr2=0x7fa440061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:52:05.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 --2- 172.21.15.32:0/2012895177 >> v2:172.21.15.55:3300/0 conn(0x7fa4400b2730 0x7fa440061190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.436 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 -- 172.21.15.32:0/2012895177 >> v2:172.21.15.32:3300/0 conn(0x7fa4400b3190 msgr2=0x7fa440060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T23:52:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 --2- 172.21.15.32:0/2012895177 >> v2:172.21.15.32:3300/0 conn(0x7fa4400b3190 0x7fa440060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.437 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 -- 172.21.15.32:0/2012895177 --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44005fbf0 con 0x7fa4400666b0 2022-08-01T23:52:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43f7fe700 1 --2- 172.21.15.32:0/2012895177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400666b0 0x7fa440066af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fa4400648f0 tx=0x7fa4380091d0).ready entity=mon.2 client_cookie=d5d9fd045d7fdca6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:52:05.438 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43e7fc700 1 -- 172.21.15.32:0/2012895177 <== mon.2 v2:172.21.15.55:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43800b5a0 con 0x7fa4400666b0 2022-08-01T23:52:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43e7fc700 1 -- 172.21.15.32:0/2012895177 <== mon.2 v2:172.21.15.55:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43800b700 con 0x7fa4400666b0 2022-08-01T23:52:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43e7fc700 1 -- 172.21.15.32:0/2012895177 <== mon.2 v2:172.21.15.55:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43800b870 con 0x7fa4400666b0 2022-08-01T23:52:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- 172.21.15.32:0/2012895177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400666b0 msgr2=0x7fa440066af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- 172.21.15.32:0/2012895177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400666b0 0x7fa440066af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fa4400648f0 tx=0x7fa4380091d0).stop 2022-08-01T23:52:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- 172.21.15.32:0/2012895177 shutdown_connections 2022-08-01T23:52:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- 172.21.15.32:0/2012895177 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400666b0 0x7fa440066af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- 172.21.15.32:0/2012895177 >> v2:172.21.15.55:3300/0 conn(0x7fa4400b2730 0x7fa440061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 --2- 172.21.15.32:0/2012895177 >> v2:172.21.15.32:3300/0 conn(0x7fa4400b3190 0x7fa440060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- 172.21.15.32:0/2012895177 >> 172.21.15.32:0/2012895177 conn(0x7fa440063870 msgr2=0x7fa440063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:52:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- 172.21.15.32:0/2012895177 shutdown_connections 2022-08-01T23:52:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.427+0000 7fa43ffff700 1 -- 172.21.15.32:0/2012895177 wait complete. 2022-08-01T23:52:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 Processor -- start 2022-08-01T23:52:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 -- start start 2022-08-01T23:52:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4400666b0 0x7fa4400a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 --2- >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400b2730 0x7fa4400add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 0x7fa4400ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.448 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4400af420 con 0x7fa4400b3190 2022-08-01T23:52:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4400a36a0 con 0x7fa4400666b0 2022-08-01T23:52:05.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43ffff700 1 -- --> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4400af650 con 0x7fa4400b2730 2022-08-01T23:52:05.450 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 0x7fa4400ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 0x7fa4400ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50748/0 (socket says 172.21.15.32:50748) 2022-08-01T23:52:05.451 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 -- 172.21.15.32:0/490230808 learned_addr learned my addr 172.21.15.32:0/490230808 (peer_addr_for_me v2:172.21.15.32:0/0) 2022-08-01T23:52:05.452 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43effd700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4400666b0 0x7fa4400a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa437fff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400b2730 0x7fa4400add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.453 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 -- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400b2730 msgr2=0x7fa4400add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.454 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400b2730 0x7fa4400add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 -- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4400666b0 msgr2=0x7fa4400a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.455 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4400666b0 0x7fa4400a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 -- 172.21.15.32:0/490230808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438009470 con 0x7fa4400b3190 2022-08-01T23:52:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43f7fe700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 0x7fa4400ae6e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fa4400648f0 tx=0x7fa438016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:52:05.457 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.431+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43800b5a0 con 0x7fa4400b3190 2022-08-01T23:52:05.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43801f5e0 con 0x7fa4400b3190 2022-08-01T23:52:05.458 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4400af8f0 con 0x7fa4400b3190 2022-08-01T23:52:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa43801f940 con 0x7fa4400b3190 2022-08-01T23:52:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4400ad0e0 con 0x7fa4400b3190 2022-08-01T23:52:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa438026030 con 0x7fa4400b3190 2022-08-01T23:52:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43cff9700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4240539e0 0x7fa424055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T23:52:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44005fe90 con 0x7fa4400b3190 2022-08-01T23:52:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa43806c890 con 0x7fa4400b3190 2022-08-01T23:52:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43effd700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4240539e0 0x7fa424055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T23:52:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.435+0000 7fa43effd700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4240539e0 0x7fa424055ea0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fa430008000 tx=0x7fa430008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T23:52:05.463 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.439+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa438015030 con 0x7fa4400b3190 2022-08-01T23:52:05.568 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.563+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fa4400574e0 con 0x7fa4400b3190 2022-08-01T23:52:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa438011e50 con 0x7fa4400b3190 2022-08-01T23:52:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43cff9700 1 -- 172.21.15.32:0/490230808 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7fa438024020 con 0x7fa4400b3190 2022-08-01T23:52:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4240539e0 msgr2=0x7fa424055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4240539e0 0x7fa424055ea0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fa430008000 tx=0x7fa430008040).stop 2022-08-01T23:52:05.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 msgr2=0x7fa4400ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T23:52:05.578 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 0x7fa4400ae6e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7fa4400648f0 tx=0x7fa438016000).stop 2022-08-01T23:52:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 shutdown_connections 2022-08-01T23:52:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:6800/22889,v1:172.21.15.55:6801/22889] conn(0x7fa4240539e0 0x7fa424055ea0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4400b3190 0x7fa4400ae6e0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3301/0,v1:172.21.15.55:6790/0] conn(0x7fa4400b2730 0x7fa4400add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 --2- 172.21.15.32:0/490230808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4400666b0 0x7fa4400a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T23:52:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 >> 172.21.15.32:0/490230808 conn(0x7fa440063870 msgr2=0x7fa4400b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T23:52:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 shutdown_connections 2022-08-01T23:52:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2022-08-01T23:52:05.571+0000 7fa43ffff700 1 -- 172.21.15.32:0/490230808 wait complete. 2022-08-01T23:52:05.587 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph.py", line 1858, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:52:05.591 INFO:teuthology.misc:Shutting down mds daemons... 2022-08-01T23:52:05.592 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T23:52:05.592 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.604 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T23:52:05.605 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T23:52:05.606 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.620 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T23:52:05.621 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T23:52:05.622 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.639 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T23:52:05.641 INFO:teuthology.misc:Shutting down osd daemons... 2022-08-01T23:52:05.642 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2022-08-01T23:52:05.643 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.740 INFO:tasks.ceph.osd.0:Stopped 2022-08-01T23:52:05.741 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2022-08-01T23:52:05.741 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.801 INFO:tasks.ceph.osd.1:Stopped 2022-08-01T23:52:05.802 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2022-08-01T23:52:05.803 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.873 INFO:tasks.ceph.osd.2:Stopped 2022-08-01T23:52:05.874 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2022-08-01T23:52:05.874 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:05.941 INFO:tasks.ceph.osd.3:Stopped 2022-08-01T23:52:05.942 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2022-08-01T23:52:05.943 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.037 INFO:tasks.ceph.osd.4:Stopped 2022-08-01T23:52:06.038 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2022-08-01T23:52:06.039 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.105 INFO:tasks.ceph.osd.5:Stopped 2022-08-01T23:52:06.106 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2022-08-01T23:52:06.107 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.181 INFO:tasks.ceph.osd.6:Stopped 2022-08-01T23:52:06.182 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2022-08-01T23:52:06.183 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.264 INFO:tasks.ceph.osd.7:Stopped 2022-08-01T23:52:06.265 INFO:teuthology.misc:Shutting down mgr daemons... 2022-08-01T23:52:06.266 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2022-08-01T23:52:06.266 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.296 INFO:tasks.ceph.mgr.y:Stopped 2022-08-01T23:52:06.297 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2022-08-01T23:52:06.298 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.323 INFO:tasks.ceph.mgr.x:Stopped 2022-08-01T23:52:06.325 INFO:teuthology.misc:Shutting down mon daemons... 2022-08-01T23:52:06.326 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2022-08-01T23:52:06.327 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.350 INFO:tasks.ceph.mon.a:Stopped 2022-08-01T23:52:06.351 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2022-08-01T23:52:06.352 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.372 INFO:tasks.ceph.mon.b:Stopped 2022-08-01T23:52:06.373 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2022-08-01T23:52:06.374 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T23:52:06.438 INFO:tasks.ceph.mon.c:Stopped 2022-08-01T23:52:06.439 INFO:tasks.ceph:Checking cluster log for badness... 2022-08-01T23:52:06.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2022-08-01T23:52:06.464 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T23:52:06.465 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2022-08-01T23:52:06.894 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T23:52:06.895 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2022-08-01T23:52:07.062 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T23:52:07.063 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2022-08-01T23:52:07.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T23:52:07.248 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2022-08-01T23:52:07.371 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T23:52:07.372 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2022-08-01T23:52:07.595 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T23:52:07.596 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2022-08-01T23:52:07.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T23:52:07.890 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2022-08-01T23:52:08.081 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T23:52:08.082 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2022-08-01T23:52:08.385 INFO:tasks.ceph:Archiving mon data... 2022-08-01T23:52:08.389 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/data/mon.a.tgz 2022-08-01T23:52:08.390 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2022-08-01T23:52:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.6DIydO6F5S 2022-08-01T23:52:08.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.6DIydO6F5S 2022-08-01T23:52:08.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.6DIydO6F5S 2022-08-01T23:52:08.687 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.6DIydO6F5S is 499KB 2022-08-01T23:52:08.759 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.6DIydO6F5S 2022-08-01T23:52:08.767 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/data/mon.b.tgz 2022-08-01T23:52:08.768 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T23:52:08.775 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.5kV9VYo3fw 2022-08-01T23:52:08.776 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.5kV9VYo3fw 2022-08-01T23:52:09.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.5kV9VYo3fw 2022-08-01T23:52:09.075 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.5kV9VYo3fw is 573KB 2022-08-01T23:52:09.139 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.5kV9VYo3fw 2022-08-01T23:52:09.147 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/data/mon.c.tgz 2022-08-01T23:52:09.148 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2022-08-01T23:52:09.197 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.Qbe5B5Smbn 2022-08-01T23:52:09.198 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Qbe5B5Smbn 2022-08-01T23:52:09.428 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.Qbe5B5Smbn 2022-08-01T23:52:09.453 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.Qbe5B5Smbn is 573KB 2022-08-01T23:52:09.517 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.Qbe5B5Smbn 2022-08-01T23:52:09.525 INFO:tasks.ceph:Cleaning ceph cluster... 2022-08-01T23:52:09.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T23:52:09.529 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T23:52:09.531 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T23:52:09.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T23:52:09.552 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2022-08-01T23:52:09.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T23:52:09.586 DEBUG:teuthology.orchestra.run.smithi055:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T23:52:09.590 DEBUG:teuthology.orchestra.run.smithi130:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T23:52:09.594 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T23:52:09.615 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T23:52:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T23:52:09.626 INFO:teuthology.orchestra.run.smithi130.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T23:52:09.632 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T23:52:09.639 INFO:tasks.ceph:Archiving crash dumps... 2022-08-01T23:52:09.643 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi032/crash 2022-08-01T23:52:09.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T23:52:09.668 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi055/crash 2022-08-01T23:52:09.671 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T23:52:09.693 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi130/crash 2022-08-01T23:52:09.698 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T23:52:09.717 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi157/crash 2022-08-01T23:52:09.719 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T23:52:09.741 INFO:tasks.ceph:Shutting down logrotate 2022-08-01T23:52:09.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.747 DEBUG:teuthology.orchestra.run.smithi130:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.750 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.818 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.835 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T23:52:09.868 INFO:tasks.ceph:Compressing logs... 2022-08-01T23:52:09.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:09.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:09.875 DEBUG:teuthology.orchestra.run.smithi130:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:09.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:22.330 INFO:tasks.ceph:Archiving logs... 2022-08-01T23:52:22.332 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi032/log 2022-08-01T23:52:22.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T23:52:25.564 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi055/log 2022-08-01T23:52:25.565 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T23:52:28.479 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi130/log 2022-08-01T23:52:28.481 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T23:52:28.870 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi157/log 2022-08-01T23:52:28.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T23:52:28.907 DEBUG:teuthology.run_tasks:Unwinding manager install 2022-08-01T23:52:28.918 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T23:52:28.922 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2022-08-01T23:52:28.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T23:52:28.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T23:52:28.928 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T23:52:28.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T23:52:29.059 DEBUG:teuthology.run_tasks:Unwinding manager clock 2022-08-01T23:52:29.076 INFO:teuthology.task.clock:Checking final clock skew... 2022-08-01T23:52:29.077 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T23:52:29.080 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T23:52:29.083 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T23:52:29.085 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T23:52:29.101 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T23:52:29.102 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2022-08-01T23:52:29.102 INFO:teuthology.orchestra.run.smithi032.stdout:+hv01.front.sepi 216.240.36.24 3 u 114 128 377 0.181 -2.260 0.436 2022-08-01T23:52:29.103 INFO:teuthology.orchestra.run.smithi032.stdout:*hv02.front.sepi 163.237.218.18 2 u 46 128 377 0.186 -2.331 0.299 2022-08-01T23:52:29.103 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 216.229.4.69 3 u 104 128 377 0.187 -3.670 0.512 2022-08-01T23:52:29.103 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T23:52:29.105 INFO:teuthology.orchestra.run.smithi055.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T23:52:29.106 INFO:teuthology.orchestra.run.smithi055.stdout:============================================================================== 2022-08-01T23:52:29.106 INFO:teuthology.orchestra.run.smithi055.stdout:+hv01.front.sepi 216.240.36.24 3 u 168 256 377 0.171 -2.661 0.739 2022-08-01T23:52:29.106 INFO:teuthology.orchestra.run.smithi055.stdout:*hv02.front.sepi 163.237.218.18 2 u 181 256 377 0.157 -1.649 0.627 2022-08-01T23:52:29.107 INFO:teuthology.orchestra.run.smithi055.stdout:+hv03.front.sepi 216.229.4.69 3 u 177 256 377 0.166 -4.125 0.455 2022-08-01T23:52:29.107 INFO:teuthology.orchestra.run.smithi055.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T23:52:29.108 INFO:teuthology.orchestra.run.smithi130.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T23:52:29.109 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================== 2022-08-01T23:52:29.109 INFO:teuthology.orchestra.run.smithi130.stdout:+hv01.front.sepi 216.240.36.24 3 u 135 128 377 0.190 -0.746 0.231 2022-08-01T23:52:29.110 INFO:teuthology.orchestra.run.smithi130.stdout:*hv02.front.sepi 163.237.218.18 2 u 46 128 377 0.149 -0.427 0.113 2022-08-01T23:52:29.110 INFO:teuthology.orchestra.run.smithi130.stdout:+hv03.front.sepi 216.229.4.69 3 u 119 128 377 0.149 -2.308 0.157 2022-08-01T23:52:29.111 INFO:teuthology.orchestra.run.smithi130.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T23:52:29.113 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T23:52:29.114 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2022-08-01T23:52:29.114 INFO:teuthology.orchestra.run.smithi157.stdout:+hv01.front.sepi 216.240.36.24 3 u 9 128 377 0.169 1.300 0.568 2022-08-01T23:52:29.115 INFO:teuthology.orchestra.run.smithi157.stdout:*hv02.front.sepi 163.237.218.18 2 u 43 128 377 0.194 0.853 0.240 2022-08-01T23:52:29.116 INFO:teuthology.orchestra.run.smithi157.stdout:+hv03.front.sepi 216.229.4.69 3 u 47 128 377 0.180 -1.035 0.236 2022-08-01T23:52:29.116 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T23:52:29.117 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2022-08-01T23:52:29.136 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2022-08-01T23:52:29.138 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2022-08-01T23:52:29.149 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2022-08-01T23:52:29.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2022-08-01T23:52:29.168 INFO:teuthology.task.internal:Duration was 3937.691607 seconds 2022-08-01T23:52:29.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2022-08-01T23:52:29.181 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2022-08-01T23:52:29.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T23:52:29.185 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T23:52:29.187 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T23:52:29.190 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T23:52:29.239 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2022-08-01T23:52:29.240 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2022-08-01T23:52:29.241 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T23:52:29.255 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2022-08-01T23:52:29.256 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T23:52:29.272 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2022-08-01T23:52:29.273 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T23:52:29.289 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2022-08-01T23:52:29.290 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T23:52:29.305 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2022-08-01T23:52:29.306 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:29.309 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:29.311 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:29.333 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T23:52:29.367 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2022-08-01T23:52:29.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T23:52:29.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T23:52:29.389 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T23:52:29.395 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T23:52:29.580 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2022-08-01T23:52:29.593 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2022-08-01T23:52:29.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T23:52:29.612 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T23:52:29.629 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T23:52:29.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T23:52:29.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2022-08-01T23:52:29.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.680 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.683 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.693 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2022-08-01T23:52:29.696 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2022-08-01T23:52:29.698 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2022-08-01T23:52:29.721 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2022-08-01T23:52:29.738 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:52:29.745 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:52:29.771 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:52:29.779 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T23:52:29.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T23:52:29.786 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2022-08-01T23:52:29.798 INFO:teuthology.task.internal:Transferring archived files... 2022-08-01T23:52:29.799 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi032 2022-08-01T23:52:29.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T23:52:29.843 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi055 2022-08-01T23:52:29.844 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T23:52:29.884 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi130 2022-08-01T23:52:29.885 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T23:52:29.923 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_22:38:49-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954891/remote/smithi157 2022-08-01T23:52:29.924 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T23:52:29.965 INFO:teuthology.task.internal:Removing archive directory... 2022-08-01T23:52:29.966 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T23:52:29.969 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T23:52:29.972 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T23:52:29.975 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T23:52:30.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2022-08-01T23:52:30.037 INFO:teuthology.task.internal:Not uploading archives. 2022-08-01T23:52:30.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2022-08-01T23:52:30.057 INFO:teuthology.task.internal:Tidying up after the test... 2022-08-01T23:52:30.058 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T23:52:30.060 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T23:52:30.063 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T23:52:30.066 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T23:52:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 23:52 /home/ubuntu/cephtest 2022-08-01T23:52:30.069 INFO:teuthology.orchestra.run.smithi055.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 23:52 /home/ubuntu/cephtest 2022-08-01T23:52:30.075 INFO:teuthology.orchestra.run.smithi157.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 23:52 /home/ubuntu/cephtest 2022-08-02T10:46:49.398 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-08-02T10:46:49.403 DEBUG:teuthology.task.console_log:Killing console logger for smithi032 2022-08-02T10:46:49.404 DEBUG:teuthology.task.console_log:Killing console logger for smithi055 2022-08-02T10:46:49.405 DEBUG:teuthology.task.console_log:Killing console logger for smithi130 2022-08-02T10:46:49.405 DEBUG:teuthology.task.console_log:Killing console logger for smithi157 2022-08-02T10:46:49.406 DEBUG:teuthology.exit:Finished running handlers